Kuphatikizana Mosalekeza kuchokera ku Mfundo Zoyamba (2025)

Kuphatikizika kopitilira muyeso ndi njira yolimbikitsira yomwe imapangitsa kuti hardware ichuluke kwambiri poyika zopempha zatsopano mu batchi yokonzekera pomwe kagawo kamamasuka, ndikuchotsa kusagwira ntchito pakati pa ntchito. Kuzimvetsetsa kuchokera ku mfundo zoyamba kumawulula chifukwa chake chakhala maziko oyambira pamakina aliwonse apamwamba kwambiri a AI omwe adayikidwa pamlingo mu 2025.

Kodi Ndendende Kubetcha Mosalekeza Ndi Chiyani Ndipo Chifukwa Chiyani Static Batching Inalephereka?

Kuti muyamikire batching mosalekeza, muyenera kumvetsetsa kaye zomwe zidalowa m'malo. Traditional static batching magulu nambala yokhazikika ya zopempha palimodzi, amazipanga ngati gawo limodzi, ndipo amangovomereza zopempha zatsopano gulu lonse likatha. Cholakwika chachikulu ndichakuti mitundu yayikulu ya zilankhulo imapanga zizindikiro zautali wosiyanasiyana - pempho limodzi litha kutha pambuyo pa ma tokeni 20 pomwe lina mugulu lomwelo limatha 2,000. GPU iliyonse mgululi imakhala yopanda kanthu kudikirira kuti nthawi yayitali kwambiri imalize ntchito iliyonse yatsopano isanayambe.

Kuphatikizana mosalekeza, kudachita upainiya pachikalata chodziwika bwino cha 2022 "Orca: Distributed Serving System for Transformer-Based Generative Models," kumathetsa chikakamizochi. Zimagwira ntchito paiteration level osati mulingo wopempha. Pambuyo podutsa kutsogolo kulikonse, wokonza mapulani amayang'ana ngati mndandanda uli wonse wafika kumapeto kwa mndandanda. Ngati zatero, kagawoko kabwezeredwanso ndikutumizidwa ku pempho lomwe lili pamzere - palibe kudikirira, palibe kuwononga. Kapangidwe ka batch kamasintha pang'onopang'ono ndi sitepe iliyonse ya decode, kusungitsa kugwiritsidwa ntchito kwa hardware kufupi ndi kuchuluka kwamalingaliro nthawi zonse.

Kodi Cache ya KV Imalumikizana Bwanji Ndi Kuyimitsa Kosalekeza pa Mulingo Wadongosolo?

Cache yamtengo wapatali ndi kamangidwe ka kukumbukira komwe kumapangitsa kuti ma transfoma azitha kuyenda bwino. Pa chizindikiro chilichonse chomwe chakonzedwa, choyimiracho chimaphatikiza makiyi osamala ndi zikhalidwe zomwe ziyenera kusungidwa kuti ma tokeni otsatira asabwerezenso kuwerengera kofunikira. Mu makina osakanikirana, kugawa kache ya KV ndikosavuta: sungani kukumbukira molingana ndi kutalika kwa mndandanda wa pempho lililonse mu batch.

Kuphatikizana mosalekeza kumapangitsa kuti izi zikhale zovuta. Chifukwa zopempha zimalowa ndikutuluka mu batch nthawi zosayembekezereka, dongosolo silingathe kugawiratu ma block blocks okhazikika. Ichi ndichifukwa chake vLLM's PagedAttention - yomwe idayambitsidwa mu 2023 - idakhala yosasiyanitsidwa ndi kuphatikizika kosalekeza pakutumiza. PagedAttention imabwereka mtundu wapaging wapachikumbutso kuchokera kumakina ogwiritsira ntchito, kugawa cache ya KV kukhala midadada yosagwirizana ndi kukula kwake. Masamba a kachesi amatha kumwazikana pamakumbukiro a GPU monga momwe masamba amakumbukiro amwazikana pa RAM. Zotsatira zake zimakhala zotayika zokumbukira zotsala pang'ono kugawikana, zomwe zimatanthawuza kuti makulidwe apamwamba a batch ndi kupititsa patsogolo popanda ndalama zowonjezera za hardware.

Kodi Njira Zazikulu Zopangira Zomwe Zimapangitsa Kuti Batch Igwire Ntchito Mosalekeza?

Zisankho zitatu zodalirana zimayang'anira makina onse osalekeza:

• Lamulo lodziletsa: Pamene kupanikizika kwa kukumbukira kuli kwakukulu ndipo pempho latsopano lofunika kwambiri lifika, wokonza ndondomeko ayenera kusankha ngati ayang'ana motsatira ndondomeko yomwe ili yofunika kwambiri, kusintha kache ya KV kukhala CPU RAM, kapena kubwereza kuyambira pachiyambi. Kusinthana kochokera ku preemption kumateteza kuwerengera koma kumadya bandwidth ya PCIe; recomputation imawononga ma GPU koma imasunga kukumbukira koyera.
• Ulamuliro wovomerezeka: Wokonza mapulani akuyenera kuneneratu ngati kache ya KV ya pempho latsopano idzakwanira m'makumbukidwe omwe amapezeka m'moyo wake wonse. Kuchepetsa kumayambitsa kuwonongeka kwa kukumbukira kwapakati; kuchulukitsidwa kumadzetsa njala pamzere mosafunikira. Machitidwe amakono amagwiritsa ntchito kugawa kwautali ndi ma buffers osungitsa kuti asamalire zoopsazi.
• Kudzaza mochulukira: Gawo lodzazatu - kukonza zomwe wogwiritsa ntchito alowetsa - silimawerengeka ndipo lingathe kulamulira GPU, kuchedwetsa kusankha masitepe amayendedwe omwe ayamba kale. Kudzaza kwachunked kumagawaniza malangizo aatali kukhala makulidwe osakhazikika omwe amalumikizidwa ndi kubwerezabwereza, kumachepetsa kuchedwa kwanthawi mpaka koyambirira kwa ogwiritsa ntchito nthawi imodzi pamtengo wotsikirako pang'ono.
• Mizere yofunika kwambiri: Pempho la magawo a Enterprise deployments ndi gawo la SLA. Latency-sensitive API imayitanitsa ntchito zoyeserera bwino kwambiri. Popanda wosanjikiza uwu, ntchito imodzi yachidule yachidule imatha kunyozetsa ogwiritsa ntchito pamagawo mazana ambiri.

"Kuphatikizika kosalekeza sikumangopititsa patsogolo ntchito - kumakonzanso ndondomeko ya zachuma ya AI. Mwa kusunga ma GPU otanganidwa ndi granularity iteration m'malo mopempha granularity, ogwiritsira ntchito amapeza 5-10 × kugwiritsa ntchito bwino kwambiri kuchokera ku hardware yofanana, yomwe ndi chiwongolero chimodzi chachikulu chomwe chilipo chochepetsera mtengo wamtundu uliwonse wa 5 mu 2p02."

Kodi Kutumiza Padziko Lonse Kumayesa Bwanji Kupindula Kwa Kachitidwe?

Zotsatira zoyeserera zochokera ku Anyscale, kuphatikiza zodziyimira pawokha m'mabanja angapo achitsanzo mu 2024, zimawonetsa mosalekeza kutulutsa pakati pa 23 × ndi 36 × kutulutsa kwapamwamba poyerekeza ndi naïve static batching pansi pa zochitika zenizeni zamagalimoto. Kupindulaku kumawonekera kwambiri pamene kusiyana kwa kutalika kwa pempho kuli kwakukulu - ndendende mikhalidwe yomwe imadziwika ndi kuchulukidwa kwa ntchito za AI pomwe mafunso a ogwiritsa ntchito amayambira pa mawu atatu ofunikira mpaka zolemba zamasamba ambiri.

Latency ikufotokoza nkhani yowonjezereka. Chizindikiro cha nthawi-to-first-token chimayenda bwino kwambiri chifukwa makinawo sadikiriranso kuti gulu lonse lokhazikika lisonkhanitsidwe lisanayambe kudzaza. Inter-token latency imakhalabe yokhazikika pansi pa katundu wocheperako koma imanyozeka mwachisangalalo m'malo mwa kugwa, chifukwa wokonza ndondomeko akupitiriza kupita patsogolo pamatsatidwe onse ogwira ntchito ngakhale mzerewo ukukula. Kwa mabizinesi omwe akupanga mawonekedwe a AI munthawi yeniyeni, njira yokhotakhota iyi nthawi zambiri imakhala yofunika kwambiri pazamalonda kuposa manambala apamwamba kwambiri.

Kodi Mabizinesi Angagwiritsire Ntchito Motani Mfundo Zotsatizana Zosalekeza Kupyolera Mlingo wa AI?

Chidziwitso cha kamangidwe ka batch mosalekeza - tenganso zothandizira pamlingo wokulirapo kwambiri ndikuzigawiranso nthawi yomweyo m'malo modikirira kuti ntchito yocheperako imalize - ndi mfundo yodziwika bwino pamakina aliwonse omwe amayang'anira ntchito zambiri. Machitidwe opangira mabizinesi amakumana ndi zovuta zomwezi: ntchito zanthawi yosiyana kwambiri zomwe zimapikisana pakugawana ntchito pa CRM workflows, malonda otopa, mapaipi a analytics, ndi ntchito za e-commerce.

Mewayz amagwiritsa ntchito nzeru imeneyi pa bizinesi yake yonse ya 207-module OS, imayendetsa ntchito zolemetsa papulatifomu yophatikizidwa yogwiritsidwa ntchito ndi mabizinesi 138,000 padziko lonse lapansi. M'malo mokakamiza magulu kuti adikire maulendo a malipoti a batch, mizere yololeza motsatizana, kapena zida zotsekera, Mewayz imayang'anira zochitika zamabizinesi mosalekeza - kudyetsa zomwe zatsirizidwa nthawi yomweyo m'ma module akumunsi momwe wokonzera ma batching mosalekeza amadyetsera ma GPU omasuka kubwerera pamzere wopempha. Zotsatira zake ndikusintha koyezeka kwa kuchuluka kwa magwiridwe antchito enieni abizinesi, osati kungoyerekeza.

Mafunso Ofunsidwa Kawirikawiri

Kodi batching mosalekeza ndi yofanana ndi ma batching amphamvu mu TensorFlow Serving?

Ayi. TensorFlow Serving's dynamic batching imasonkhanitsa zopempha m'magulu amitundu yosiyanasiyana kutengera nthawi ya windows ndi kuya kwa mzere, koma imayendetsabe gulu lililonse kuchokera koyambira mpaka kumapeto. Kuphatikizika kosalekeza kumagwira ntchito pakupanga ma tokeni, kulola kuti ma batch asinthe kupita patsogolo kulikonse. Kusiyana kwa granularity ndichifukwa chake batching mosalekeza imakwaniritsa zochulukira kwambiri pakuchulukira kwa ntchito zamtundu wa autoregressive makamaka.

Kodi batching mosalekeza imafuna masinthidwe amitundu?

Mamangidwe amtundu wa thiransifoma safuna kusinthidwa. Kuphatikizika kosalekeza kumayendetsedwa kwathunthu pagawo lothandizira kudzera pakusintha kwa inference schedule, memory manager, ndi kernel. Komabe, kukhathamiritsa kwina - makamaka PagedAttention - kumafunikira ma maso a CUDA omwe amalowa m'malo mwa kukhazikitsidwa kokhazikika, ndichifukwa chake mapangidwe opitilira muyeso ngati vLLM ndi TensorRT-LLM salowa m'malo mwa maseva olozera zolinga wamba.

Ndi zovuta zotani za Hardware zomwe zimalepheretsa batching mosalekeza?

GPU HBM bandwidth ndi kuchuluka kwa VRAM ndizovuta zazikulu. Zosungira zazikulu za KV zimafunikira kukumbukira zambiri, kuletsa kuyanjana kwakukulu. Kulumikizana kwakukulu kwa bandwidth (NVLink, Infiniband) kumakhala kofunikira pakutumiza kwa ma GPU angapo komwe KV cache iyenera kugawidwa pazida zonse. M'malo osakumbukika, kuchulukitsidwa kwaukali kwa KV cache values (kuchokera ku FP16 mpaka INT8 kapena INT4) kumabwezeretsanso mphamvu pamtengo wowonongeka pang'ono womwe ndi wovomerezeka pazogulitsa zambiri.

Kaya mukupanga zinthu zoyendetsedwa ndi AI kapena mukupanga mabizinesi ovuta m'gulu lanu lonse, mfundo yayikulu ndi yofanana: chotsani nthawi yopanda ntchito, bwezerani mphamvu mosalekeza, ndikukonza ntchito yochulukirapo ndi zinthu zomwe muli nazo kale. Mewayz amayika mfundo imeneyi m'magawo 207 ophatikizika - kuchokera ku CRM ndi e-commerce kupita ku analytics ndi mgwirizano wamagulu - kuyambira $19 pamwezi.

Mwakonzeka kuyendetsa bizinesi yanu mokwanira? Yambani kuyesa kwanu kwaulere pa app.mewayz.com ndikuwona momwe mabizinesi 138,000 akuchitira mwanzeru ndi Mewayz.