Kazeia
/
Kazeia-engine
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

R&D: battre pte=lm_head HMX INT4+spec decode+ctx binary; mur=lm_head CPU

This commit is contained in:
Richard Loyer 2026-05-25 18:46:28 +02:00
parent 025bdd87bd
commit 9b3262a8aa
1 changed files with 3 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
CLAUDE.md
View File

@ -129,3 +129,6 @@ pte NPU: 15tok/s, 1.7s, 2.2GB. GGUF bridge: 0.32tok/s (1thread+HTP res), 4.5GB.
## Briques final 2B/4B/9B ## Briques final 2B/4B/9B
2B=13, 4B=7, 9B=2.1tok/s (NPU stack+CPU lm_head). RAM 2.5/3/6.8GB. 9B P4 233ms vocab248k bad. 2B briques 13 sweet spot > engine 4B 9.8/9B 5.2. pour Pierre=2B. JNI206 reste. 2B=13, 4B=7, 9B=2.1tok/s (NPU stack+CPU lm_head). RAM 2.5/3/6.8GB. 9B P4 233ms vocab248k bad. 2B briques 13 sweet spot > engine 4B 9.8/9B 5.2. pour Pierre=2B. JNI206 reste.
## Plan R&D battre pte
V79 INT4/8/FP8/16, 8MB TCM, HMX fp16 tile32x32. .pte vite=tout-NPU INT4. mur=lm_head CPU 40% (NPU 3.5GB 32bit addr). 1.lm_head HMX INT4 group+LUT vocab tile. 2.spec decode 0.8B->2B. 3.QNN ctx binary partage poids. 4.drop bit-exact fp16. memory-wall=90% decode. briques 2B 13 base. effort mois, frontiere kernel aide partielle.