Kazeia
/
Kazeia-engine
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

GDN chunkwise: doc P1 math+structure+gate (refactor chunks C=64 phase1)

This commit is contained in:
Richard Loyer 2026-05-26 16:13:34 +02:00
parent aa363e652f
commit 837d9de766
1 changed files with 5 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
docs/04_gdn_chunkwise.md
View File

@ -16,3 +16,8 @@ Chunk C=64: intra-chunk = matmuls C×S triangulaire, inter-chunk = state. Les ra
Risque: HMX fp16 accum non bit-exact — OK prefill (decode CPU exact). Frontière, aide partielle structure/debug.
## P0 fait (26/05): oracle = `test-backend-ops -o GATED_DELTA_NET -b HTP0` = 28/28 OK (hs64, 256tok, kda). Gate régression. K>1 non-sup=decode, hors prefill. baseline pp512=93. build: ninja .../libggml-htp-v79.so. P1 en cours.
## P1 — chunk f32 (réf code: pp_thread:573-695)
Récurrence/token: S(SxS)*=gate ; sum=S·k ; δ=(v-sum)·β ; S+=k⊗δ ; out=S·q. Helpers: gdn_mul_dot4_f32 (decay+S·k), gdn_add_scaled_dot4_f32 (rank1+S·q), kda=vecteur/scalar gate. Sériel sur n_tokens = ancre.
Chunkwise C=64: A=tri(K K^T) corrigé β,gate → W=(I-A)^-1 ; intra: out += (Q K^T tri)·V ; inter: out += Q·S, S=gate·S+K^T(βV). 3 GEMM CxS au lieu de 3·S² rank1/tok. f32 d'abord, bit-exact oracle, gain cache. C=64 mappe tuile HMX. Gate: GGML_HEXAGON_GDN_CHUNK=1 opt-in tant que pas validé.
Étapes: (1) refactor pp en boucle de chunks C=64 (1 chunk=ancien path) -> 28/28 inchangé. (2) intra-chunk parallèle f32. (3) bench. risque=δ inter-chunk gate, garder pp_thread comme fallback C=1.