43 lines
3.0 KiB
Markdown
43 lines
3.0 KiB
Markdown
# Kazeia-Engine — intégration kazeia-android
|
|
|
|
Moteur LLM (+ TTS) **GGUF, sans `.pte`**, fork llama.cpp upstream + backend Hexagon.
|
|
Remplace ExecuTorch/Genie pour le LLM. **STT reste ORT-QAIRT** (inchangé). Prefill NPU /
|
|
decode CPU. Modèle dense (Qwen3) plein NPU ; hybride (Qwen3.5 DDDA) GDN sur NPU corrigé.
|
|
|
|
## 0. Périmètre
|
|
| Brique | Avant | Après |
|
|
|---|---|---|
|
|
| LLM Speaker/Thinker | ExecuTorch `.pte` | **Kazeia-Engine GGUF** |
|
|
| TTS Talker/CP | ggml-cpu | engine (Talker prefill HTP option.) |
|
|
| TTS Decoder | libtts_decoder_ggml | inchangé |
|
|
| STT Whisper | ORT-QAIRT | **inchangé** |
|
|
|
|
## 1. Contenu du paquet
|
|
- `lib/` : `libkazeia_engine.so` (bridge JNI) + `libllama.so` + `libggml{,-base,-cpu,-hexagon}.so` + `libggml-htp-v68/69/73/75/79/81.so` (sélection auto = V79 sur Pad3). 168 MB.
|
|
- `jni/` : `kazeia_engine_jni.cpp`, `EngineLlmEngine.kt`. `include/`, `CMakeLists.txt`.
|
|
- `INTEGRATION.md` (steps), `TTS.md`, `MODELS.md`, `PERF.md`, `PITFALLS.md`.
|
|
|
|
## 2. Build (5 étapes)
|
|
1. `lib/*.so` → `kazeia-android/app/src/main/jniLibs/arm64-v8a/`
|
|
2. `jni/kazeia_engine_jni.cpp` → `app/src/main/jni/`, `include/*.h` à côté
|
|
3. `EngineLlmEngine.kt` → `com/kazeia/llm/`, `System.loadLibrary("kazeia_engine")`
|
|
4. CMake: lib avec libllama+libggml+libggml-base, `-march=armv8.6-a+dotprod+fp16+i8mm+bf16`
|
|
5. GGUF → external storage, paths via `KazeiaApplication.LLM_DIR`
|
|
|
|
## 3. Cascade — remplace LlmProcessor cascade
|
|
Thinker Guard-4B → 4 bullets ; Speaker 9B = SYS_KAZEIA+bullets. Mono-moteur, `reset()` entre tours. Prompts: voir RAPPORT_KAZEIA §8. `--reasoning-budget 0` impératif (sinon ramble anglais) : EngineLlmEngine met thinking off. Mesuré: 9B>4B qualité, Speaker=9B.
|
|
|
|
→ MODELS.md (choix), PERF.md (chiffres), PITFALLS.md (params_fit, no tty, OOM 30B+), INTEGRATION.md (détail API). API: load/generate/reset/free.
|
|
|
|
## VALIDÉ DEVICE 24/05
|
|
test_native (=logique generate bridge) sur Pad3: load 4B HTP+decode CPU+detok = OUT propre, exit0. Pipeline prefill-NPU/decode-CPU prouve end-to-end. dist/jni/test_native.cpp = harness reproductible. Bridge so: 4 symboles JNI + deps ok. Reste app: gradle+jniLibs+template chat.
|
|
|
|
## v2 STATIC (collision libllama TTS resolue)
|
|
TTS Talker/CP linke vs son libllama.so -> conflit ABI. FIX: libkazeia_engine.so STATIC (llama+ggml+ggml-cpu+hexagon en .a, --whole-archive hexagon). NEEDED= libm/log/dl/c only, 42MB. Coexiste avec libllama TTS. Drop: libkazeia_engine.so + libggml-htp-v79.so. Valide device 4B HTP. Build: bstatic BUILD_SHARED_LIBS=OFF.
|
|
|
|
## v3 utilisable: thinking-off bridge
|
|
generate(sys,usr,max): wrap ChatML + <think></think> vide = stop reasoning, decode 4B 15tok/s ~5s/tour. test 4B: "Je suis desole..." 40s(load)+gen. signature 2-arg. cap maxTok 64. dist=libkazeia_engine.so static + htp-v79. #277 fix livre.
|
|
|
|
## v4 USABLE: ngl0 CPU decode (vrai fix #277)
|
|
Hang in-app=0.21tok/s: ngl99 -> decode ping-pong NPU/GDN-CPU. FIX: n_gpu_layers=0 decode CPU pur 14tok/s. test 4B: reponse FR 4s/tour. dev avait raison: pas thinking. cap64 garde. Speaker=4B CPU ngl0. RAM 4.5G. dist=libkazeia_engine.so. utilisable, #277 debloque.
|