Kazeia
/
Kazeia-engine
3
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
Kazeia-engine/dist/jni
History
Richard Loyer e42e8f8464 SE.1+SE.2+SE.3 partiel : speaker encoder pipeline POC bout-en-bout 
Chantier Option C : embarquer le speaker encoder Qwen3-TTS dans Kazeia-Engine
pour permettre le clonage vocal depuis n'importe quel WAV (5-10s) en
tout-tablette sans étape Python offline.

SE.1 Audit : speaker_encoder_weights.pt = ECAPA-TDNN classique
  - 76 tenseurs, 8.85M params (33.8 MB f32 -> 17 MB f16)
  - blocks.0 Conv1d(128->512 k=5) -> 3x SE-Res2Net (tdnn1+res2net+tdnn2+se)
    -> MFA concat(1536) -> ASP(stats pooling) -> FC(3072->1024)
  - Toutes ops supportées par ggml-cpu (Conv1d via im2col+matmul)

SE.2 Convert : dist/scripts/convert_speaker_encoder_to_gguf.py
  produit speaker_encoder.gguf (17 MB f16) + metadata mel config :
    n_mels=128, sr=24000, n_fft=1024, hop=256, win=1024, fmin=0, fmax=12000
  (bit-exact qwen_tts.core.models.modeling_qwen3_tts.mel_spectrogram)

SE.3 partiel : dist/jni/speaker_encoder.cpp + build_speaker_encoder.sh
  POC bout-en-bout sur tablette :
    WAV damien_5s (24kHz mono, 120k samples = 5s)
      -> mel 128 x 468 frames (FFT radix-2 + Hann + reflect pad + librosa basis)
      -> GGUF loaded (76 tensors)
      -> [TODO session suivante : forward ECAPA-TDNN complet, ~300-400 lignes]
      -> stub x_vector[1024] zeros écrit pour valider l'I/O

  Binaire : 53 KB statique, dépend juste de libggml dynamic.

  mel_basis librosa pré-calculé (24kHz/128 mel/0-12000Hz) dumpé en
  dist/models/mel_basis_qwen3tts.bin (262 KB f32 [128,513]) pour
  reproduire à l'identique torch.stft + librosa filter_bank.

À faire next session :
  - Valider mel C++ bit-exact vs torch.stft Python (sanity)
  - Implémenter forward ECAPA-TDNN : ggml_conv_1d, SE block (squeeze/excite),
    ASP (Attentive Statistics Pooling = stats globaux + tdnn attention),
    FC final.
  - Bit-match damien_xvector.bin référence Python.
  - JNI bridge + Kotlin cloneVoice(refWavPath): FloatArray

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 		2026-05-31 11:37:14 +02:00
..
EngineLlmEngine.kt chantier B TTS #1: API JNI embeds-only + M-RoPE qwen3 dense 2026-05-28 13:37:17 +02:00
TtsEngine.kt TU sampler tuning : top_p 1.0->0.95 + rep_penalty 1.05->1.10 nouveaux défauts 2026-05-31 11:18:25 +02:00
build_prefill.cpp chantier B TTS #4: P3.1+P3.2 = x_vector + prefill_embeds depuis input_ids (bit-exact) 2026-05-28 15:05:23 +02:00
cp_inference.cpp T2.1 CP heads NEON 16-way: CP -33%, pipeline -16%, bit-exact 2026-05-29 17:26:54 +02:00
cp_inference.h chantier F (CP HMX) : path sched livré, gain absent (overhead per-call) 2026-05-29 16:03:41 +02:00
cp_validate.cpp chantier B TTS #5: P3.3 = vrai CP runner intégré (bit-exact) 2026-05-28 15:11:07 +02:00
decoder_subprocess.cpp Decoder chraac en sous-process : -20% decoder, -9% pipeline, talker intact 2026-05-30 18:14:09 +02:00
dual_ctx.cpp dist: prefill HTP = CASSÉ (sortie charabia), prefill obligatoirement CPU 2026-05-27 09:42:59 +02:00
dual_ctx_mt.cpp dist: JNI option C câblé (prefill HTP / decode CPU), validé multi-tour 2026-05-27 10:24:55 +02:00
kazeia_engine_jni.cpp chantier B TTS #2: fix M-RoPE positions en embeds-mode + validation bit-exact 2026-05-28 13:54:03 +02:00
kazeia_text_tokenizer.cpp chantier B TTS #10: tokenizer Qwen3 BPE via llama_tokenize, texte arbitraire 2026-05-28 21:21:53 +02:00
kazeia_text_tokenizer.h chantier B TTS #10: tokenizer Qwen3 BPE via llama_tokenize, texte arbitraire 2026-05-28 21:21:53 +02:00
kazeia_tts_jni.cpp chantier B TTS #11: libkazeia_tts.so + Kotlin wrapper, JNI validé bout-en-bout 2026-05-28 21:54:21 +02:00
sampler.cpp chantier B TTS #7: sampler HF-style top_k+top_p+rep_penalty (générique) 2026-05-28 16:55:05 +02:00
sampler.h chantier B TTS #7: sampler HF-style top_k+top_p+rep_penalty (générique) 2026-05-28 16:55:05 +02:00
speaker_encoder.cpp SE.1+SE.2+SE.3 partiel : speaker encoder pipeline POC bout-en-bout 2026-05-31 11:37:14 +02:00
test_engine.c Valide device: generate() bout-en-bout OK (test_native), bridge pret integration 2026-05-24 22:05:44 +02:00
test_engine.cpp Valide device: generate() bout-en-bout OK (test_native), bridge pret integration 2026-05-24 22:05:44 +02:00
test_engine_2calls.cpp chantier B TTS #11: libkazeia_tts.so + Kotlin wrapper, JNI validé bout-en-bout 2026-05-28 21:54:21 +02:00
test_jni_native.cpp dist: multi-tour propre (generateRaw + ChatSession) + JNI validé end-to-end natif 2026-05-27 11:18:20 +02:00
test_jni_tts.cpp chantier B TTS #11: libkazeia_tts.so + Kotlin wrapper, JNI validé bout-en-bout 2026-05-28 21:54:21 +02:00
test_native.cpp v4 fix #277: ngl0 CPU decode 14tok/s, 4s/tour, usable 2026-05-25 10:18:31 +02:00
test_talker.cpp chantier B TTS #1: API JNI embeds-only + M-RoPE qwen3 dense 2026-05-28 13:37:17 +02:00
test_talker_replay.cpp chantier B TTS #2: fix M-RoPE positions en embeds-mode + validation bit-exact 2026-05-28 13:54:03 +02:00
test_tokenizer.cpp chantier B TTS #10: tokenizer Qwen3 BPE via llama_tokenize, texte arbitraire 2026-05-28 21:21:53 +02:00
tts_engine.cpp TU sampler tuning : top_p 1.0->0.95 + rep_penalty 1.05->1.10 nouveaux défauts 2026-05-31 11:18:25 +02:00
tts_engine.h TU sampler tuning : top_p 1.0->0.95 + rep_penalty 1.05->1.10 nouveaux défauts 2026-05-31 11:18:25 +02:00
tts_orchestrate.cpp chantier B TTS #6: P3.4+P3.5 = pipeline complet 100% tablette bout-en-bout 2026-05-28 15:25:00 +02:00
tts_pipeline.cpp TU sampler tuning : top_p 1.0->0.95 + rep_penalty 1.05->1.10 nouveaux défauts 2026-05-31 11:18:25 +02:00