Go to file
Richard Loyer 7a998dec6b chantier B TTS #4: P3.1+P3.2 = x_vector + prefill_embeds depuis input_ids (bit-exact)
P3.1 - x_vector Damien dumpe offline : tts.model.extract_speaker_embedding(damien_15s_24k.wav)
-> f32[1024] = 4 KB. Embed dans l'app comme ressource. Pas de port du speaker
encoder Python (8.9M params), voix fixe = pre-calcul trivial.

P3.2 - reproduction C++ de la construction Python des talker_input_embeds.
Reference modeling_qwen3_tts.py:2124-2233 (mode x_vector_only + non_streaming).
Sequence prefill (19 positions pour 'Bonjour je m'appelle Kazeia', 8 tokens texte):
  0..2  : text_projection(text_embed[input_ids[:3]])                  role <im_start>assistant\n
  3..7  : tts_pad + tok_embd[think/think_bos/lang_fr/think_eos/x_vec]  5 positions codec prefix
  8     : tts_bos + tok_embd[codec_pad]                                 1 position
  9..N+8: text_projection(text_embed[input_ids[3:-5]]) + tok_embd[codec_pad]   text body
  N+9   : tts_eos + tok_embd[codec_pad]
  N+10  : tts_pad + tok_embd[codec_bos]
ou tts_*_embed = text_projection(text_embed[tts_bos/eos/pad_token_id]).chunk(3).

Composants : text_embed [151936, 2048] f32 (1.21 GB, a quantiser en P3.5),
text_projection ResizeMLP(2048->2048->1024, SiLU, bias=True), tok_embd talker
[3072, 1024], x_vector [1024], constants (special token ids).

Validation host (g++ -O2) sur phrase 'Bonjour je m'appelle Kazeia' :
  mean_rmse = 3.2e-8, max_abs = 1.8e-6 sur les 19 positions vs prefill_embeds
  capture du dump Python -> noise pure du f32 (eps ~1.2e-7).
Construction bit-exact.

Reste pour usage texte arbitraire : tokenizer BPE Qwen3 en C++ (llama_vocab
supporte cela en standard, a brancher en P3.4 quand on assemble la lib finale).
Pour l'instant input_ids dumpes depuis Python pour 1 phrase fixe = suffisant
pour P3.3 (integration CP) et P3.4 (assemblage).

Fixtures /opt/Kazeia/tts_talker_dump/ : text_embed.bin, tp_fc1_w/b.bin,
tp_fc2_w/b.bin, damien_xvector.bin, input_ids_full.bin, manifest_text.txt.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-28 15:05:23 +02:00
.claude Rapport R&D complet: synthese session, perf, cascade, #277, briques, reste a faire 2026-05-25 22:27:34 +02:00
dist chantier B TTS #4: P3.1+P3.2 = x_vector + prefill_embeds depuis input_ids (bit-exact) 2026-05-28 15:05:23 +02:00
docs GDN chunkwise P1.2: VTCM state prefill=neutre, prouve compute-bound 3S2/tok -> gain=WY matmul (palier dur). oracle 28/28, pp98 2026-05-26 16:30:08 +02:00
eval eval: verdict (assisté, à valider) -> ship Qwen3.5-lmq4 2026-05-26 23:23:00 +02:00
ql@6e0edba837 ql: bump pour qwen3 dense IMROPE (ql:6e0edba) 2026-05-28 13:54:27 +02:00
.gitignore eval: kit qualité A/B aveugle Qwen3.5-lmq4 vs dense (decode CPU, template, reasoning off) 2026-05-26 22:24:23 +02:00
CHANTIER_HMX_DENSE.md chantier dense-HMX #4: 0x2e décodé = AEE_ECONNRESET (reset watchdog CDSP), pas une corruption de valeurs 2026-05-27 19:20:28 +02:00
CLAUDE.md R&D: battre pte=lm_head HMX INT4+spec decode+ctx binary; mur=lm_head CPU 2026-05-25 18:46:28 +02:00
RAPPORT_RD.md RAPPORT §7: piste kernel GDN fermee — GDN-compute != goulot prefill (fork ql) 2026-05-26 21:46:05 +02:00
c4.sh Dist Kazeia-Engine: libs arm64+headers+JNI bridge+INTEGRATION/TTS doc; STT reste ORT 2026-05-24 21:55:01 +02:00
casc.sh Dist Kazeia-Engine: libs arm64+headers+JNI bridge+INTEGRATION/TTS doc; STT reste ORT 2026-05-24 21:55:01 +02:00
casc2.sh Dist Kazeia-Engine: libs arm64+headers+JNI bridge+INTEGRATION/TTS doc; STT reste ORT 2026-05-24 21:55:01 +02:00
casc3.sh Dist Kazeia-Engine: libs arm64+headers+JNI bridge+INTEGRATION/TTS doc; STT reste ORT 2026-05-24 21:55:01 +02:00
cascade_test.sh Dist Kazeia-Engine: libs arm64+headers+JNI bridge+INTEGRATION/TTS doc; STT reste ORT 2026-05-24 21:55:01 +02:00
cf.sh Dist Kazeia-Engine: libs arm64+headers+JNI bridge+INTEGRATION/TTS doc; STT reste ORT 2026-05-24 21:55:01 +02:00
qcmp.sh Dist Kazeia-Engine: libs arm64+headers+JNI bridge+INTEGRATION/TTS doc; STT reste ORT 2026-05-24 21:55:01 +02:00
spk.sh Dist Kazeia-Engine: libs arm64+headers+JNI bridge+INTEGRATION/TTS doc; STT reste ORT 2026-05-24 21:55:01 +02:00