Deux bugs de prod trouvés en validant le cycle OTA corpus r2 :
1. ttsEngine default "prod" → toute install fraîche (release patient) chargeait
Qwen3TtsEngine legacy → SIGSEGV déterministe dans llama_context ctor
(libtts_talker_cpu compilé contre llama-upstream, libllama.so embarqué =
fork ql depuis 2026-06-02, dérive ABI llama_context_params). La tablette
release crash-loopait au démarrage du service depuis la migration ; la
tablette dev ne le voyait pas (config tts_engine=lib persistée).
→ default "lib" (seul backend compatible avec les jniLibs livrés).
2. Le corpus RAG était résolu via MODELS_DIR/../rag_corpus : sur tablette dev,
MODELS_DIR = legacy /data/local/tmp (modèles adb) → syncDir lisait l'ancien
corpus alors que l'OTA dépose le composant rag_corpus dans le stockage
externe. → KazeiaPaths.ragCorpusDir avec priorité INVERSE des modèles :
l'externe (installeur/OTA) prime dès qu'il est non vide, fallback legacy.
versionCode 4 / 0.1.3, publié catalog v6.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
L'auto-ingestion ne jouait qu'à base vide (1er démarrage) : une MAJ OTA du
composant rag_corpus déposait les nouveaux fichiers mais le RAG continuait de
servir l'ancien corpus, sans aucun déclencheur (sauf intent manuel rag_ingest).
Rag.syncDir(dir) : ingère tout fichier .txt/.md absent de la base ou dont le
texte diffère du brut stocké (comparaison exacte). Idempotent, n'embed que le
delta → appelé à CHAQUE buildRag, remplace le cas spécial « base vide ».
Ne supprime jamais (docs admin et docs retirés du dossier restent — suppression
via l'admin).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Le « corpus de Damien » (/opt/Kazeia/Kaz/knowledge_base) s'est révélé être des
fichiers de test de la machinerie RAG (licornes, Wikipédia Musk, scrape Tomatis)
— aucune valeur clinique. Corpus écrit ici à la place : +12 fiches (crise
d'angoisse, pensées automatiques, émotions, estime de soi, auto-compassion,
colère, deuil, stress, relaxation musculaire progressive, pleine conscience,
anxiété sociale, demander de l'aide) en plus des 6 existantes.
Lignes éditoriales (README) : une fiche = un thème + une technique concrète,
~1 chunk (maxChars=1200), vouvoiement neutre, jamais de médication/diagnostic/
crise suicidaire (CrisisGuard gère la crise hors LLM). À valider par le
clinicien avant activation patient.
Publié : composant catalogue rag_corpus → corpus-seed-r2, catalog v4 en ligne.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Code-review (2e passe) :
- Faux négatif CrisisGuard : « je serais/suis mieux mort », « mieux mort que
vivant » ne déclenchaient aucun motif. Ajout du motif « mieux mort » + 3 cas
de test. (Sécurité vitale — sensibilité d'abord.)
- normalize() recompilait 3 Regex à chaque appel (1×/message patient) :
hoistées en constantes (MARKS/APOS/SPACES).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rag.ingest : embed D'ABORD, n'écrit la base qu'ensuite, via la nouvelle
RagDb.replaceSource (delete+insert dans UNE transaction). Un échec transitoire
d'embedding ne détruit plus les chunks existants (auparavant deleteSource était
appelé avant la boucle d'embed → corpus tronqué/vidé en silence, non rattrapé
par reingestMissing qui ne reprend que les docs à zéro chunk).
KazeiaService.buildRag/teardownRag : synchronized(ragLock) + buildRag idempotent
(retourne l'instance existante si déjà active). onCreate et un toggle config RAG
peuvent se chevaucher (deux coroutines serviceScope) → sans verrou, deux
EngineEmbedder natifs étaient créés et le perdant fuyait (jamais close()).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CrisisGuard (sécurité vitale) :
- normalize() : suppression accents (NFD) + apostrophes→espace + compactage,
pour matcher la sortie STT FR (souvent sans accents) et la frappe libre.
- PATTERNS étendus : formulations directes manquantes (« je veux mourir »,
« idées suicidaires/noires », « disparaître », « me pendre », « passer à
l'acte », « plus la force de vivre »…). « me pendre » autonome (aucun usage
bénin) ; « me noyer » reste gardé (idiome « se noyer dans le travail »).
- Tests étendus : 12 positifs + 4 négatifs (idiomes « mourir de rire/faim »,
« idées plein la tête », « passe à la pharmacie »). 11/11 verts.
Rag.retrieve() : un chunk plus long que le budget ne fait plus retomber tout
le bloc à null — on garantit ≥1 hit (tronqué si besoin) puis on empile tant
qu'il reste du budget. Évite la perte de contexte pertinent.
Rag.close() : @Synchronized — sérialise avec retrieve()/searchScored() pour
qu'un teardown (toggle RAG off) ne libère pas le contexte natif de l'embedder
pendant une requête provider en cours (évite le SIGSEGV).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Migration tablette dev debug→release + cycle OTA complet exercé de bout en bout
pour la première fois :
- uninstall debug → install release v1 (signé clé Kazeia, cf RELEASE_SIGNING.md)
- bump versionCode 2 / versionName 0.1.1, assembleRelease (verifyJniLibs OK)
- publication Nextcloud : APK v2 + composants RAG (e5 + corpus) + catalog v3
- sur device : Onboarding détecte « Mise à jour disponible v0.1.1 », télécharge
(106 Mo, SHA-256 vérifié), PackageInstaller installe in-place → versionCode=2.
PREUVE CLÉ (la raison d'être de la signature release) : un marqueur de config
posé en v1 (rag_enabled=1, rag_threshold=0.85) a SURVÉCU à la mise à jour v2 →
signature cohérente entre versions → une MAJ ne détruit pas les données patient.
Note : catalog.spec.json/dist non suivis (miroir data). versionCode=2 est la
seule trace git ; la prochaine release repart de là.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Les 23 .so de app/src/main/jniLibs/ (474 Mo, 5 chaînes de build : kazeia-engine,
ExecuTorch, QNN SDK 2.42, Genie, TTS pipeline) sont des artefacts gitignorés
posés à la main — « ça ne buildait que sur cette machine ». On ne les versionne
pas (l'historique gonflerait de ~500 Mo par update engine) ; on versionne leur
DÉFINITION :
- scripts/jnilibs.MANIFEST.sha256 : état attendu (sha256 + provenances).
- scripts/jnilibs.sh : verify (CI/Gradle) | sync-engine (rafraîchit le
sous-ensemble kazeia-engine depuis /opt/Kazeia-engine/dist) | update-manifest
(re-fige, à committer) | export/import (tarball pour autre machine).
- Garde Gradle : preReleaseBuild dépend de verifyJniLibs → un build RELEASE avec
des libs manquantes/modifiées/non déclarées ÉCHOUE. Debug reste libre.
Validé : release passe avec libs conformes ; échoue sur lib altérée (MODIFIÉ
libomp.so) ; verify OK après restauration. Ménage au passage : 3 .bak morts
(~293 Mo) sortis de jniLibs vers _jnilibs_backup_baks/.
Nouvelle machine : git clone + ./scripts/jnilibs.sh import <tarball> (export
déposé sur box.kazeia.com privé) → build garanti conforme.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Premier harnais de tests du projet. testImplementation junit:4.13.2, src/test/.
CrisisGuardTest — NON-RÉGRESSION DE SÉCURITÉ : 26 formulations d'idéation qui
DOIVENT déclencher (toutes les familles de motifs, casse, variantes STT collées
validées device), 10 messages de détresse ordinaire qui ne doivent PAS déclencher
(tristesse, insomnie, colère, deuil d'autrui, « envie de dormir »…), 2 limites
ASSUMÉES documentées (sensibilité d'abord : « en finir avec ces insomnies » et
mention indirecte du suicide déclenchent), invariants de la réponse (112,
honnêteté « programme », anti-isolement). Toute retouche des PATTERNS doit garder
ces tests verts ; le clinicien ÉTEND les cas, ne supprime pas de positif.
VectorIndexTest : classement cosinus, top-k, seuil, index vide/dim incohérente.
ChunkerTest : tag source, découpe+overlap sous la limite ubatch, texte vide.
./gradlew :app:testDebugUnitTest → 11 tests, 0 échec, <5 s.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Décision : le dépôt git.kazeia.com étant privé (Gitea auto-hébergé, équipe
restreinte), le keystore + credentials y sont versionnés comme sauvegarde —
une panne de la machine de build ne peut plus coûter la clé.
Conséquence assumée : tout accès au dépôt = capacité de signer des mises à
jour Kazeia ; la clé reste dans l'historique git définitivement. Si le dépôt
devait un jour être ouvert/partagé plus largement, faire une rotation de clé
AVANT (cf docs/RELEASE_SIGNING.md §7).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Keystore release créé : /opt/Kazeia/keystore/kazeia-release.jks (RSA 4096,
alias kazeia, valide 2056), credentials dans keystore/credentials.properties
(chmod 600, hors git — la liste blanche du .gitignore racine l'exclut d'office).
Les deux modules lisent credentials.properties : présent → assembleRelease signe
en release ; absent (autre machine/CI) → repli debug + warning, le build ne casse
pas. MÊME clé pour com.kazeia et com.kazeia.admin → permettra la
signature-permission sur le ContentProvider (plan admin).
Validé : assembleRelease des deux apps OK, apksigner confirme le certificat
CN=Kazeia (SHA-256 4b408d9d…) sur les deux APK.
docs/RELEASE_SIGNING.md : gestion complète — câblage, procédure de release via
catalogue (bump versionCode obligatoire), SAUVEGARDE du keystore (le point qui
ne pardonne pas), scénarios de panne (perte clé/mdp/machine/compromission),
migration one-shot des tablettes debug→release, dev quotidien reste en debug.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fusionne les garde-fous du prompt de Damien dans la voix chaleureuse de Kazeia,
et ajoute un filet de sécurité crise indépendant du LLM.
Bornes "soft" (niveau prompt, DEFAULT_SPEAKER_PROMPT + DEFAULT_SYSTEM_PROMPT +
copie admin, synchronisés) : pas de diagnostic / pas de pathologie nommée ;
médicament/dose/traitement -> décision médicale, renvoi médecin/psychiatre ;
refus des jeux de rôle/simulations ; réponse cadrée sur le stockage local.
Tutoiement + ton chaleureux préservés (≠ style clinique/vouvoiement de Damien).
Borne "hard" (CrisisGuard, DÉTERMINISTE) : on ne laisse jamais un 4B sous
contrainte de brièveté improviser sur une idéation suicidaire. Détection par
motifs FR (tunables clinicien) -> court-circuite Thinker+Speaker -> réponse fixe,
validée, honnête (Kazeia = programme), oriente vers l'humain + 112 (urgence EU,
valable FR et Luxembourg). Placé en tête de processLlmResponse.
Validé device : "envie d'en finir / me faire du mal / vaudrait mieux d'en finir"
-> [CRISIS] -> réponse de sécurité synthétisée (TTS frames=192) + jouée.
"arrêter mes médicaments" -> redirection médecin (prompt, pas crise). "triste et
fatigué" -> réponse normale, AUCUN faux positif crise.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Active le RAG de bout en bout dans la vraie conversation :
- Rag.ingestDir(dir) : ingère tous les .txt/.md d'un dossier (source = nom fichier).
- KazeiaService : auto-ingestion du corpus au 1er démarrage si base vide
(.../kazeia/rag_corpus/), + intent `rag_ingest` (clear+reingest, brique admin future).
- Injection live : seuil relevé à 0.82 (calé e5 : base ~0.80 / pertinent ~0.88), k=3,
pour ne PAS injecter de hors-sujet.
- Corpus seed FR de psychoéducation (6 fichiers) dans docs/rag_corpus_seed/.
Validé device (ragEnabled ON, Speaker lib qwen3.5-4b) : 6 chunks ingérés, tour
patient « je n'arrive pas à dormir… » -> retrieve 1 hit (top 0.84, le chunk sommeil ;
les 5 autres filtrés par le seuil) -> bloc CONTEXTE injecté (577 c) -> réponse ancrée.
Aucun crash, coexistence RAM OK. ragEnabled reste OFF par défaut (opt-in clinicien).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Le dev kazeia-engine a livré le bridge CPU-only avec embedText (commit engine
655d65a) + un GGUF multilingual-e5-small fonctionnel (converter patché pour le
tokenizer XLM-R/unigram). Bridge swappé dans jniLibs (ABI cohérente : libllama/
libggml déjà = build CPU-only du 2 juin).
- Rag.ingest : les passages portent désormais le préfixe "passage: " (e5), en
symétrie du "query: " de retrieve (EngineEmbedder.embedPassage).
- Intent `rag_real_test` : valide le VRAI EngineEmbedder (e5) in-app.
Validé device (CPU-only, untrusted_app) : embedder ready=true dim=384, AUCUN
crash SELinux/fastrpc (la .so ne tire plus hexagon), retrieval FR correct —
requête sommeil → doc insomnie en tête (top=0.88). embedText fonctionne in-app
end-to-end. Le .so (gitignored) est un artefact build, hors git.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Module com.kazeia.rag, conçu d'après le RAG PC de Damien mais porté on-device :
- Embedder : interface + FakeEmbedder (déterministe, test hors engine) +
EngineEmbedder (réel, branché sur EngineJni.loadEmbedder/embedText — inerte
via UnsatisfiedLinkError tant que la lib n'expose pas ces symboles).
- VectorIndex : recherche exhaustive cosinus (produit scalaire, vecteurs L2),
top-k + seuil. Pas d'ANN : injustifié à l'échelle (10^2–10^3 chunks).
- Chunker : découpage par phrases + overlap, tag [Source: x] (mieux que le
découpage au caractère du PC).
- RagDb : SQLite (vecteurs en BLOB float32 LE), garde-fou modèle+dim.
- Rag : façade ingest/loadIndex/retrieve -> bloc CONTEXTE budgété en tokens.
Bindings JNI embedder ajoutés à EngineJni (loadEmbedder/embedText/freeEmbedder).
Flag ragEnabled (ConfigStore + provider rag_enabled), DEFAULT OFF -> production
inchangée. Câblage live dans KazeiaService : si activé + embedder prêt + contexte
au-dessus du seuil, préfixe le bloc CONTEXTE au prompt patient (défensif, le RAG
ne porte jamais la gestion de crise).
Validé device via l'intent `rag_test` (FakeEmbedder) : ingest 2 docs -> SQLite ->
index -> retrieve classe correctement (requête sommeil -> doc insomnie en tête).
Démarrage prod : "[RAG] désactivé (config)", rag_enabled=0. Aucune régression.
Reste (dépend de l'engine) : embedText livré + modèle e5/bge FR + ingestion d'un
vrai corpus côté admin + entrée catalogue. Cf docs/RAG_EMBEDDINGS_ENGINE_SPEC.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ajoute docs/RAG_EMBEDDINGS_ENGINE_SPEC.md : instructions à transmettre au dev
de kazeia-engine pour exposer un entrypoint texte→vecteur poolé (loadEmbedder/
embedText/freeEmbedder) réutilisant la machinerie d'embedding déjà présente dans
le fork libllama. Calée sur dist/jni/kazeia_engine_jni.cpp réel.
Whitelist .gitignore élargie à /docs/.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Le working tree contenait ~21 k fichiers non suivis (modèles, backups, SDK,
venvs, arbres expérimentaux, intouchables beta_kazeia/root_oneplus, Kaz de
Damien…) qui polluaient `git status`. Remplace la blacklist fuyante par une
LISTE BLANCHE : on n'ignore plus au cas par cas, on ignore tout à la racine
sauf l'app mobile (kazeia-android), scripts/, executorch-*, et les docs *.md/
*.txt. Rien n'est supprimé du disque — juste sorti du suivi.
Résultat : non-suivis 21389 → 0. Ajoute au passage des fichiers légitimes qui
traînaient non suivis (FROZEN.md = contrat de stack figée, scripts d'export TTS,
rapport). Le moteur unifié reste un dépôt séparé (/opt/Kazeia-engine).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Capture l'état courant de kazeia-android (migration vers le moteur lib unifié :
suppression des anciens AudioCaptureManager/VadEngine/SileroVad, WhisperJni/
WhisperLiteRt/WhisperNpu/AndroidStt, KazeiaLlmJni/LlamaCppLlmEngine,
AndroidTts/ChatterboxTts ; édits app/JNI/gradle associés). L'app construit et
tourne sur device dans cet état. Commit de préservation pour ne rien perdre
avant de restreindre le périmètre de suivi du dépôt.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ModelRegistry devient backend-aware (PTE | GGUF) : enum Backend, ggufFile +
ggufPath() sous MODELS_DIR, fileExists() par backend, et un type de template
ChatTemplate (QWEN35_THINKOFF | PLAIN_CHATML). Ajout de deux entrées Speaker
GGUF servies par le moteur lib : Qwen3.5-4B (q35-lmq4, le défaut lib jusqu'ici
codé en dur, désormais visible) et Qwen2.5-7B (Q4_K_M, arch qwen2 dense).
KazeiaService : le path lib route le GGUF du Speaker sélectionné (sp.ggufPath())
au lieu de q35-lmq4 en dur, et passe plainChatml selon le template. Pour les
modèles sans thinking (Qwen2.5), EngineLlmAdapter construit un ChatML standard
sans bloc <think> via generateRaw (le natif l'injecterait sinon).
Affordances de test/maintenance (pilotage adb sans UI) :
- llm_text / llm_max : génération LLM pure loggée (tok/s, chars), hors pipeline TTS.
- cfg_set : persiste engine+model+prompt via am --es (contourne content --bind
qui casse sur ':'); cfg_sys=DEFAULT résout le prompt thérapeutique en interne.
Provider /models : expose le chemin GGUF pour les entrées GGUF.
Validé device : les deux GGUF chargent, FR cohérent, templates propres.
Bench : Qwen3.5-4B ~13.5 tok/s, Qwen2.5-7B ~6.8 tok/s (CPU-only) ; qualité FR
thérapeutique nettement en faveur du Qwen3.5-4B.
Note : KazeiaService.kt et les 3 fichiers nouveaux portent aussi du travail
antérieur non commité (migration moteur lib + app admin) ; ce commit en capture
l'état courant en plus de la fonctionnalité Speaker GGUF.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New tool + generated artefacts so the on-device voice spinner can now
hot-swap between all 8 voices — previously only Damien's prefix/suffix
were present in the model dir, and the tablet fell back to him
regardless of selection.
scripts/export_voice_prefix_suffix.py runs Qwen3TTS's voice-clone
path under a forward hook, captures the first prefill call's 1024-dim
talker input embeddings, aborts the rest of the (very slow on CPU)
decode via a sentinel exception, and slices out the first 9 vectors
as <name>_voice_prefix.bin and the last 2 as <name>_voice_suffix.bin.
Validated against the shipped damien_voice_prefix.bin: using
damien_15s_24k.wav as the reference audio, max|diff| = 0, so the
extraction matches the original tooling bit-for-bit.
Generated and adb-pushed to
/data/local/tmp/kazeia/models/qwen3-tts-npu/:
amir / didier / elodie / jerome / richard / sid / zelda
(+ re-generated damien from the canonical 15s_24k reference)
Qwen3TtsEngine.setVoice (already wired) reads <voice>_voice_prefix.bin
/ <voice>_voice_suffix.bin by basename, so voice changes now take
effect from the next synthesized segment with no app restart.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three tightly-coupled UX fixes the user flagged during live testing.
**Voice hot-swap (Qwen3TtsEngine.setVoice)**: previously a no-op
stub — the spinner callback updated the orb color but the actual
audio kept using Damien's cached prefix/suffix embeddings. Now we
derive the voice id from the WAV basename (elodie.wav → 'elodie'),
look up `<id>_voice_prefix.bin` + `<id>_voice_suffix.bin` in the
model dir, parse their headers, and atomically replace the embedding
arrays so the NEXT synthesized segment uses the new voice. If the
files aren't present we log a clear warning pointing at
prepare_tts_native.py — the hot-swap is wired, but per-voice prefix/
suffix still need to be generated offline and adb-pushed.
KazeiaService.setVoice now forwards to Qwen3TtsEngine in addition to
the Chatterbox branch.
**Emoji stripping**: the model loves closing on "😊" and it was
reaching TTS as a standalone segment that synthesized a fraction of
a second of junk. KazeiaPipeline.speakText now runs each sentence
through stripNonSpeakable before enqueueing — drops Unicode emoji /
dingbat / pictograph / flag blocks plus variation selectors and
zero-width joiners, then trims. Empty-after-strip sentences are
skipped entirely. The chat bubble still shows the original text
(with emojis) — only the audio path drops them.
**Typing dots indicator**: while LLM is done but TTS synthesis is
still running (~3–5 s for the first segment), the Kazeia bubble now
shows an animated ". / .. / ..." cycle at 400 ms cadence instead of
sitting empty. The moment the first segment actually starts playing,
the cycle cancels, the bubble resets to empty, and the existing
word-by-word reveal takes over. A defensive finally block also
cancels the job when no segment ever fires (e.g. all-emoji reply).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Dropped the internal waveform lines — not what we wanted visually —
and replaced them with a spectrum-driven deformation of the sphere
outline itself. Each of the 12 log-spaced bands drives one Fourier
mode of the perimeter (band b → mode b + 2, so modes 0/1 stay
circular and higher bands produce tighter ripples). Low bands pull
the shape into wide asymmetric bumps that feel like formants; high
bands add quick sibilant-like tremors. Phase advances faster for
higher modes so tight ripples visually match high-frequency content.
Overall displacement is gated by the RMS envelope so silence is
quiet and loud syllables distort strongly. Fill + highlight are
clipped to the deformed path so the gradient follows the shape and
it reads as a single living object rather than a circle with stuff
bolted on.
Removed drawSpectrumBars and drawWaveformLine.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
**Regression fix**: when synthesis of segment N+1 ran longer than the
playback of segment N (e.g. 5 s synth for a 1.5 s "Bonjour !"), the
previous MediaPlayer was already in the Completed state by the time
we queued the next one. setNextMediaPlayer() on a Completed player is
a documented silent no-op — so the second sentence never started and
the user only heard the first part of the reply.
Rewrote playChainedMediaPlayers with per-player CompletableDeferred
tracking: before calling setNext we check whether current's done has
fired; after awaiting completion we verify next really auto-started
(checking isPlaying / currentPosition) and call start() explicitly if
the chain missed. Belt-and-suspenders against the race either way.
Removed the now-unused waitForPlaybackCompletion helper.
**Visual change**: in-sphere spectrum bars replaced with three
superimposed Bézier "deforming lines", mirrored above/below a central
baseline, with a soft cosine taper so the curves decay to zero at the
sphere's left/right edges (matches the circular mask). Each line has
its own slow-moving phase + gain + thickness + alpha so the three
overlap to give depth — closer to an oscilloscope trace than an EQ.
Low-level sin jitter keeps the lines alive during quiet passages,
amplitude-gated so true silence is a flat line.
User-facing change: no bars anymore. The sphere now "breathes" with
flowing waveforms matching its voice.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Complete redesign of AudioVisualizerView based on feedback: the orb
is now the app's visual face, takes the top ~60% of the chat area,
and has clearly distinct behaviour in each state.
- **Idle**: slow 5 s breathing (scale 0.88 → 1.00 via cos easing),
pure round shape, soft halo in phase. No high-frequency motion.
- **Listening**: organic blob outline built from 8 Fourier modes
whose amplitude scales with live mic RMS; a thin shimmering arc
rotates around the orb while mic energy is present; continuous
micro-ripples pulse outward. Looks clearly 'alive and attentive'
vs Idle's static breathing.
- **Speaking**: the orb becomes a contained spectrometer. A pre-
computed log-spaced spectrogram (12 bands, 120 Hz–4 kHz,
Hann-windowed FFT, one column per 50 ms of audio) is rendered as
vertical rounded-rectangle bars CLIPPED to the sphere outline so
they really look like the sphere itself speaking. Bar heights
interpolate between spectrogram frames and exponentially smooth
toward the target for fluid 60 fps motion. Outer halo pulses with
the RMS envelope; ripples release on envelope peaks.
- **Per-voice color**. Eight-entry palette (Damien lavender,
Elodie rose, Jerome aqua, Richard amber, Amir emerald, Didier
indigo, Sid peach, Zelda periwinkle). Halo, accent, bars, ring,
and ripples are all derived from a single voiceColor so switching
the voice spinner tweens the entire scene to the new identity
over a few frames. Color stored on both KazeiaService (for
persistence across process/view rebinds) and pushed directly to
the view for instant feedback at selection time.
Sidecar pipeline changes:
- Qwen3TtsEngine now computes per-segment spectrogram alongside the
RMS envelope (new computeSpectrogram + an in-place radix-2 FFT).
FFT_SIZE = 1024, hop = 50 ms, 12 log-spaced bands. SegmentReady
carries both arrays; onSegmentPlaying is (sentence, durationMs,
rmsEnvelope, spectrogram).
- KazeiaPipeline.speakText forwards the new callback shape.
- KazeiaService.VisualizerSignal.Speaking now carries the
spectrogram and the new voiceColor StateFlow.
- ChatActivity passes both to the view and collects voiceColor.
Layout: vertical chain between audioViz (weight 3) and rvMessages
(weight 2) so the orb owns ~60% of the chat panel and the chat list
takes the remainder. Removed the fixed 140 dp constraint.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds a breathing lavender orb centred above the chat list that tracks
the actual audio state of the app:
- **Idle**: slow respiratory pulsation (~4 s cycle) at 20 fps. The
chatbot is visually "awake" without animating loudly.
- **Listening**: halo swells with live mic RMS from the VAD loop, so
the user sees Kazeia hearing them even before Whisper has produced
any transcription. Mic RMS is normalised with the same sqrt
squashing the TTS envelope uses so quiet speech still reads visibly.
- **Speaking**: amplitude + halo driven by a pre-computed RMS envelope
(50 ms windows, sqrt-normalised) produced at synthesis time. Ripples
fire on local peaks above 0.35 — matches speech rhythm without
overwhelming. Timer is internal to the view, synced to the segment's
durationMs; no MediaPlayer position polling.
Architecture:
- Sidecar RMS envelope. Computed in Qwen3TtsEngine.generateSegmentAudioVC
right after PCM is available, packed into SegmentReady, and handed to
onSegmentPlaying(sentence, durationMs, rmsEnvelope) when each MediaPlayer
starts. Zero extra IO — runs on the same PCM we already write to WAV.
- KazeiaService exposes VisualizerSignal (Idle | Listening(rms) |
Speaking(env, dur)) as a StateFlow. The VAD loop pushes Listening,
processLlmResponse pushes Speaking from the per-segment TTS callback,
and finally clears to Idle when no mic is open.
- AudioVisualizerView renders via Choreographer.FrameCallback, self-
throttled to 20 fps at Idle and full refresh during Listening/
Speaking. Hardware layer. Pure Kotlin + Canvas, no deps. ~280 LOC.
Layout: 140 dp strip between voiceBar and rvMessages in activity_chat.xml.
No 3D engine, no Unity, no splash extension. The avatar design work
remains on disk for a later phase when the TTS+streaming pipeline
stabilises enough to spend time on DECA/FLAME integration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Matches the 'conversation' feel the user asked for. Previously the
full LLM response appeared in the chat as soon as generation
finished, then audio played 5–10 s later — text and sound felt
decoupled. Now:
- The KAZEIA bubble is created empty and only starts filling when
the first TTS segment actually starts playing through the speaker
(we already split the response by sentence for the chained-
MediaPlayer pipeline; that split drives the reveal too).
- Inside each sentence, words are appended one by one at a cadence
of (audio duration / word count) — slower sentences reveal slower,
matching speech pacing. The first word of each sentence appears
immediately so audio and text stay aligned at the start.
Implementation:
- Qwen3TtsEngine: added `onSegmentPlaying(sentence, durationMs)`
listener, invoked from the chained-MediaPlayer worker the moment
each segment's MediaPlayer.start() lands. Sentence + duration are
carried end-to-end via a new SegmentReady data class.
- KazeiaPipeline.speakText: forwards an optional listener down to
the TTS engine, same signature.
- KazeiaService: new updateMessageText(id, text) helper. In
processLlmResponse, the bubble is added empty before speakText and
grown by a reveal coroutine per sentence; after speakText returns
we snap to the full text as a safety net.
No change to the stream_llm debug intent path — it still uses the
old enqueueSentence flow directly and doesn't need the reveal (no
UI bubble there).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The kill-background approach worked at startup (−1.6 GB) but respawns
pulled most of that back within 1–3 min, and the periodic sweep only
kept the first sweep's reclaim stable rather than going lower. Net
practical benefit over "just let Android manage it" is small, not
worth a custom optimizer + normal-perm + file maintenance.
Removes:
- MemoryOptimizer.kt
- KazeiaService.onCreate calls to freeRamForModels / startPeriodicOptimizer
- KILL_BACKGROUND_PROCESSES permission from the manifest
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
First sweep reclaimed 1.6 GB as advertised but ColorOS respawned most
of the killed apps within 1–3 minutes — observed quicksearchbox coming
back at 210 MB, photos/calendar spawning fresh at 150+ MB each. Two
changes:
1. Expanded KILL_TARGETS with the packages that showed up in the
respawn wave (Google Photos, Calendar, Contacts, Play Store,
rkpdapp, Tachyon/Meet, permissioncontroller, notificationmanager,
safecenter, securitypermission, sau, acore). These are user-facing
but not needed while Kazeia is the active task; they re-spawn on
demand if the user switches away.
2. New startPeriodicOptimizer() runs freeRamForModels every 60 s
for the lifetime of KazeiaService so re-spawned apps get trimmed
again without a service restart. Tied to serviceScope so it stops
cleanly on destroy.
Net effect observed: avail RAM stays ~1.2–1.5 GB higher than without
the sweep. Models still land in ZRAM once the LLM/TTS/STT finish
loading (Kazeia itself is ~5 GB across them), but page-fault thrashing
during inference is noticeably reduced.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three unrelated fixes rolled into one so testing on the tablet stayed
coherent. All were driven by what the user was observing during live
audio tests, not by pre-planned refactors.
1. **Audio playback actually audible.** ColorOS's AudioFlinger
silently muted our AudioTrack ~600 ms after play() every time
(dumpsys audio showed `event:muted updated source:clientVolume`
and playbackHeadPosition stuck at 0), regardless of USAGE_MEDIA /
USAGE_ASSISTANT / USAGE_VOICE_COMMUNICATION, regardless of audio
focus grant, regardless of FGS type including mediaPlayback. A
MediaPlayer path using the SAME usage attributes works because it
routes through a different AudioFlinger thread that isn't under
the same background-hardening policy. `USE_MEDIAPLAYER_FALLBACK`
in Qwen3TtsEngine.kt flips playback to a WAV-per-segment pipeline.
Two MediaPlayer instances are chained via `setNextMediaPlayer()`
so segments transition without re-arming the DAC (that re-arm was
audible as "beg beg" pops between sentences). Synth of seg N+1
runs in parallel with playback of seg N via a capacity-2 Channel,
hiding synthesis latency behind playback for all but the first seg.
2. **Mic no longer loops TTS back into STT.** The continuous-
listening VAD in KazeiaService already had a guard to drop frames
while `pipelineState is Speaking`, but that state was never set by
any caller — so the mic kept recording during playback and fed our
own speaker output back to Whisper, creating the infinite
"Kazeia talks to Kazeia" loop the user observed. Both the
stream_llm intent path and the main `processLlmResponse` TTS path
now wrap the TTS call with `Speaking → Idle/Listening`.
3. **Free 1.6 GB of RAM at service start.** The OnePlus Pad 3 with
ColorOS keeps ~7 GB of Google + OPLUS background services
resident at idle. With Qwen3-4B (3.2 GB) + Qwen3-TTS (1 GB) +
Whisper (0.5 GB) on top, most of our model weights were going to
ZRAM swap — "the NPU is stuck" reports were actually page faults
paging 3 GB of LLM weights back in before each inference. New
`MemoryOptimizer` kills 30-ish non-essential background packages
(Google optional: YouTube, Wallet, Chromecast, Messaging, AICore,
Quicksearchbox; OPLUS optional: smartsidebar, cosa, pantanal,
nhs, midas, …) via `ActivityManager.killBackgroundProcesses`.
Measured reclaim on first run: **avail RAM 8468 MB → 10112 MB,
+1644 MB**. Uses KILL_BACKGROUND_PROCESSES (normal perm, no user
prompt); system-critical packages and the launcher/systemui are
explicitly excluded from the target list.
Collateral changes:
- Added FOREGROUND_SERVICE_MEDIA_PLAYBACK permission + fgsType flag
(didn't fix the mute on its own, but it's correct per Android 14
policy and leaving it without would be a latent compliance risk).
- Kept `USE_STREAMING_DECODE` + CP↔BigVGAN overlap code intact
behind the MediaPlayer-fallback branch so reverting to the
AudioTrack streaming path is a single-const flip if ColorOS ever
lifts the hardening (or we move to a device without it).
- New AudioTrack path has a keep-alive silence watchdog and a
playback-head drain wait on stop. Both were attempts to fix the
mute that didn't pan out on their own; leaving them in so the
streaming path stays usable on non-hardened devices.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ResourceMonitor.init ran `su -c id` at every ChatActivity launch to see
if root was available, then used root to read /sys/class/kgsl/... and
/sys/bus/platform/devices/soc:qcom,msm-cdsp-rm/... for GPU/NPU usage %.
That probe was the only thing still triggering the Magisk auth dialog
on each app start after the no-root LLM migration.
Remove the root probe and the execRoot helper. GPU/NPU reads now return
-1 (UI already renders "—" for negative values). The non-root
/sys/class/kgsl/kgsl-3d0/gpubusy path is kept as a best-effort — it's
world-readable on some devices, silently fails otherwise. CPU and RAM
readouts are unaffected (never needed root).
Dead-code `su -c ...` calls remain in Qwen3TtsEngine (hexStartRunner,
hexStartCpRunner, hexStopRunner, etc.) and WhisperNpuSttEngine, but all
are gated behind fallback paths that don't execute under the current
PTE-only config (talkerPteModule != null && cpPteModule != null short-
circuits before any su call). Left in place to avoid churning the TTS
Hexagon fallback; can be purged in a later cleanup pass if needed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Streaming variant of the per-segment decode pipeline. As soon as SEQ_LEN
codes are accumulated from the talker/CP loop, BigVGAN is dispatched on
a background coroutine while the producer keeps generating the rest of
the segment. The BigVGAN consumer feeds a streaming crossfader that
emits stable audio as it arrives and holds back overlapSamples for the
next chunk's blend.
Mirrors decodeChunked's semantics exactly so final audio is bit-identical
modulo the fadeOut application location (now applied to the final
emission tail instead of the full buffer; the last 40ms still get faded).
Validated A/B on the same prompt 3 used in the recent benchmark:
prompt: "Je me sens un peu triste aujourdhui…"
seg 0 first audio: 14 485 ms → 10 936 ms (−3.5 s)
end-to-end first audio (LLM trigger → audio): 16.2 s → 12.7 s
Stream LLM total: 33 234 ms → 28 594 ms (−4.6 s)
Short segments (<SEQ_LEN codes) and the legacy non-streaming callers
(generateSegmentAudioVC, decodeChunked, multi-segment pipelines, etc.)
are untouched. The new path is gated behind USE_STREAMING_DECODE so it
can be reverted by flipping a single const if a regression is found.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Re-exported Qwen3-4B in hybrid mode (prefill_forward + kv_forward) with
num_sharding=1 after discovering that sharding=2 produces a multi-context
.pte that the LlmModule loader cannot restore (error 5010 "Context group 1
does not exist"). Single-context hybrid .pte loads cleanly through the JNI
runner and the auto-detected eval_mode=1 path.
The peak RAM during export hit 49 GB, which is why sharding=2 was used
originally — the /swapfile (192 GB) now absorbs it. Compile wall time with
sharding=1 + hybrid is ~73 min (two graphs) vs ~30 min for sharding=2 +
kv-only (one graph).
End-to-end on tablet, same 'Bonjour, comment vas-tu ?' prompt:
Before (kv-only, short prompt): TTFT 2865 ms, total 4034 ms
After (hybrid, short prompt): TTFT 113 ms, total 1471 ms
Gain: -2752 ms TTFT (96% reduction, 25× faster)
Response: "Bonjour ! Je vais bien, merci de me demander. Comment vas-tu ?"
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
#2 BigVGAN GPU experiment: ORT-QNN GPU EP loaded the v2_decoder_conv ONNX
model successfully (session creation 463 ms, no fallback warnings) but
per-phrase inference jumped to ~3.5 s vs ~2 s on CPU 8-thread. The GPU/CPU
memory transfer cost dominates for this conv-heavy decoder, and the
optimization went the wrong way. Comment block updated to record both the
HTP and GPU paths as tried-and-rejected so future passes don't re-walk the
same ground.
LLM streaming filter: extend the lookahead-based <think>…</think>
suppressor to also strip singleton special tokens (<|im_start|>,
<|im_end|>, <|endoftext|>). Previously the closing <|im_end|> at end of
the assistant's turn leaked into the SentenceStreamer and ended up as a
spurious sentence at the end of the TTS output. Same lookahead-buffer
trick handles split tokens.
Validated end-to-end: 'Bonjour, comment vas-tu ?' → "Bonjour ! Je vais
bien, merci. Comment vas-tu ?" → seg 0 "Bonjour !", seg 1 "Je vais bien,
merci." (no <|im_end|>), BigVGAN back to 1.8 s/phrase.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the hardcoded eval_mode=0 in the QNN_LLAMA branch with a runtime
check on the loaded module's method names: if the .pte exposes a
prefill_forward graph, switch to EvalMode::kHybrid (1) — the runner can
then batch the entire prompt through prefill_forward in one parallel pass
instead of running 52 ms/token sequentially through kv_forward. Falls
back to kKVCached (0) when only kv_forward exists, matching the current
.pte behaviour exactly so this is a safe in-place upgrade ahead of the
hybrid re-export.
Sanity-tested with the kv-only Qwen3-4B .pte already on the tablet:
Prompt 'Bonjour, ça va ?' → "Bonjour ! Ça va, merci de me demander ça.
Tu as une question ?", TTFT 2728 ms, total 4158 ms — no change vs the
hardcoded eval_mode=0 build.
Once the hybrid Qwen3-4B export finishes (~50 min compile, both
prefill_forward + kv_forward graphs), the same JNI binary will pick up
the new .pte and TTFT should drop to <1 s.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The verbose 55-token system prompt was the cheapest TTFT win on the
kv-only path (52 ms per prefill token). Compacting it to 25 tokens while
keeping the three load-bearing constraints — Kazeia identity, French only,
short replies, /no_think — measurably improved end-to-end latency.
Validated 'Bonjour, comment vas-tu ?' on tablet:
Before: prompt_tokens=80, TTFT=4202ms, total=5716ms
After: prompt_tokens=53, TTFT=2865ms, total=4034ms (-1.3s, -32% TTFT)
Reply quality preserved: "Bonjour ! Je vais bien, merci. Comment vas-tu ?"
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Concrete measurements taken 2026-04-14 on the same Qwen3-4B .pte and the
same C++ runner — only the invocation path differs (subprocess su -c vs
in-process LlmModule JNI). Confirms no LLM regression and a measurable
speedup on the TTS path thanks to the shared QNN context (Talker 37 ms/step
vs 45-65 ms/step before).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The root cause was process-credential loss across fork+exec, not the QNN
SDK version mismatch I had hypothesized. Switching the LLM to in-process
ExecuTorch LlmModule (Zygote-forked context, accepted by adsprpcd's
FastRPC credential check) eliminated the su requirement.
The original investigation sections are kept verbatim for reference; the
new section 10 documents the actual fix, the patches applied to ExecuTorch,
the metrics validated end-to-end, and pointers to the project memory entry.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Even with /no_think in the system prompt Qwen3 still emits an empty
<think>…</think> wrapper before the real answer. Without filtering, the
SentenceStreamer treats '<think>' as a sentence boundary and feeds three
tokens of XML into the TTS, producing audible parasites at the start of
each reply.
The new in-callback filter buffers a small lookahead (just enough to span
"</think>"), suppresses everything between the open and close tags, and
flushes the surrounding prose to onToken in order. With the lookahead, tags
that arrive split across decoded pieces ("<thi"+"nk>") still match.
Validated end-to-end: prompt 'Bonjour, comment vas-tu ?' now streams
sentence-by-sentence to the TTS — first segment "Bonjour !" reaches the
talker at 4.6 s, no <think> sneak-through.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
End-to-end validation on OnePlus Pad 3 with stream_llm intent:
Prompt: 'Bonjour, comment vas-tu ?'
Response: 'Bonjour ! Je suis là pour t'écouter. Comment vas-tu aujourd'hui ?'
TTS: Talker(PTE) 37ms/step, CP(PTE) 73ms/step, audio synthesized.
No su, no Magisk prompts.
Two fixes since the previous commit:
1. ExecuTorchLlmEngine: pass echo=false to LlmModule.generate() — by default
the runner echoes the prompt tokens back via the callback, which fed the
ChatML wrap (<|im_start|>user …) into the SentenceStreamer and TTS.
2. jni_layer_llama.cpp: pick Runner<uint8_t> vs Runner<uint16_t> based on the
model's get_kv_io_bit_width metadata, mirroring qnn_llama_runner.cpp main().
The hard-coded uint16_t was wrong for our Qwen3-4B export (which uses 8-bit
KV I/O) and produced fluent-looking but completely random tokens
("blocked罩ug darkestSOLEQuotes作者本人 …") — same symptom whether greedy or
sampled, the smoking gun for a width-mismatched KV cache reinterpretation.
Other tweaks:
- temperature=0.0 in the QNN_LLAMA branch of jni_layer_llama.cpp (greedy,
matches the working qnn_llama_runner --temperature 0 invocation)
- shared_buffer=true (same as binary defaults)
- Kotlin chat template mirrors qnn_llama_runner.cpp's get_formatted_prompt for
Qwen3 (user-first, then optional system, then "<|im_start|>assistant" with
no trailing newline — that quirky ordering is what the .pte was trained on)
TFTT is ~4 s for a 77-token prompt on kv-only mode (sequential prefill, one
forward per token). To get a sub-second TTFT we'd need to re-export the model
in --model_mode hybrid which adds a parallel prefill_forward graph; not
required for the conversational use case.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The root cause of the previous su-c requirement was that Qualcomm's FastRPC
kernel driver rejects processes spawned via ProcessBuilder fork+exec because
they lose supplementary GIDs on exec. Zygote-forked app processes retain the
proper init-configured credentials and are accepted by the adsprpcd service,
which is why ORT-QNN (Whisper, in-process) worked while the subprocess
qnn_llama_runner did not. Running the LLM in-process via ExecuTorch's
LlmModule bypasses the fork+exec path entirely.
What this commit does:
- ExecuTorchLlmEngine now uses org.pytorch.executorch.extension.llm.LlmModule
with MODEL_TYPE_QNN_LLAMA=4 (routes to example::Runner in jni_layer_llama.cpp,
the same C++ runner that qnn_llama_runner embeds).
- All su, ProcessBuilder, file-based prompt/response plumbing, and run_llm.sh
gone. ChatML template is built in Kotlin; tokens stream in via LlmCallback.
Supporting changes under executorch-patches/llm_in_process_jni.patch:
1. backends/qualcomm/CMakeLists.txt — gate PyQnnManagerAdaptor on NOT ANDROID.
The original guard (CMAKE_SYSTEM_PROCESSOR MATCHES x86_64) misfires in a
nested scope during Android cross-compile and tried to build the host
Python bindings.
2. extension/android/jni/jni_layer_llama.cpp — hardcode decoder_model="qwen3"
(was "llama3") and pass eval_mode=0 (EvalMode::kKVCached) + shared_buffer=true
to match our hybrid_llama_qnn.pte which only contains kv_forward, not
prefill_forward.
Build: scripts/build_android_library.sh arm64-v8a with QNN_SDK_ROOT pointing
to /opt/Kazeia/qnn_sdk_242/qairt/2.42.0.251225 and EXECUTORCH_BUILD_QNN=ON.
Produces libexecutorch_jni.so (192 MB) with QNN v2.42 backend + the llama
runner code, plus libqnn_executorch_backend.so. Both staged in jniLibs.
Validated on OnePlus Pad 3: LlmModule.load() completes in 4.2 s, no su
prompts, Pipeline ready with STT(WhisperHybridEngine) → [VoiceCommands →
LLM] → TTS(Qwen3TtsEngine). TTS .pte still loads with the upgraded v2.42
runtime — no regression.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Commit de sauvegarde avant la tentative d'unification QNN SDK v2.37 et
suppression du su -c pour le LLM. État actuel fonctionnel :
- LLM Qwen3-4B via su -c qnn_llama_runner (v2.42 dans /data/local/tmp/kazeia-et/)
- TTS talker + CP via ExecuTorch .pte JNI (v2.31 dans jniLibs)
- STT Whisper via ORT-QNN 1.24.3
Le rapport kazeia-no-root-report.md documente en détail les tentatives de
no-root et leurs échecs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ExecuTorchLlmEngine: system prompt forces French, 1-2 short sentences,
/no_think so the full budget goes to the answer (Qwen3 was consuming
120+ tokens on <think>); eval_mode 0 matches our kv-mode export.
- Qwen3TtsEngine.generateSegmentAudioVC: when the Hexagon talker socket
isn't open, fall back to runInterleavedPteFromEmbeds so the Stage 3
streaming session still produces audio. Without this the session opened,
accepted sentences, and silently emitted empty PCM.
Documents the QNN SDK version-skew pitfall in ExecuTorchLlmEngine.kt
ahead of the upcoming migration to a unified v2.42 toolchain.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ExecuTorchLlmEngine: eval_mode 0 (our .pte is kv-mode, not hybrid)
- KazeiaService: call llm.load() after TTS init; try/catch falls back
to echo mode if the runner or .pte are missing.
Pipeline on device: STT(WhisperHybridEngine) → [VoiceCommands → LLM] → TTS(Qwen3TtsEngine).
Validated on OnePlus Pad 3: LLM ready in ~8 s, gen 21.3 tok/s, RSS 1.76 GB in the
qnn_llama_runner subprocess (out-of-process from the Kazeia app).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds executorch-patches/ with the local modifications to /opt/Kazeia/executorch
(upstream pytorch/executorch v1.2.0) required to export Qwen3-4B to QNN for the
OnePlus Pad 3 Hexagon V79. Tablet runs 18.2 tok/s (gen), TTFT 0.9 s, RSS 1.76 GB.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaces the fixed maxGen + length-based boost with a fully dynamic
end-of-utterance detector that watches the model's own EOS logit rank.
End result on the Baer 3-segment monologue, validated by user as
"FORMIDABLE" / "impeccable" with both Damien and Zelda voices:
- All 3 segments terminate via EOS (no maxGen cap hit)
- No "page beg beg" filler tail
- No abrupt cuts between segments
- Audio durations 5-8 s per segment, matching Python within ~10 %
How it works (runHexGenWithPrefill, in tts/Qwen3TtsEngine.kt):
1. At every decode step, compute the rank of CODEC_EOS in the
repetition-penalised logits. Mid-utterance the rank sits at
150-700 (model is committed to producing speech). Approaching
the natural end, the rank dips toward top-50.
2. Arm the boost only when EOS rank stays below eosRankTrigger=60
for THREE consecutive steps. The 3-step requirement filters out
transient single-step dips that occur during low-energy phonemes
mid-sentence (without it, short sentences would terminate after
~3 s). Arming is also gated by eosBoostMinStep (50 % of expected
speech length) so we never arm in the very first frames.
3. Once armed, the boost increments monotonically: each subsequent
step adds boostStepsActive * eosBoostScale to the EOS logit. The
accumulated boost lifts EOS above top-1 within 1-3 steps, the
argmax check fires, and the loop breaks. Scale=4 gives the model
a small natural decay before termination; scale=5 was perfect-but-
slightly-clipping, scale=3 wasn't strong enough to outpace the
growing top-1 logit.
Other tweaks bundled in this commit because they all contribute to
the clean output:
* Inter-segment gap 120 → 250 ms — gives the listener a perceived
sentence boundary instead of a hard concatenation.
* fadeOut(audio, 40) on every segment — cosine roll-off over the
last 40 ms so the EOS-clipped tail decays naturally instead of
sample-clipping.
* top_k 50 → 200 in the fallback sample call — wider pool to keep
EOS reachable when the boost just fails to hit argmax.
Voice swap is a 45 KB file push (damien_voice_prefix.bin and
damien_voice_suffix.bin). Successfully tested today with Elodie
(female, norm 10.12) and Zelda (norm 9.39) using Damien (norm 10.36)
as the baseline — same Kotlin code, no rebuild needed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Kazeia Team