Replace the hardcoded eval_mode=0 in the QNN_LLAMA branch with a runtime
check on the loaded module's method names: if the .pte exposes a
prefill_forward graph, switch to EvalMode::kHybrid (1) — the runner can
then batch the entire prompt through prefill_forward in one parallel pass
instead of running 52 ms/token sequentially through kv_forward. Falls
back to kKVCached (0) when only kv_forward exists, matching the current
.pte behaviour exactly so this is a safe in-place upgrade ahead of the
hybrid re-export.
Sanity-tested with the kv-only Qwen3-4B .pte already on the tablet:
Prompt 'Bonjour, ça va ?' → "Bonjour ! Ça va, merci de me demander ça.
Tu as une question ?", TTFT 2728 ms, total 4158 ms — no change vs the
hardcoded eval_mode=0 build.
Once the hybrid Qwen3-4B export finishes (~50 min compile, both
prefill_forward + kv_forward graphs), the same JNI binary will pick up
the new .pte and TTFT should drop to <1 s.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
f4b15a72a7