ExecuTorch Android measurements — November 5, 2026

Device: Google Pixel 8a, Tensor G3, Android 17 / API 37, ARM64.
Build: debug, version 1.0; org.pytorch:executorch-android:1.5.1.
Source commit: 62a53e87c67c54b959fd0e89d6668563005d8f28
Source snapshot (includes existing uncommitted files): a9cea59492a1ae8b7592a939b9d28e5b5fd1d9229d8b64fdf8a8e18881276597
APK SHA-256: c8af30cfa23611393ff756bdd238313d5c3524e6a3fbc0a82d8376adb02979a1
Android source was not changed for these measurements.

Model: Llama 3.2 1B Instruct SpinQuant INT4 EO8.
Model bytes: 1135951488; SHA-256: 8715cdba9e91f6bede00cc5f2d6b12397b95225da5630c4972a8da03001cda3b
Tokenizer bytes: 2183982; SHA-256: 82e9d31979e92ab929cd544440f129d9ecd797b69e327f80f17e1c50d5551b55
Matching repository revision: 911221e0f49d3354cbd4697a9c09feee25cadec7.
Repository: https://huggingface.co/executorch-community/Llama-3.2-1B-Instruct-SpinQuant_INT4_EO8-ET
Runtime metadata: TikToken, 2048 context/sequence length, EOS IDs 128009 and 128001.
Native initialization reports an eight-thread pool. PTE contains XnnpackBackend.
Temperature 0.7, no fixed seed. App callback ceiling 512, not an independently verified native max-new-token setting.

System prompt:
You are a helpful, concise AI assistant running fully offline on an Android device using PyTorch ExecuTorch.

Method: three fixed prompts, one process-first and five warm runs each (18 total).
Clear chat between independent prompts; keep model loaded across warm runs.
Rotate workload order. At least 45 seconds between completion and next preparation;
UI automation adds more time. Charging and screen on. Thermal status 0 before
all 18 baseline runs. No other control of temperature, background load, or OS cache.

Native TTFT: native inference start to first generated token.
Native total: prompt processing plus decode; already includes TTFT.
Native decode rate: native generated-token count divided by native decode interval.
These exclude explicit model load, ADB typing, Compose rendering, and true tap-to-result time.
Explicit load log intervals: 1632, 1428, 1538 ms; exclude construction. OS cache not flushed.
Medians use only the 15 warm runs. Response lengths vary; inspect token counts.

A separate nine-turn chat dataset uses three independent three-turn conversations.
It has no intentional 45-second gaps between turns. One thermal pause preceded
the third sequence; all recorded turns started at thermal status 0. Do not pool
chat timings with the single-prompt baseline.

Outputs are rendered UI text. Separate code-block fragments are included, but
Markdown fences are removed by the renderer. quality-review.csv contains manual
checks of the particular fixed prompts, not a general accuracy score.

Offline verification: Wi-Fi off and Android active default network reported none.
The global mobile_data flag remained 1 and was not used to prove disconnection.
APK declares no INTERNET permission. Original connectivity settings were restored.

Memory snapshots: missing-tokenizer baseline PSS 155.1 MiB; loaded PSS 1294.3 MiB;
after profiled long reply PSS 1381.7 MiB; maximum sampled PSS 1382.2 MiB (~1.35 GiB).
Point-in-time dumpsys samples, not a guaranteed peak. Profiling/recording runs are
excluded from the timing baseline. Long response: 2384 characters, 511 reported
native output tokens, ending mid-example near the 512-callback ceiling; explicit
stop reason was not logged. Entire generated text is accessible by scrolling.

Screenshots: summary-result.webp corresponds to summary-first (wrong deadline).
chat-result.webp is an initial exploratory run outside the baseline.
multi-turn.webp shows chat-3-turn-3 after scrolling.
demo.mp4 is a separate illustrative offline summary, about 8.5 seconds, with
recording overhead. It shows the full input, streamed output, and completion.
Its UI numbers are not benchmark-table samples.

Build and existing unit-test task succeeded with up-to-date tasks (23-test report
reused). Release, rapid cancellation races, oversized-input safety, GPU/NPU,
battery drain, and broad device/model accuracy remain unverified.
Do not compare runtime rankings with the earlier Qwen/llama.cpp experiment.

Demo source: https://github.com/meet30997/executorch-android-offline-chat