Tools / Phase

Phase

Energy per token for local AI models. Prompt and answer measured apart, at the chip and the whole machine. No extra hardware.

github.com/splicedso/phase
A processor package
Method

Small enough to read in one sitting.

# brew install macmon llama.cpp
uv run phase model.gguf --out results/run.json
  1. Two test points. TP2 is the chip, sampled every 100 to 200 ms. TP1 is the board's DC input, which includes memory, smoothed to about 1 Hz and so only used over long windows.
  2. Baseline first. Idle power before anything runs, so every figure can be reported gross and net.
  3. Phases apart. Prefill (512 prompt tokens) and decode (128 generated tokens) run as separate tests.
  4. Sized runs. A short probe sets the repetition count so each phase has about twenty timed seconds.
  5. Steady window only. The first quarter of the compute window is dropped for clock ramp and sensor lag.
  6. Watts over throughput. Mean power across the window divided by measured tokens per second gives joules per token.
Baseline results
modelphasemillijoules per token, chip and boardtok/s · chip / board
TP2 chipTP1 board DC input

Apple M5 Pro, 24 GB, on AC power, llama.cpp build 11669, Q4_K_M, all layers on the GPU. The 7B rows are a single run.

Limits

Not wall power

TP1 is DC input as the SMC reports it. It leaves out the adapter's own loss.

Three platforms

Apple Silicon, NVIDIA and AMD, no extra hardware.

Repeated

Every run is repeated and its spread reported.