T1 Tools / joule

joule

Energy per token for local LLM inference, measured at the chip and at the board, with prefill and decode reported apart. No sudo, no extra hardware.

github.com/splicedso/joule
T2 Method

Small enough to read in one sitting.

# brew install macmon llama.cpp
uv run joule model.gguf --out results/run.json
  1. Two test points. TP2 is the chip, sampled every 100 to 200 ms. TP1 is the board's DC input, which includes memory, smoothed to about 1 Hz and so only used over long windows.
  2. Baseline first. Idle power before anything runs, so every figure can be reported gross and net.
  3. Phases apart. Prefill (512 prompt tokens) and decode (128 generated tokens) run as separate tests.
  4. Sized runs. A short probe sets the repetition count so each phase has about twenty timed seconds.
  5. Steady window only. The first quarter of the compute window is dropped for clock ramp and sensor lag.
  6. Watts over throughput. Mean power across the window divided by measured tokens per second gives joules per token.
T3 Baseline results
modelphasemillijoules per token, chip and boardtok/s · chip / board
TP2 chipTP1 board DC input

Apple M5 Pro, 24 GB, on AC power, llama.cpp build 11669, Q4_K_M, all layers on the GPU. The 7B rows are a single run.

T4 Limits

Not wall power

TP1 is DC input as the SMC reports it. It leaves out the adapter's own loss.

Apple Silicon first

macOS today. A Windows backend, reading battery discharge and GPU power, is being written.

Laptop, not bench

Runs happen on a working machine. Every note repeats its runs and reports how far they agree.