§1Name the category before the benchmark
| Category | What it means | What it does not promise |
|---|---|---|
| On-device | Runs on the laptop or phone in front of you. | Frontier capability or long context. |
| Private | Runs inside infrastructure you control. | One-machine simplicity. |
| Open weights | The model weights can be inspected or deployed. | Affordable local inference or identical provider behaviour. |
§2Current experiment: LM Studio + Gemma
The current use case is intentionally narrow: a private second reviewer for drafts, code explanations, and contradiction finding. The local model does not replace the main implementation agent. Its value is independence, privacy, and a different error profile.
FIELD NOTE · 17 JUL 2026
- Runner
- LM Studio
- Model family
- Gemma
- Job
- Second review and bounded private analysis
- Success test
- Find a material contradiction and point to the exact passage
- Stop
- No unsupported rewrite after one focused revision
- Status
- Testing; no final verdict
§3Advertised context is not usable context
The useful number is the largest context that preserves recall, instruction following, and output quality on the actual task. Field notes therefore record four separate limits: the configured window, what fits in memory, what completes at acceptable latency, and what remains reliable under retrieval questions.
§4Compare jobs, not personalities
- Local wins when privacy, offline access, predictable marginal cost, or independence matters most.
- Cloud wins when tool use, very long reliable context, frontier reasoning, or low setup cost matters most.
- Hybrid wins when a cloud agent implements and a local model provides a genuinely separate review.
§5Field-note contract
DATE / HARDWARE / RUNNER
MODEL + QUANTIZATION
CONFIGURED CONTEXT
TASK AND INPUT SHAPE
LATENCY / MEMORY
PASS CONDITION
OBSERVED FAILURE
CLOUD COMPARISON
DECISION: ADOPT / TEST / DROPCurrent verdict
- Use local models for bounded jobs with measurable pass conditions.
- Record exact test conditions before comparing results.
- Do not equate open weights with one-machine inference.
- Keep model claims dated; the stack changes faster than the method.