# Local-or-cloud routing card

Version: 1.0
Updated: 2026-10-02
Method: https://suhailmohebi.com/notes/local-uncensored-models/

Purpose: decide, job by job, whether work runs on a local model, a cloud model, either, or stays with a person, before data or authority moves.

Output: a dated routing table with a verdict and reason per job, an escalation rule and the triggers that force a re-test.

## Use when

You are adding a local model beside cloud models, or a new job, data class, model, price or policy raises the question of where work should run.

## Quick start

1. List the recurring jobs and the most sensitive data each one touches.
2. Mark the steps that need authority, then answer the data question (may this leave the machine?), then score the rest from your own measurements.
3. Write one verdict per job (Local, Cloud, Either or Human), the escalation rule and the date.

A local model keeps inference local; the agent, tools and logs around it may still reach the network.

## 1. Job inventory

| Job | Input size (tokens) | Most sensitive data | Volume per week | Cost of a wrong answer |
|---|---|---|---|---|
|  |  |  |  |  |
|  |  |  |  |  |

## 2. Routing criteria

| Criterion | Question | Points to local | Points to cloud |
|---|---|---|---|
| Data sensitivity | May the input leave the machine? | No | Yes, under your data policy |
| Quality bar | Pass rate on your own graded eval for this job | Local meets the bar | Only cloud meets it |
| Latency, throughput | Long inputs or short interactive turns? | Short prompts; stable prefixes the cache reuses | Long inputs the local machine reads too slowly |
| Context length | Measured usable context, not advertised | The job fits | The job needs more |
| Tool-use reliability | Graded tool-call pass rate at the needed depth | Passes there | Fails or untested |
| Cost | Cloud: tokens × price. Local: hardware share, energy, upkeep | High volume, low stakes | Low volume, no suitable hardware |
| Availability | Offline, outage or single-vendor risk? | Must keep working | Not required |
| Licence and terms | Does this use comply? | The model licence and acceptable-use policy permit it | The provider’s terms permit it |

- Benchmarks are not your workload: fill the quality rows from your own eval, with the same grader on both sides.
- With more than one local machine, route by phase as well: prefill-heavy jobs go to the machine that reads prompts fastest.
- Running locally changes where inference happens, not what you may do. Licences, acceptable-use policies and the law still apply; a person reviews output where consequences rise.

## 3. Decision table

Apply in order:

1. Human: the step needs authority (send, publish, pay, approve, delete), no permitted model meets the bar (an offline job permits only local), or the licence position is unclear.
2. Local: local meets the bar at the needed context, and the data must not leave or the job must work offline.
3. Cloud: the data may leave, local misses the quality, tool-use, context or latency bar, and cloud meets it.
4. Either: the data may leave and local meets the bar. Default to the cheaper or faster; keep the other as fallback once it passes the same eval. A strong split: cloud implements, local gives an independent review.

| Job | Data may leave? | Local pass rate | Cloud pass rate | Verdict | Reason |
|---|---|---|---|---|---|
|  | yes / no |  |  | Local / Cloud / Either / Human |  |

## 4. Escalation rule

```text
1. Run [job] locally first.
2. If the result fails [check], retry once with [one change], then escalate.
3. Escalate to cloud only for jobs whose data may leave, after removing [fields] with [method]
   and confirming nothing on the must-not-leave list remains. Every other failure goes to a person.
4. Log what was sent, when and why.
5. [Decision] stays with a person; no model output is final for it.
```

Redaction is a control, not a guarantee. If the job depends on the sensitive part, it cannot be redacted into a cloud job. Serve the local model behind the same API shape as the cloud one, so a route change is a configuration change, not a code change.

## 5. Re-test triggers

Re-run the job’s eval and re-date the card when any of these change:

- [ ] Model or model file (new hash)
- [ ] Quantization or KV-cache precision
- [ ] Runtime version or serving flags
- [ ] Hardware, memory or configured context
- [ ] The job’s input size, data class or quality bar
- [ ] Cloud model, price or terms; model licence or acceptable-use policy

## Worked example

Quality bar: 90% of graded checks over three runs. Card dated 2026-10-02.

| Job | Data may leave? | Local pass rate | Cloud pass rate | Verdict | Reason |
|---|---|---|---|---|---|
| Review unreleased design notes, ~8k tokens | no | 34/36 | not run | Local | Private; local meets the bar |
| Refactor plan over a 150k-token repository dump | yes | n/a: usable context 16k | 54/60 | Cloud | Beyond local usable context and prefill budget |
| Tag 20,000 public help articles a week | yes | 96% | 97% | Either, local default | Volume favours local; cloud is the fallback |
| Draft customer email replies with personal data | no | 33/36 | not run | Local | Data stays local; a person edits and sends |
| Approve a production deploy | n/a | n/a | n/a | Human | Needs authority, not inference |

Escalation: the refactor job goes straight to cloud; the other model jobs run locally first, with cloud fallback only for the help-article job, whose data is public. A failed review or email draft goes to a person, never to cloud: redaction would remove what the job needs. Re-test triggers: a new local model file or runtime version re-opens the first four rows.

## Stop if

- A job’s data class is unknown: treat it as must-not-leave until someone classifies it.
- A job is about to be routed locally without an eval at the input size it needs.
- An escalation would send must-not-leave data, redacted or not.
- The model licence, acceptable-use policy or provider terms do not clearly permit the use.
- The surrounding agent or tools make network calls you have not accounted for.
- A refusal is being routed around by switching models: confirm the work is lawful and permitted, and keep a person on the output.