The AI-native inference runtime
The OS owns inference orchestration, residency, scheduling, batching, metering and backend selection end to end down to the hardware backend boundary. Vendor drivers and accelerator firmware remain vendor-controlled below that boundary.
Contract beneath ModelProvider
InferenceRuntime.run(model_id, tensors, budget, priority, meter_id) -> (tensors, metrics)L2.5 sees tensors plus opaque execution metadata. It receives no plaintext prompt, agent identity, capability, zone, policy or user-semantic context. It never writes to the audit log; metrics go to L2 over a restricted typed MeteringSubmit IPC that only the authenticated L2.5 service may call.
Build sequence
Stages are compared only against their accepted predecessor on identical target, model, corpus and decoding settings. R0-PoC numbers are never an optimisation baseline.
| Stage | Content | Compared against |
|---|---|---|
R0-PoC | Metering wrapper around the remote proof-of-concept path. Establishes L10 compliance for the exit proof. | — |
R0-Local | Metering wrapper around the stock on-device path, unchanged. The optimisation baseline. | — |
R1 | Residency and batching on CPU. | R0-Local |
R2 | DSP/NPU placement by cost table. | R1 |
R3 | Energy-aware scheduling: defer eligible compute to charging-and-idle. | R2 |
Regression gate
Fixed prompt corpus, fixed decoding settings, repeated runs, tolerance bands with confidence intervals. A stage fails only when tokens per joule or p95 latency regresses beyond the band. Cross-intent isolation of KV-cache, tensors and buffers is mandatory.
What the runtime cannot do
- It cannot authorise anything. Compute optimises; it never permits.
- It cannot read the audit log or write to it directly.
- It cannot see who is asking, why, or under which policy.
- It cannot let one intent's buffers be observed by another.