ODINO 3.11 · NEMOTRON 3.5 LIGHTNING · PRODUCTION LIVE
Thirty-billion-parameter capacity.Sustained at the sovereign edge.
Odino 3.11 promotes NVIDIA Nemotron 3.5 Lightning 30B-A3B NVFP4 to production on a single NVIDIA Jetson AGX Thor. The 30B Mixture-of-Experts model activates approximately 3B parameters per token: 7.5× the total capacity of the former Nano-4B model, with 72.623 tok/s sustained production throughput and 68.072 ms median time to first content.
THE CONSTRAINT
More capability, inside the same sovereign machine.
Jetson AGX Thor 5000 provides 128 GiB of unified memory — substantial for an edge node, finite for a complete multimodal cognitive system. Odino began as the answer to that hard resource boundary. Its current production engine now holds a 30B MoE model at 19,102 MiB GPU UMA residency while Nemotron Omni, Riva speech and the governed service fleet remain live.
The advance is not a like-for-like model swap. Nemotron 3.5 Lightning brings 7.5× greater total parameter capacity than Nano-4B while activating about 3B parameters per token. This preserves a relatively contained active-compute footprint and materially expands the system's scientific reasoning capacity.
Q19 · PRODUCTION PERFORMANCE · 23 AUGUST 2026
+28% sustained. +36% qualified. Up to +63% native.
The headline uses the last sustained, comparable Odino Nano result as its baseline. All measurements run on the same Jetson AGX Thor; qualified HTTP and native results use the fixed Q18 workload, while the production figure is the mean of five HTTP runs.
72.623
TOK/S · PRODUCTION
77.104
TOK/S · HTTP QUALIFIED
92.8
TOK/S · NATIVE PEAK
68.072 ms
MEDIAN FIRST CONTENT
| Workload | Comparable Nano baseline | Odino 3.11 | Improvement |
|---|---|---|---|
| Sustained production · five-run HTTP | 56.895 tok/s | 72.623 tok/s | +27.6% |
| Qualified HTTP · fixed Q18 workload | 56.895 tok/s | 77.104 tok/s | +35.5% |
| Native engine · fixed Q18 workload | 56.895 tok/s | 92.8 tok/s | +63.1% |
| Time to first content · production median | — | 68.072 ms | CURRENT |
Method boundary: an older Q1 run recorded 97.651 tok/s on a short, mixed workload. It is not used as a direct comparator because prompt, model, runtime and methodology differ.
SYSTEM OUTCOME · SAME Q9 SCIENTIFIC MANIFEST
The larger result is capability and scientific reliability.
On the fixed bilingual scientific and adversarial benchmark, Odino now closes every measured case. Deterministic Direct Evidence handles exact graph, process and numeric questions with provenance gates and zero LLM calls.
| Metric | Q9 baseline | Current system | Change |
|---|---|---|---|
| Scientific accuracy · fixed Q9 manifest | 63.33% | 100% | +36.67 points · +57.9% relative |
| Direct Evidence latency · p50 | 3,556.867 ms | 31.370 ms | −99.1% · 113.38× faster |
| Grounded catalogue request | HTTP 413 | HTTP 200 · 5.73 s | Real tool trace · 236 rows |
Knowledge Graph, claim-level provenance, evidence assessment, local UQ/OOD, grounded tools and specialist read-only consumers now form a single governed evidence path. The production graph contains 259 nodes and 246 edges.
These capabilities were added without enabling mutation, export, ingest or autonomous experiment execution. Workbench and vector-store invariants, Guardian policy, kill switch and human-authority boundaries remain intact.
HISTORICAL FOUNDATION · PHASE 1
The numbers that justified the build.
The table below compares Odino Phase 1 to the prior vLLM configuration on the same Jetson AGX Thor 5000 hardware, executing the same FP8-quantized chat model. Validated 2025-12-06.
| Metric | vLLM (prior) | Odino Phase 1 (validated) | Δ vs vLLM | Odino Sprint 3 streaming |
|---|---|---|---|---|
| Memory occupation (idle) | ~67 GiB | 381 MiB | −99.4% | ~381 MiB (unchanged) |
| Memory occupation (loaded) | ~70 GiB (saturated) | ~15 GiB | −78% | ~15 GiB (unchanged) |
| Cold-start time | 30–60 s | < 2 s | −96% | < 2 s engine + first-call warmup |
| Streaming TTFT | n/a | n/a | n/a | warm steady-state within sub-200 ms band |
| Interactive support | n/a | synchronous only | n/a | token-by-token SSE |
The recovered memory — approximately 50 GiB freed by the loaded-state comparison — is what allows the Tin Man cluster to operate Vision (RADIO ViT-H/16), Audio (Parakeet Conformer), and Real-time perception cores alongside the reasoning core on a single Jetson Thor node.
ENGINEERING PRINCIPLES
Native compilation. Bounded residency. Governed operation.
Three architectural decisions, applied uniformly:
- Thor-native compilationA TensorRT Edge-LLM 0.10.0 engine compiled for Jetson AGX Thor runs Nemotron 3.5 Lightning in NVFP4, without a general-purpose serving framework in the inference path.
- Bounded residencyThe production engine holds full residency at 19,102 MiB of GPU unified memory while the wider multimodal stack remains operational.
- Governed production pathThe canonical model endpoint is fixed, the former Nano engine is inactive, mutating tools remain disabled, and supervised readiness is tested across the complete stack.
Each decision reflects Odino's role as a purpose-built reasoning runtime for Tin Man on this specific hardware, rather than a general-purpose inference framework.
DETERMINISM
Evidence paths before generative paths.
Beyond raw throughput, Odino routes exact graph, numeric and process questions through a deterministic Direct Evidence path. Those answers carry bounded local evidence and provenance, bypass generation, and require zero LLM calls. Ambiguous reasoning remains on the governed model path rather than being misrepresented as deterministic evidence.
The architecture keeps the distinction visible: evidence-backed facts, model reasoning and tool results retain separate provenance. Shield Brain and the human-authority boundary apply across every route, while mutating tools and autonomous experiment execution stay disabled.
INTEGRATION
A component, not a product.
Odino is not delivered as a standalone product. It is the Chat Core of the Tin Man cognitive cluster — the reasoning layer that interprets text-mode inputs and produces the multimodal prompt assembly that downstream cores consume. Its value is realized through the integration with Vision, Audio, Brainstem, Memory, and Realtime cores under the Shield Brain control architecture.
The 3.11 production line exposes one canonical Nemotron 3.5 endpoint with no automatic model fallback. Its read-only scientific consumers, grounded tools and deterministic evidence routes share the same Guardian and human-authority boundary; mutating operations remain disabled.
SPRINT 3 ENHANCEMENT · MAY 2026
Streaming token-by-token delivery.
In May 2026 the Odino runtime was extended with server-side streaming support, enabling the Tin Man cluster to pipeline language-model token generation with downstream synthesis layers (text-to-speech) and reduce end-to-end voice-loop latency well below 1 second.
Backward compatibility with the Phase 1 synchronous endpoint is preserved. Streaming is implemented via OpenAI-compatible Server-Sent Events, delivering per-token incremental output.
Empirical performance:
- Warm steady-state Time-To-First-Token: well within the sub-200 ms target band
- Coherent response on drone-command corpus at interactive grade
The Golden Master engine plan of Phase 1 is preserved unchanged; the streaming behavior is achieved through a variant server harness that binds the same engine plan at runtime — a provenance pattern designed to keep validated assets stable across iterative enhancements.
ENGAGE
Engineering deep dives are released under partnership.
The full Odino technical report, benchmark manifests, integration notes for the Tin Man cluster, engine-build evidence, and the embedded-AI engineering pathway are released under non-disclosure agreement. Engage to begin a conversation.
Direct: engage@reinventy-solutions.ca