vLLM Semantic Router v0.2 Athena: ClawOS, Model Refresh, and the System Brain
Since v0.1 Iris, vLLM Semantic Router has made a large jump. In one release cycle, the project rebuilt its model stack, expanded routing into safety, semantic caching, memory, retrieval, and long-context signal handling, and started pushing toward a broader ambition: turning semantic routing into the system brain for mixture-of-models and multi-agent deployments.
Athena is where that shift becomes visible. v0.2 ships a complete model refresh and a much stronger routing runtime, but one of its boldest new bets is ClawOS: an experimental operating layer where Semantic Router can orchestrate multiple OpenClaw systems through routing, memory, safety, and chat-driven team management. If Iris established the bridge between users and models, Athena starts turning that bridge into an operating surface for model teams.

Why Athena?
In Greek mythology, Athena represents wisdom, strategy, and disciplined craft. That symbolism fits this release precisely. v0.2 is not just about routing requests faster or adding more plugins. It is about making semantic routing more strategic: learning which model to choose, coordinating teams of OpenClaw workers, remembering what matters across turns, exposing decisions through better tooling, and turning a powerful runtime into something teams can actually operate.

What's New in v0.2 Athena?
1. A Complete Model Refresh Rebuilds the MoM Foundation
The most consequential change in Athena sits below the UI and below the routing DSL: the model stack was rebuilt.
Athena now centers on a new long-context multilingual base, mmbert-embed-32k-2d-matryoshka, and a new classifier family collected under mom-multilingual-class. In practice, that means the router's embedding, intent, jailbreak, PII, feedback, fact-check, and related classifier surfaces are moving onto a shared mmBERT-derived foundation instead of a more fragmented base-model story. Just as importantly, that refreshed family now lines up with the same ONNX + Flash Attention acceleration path.
Athena also introduces multi-modal-embed-small, a standalone embedding model that puts text, images, and audio into one shared 384-dimensional space. It is designed for true cross-modal retrieval, so the system can search images with text, find audio with text descriptions, and align content across all three modalities. Just as importantly, it keeps the deployment story simple: it can be loaded with transformers and torch without custom runtime dependencies.

This new model layer brings three changes that matter immediately:
- Multi-Modal Embed Small gives Athena a compact cross-modal primitive at ~120M parameters, with a shared 384d space, strong image-text alignment, 2D Matryoshka controls, sub-100ms inference targets, and reported Audio-Text Retrieval R@1 = 36.4%
- mmBERT-Embed-32K-2D-Matryoshka gives the router a production-ready multilingual long-context backbone: 32K context, 1800+ languages, 307M parameters, STS 80.5, 768d -> 256d truncation with ~99% quality retention, and 22L -> 6L early exit for roughly 3.3x speedups
- the mom-multilingual-class collection turns that backbone into a coherent classifier family, so long-context multilingual routing and safety tasks can share the same base-model assumptions and the same ONNX acceleration path
At the time of this release, the mom-multilingual-class collection spans five core routing and safety tasks, each exposed in both merged and LoRA form:
| Task | Merged model | LoRA model |
|---|---|---|
| Intent | mmbert32k-intent-classifier-merged | mmbert32k-intent-classifier-lora |
| Jailbreak | mmbert32k-jailbreak-detector-merged | mmbert32k-jailbreak-detector-lora |
| PII | mmbert32k-pii-detector-merged | mmbert32k-pii-detector-lora |
| Fact-check | mmbert32k-factcheck-classifier-merged | mmbert32k-factcheck-classifier-lora |
| Feedback | mmbert32k-feedback-detector-merged | mmbert32k-feedback-detector-lora |
That classifier collection is only one part of the refresh. Athena also pairs it with a new embedding backbone, a new multimodal embedding model, and a much stronger production acceleration path. At a higher level, the model refresh in v0.2 looks like this:
| New foundation | What Athena changes |
|---|---|
multi-modal-embed-small | Unified text-image-audio embeddings in one 384d semantic space |
mmbert-embed-32k-2d-matryoshka | 32K context, 1800+ languages, 2D Matryoshka runtime controls |
| ONNX + CK Flash Attention | The refreshed model stack becomes materially faster in production, not just newer on paper |

This matters because Athena's model refresh is also a runtime refresh. The ONNX path, ROCm support, and CK Flash Attention work turn the new foundation into a deployable latency story.
In our three-way benchmark on AMD Instinct MI300X with the real router path Envoy (:8801) -> ext_proc -> SR (:50051), the end-to-end latency profile changed dramatically:
| Request size | ONNX + GPU avg | ONNX + CPU avg | Candle + CPU avg |
|---|---|---|---|
| ~500 tokens | 22 ms | 853 ms | 1053 ms |
| ~2000 tokens | 31 ms | 1814 ms | 1805 ms |
| ~8000 tokens | 128 ms | 4796 ms | 1830 ms |
At the signal level, the gains are even clearer. For domain extraction, ONNX+GPU ran at 10.2 ms on ~500 tokens, 16.3 ms on ~2000 tokens, and 36.1 ms on ~8000 tokens, versus 630.4 / 833.3 / 743.9 ms on ONNX+CPU and 849.0 / 1304.9 / 1311.5 ms on Candle+CPU. For PII extraction, ONNX+GPU reached 8.4 ms, 19.0 ms, and 118.8 ms at those same lengths, versus 729.5 / 1781.8 / 4783.9 ms on ONNX+CPU and 854.2 / 1299.8 / 1327.8 ms on Candle+CPU.
The Flash Attention story is just as important. With three classifiers loaded concurrently on MI300X, the old SDPA path hit a memory wall, while the new CK Flash Attention path kept scaling:
| Sequence length | SDPA | CK Flash Attention | Result |
|---|---|---|---|
| 4096 | 167 ms | 51 ms | 3.3x faster |
| 8192 | OOM | 105 ms | SDPA fails, FA works |
| 16384 | OOM | 259 ms | FA works at 16K |
| 32768 | OOM | 756 ms | FA reaches full 32K |
What makes this especially important is how FA is supported. Under onnx-binding/ort-ck-flash-attn, Athena adds a standalone ONNX Runtime custom-op library that registers com.ck::CKFlashAttention on ROCm and calls AMD Composable Kernel tiled FMHA kernels directly. A graph-rewrite step then rewrites mmBERT ONNX graphs layer by layer, replacing the dense SDPA attention subgraph with a single CK Flash Attention node.
That rewrite is where much of the systems gain comes from. Instead of materializing a dense [1, 1, S, S] attention mask, the rewritten graph derives a lightweight [B, 1, 1, S] padding bias from attention_mask and passes sliding-window settings directly into the kernel. Local-attention layers use CK's built-in window parameters, while global-attention layers switch back to full attention with unlimited windows. In other words, Athena's FA path is not just a backend toggle. It is a model-aware ONNX rewrite plus a custom ROCm kernel path built specifically for long-context mmBERT inference.
Under heavier load, CK Flash Attention still completed 20 concurrent 32K-token requests at 9872 ms median / 14862 ms p95 with zero OOMs, while preserving identical classification outcomes across the validation queries. That is why the model reset belongs at the front of this release: Athena did not just add features around the router. It changed the computational foundation underneath it.
2. Model Selection Becomes a First-Class Routing Primitive
The biggest leap in Athena is that model selection is no longer just a roadmap item. It is now a concrete part of the system, spanning both trainable ML selectors and advanced runtime selection strategies.
Just as importantly, Athena makes its position in the routing pipeline explicit. Model selection does not replace signal extraction or decision matching. The system first extracts signals, then evaluates decisions, and only after a decision matches does a per-decision algorithm choose among that decision's modelRefs. In other words, model selection becomes the last strategic step between "this request belongs to this decision" and "this exact model should serve it."
This matters because modern LLM systems do not just need to decide whether a request belongs to a route. They need to decide which model should handle it under changing tradeoffs in quality, latency, cost, and specialization. Athena makes that strategic layer visible and programmable.
| Family | Method | What it does |
|---|---|---|
| ML-based | KNN | Finds similar historical queries and lets nearby examples vote for the best model. |
| ML-based | KMeans | Clusters requests and assigns models based on cluster-level quality and efficiency patterns. |
| ML-based | SVM | Learns nonlinear decision boundaries between model preferences using an RBF classifier. |
| ML-based | MLP | Uses a neural router to predict the best model from embeddings, with efficient inference through Candle. |
| Advanced | Static | Uses a fixed default model when predictability matters more than adaptation. |
| Advanced | Latency-Aware | Selects the fastest candidate from TPOT and TTFT percentile data when latency budgets dominate. |
| Advanced | Elo | Learns from user feedback and pairwise preferences using Bradley-Terry style rating updates. |
| Advanced | RouterDC | Matches queries to model descriptions with dual-contrastive embedding similarity. |
| Advanced | AutoMix | Starts with cheaper models and escalates based on self-verification to balance cost and quality. |
| Advanced | Hybrid | Blends multiple methods such as quality, similarity, and cost with configurable weights. |
| Advanced | Thompson Sampling | Balances exploration and exploitation online so routing can keep learning while serving production traffic. |
| Advanced | GMTRouter | Personalizes model choice from multi-turn interaction history with graph-based routing. |
| Advanced | Router-R1 | Uses an external router model to reason about the request before choosing a downstream model. |

Athena also adds the operational layer around these algorithms: setup wizard support for ML training and config generation, CLI and runtime integration, metrics, E2E coverage, and Elo feedback surfaces in the dashboard for human-in-the-loop refinement.