Architecture, models, and performance.

Choose hosted API access or a deployment tuned to your model and hardware.

Choose a serving path

Hosted API: IonRouter provides an OpenAI-compatible API with published models and token pricing. Create an account, add credits, and create an API key.
Models and pricing
Dedicated deployment: Choose Cumulus infrastructure or your own hardware. Specify the model checkpoint, precision, context length, traffic, and latency target.
Discuss your configuration

Ionattention is a C++ runtime built for NVIDIA GH200. Its coherent-memory optimizations use the Grace CPU, Hopper GPU, and NVLink-C2C link.

Published Ionattention model, backend, hardware, and precision configurations
Model	Backend	Hardware	Precision	Published scope	Source
Qwen2.5-7B-Instruct	Ionattention	1 × GH200	Not specified in measurement	Continuous batching; no tensor parallelism	Measurements
Qwen3-VL-8B	Ionattention	1 × GH200	Not specified in measurement	Vision encoder + language model; no tensor parallelism	Measurements
Qwen3.5-27B	Ionattention	GH200	Selective FP8	Dense model; custom adapter and recurrent-state execution	Adapter details
Qwen3.5-35B-A3B	Ionattention	GH200	BF16 weights, selectively quantized to FP8	Mixture of experts; custom adapter and mixed-model serving	Adapter details

Other hardware and models: Use the hosted catalogue for API availability. A dedicated deployment specifies its backend and hardware; GH200 coherent-memory techniques require a coherent CPU–GPU interconnect.
Hosted model catalogue

Model weights: The parameters used to execute a model. Supported models move from system memory to GPU memory as the active model changes.
Request KV state: Attention state belonging to an individual request. Completed KV blocks are mirrored to Grace memory in the background, so eviction can update metadata instead of waiting for a full transfer.
KV writeback and model switching
Grace CPU work: The published runtime uses CPU-side speculative draft models and vision-encoder caching alongside GPU execution.
CPU and GPU execution

A deployment can drain active requests or pause and resume them. Choose the mode around your model set, memory budget, and latency target.

Drain: Complete active requests before loading the next model. Queued requests keep their selected model and start when it is ready.
Pause and resume: Pause active requests, then resume them when their original model is resident again. KV state is preserved by default and can be recomputed when needed. Queued requests retain their selected model.

Published Ion throughput results and their measurement conditions
Model	Backend / hardware	Workload	Throughput	p50 latency	Source
Qwen2.5-7B-Instruct	Ionattention / 1 × GH200	68 input tokens; 128 generated tokens; concurrency 128; continuous batching	7,167 tokens/s	Not specified	Concurrency sweep
Qwen3-VL-8B	Ionattention / 1 × GH200	Real video clips; full vision encoder + language-model pipeline	588 tokens/s	1.46 seconds; article reports an increase from 0.74 seconds	Video results

Your service target: Evaluate your input/output lengths, precision, arrival rate, concurrency, time to first token, time per output token, and p95/p99 latency. Include warm, cold, and mixed-model traffic.

Start with a hosted model and your first API request.