Tutticore is an inference runtime that runs any model across GPUs, TPUs, ASICs and whatever accelerator comes next, as one machine. Memory orchestration is how.
Attention, feed-forward, and cache traffic all have different memory shapes. Most stacks ignore that and pin the whole graph to one vendor.
Bandwidth-bound, compute-bound, and capacity-bound operators are separated before placement. No kernel rewrites.
Attention lands on high-bandwidth memory. Dense MLP on NPU SRAM. The KV cache spills into pooled host memory instead of starving the GPU.
One address space, with pages migrating while tokens stream. Mixed fleets stop behaving like a collection of islands.
Attention, feed-forward, and cache traffic all have different memory shapes. Most stacks ignore that and pin the whole graph to one vendor.
Bandwidth-bound, compute-bound, and capacity-bound operators are separated before placement. No kernel rewrites.
Attention lands on high-bandwidth memory. Dense MLP on NPU SRAM. The KV cache spills into pooled host memory instead of starving the GPU.
One address space, with pages migrating while tokens stream. Mixed fleets stop behaving like a collection of islands.
Tutticore sits between your serving layer and the silicon. It owns memory placement, so the same model runs on a rack of mixed accelerators as if it were one device.
On-chip memory, device memory and host memory are exposed as one tiered address space, whatever the silicon underneath. Pages move by bandwidth need, not by which chip they happen to sit on.
KV pages are placed across accelerator memory and host DRAM by access pattern, and shared between devices. Long contexts stop evicting each other.
Speaks the OpenAI-compatible chat API on your own host, so existing clients work unchanged.
curl http://<your-tutticore-host>/v1/chat/completions \
-d '{ "model": "your-model",
"messages": [{"role":"user",
"content":"hello"}] }'Operators are placed by memory profile with a live cost model. When a device saturates, work spills sideways instead of queueing.
Vision
The next decade of inference will not run on one vendor's chips. It will run on whatever silicon is available, mixed by generation, by vendor and by design. Today each of those is an island with its own stack, and the model is rewritten for each one.
Tutticore's bet is that the runtime should absorb that difference, not the model. Treat mixed silicon as one target, and the question of which chip wins stops mattering to the software.
Any model
Whatever you serve today, or ship next quarter. Tutticore profiles the graph it is given; it does not ask for a rewrite.
Every hardware
GPUs beside TPUs, ASICs built for one workload, and whatever ships next. Whatever mix you can source, in whatever ratio.
One machine
The fleet is addressed as one target with one memory space. Placement and migration live in the runtime, so serving code stays the same when the fleet changes.
Three steps, shown with illustrative commands. The runtime is not publicly available yet.
Point Tutticore at whatever you have: GPUs, TPUs, ASICs, or any other accelerator. Devices register with their bandwidth and capacity profile.
$ tutticore fleet add --discover
Pull weights once. Tutticore profiles the graph, regroups operators by memory shape and writes the placement plan.
$ tutticore deploy <model> --plan auto
Expose an OpenAI-compatible endpoint on your own host. Placement and migration stay observable, and the plan re-balances when hardware is added.
$ tutticore serve
Tutticore is not publicly available yet. If you run mixed accelerators, build them, or fund the people who do, we would like to talk.