homearchitecture
Architecture
Cue Cloud stack
Open coding models on Mac Studios we operate. OpenAI-compatible API. Flat seat, unlimited tokens per user. Below: where prompts go, why Studios, how a pod decodes, and why buying boxes yourself is a different job.
Mac Studiospipeline-parallelcontinuous batchingOpenAI-compatible
01 · Billing
Agent loops break metered APIs.
Tools, retries, and long contexts multiply tokens. On a metered API that shows up as a climbing invoice. Cue Cloud sells a seat: one engineer, unlimited tokens, fixed $1,500/mo.
02 · Hosting
Your prompts go somewhere. Pick deliberately.
Open coding models are usable. The question is who runs them when you hit an endpoint: a vendor’s cloud, an opaque router, or Mac Studios with a known operator. Cue Cloud is the third — we rack and run the machines behind an OpenAI-compatible API.
03 · Hardware
Why Mac Studios.
Generating a token is mostly reading model weights from memory, not peak FLOPs. Big open MoEs keep huge parameter sets resident and only activate a slice per token — so memory bandwidth and capacity dominate. A Mac Studio’s unified memory is one pool shared by CPU and GPU: no “copy the model into VRAM” step. That shape fits a small rack of Studios serving frontier-class open coding weights.
04 · Path
What happens when you call the API.
Same path from CueCode or any OpenAI-compatible client: your request hits the gateway, the scheduler batches it with other live sessions, a Mac Studio pod decodes, tokens stream back.
05 · Pod
One model, split across Mac Studios.
Studio 1 holds early layers, Studio 2 the next, and so on. Weights stay on each Studio; only small activation tensors move between them. That is pipeline parallelism — how a multi-Studio pod serves one large MoE without shuffling the full weights every token.
06 · Stack
The chat API is not the hard part.
Anyone can put an OpenAI-compatible gateway in front of machines. The scarce work is running many concurrent agent sessions on the same Mac Studio pod without melting latency — continuous batching, KV/context placement, keeping the pipeline fed. That decode layer is what a Cue Cloud seat buys.
07 · Capacity
Most teams share a pod. Some need their own.
Default: your seat shares batching capacity on our Mac Studio fleet, isolated by org/API key. Need stronger isolation? Dedicated Studio capacity — still operated by us, not a rack in your office.
many seats → same Mac Studio fleet (isolated by key/org)08 · Build vs buy
Buying Mac Studios ≠ running Cue Cloud.
Buying Studios gets you hardware. You still have to build continuous batching, multi-user scheduling, model upgrades, auth, metering, and an OpenAI-compatible edge that survives agent loops — plus on-call. Cue Cloud is that stack on Mac Studios we operate: a flat seat instead of capex and a serving team.