Chat demos
- One thread
- Short context
- Reply and done
productagents.md
Agents
Why Cue Cloud focuses on coding agents: the loop shape, latency and throughput, and how a 1B pack keeps interactive turns alive.
Why we focus here
Coding agents are the load that matters
Chat is one shot → reply. Coding agents plan, read the repo, call tools, patch, verify, and often run concurrent turns with fat context. Cue Cloud is sized for that pattern — not demos that look fast on a single bubble.
Coding agents
- Multi-turn tool loops
- Growing context + retries
- Parallel work, long sessions
What an agent loop actually does
Context grows. Tools fan out. Idle waits happen. Then decode bursts. That is why chat-optimized infra feels wrong here.
Latency and throughput
Agents need both: snappy first tokens so the loop stays interactive, and sustained decode so patches and parallel tool work do not stall.
Latency
Time to first useful token
The loop waits on the first decision. Snappy TTFT keeps plan / tool choice interactive.
Throughput
Sustained decode under load
Long patches and parallel tools need tokens that keep coming — not a burst that dies when concurrency rises.
How we keep latency good
Good agent latency is time-to-first-useful-token plus steady decode while tools are in flight. Chat benchmarks miss half of that. These are the levers — not unlabeled SLAs.
Path we run
Tokens decode on capacity we operate, not a thin coat on a hyperscaler chat API you cannot tune for agents.
Workload fit
Packing is aimed at tool loops and concurrency, so interactive turns are not treated like one-off chat demos.
CueCode density
When CueCode declares agent structure, the stack can pack work without serializing everything — latency under load, not empty-queue vanity.
Workspace isolation
One workspace, one cue_… key. You are not fighting mystery org traffic for the same queue.
Live latency / throughput probes stay labeled until the fleet is public. See the home proof stage for the demo cadence.
How the system works
Open Hub models on capacity we operate. You bring a cue_… key. Tokens stream back into the agent loop. Not a reseller badge on someone else’s chat endpoint.
- 1IDE / agent
CueCode or OpenAI-compatible client
- 2Workspace key
cue_… key on a live pack
- 3Cue Cloud
Auth · pack · route
- 4Managed inference
Capacity we operate
- 5Open models
DeepSeek · Kimi · GLM
- 6Stream back
Tokens + tool turns into the loop
CueCode and the API
CueCode
Our IDE + harness can declare agent structure so the stack packs work more efficiently. Better interactive feel under real agent load.
OpenAI-compatible API
Cursor, Claude Code, scripts. Same Cue Cloud models and pack include. Conservative defaults. Works everywhere.
Same Cue Cloud. CueCode unlocks extra agent optimizations; the API works everywhere. See api →
Packs are agent fuel
Multi-step IDE loops stretch. On usage-based OpenAI / Anthropic APIs, that is a bill spike. On Cue Cloud it is $500 × packs for a 1B include this period.