Chat demos
- One thread
- Short context
- Reply and done
productagents.md
Agents
Why Cue Cloud focuses on coding agents: the loop shape, latency and throughput, and how owned decode keeps interactive turns alive.
Why we focus here
Coding agents are the load that matters
Chat is one shot → reply. Coding agents plan, read the repo, call tools, patch, verify, and often run concurrent turns with fat context. Cue Cloud is sized for that pattern — not demos that look fast on a single bubble.
Coding agents
- Multi-turn tool loops
- Growing context + retries
- Parallel work, long sessions
What an agent loop actually does
Context grows. Tools fan out. Idle waits happen. Then decode bursts. That is why chat-optimized infra feels wrong here.
Latency and throughput
Agents need both: snappy first tokens so the loop stays interactive, and sustained decode so patches and parallel tool work do not stall.
Latency
Time to first useful token
The loop waits on the first decision. Snappy TTFT keeps plan / tool choice interactive.
Throughput
Sustained decode under load
Long patches and parallel tools need tokens that keep coming — not a burst that dies when concurrency rises.
How we keep latency good
Good agent latency is time-to-first-useful-token plus steady decode while tools are in flight. Chat benchmarks miss half of that. These are the levers — not unlabeled SLAs.
Owned decode path
Tokens decode on hardware we run — not a thin coat on a hyperscaler chat API you cannot tune for agents.
Workload fit
Packing is aimed at tool loops and concurrency, so interactive turns are not treated like one-off chat demos.
CueCode density
When CueCode declares agent structure, the stack can pack work without serializing everything — latency under load, not empty-queue vanity.
Per-seat isolation
One engineer ↔ one seat ↔ one cue_… key. You are not fighting mystery org traffic for the same queue.
Live latency / throughput probes stay labeled until the fleet is public. See the home proof stage for the demo cadence.
How the system and hardware work
Owned coding inference: open Hub models on Mac Studio–class hardware we operate. You bring a seat key. Tokens stream back into the agent loop. Not a reseller badge on someone else’s chat endpoint.
- 1IDE / agent
CueCode or OpenAI-compatible client
- 2Seat key
cue_… per engineer
- 3Cue Cloud
Auth · seat · route
- 4Owned fleet
Mac Studio–class decode we operate
- 5Open models
DeepSeek · Kimi · GLM
- 6Stream back
Tokens + tool turns into the loop
CueCode on the density path
CueCode
Our IDE + harness can declare agent structure so the stack packs work more efficiently — better interactive feel under real agent load on this fleet.
OpenAI-compatible API
Cursor, Claude Code, scripts — same Cue Cloud models and flat seat. Conservative defaults; they still get the owned decode path.
Same inference plane. CueCode unlocks extra agent optimizations; the API works everywhere. See api →
Flat seats are agent fuel
Multi-step IDE loops stretch. On usage-based OpenAI / Anthropic APIs, that is a bill spike. On Cue Cloud it is still $1,500/mo per seat · unlimited tokens for that user.