Dedicated Deployments (api.cyan.kiwi/serverless)
A Dedicated Deployment is a model the cyankiwi agent deploys for you from the workspace, served as a private, OpenAI-compatible endpoint. To make your first request with curl or the OpenAI SDKs, see the API Usage guide.
Base URL: https://api.cyan.kiwi/serverlessYours alone. Each deployment runs on its own workers, which serve that deployment only. They are not shared with other customers or with Shared Inference. Only requests made with your API key reach your deployment: another account sending the same model name gets the same model_not_found as a typo, so it cannot even tell your deployment exists. Dedicated Deployments also have their own base URL, separate from Shared Inference (/v1), so the two never mix.
Zero data retention. Your deployment’s workers do not log prompts or completions, and we do not store them; they are never used for training. We keep only usage metadata for each request (timing and status) for billing. See the Privacy Notice for details.
Routes
Only these two routes exist: POST /serverless/chat/completions and GET /serverless/models. Text completions and embeddings are not served on deployments. Any OpenAI SDK or tool works with these two routes.
GET /serverless/models lists your deployments that are starting or serving. Each id is the value to pass as model, the name shown on the Deployment tab. The extra state field is the deployment’s current status; it does not guarantee that a worker is already running. A new deployment is listed as soon as it is set up and accepts requests right away: its first request starts a worker, so that request waits for the model to load or answers 503 cold_start with Retry-After.
A response has a time limit, shorter for a stream than for a non-streamed response, and the time a cold deployment spends starting counts toward it. There is also an overall limit on one request. For longer outputs, lower max_tokens or split the request.
If your client disconnects, the request still runs to its end on the worker and that time is billed.
Concurrency and bursts
The configured per-deployment concurrency ceiling is workers_max × max_num_seqs (maximum workers times sequences per worker), not a guarantee that all those requests can start or finish at once. Stay under the ceiling and, if a request gets 429, retry after the pause in the Retry-After header; on a cold deployment, expect the first requests to wait for workers to start. If streams are interrupted under load, send fewer requests at once.
Request limits
A refused request never reaches your deployment and never starts a worker. When a parameter is refused, the param field of the error names it.
| Limit | code (status 400 unless noted) |
|---|---|
| Request body at most 2 MiB | request_too_large (413) |
logprobs, top_logprobs, prompt_logprobs are not accepted | logprobs_not_supported |
chat_template, or chat_template / tokenize inside chat_template_kwargs, are not accepted | template_override_not_permitted |
Engine-specific control fields (guided_json, guided_regex, guided_grammar, guided_choice, structural_tag, prompt_embeds, prompt_token_ids and similar) are not accepted | unsupported_parameter |
n and best_of must be positive integers, at most 8 | invalid_parameter, too_many_completions |
max_tokens / max_completion_tokens, when set, must be positive integers, no larger than the deployment’s context length | invalid_parameter, output_limit_exceeds_context |
| An output limit sent to a deployment with no recorded context length (omit it, or redeploy) | output_limit_unverifiable |
Cold starts and billing
A deployment scales to zero workers when it is idle, and you are not billed while it has no workers. The first request after a quiet period starts a worker, so it can take noticeably longer, or answer 503 with Retry-After while the worker starts. Honor Retry-After and retry.
You are billed for the time workers are running, including the time a worker spends loading the model and a short idle period after the last request, before the worker stops. That idle period is the idle timeout: the agent tells you its value when it plans the deployment, and you can ask for a different one then; the agent tells you the range it accepts.
A deployment keeps serving until you ask the agent to stop it, or until it has gone unused for a long time, when it is stopped automatically (ask the agent when that would be). What you have spent is on the Account tab.
Errors
Errors use the standard OpenAI envelope ({"error": {"message", "type", "code"}}), so SDK exception handling works unchanged.
| Situation | Status | code |
|---|---|---|
| Missing or invalid API key | 401 | invalid_api_key |
No model in the request, or the body is not a JSON object | 400 | model_required |
| No deployment of yours has that model name | 404 | model_not_found |
| The deployment was stopped and removed | 410 | model_deleted |
| Credits exhausted | 429 | insufficient_credits, with no Retry-After |
| Cold-starting (the deployment was starting; start-up time counts toward the non-stream time limit) | 503 | cold_start, with Retry-After |
| No GPU capacity was available to start a worker | 503 | gpu_unavailable, with Retry-After |
| A stream could not start within the streaming time limit (workers busy or still starting) | 503 | deployment_busy, with Retry-After |
| A non-streamed request ran past the non-stream time limit | 504 | response_time_limit |
| A request ran past the overall time limit | 504 (before any output) or an in-stream error event | response_time_limit |
| A stream reached the streaming time limit | in-stream error event | response_time_limit |
| A stream was interrupted before it finished | in-stream error event | stream_interrupted |
The deployment returned no output (usually the prompt plus max_tokens is above the context length) | 502 | empty_response |
The prompt (estimated) plus max_tokens is above the context length | 400 | context_length_exceeded |
| The deployment rejected the request | its 4xx status | the deployment’s own error code when it gives one, else request_rejected |
| The deployment is at capacity; reduce simultaneous requests and retry after a pause | 429 | deployment_at_capacity, with Retry-After (follow its value) |
| The deployment failed and is not serving | 503 | deployment_failed |
| Temporarily unavailable | 503 | registry_unavailable, service_unavailable |
| The deployment request failed | 502, or the deployment’s own 5xx status | model_error |
A streamed response that ends early finishes with an error event instead of [DONE], for example data: {"error": {"message": "...", "type": "api_error", "param": null, "code": "response_time_limit"}}. Messages say what went wrong and, where it helps, what to do next: lower max_tokens, shorten the prompt, send fewer requests at once, or retry once the deployment is warm. The context-length check on the prompt is an estimate; the deployment makes the exact check.
gpu_unavailable: No GPU of this deployment’s class was free to start a worker. Retry in a few minutes; if it keeps happening, ask the agent to stop this deployment and plan it again. If a deployment fails to start because no GPU became available before its start deadline (reason start_no_capacity), plan it again later.
A 429 with code insufficient_credits means your balance ran out: top up on the Account tab to resume the deployment; retrying will not help. Tell it apart from deployment_at_capacity (also 429) by the code. deployment_failed means the deployment is not serving: ask the agent to check it, then stop it and deploy again in chat.
Managing a deployment
The Deployment tab shows planned and active deployments, and shows failed or stopped deployments for 24 hours after they end; status of any deployment remains available by asking the agent.
Planning, starting and stopping deployments happen in the workspace chat. Ask the agent for status at any time; to stop a deployment, name it and confirm in chat.