Vidu‑S is the real-time interactive avatar API built on Vidu S1, the world-leading streaming video generation model. Bring one image and one line of persona — get a character that talks, performs and perceives, for minutes or hours without breaking.
6-step integration · works for web & native apps · audio-only or full video
The product
Vidu‑S generates an unlimited, real-time video stream of your character — a real person, an anime figure, even a pet — and lets your users talk with it face to face.
Real people, anime, mascots or pets — full-body or half-body, any art style.
It doesn't just respond — it watches and listens, picking up your users' appearance and emotions.
Speak to steer its actions in real time — strong instruction following, industry-fastest inference.
Custom personas with short-term memory make every session personal — a companion, not a widget.
Why Vidu-S
The first interactive digital human that can converse, perform, and perceive in both directions — ready for production, not just demos.
The world's first generative video technology with unlimited interaction time — from one minute to two hours with zero quality degradation.
Deep semantic understanding and the fastest inference in the industry deliver near-real-time, cross-screen interaction that feels alive.
Custom looks and personalities — human, anime or pet — plus short-term memory, for company that feels personal, not scripted.
Voice, text and video in and out. It precisely reads what it sees and hears — including your users' looks and moods.
High-resolution frames generated live and streamed at a steady 540P / 25FPS — crisp enough for commerce, smooth enough for conversation.
Where it lands
Ship characters that can interact, perform, perceive and keep users company — in social, e-commerce, gaming, education and beyond.
Always-on characters that remember, comfort, and grow closer over time.
One-on-one fan meet-and-greets that scale to millions without losing the sparkle.
Endlessly patient tutors and role-play partners for any teaching scenario.
A face for your support — warmer than a chat bubble, faster than a queue.
Hosts that pitch, react to comments, and answer buyers in real time, 24/7.
NPCs and story characters that improvise with the audience instead of replaying lines.
How it works
You bring a key, an image and a persona. Vidu‑S handles generation, understanding and streaming.
Token vda_xxxOne POST /live/v1/lives with your call mode and avatar opens a "room". You get back the room id and an entry ticket for the media channel.
The avatar's video and voice travel over real-time audio/video, not HTTP. Join with the ticket, publish your microphone (plus camera in video mode), and subscribe to the avatar's stream.
A persistent signalling line to the server. Connect with your live_id and immediately send the conn_init "I'm ready" message.
success: true means your avatar is on-line — start talking. A NOT_READY reply is normal in video mode: reconnect with exponential backoff until it flips to ready.
The server pings every 5 seconds; answer with any message within 15 or be dropped. Listen for forced-hangup signals and handle them gracefully.
Send the hangup signal, leave the RTC channel, then query the session for exact billed seconds and credits spent.
API reference
Everything below is live documentation from the Vidu‑S private beta. Pick your region — all examples update.
Every request carries your key in the Authorization header. Keys are issued from the Vidu platform workbench.
Authorization: Token vda_xxxxxxxxxxxxxxxx
| Method | Path | Purpose |
|---|---|---|
| POST | /live/v1/lives | Create an avatar session |
| GET | /live/v1/lives/{live_id} | Query session status & billing |
| WS | /live/ws/live/connect | Control signalling (start / hangup) |
| POST | /live/v1/voices/clone | Clone a new voice |
| GET | /live/v1/voices | List your custom voices |
Hosts — Global: api.vidu.com · China: api.vidu.cn
Opens the room. Returns the live.id you'll use everywhere, plus RTC credentials for the media channel.
POST https://api.vidu.com/live/v1/lives
Authorization: Token vda_xxx
Content-Type: application/json
{
"call_mode": "video",
"avatar": {
"persona": "You are a friendly host. Chat with users naturally, in real time.",
"image_uri": "https://your-cdn.com/avatar.png",
"name": "Tina",
"voice": ""
}
}| Field | Type | Required | Notes |
|---|---|---|---|
call_mode | String | Yes | audio voice-only · video audio + video |
avatar.persona | String | Yes | Who the character is. No length limit. |
avatar.image_uri | String | Yes | One single-person image (full or half body, any style). URL or Base64. PNG / JPG / JPEG / WEBP, up to 50 MB. Base64 must decode to <20 MB and include a content-type prefix, e.g. data:image/png;base64,… |
avatar.name | String | No | Display name, ≤20 chars recommended |
avatar.voice | String | No | Any voice id from the voice library or your clone. Default: Tina |
{
"live": {
"id": "123456789",
"status": "waiting",
"live_duration": 600,
"call_mode": "video"
},
"rtc": {
"app_id": "xxxx",
"channel_id": "live-user-123456789",
"user_id": "live-user-1001-123456789",
"token": "base64-token...",
"token_expire_at": "1750003600"
}
}| Field | Notes |
|---|---|
live.id | The room id. Every later step needs it — save it. |
live.status | waiting — the room exists, the avatar isn't in yet |
live.live_duration | Max session length in seconds (auto-disconnect; cap 600) |
rtc.token | Media-channel ticket, valid ~1 hour. Expired → create a new session. |
rtc.user_id | Your identity inside the RTC channel |
Audio and video are carried by Aliyun real-time channels (ARTC), not HTTP — you must join before you can see or hear the avatar. Download the ARTC SDK ↗
// Join with the rtc.token and rtc.user_id from step 1
await aliRtc.joinChannel(rtc.token, rtc.user_id);
// Publish your microphone (required — or the avatar can't hear you)
await aliRtc.publishLocalAudioStream(true);
// Publish your camera (video mode only)
await aliRtc.publishLocalVideoStream(true);
// Subscribe to the avatar's audio & video
// Its user id is always: live-bot-{creatorID}-{liveID}
aliRtc.on('videoSubscribeStateChanged', (uid, newState) => {
if (newState === 'subscribed') {
// render the stream into your page
}
});| Role | User id format |
|---|---|
| You (the user) | live-user-{creatorID}-{liveID} |
| The avatar | live-bot-{creatorID}-{liveID} |
| Video push stream | live-video-push-{creatorID}-{liveID} |
### audio mode
join channel: live-audio-{liveID}
my uid: live-user-{creatorID}-{liveID}
publish: mic audio -> live-user-{creatorID}-{liveID}
subscribe: avatar audio -> live-bot-{creatorID}-{liveID}
### video mode
join channel: live-user-{liveID}
my uid: live-user-{creatorID}-{liveID}
publish: mic audio -> live-user-{creatorID}-{liveID}
camera video -> live-user-{creatorID}-{liveID}
subscribe: avatar video -> live-video-push-{creatorID}-{liveID}A persistent control line used to say "I'm ready", and later to hang up.
wss://api.vidu.com/live/ws/live/connect?live_id={live_id}
Authorization: Token vda_xxxAs soon as the socket opens, send:
{
"type": 1,
"live_id": "123456789",
"seq_id": 1,
"payload": {
"conn_init": { "version": 1 } // version is always 1
}
}The server answers your conn_init with one of three outcomes:
conn_init_ack { success: true }
The avatar is live — start talking. Billing starts at this moment.
success: false, error_code: "NOT_READY"
Normal in video mode. Close the socket, wait 2–3 s, reconnect (step 3). Use exponential backoff: 2s → 4s → 8s.
error_code: "LIVE_CONN_INIT_FAILED"
Not transient. Go back to step 1 and create a fresh session.
The server pings every 5 seconds. Your client must produce some message within 15 seconds or it's treated as dead and force-disconnected. Most WebSocket libraries auto-reply to ping — verify yours does.
The server can end the call at any time. This message means you've been disconnected:
{
"type": 6,
"payload": {
"hangup": {
"hangup_reason": "xxxx"
}
}
}| hangup_reason | Trigger |
|---|---|
user_end | User hung up voluntarily |
timeout | Session hit its time limit |
audit_violation | Content-safety system ended the call |
credit_insufficient | Account ran out of credits |
sip_closed | SIP / provider side closed |
provider_closed | Provider actively closed |
sip_reconnect_timeout | SIP dropped and missed the reconnect grace window |
client_reconnect_timeout | App client dropped and missed the reconnect grace window |
prepared_sip_disconnected | SIP dropped while the session was still prepared (app never went ready) |
owner_taken_over | Internal: session owner taken over by another node |
owner_lease_lost | Internal: owner lease renewal failed |
ai_output_closed | Internal: AI output channel closed |
external | Broadcast close (live:close) |
type: 6, also watch the raw WebSocket close/error events as a fallback. Notes: WS messages cap at 64 KB; after a balance or safety hangup, don't auto-reconnect.When you're done, send the hangup over the WebSocket, then close it and call leaveChannel() on the RTC SDK.
{
"type": 5,
"live_id": "123456789",
"seq_id": 2,
"payload": {
"hangup": { "hangup_reason": "user_end" }
}
}GET https://api.vidu.com/live/v1/lives/{live_id}
Authorization: Token vda_xxx
{
"live": {
"id": "969824102288199680",
"status": "ended",
"call_mode": "video",
"avatar": { "name": "Tina", "voice": "Tina", "persona": "..." },
"billed_seconds": 18,
"credits_cost": 27,
"created_at": "1782895599"
}
}| Field | Notes |
|---|---|
status | ended once the session is over |
billed_seconds | Billable time, counted from the moment the avatar was ready |
started_at / ended_at | Billing window (Unix timestamps) |
credits_cost | Credits deducted for this session |
Turn any mp3/wav sample into a reusable voice, then reference it by name in avatar.voice.
POST https://api.vidu.com/live/v1/voices/clone
Authorization: Token vda_xxx
{
"audio_url": "https://your-cdn.com/sample.mp3",
"voice": "brand_voice_01",
"language": "en"
}
{ "voice": "brand_voice_01" }| Field | Type | Required | Notes |
|---|---|---|---|
audio_url | String | Yes | Link to the audio to clone — mp3 or wav |
voice | String | Yes | Your custom voice name |
language | String | No | Language hint, e.g. en, zh |
List the clones you've created (system voices are in the gallery):
GET https://api.vidu.com/live/v1/voices
Authorization: Token vda_xxx
{ "voices": [ { "voice": "brand_voice_01" } ] }| Error | Cause | Fix |
|---|---|---|
HTTP 401 | Missing or invalid API key | Check the Authorization header |
HTTP 403 | Not the session's creator | Use the same key that created the session |
HTTP 404 | Session doesn't exist or expired | Create a new session |
WS NOT_READY | Avatar pipeline still warming up | Reconnect with exponential backoff |
WS LIVE_CONN_INIT_FAILED | Initialization failed for good | Create a new session |
WS auth uses a header — terminate the socket on your server so the Vidu key never ships to a browser.
Aliyun RTC must be integrated and downloaded separately — budget for it in your build.
It's a normal warm-up signal, not an error. Ship retry logic with backoff from day one.
Credits are deducted continuously during a call; at zero the session cuts off. Starting a session requires ≥45 credits (a 30-second runway).
The model itself sustains up to 2 hours — talk to us about longer windows for your use case.
Voice library
Every built-in voice speaks 29+ languages — English, Spanish, Japanese, Arabic, Hindi and more. Audition below, then set voice in your create-session call. Default voice: Tina.
One audio file → a production voice via POST /live/v1/voices/clone. 899 credits per clone — your first 10 are free.
Pricing
Usage-based credits. Audio and video modes cost the same. Billing runs only while the avatar is actually live.
Live interaction
≈ 90 credits per live minute · audio & video billed identically
on_live — the success: true ack).Your first 10 clones are free on every account.
New accounts get ~11 live minutes on us. Existing customers: ask your account manager. Volume & enterprise pricing available.
Request accessJoin the Vidu‑S private beta and ship an interactive digital human in days, not quarters.