Streaming Best Practices

Build production-ready realtime video integrations. This guide covers camera setup, connection management, prompt strategy, error handling, and mobile-specific considerations.

Camera setup

Always use the model’s built-in constraints when requesting camera access. Each model defines the fps, width, and height it expects — passing these directly avoids scaling artifacts and wasted bandwidth.

import { models } from "@decartai/sdk";

const model = models.realtime("lucy_2_rt");

const stream = await navigator.mediaDevices.getUserMedia({
  video: {
    frameRate: model.fps,
    width: model.width,
    height: model.height,
  },
});

Requesting a resolution that doesn’t match the model’s expectations forces the browser to scale the video. This adds latency and can reduce output quality.

Handling constraint failures

Not every device supports every resolution. Use ideal constraints as a fallback when targeting a wide range of hardware:

const stream = await navigator.mediaDevices.getUserMedia({
  video: {
    frameRate: { ideal: model.fps },
    width: { ideal: model.width },
    height: { ideal: model.height },
  },
});

Front vs back camera

On mobile devices, specify the facingMode to select the camera:

const stream = await navigator.mediaDevices.getUserMedia({
  video: {
    facingMode: "user", // front camera
    // facingMode: "environment" // back camera
    frameRate: model.fps,
    width: model.width,
    height: model.height,
  },
});

When switching between front and back cameras, recapture the stream — each camera has different hardware capabilities that affect resolution and frame rate.

Connection management

Connect once, update prompts

Establishing a WebRTC connection takes a few hundred milliseconds. Don’t reconnect just to change the style or prompt — use set() or setPrompt() instead:

// Good: update prompt without reconnecting
await realtimeClient.set({ prompt: "New style" });

// Avoid: disconnecting and reconnecting for every prompt change
realtimeClient.disconnect(); // unnecessary
await client.realtime.connect(stream, { model, ... }); // slow

When to reconnect

Reconnect only when you need to:

Switch models (e.g., from Mirage to Lucy 2) — each model runs on a different pipeline
Switch cameras — the stream changes, so you need a new connection
Recover from a failed connection after auto-reconnect gives up

async function switchModel(newModel: ModelDefinition) {
  realtimeClient.disconnect();
  await new Promise((r) => setTimeout(r, 100)); // allow cleanup
  realtimeClient = await client.realtime.connect(stream, {
    model: newModel,
    onRemoteStream: (s) => { videoElement.srcObject = s; },
  });
}

Track connection state

Always listen to connectionChange events to update your UI:

realtimeClient.on("connectionChange", (state) => {
  switch (state) {
    case "connecting":
      showSpinner();
      break;
    case "connected":
    case "generating":
      hideSpinner();
      break;
    case "reconnecting":
      showReconnectingBanner();
      break;
    case "disconnected":
      showDisconnectedUI();
      break;
  }
});

Connection states:

State	Meaning	User-visible action
`connecting`	Initial connection in progress	Show loading spinner
`connected`	Ready to send prompts	Show connected UI
`generating`	Actively producing transformed video	Show “live” indicator
`reconnecting`	Auto-reconnecting after drop	Show “reconnecting” banner
`disconnected`	Not connected	Show reconnect button

Prompt strategy

Use prompt enhancement

Enable enhance: true (the default) to let Decart expand short prompts into detailed descriptions. This dramatically improves output quality for simple inputs:

// Short prompt → enhanced automatically
realtimeClient.setPrompt("Anime style"); // enhanced behind the scenes

// Detailed prompt → disable enhancement for full control
realtimeClient.setPrompt(
  "A highly detailed 2D anime character with smooth cel-shaded lines, soft pastel highlights, large expressive eyes",
  { enhance: false }
);

Start with enhancement enabled. Only disable it when you need exact control over the prompt text.

Atomic updates with `set()`

When using Lucy 2 with both a prompt and reference image, always update them together using set(). This prevents intermediate states where only one value has changed:

// Good: atomic update
await realtimeClient.set({
  prompt: "Transform into this character",
  image: characterPhoto,
  enhance: true,
});

// Avoid: separate calls can cause flickering
await realtimeClient.setPrompt("Transform into this character");
await realtimeClient.setImage(characterPhoto); // brief mismatch between prompt and image

Debounce rapid prompt changes

If your UI lets users type prompts in a text field, debounce the input to avoid sending dozens of rapid updates:

let debounceTimer: ReturnType<typeof setTimeout>;

function onPromptInput(text: string) {
  clearTimeout(debounceTimer);
  debounceTimer = setTimeout(() => {
    realtimeClient.setPrompt(text, { enhance: true });
  }, 300); // wait 300ms after the user stops typing
}

Error handling

Listen for errors

Always register an error handler. Without one, connection failures go unnoticed:

import type { DecartSDKError } from "@decartai/sdk";

realtimeClient.on("error", (error: DecartSDKError) => {
  console.error(`[${error.code}] ${error.message}`);

  switch (error.code) {
    case "INVALID_API_KEY":
      redirectToLogin();
      break;
    case "WEB_RTC_ERROR":
      showNetworkError();
      break;
    case "MODEL_NOT_FOUND":
      showModelError();
      break;
    default:
      showGenericError(error.message);
  }
});

Auto-reconnect

The SDK automatically reconnects when an unexpected disconnection occurs (e.g., network interruption). It retries with exponential backoff up to 5 times:

Connection drops → state moves to "reconnecting"
SDK retries with increasing delays
If reconnection succeeds → state moves back to "generating"
If all retries fail → state moves to "disconnected" and an error event fires

You don’t need to implement reconnection logic yourself — but you should update your UI to reflect the reconnecting state.

Wrap setup in try/catch

Camera access and connection can both fail. Catch errors early:

async function startRealtime() {
  try {
    const stream = await navigator.mediaDevices.getUserMedia({
      video: { frameRate: model.fps, width: model.width, height: model.height },
    });
    const realtimeClient = await client.realtime.connect(stream, {
      model,
      onRemoteStream: (s) => { videoElement.srcObject = s; },
    });
    return realtimeClient;
  } catch (error) {
    if (error instanceof DOMException && error.name === "NotAllowedError") {
      showCameraPermissionError();
    } else {
      showConnectionError(error);
    }
  }
}

Cleanup

Always release resources when the session ends or the component unmounts:

function cleanup() {
  // 1. Disconnect from Decart
  realtimeClient.disconnect();

  // 2. Remove event listeners
  realtimeClient.off("connectionChange", onConnectionChange);
  realtimeClient.off("error", onError);

  // 3. Stop the camera
  stream.getTracks().forEach((track) => track.stop());
}

// Browser: cleanup on page unload
window.addEventListener("beforeunload", cleanup);

// React: cleanup on unmount
useEffect(() => {
  return () => cleanup();
}, []);

Forgetting to call disconnect() leaves WebRTC connections open, consuming bandwidth and server resources. Always clean up.

Client-side authentication

Never expose your permanent API key in client-side code. Use short-lived client tokens instead:

Create a backend endpoint

// Your server (Express, Next.js, etc.)
const token = await client.tokens.create();
res.json(token);

Fetch and use the token in the browser

const { apiKey } = await fetch("/api/realtime-token", { method: "POST" }).then((r) => r.json());
const client = createDecartClient({ apiKey }); // short-lived token, not your permanent key

Client tokens only prevent new connections after expiry. Active WebRTC sessions continue working even after the token expires.

Mobile considerations

React Native / Expo

React Native’s WebRTC implementation differs from the browser:

Force VP8 codec. React Native works best with VP8. Use customizeOffer to reorder the SDP:

const realtimeClient = await client.realtime.connect(stream, {
  model,
  onRemoteStream: (s) => setRemoteStream(s),
  customizeOffer: (offer: RTCSessionDescriptionInit) => {
    forceCodecInSDP(offer, "VP8");
  },
});

Type casting. React Native’s MediaStream type differs from globalThis.MediaStream. Cast when needed:

const realtimeClient = await client.realtime.connect(
  stream as unknown as globalThis.MediaStream,
  { ... }
);

Handle app lifecycle. Disconnect when backgrounded, reconnect when foregrounded:

const handleAppState = (nextState: AppStateStatus) => {
  if (nextState === "active") {
    initializeWebRTC({ model, facingMode });
  } else {
    cleanupWebRTC();
  }
};

AppState.addEventListener("change", handleAppState);

See the Realtime Mobile App walkthrough for a complete Expo integration with model switching and style presets.

Battery and bandwidth

Realtime WebRTC streaming consumes significant resources on mobile:

Disconnect when not visible. Use app lifecycle events to pause streaming in the background.
Match camera constraints to model. Don’t request 4K when the model expects 720p — this wastes encoding power and bandwidth.
Use front camera when possible. Front cameras typically have lower resolution, which reduces encoding overhead.

Session tracking

Use the generationTick event to track session duration and display billing information to users:

realtimeClient.on("generationTick", ({ seconds }) => {
  document.getElementById("timer").textContent = `${seconds}s`;
});

Quick reference

Topic	Recommendation
Camera setup	Use `model.fps`, `model.width`, `model.height`
Prompt changes	Use `set()` or `setPrompt()` — don’t reconnect
Prompt + image	Use `set()` atomically — avoid separate calls
Enhancement	Keep `enhance: true` unless you need exact control
Error handling	Register `error` and `connectionChange` listeners
Reconnection	Handled by SDK automatically (5 retries, backoff)
Cleanup	Always `disconnect()` + stop media tracks
Auth	Use client tokens in the browser, never your API key
Mobile	Force VP8, handle app lifecycle, match camera to model

Next steps

Lucy 2 Guide

Character transformation with reference images

Reference Image Guide

Best practices for character and style reference images

JavaScript SDK

Full SDK reference for realtime features

Mobile App

Complete Expo walkthrough

Getting started

Image

Video

Realtime

Libraries & SDKs

Resources

Camera setup

Handling constraint failures

Front vs back camera

Connection management

Connect once, update prompts

When to reconnect

Track connection state

Prompt strategy

Use prompt enhancement

Atomic updates with `set()`

Debounce rapid prompt changes

Error handling

Listen for errors

Auto-reconnect

Wrap setup in try/catch

Cleanup

Client-side authentication

Mobile considerations

React Native / Expo

Battery and bandwidth

Session tracking

Quick reference

Next steps

Lucy 2 Guide

Reference Image Guide

JavaScript SDK

Mobile App

Getting started

Image

Video

Realtime

Libraries & SDKs

Resources

​Camera setup

​Handling constraint failures

​Front vs back camera

​Connection management

​Connect once, update prompts

​When to reconnect

​Track connection state

​Prompt strategy

​Use prompt enhancement

​Atomic updates with set()

​Debounce rapid prompt changes

​Error handling

​Listen for errors

​Auto-reconnect

​Wrap setup in try/catch

​Cleanup

​Client-side authentication

​Mobile considerations

​React Native / Expo

​Battery and bandwidth

​Session tracking

​Quick reference

​Next steps

Lucy 2 Guide

Reference Image Guide

JavaScript SDK

Mobile App

Camera setup

Handling constraint failures

Front vs back camera

Connection management

Connect once, update prompts

When to reconnect

Track connection state

Prompt strategy

Use prompt enhancement

Atomic updates with `set()`

Debounce rapid prompt changes

Error handling

Listen for errors

Auto-reconnect

Wrap setup in try/catch

Cleanup

Client-side authentication

Mobile considerations

React Native / Expo

Battery and bandwidth

Session tracking

Quick reference

Next steps