> ## Documentation Index
> Fetch the complete documentation index at: https://docs.platform.decart.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Streaming Best Practices

> Optimize your realtime video integration for quality, performance, and reliability

Build production-ready realtime video integrations. This guide covers camera setup, connection management, prompt strategy, error handling, and mobile-specific considerations.

## Camera setup

Always use the model's built-in constraints when requesting camera access. Each model defines the `fps`, `width`, and `height` it expects — passing these directly avoids scaling artifacts and wasted bandwidth.

```typescript theme={null}
import { models } from "@decartai/sdk";

const model = models.realtime("lucy-latest");

const stream = await navigator.mediaDevices.getUserMedia({
  video: {
    frameRate: model.fps,
    width: model.width,
    height: model.height,
  },
});
```

<Warning>
  Requesting a resolution that doesn't match the model's expectations forces the browser to scale the video. This adds latency and can reduce output quality.
</Warning>

### Handling constraint failures

Not every device supports every resolution. Use `ideal` constraints as a fallback when targeting a wide range of hardware:

```typescript theme={null}
const stream = await navigator.mediaDevices.getUserMedia({
  video: {
    frameRate: { ideal: model.fps },
    width: { ideal: model.width },
    height: { ideal: model.height },
  },
});
```

### Front vs back camera

On mobile devices, specify the `facingMode` to select the camera:

```typescript theme={null}
const stream = await navigator.mediaDevices.getUserMedia({
  video: {
    facingMode: "user", // front camera
    // facingMode: "environment" // back camera
    frameRate: model.fps,
    width: model.width,
    height: model.height,
  },
});
```

<Tip>
  When switching between front and back cameras, recapture the stream — each camera has different hardware capabilities that affect resolution and frame rate.
</Tip>

## Front-camera mirroring

For selfie streams, pre-flip the input with `mirror`. Server-baked pixels (watermarks, overlays) then come out in display orientation, and the output renders as-is.

```typescript theme={null}
const realtimeClient = await client.realtime.connect(stream, {
  model,
  mirror: "auto",
  onRemoteStream: (s) => { videoElement.srcObject = s; },
});
```

* `"auto"` mirrors when the input track reports `facingMode: "user"`.
* Pass `true` when your app already knows the camera is front-facing (desktop webcams often don't report `facingMode`).

The SDK only flips the stream it sends — not your local preview.

## Connection management

### Connect once, update prompts

Establishing a WebRTC connection takes a few hundred milliseconds. Don't reconnect just to change the style or prompt — use `set()` or `setPrompt()` instead:

```typescript theme={null}
// Good: update prompt without reconnecting
await realtimeClient.set({ prompt: "New style" });

// Avoid: disconnecting and reconnecting for every prompt change
realtimeClient.disconnect(); // unnecessary
await client.realtime.connect(stream, { model, ... }); // slow
```

### When to reconnect

Reconnect only when you need to:

* **Switch models** (e.g., from Lucy Restyle Live to Lucy 2.1) — each model runs on a different pipeline
* **Switch cameras** — the stream changes, so you need a new connection
* **Recover from a failed connection** after auto-reconnect gives up

```typescript theme={null}
async function switchModel(newModel: ModelDefinition) {
  realtimeClient.disconnect();
  await new Promise((r) => setTimeout(r, 100)); // allow cleanup
  realtimeClient = await client.realtime.connect(stream, {
    model: newModel,
    onRemoteStream: (s) => { videoElement.srcObject = s; },
  });
}
```

### Track connection state

Always listen to `connectionChange` events to update your UI:

```typescript theme={null}
realtimeClient.on("connectionChange", (state) => {
  switch (state) {
    case "connecting":
      showSpinner();
      break;
    case "connected":
    case "generating":
      hideSpinner();
      break;
    case "reconnecting":
      showReconnectingBanner();
      break;
    case "disconnected":
      showDisconnectedUI();
      break;
  }
});
```

**Connection states:**

| State          | Meaning                              | User-visible action        |
| -------------- | ------------------------------------ | -------------------------- |
| `connecting`   | Initial connection in progress       | Show loading spinner       |
| `connected`    | Ready to send prompts                | Show connected UI          |
| `generating`   | Actively producing transformed video | Show "live" indicator      |
| `reconnecting` | Auto-reconnecting after drop         | Show "reconnecting" banner |
| `disconnected` | Not connected                        | Show reconnect button      |

## Prompt strategy

### Use prompt enhancement

Enable `enhance: true` (the default) to let Decart expand short prompts into detailed descriptions. This dramatically improves output quality for simple inputs:

```typescript theme={null}
// Short prompt → enhanced automatically
realtimeClient.setPrompt("Anime style"); // enhanced behind the scenes

// Detailed prompt → disable enhancement for full control
realtimeClient.setPrompt(
  "A highly detailed 2D anime character with smooth cel-shaded lines, soft pastel highlights, large expressive eyes",
  { enhance: false }
);
```

<Tip>
  Start with enhancement enabled. Only disable it when you need exact control over the prompt text.
</Tip>

### Atomic updates with `set()`

Since `set()` replaces the entire state, always include every field you want to keep. When using Lucy 2.1 with both a prompt and reference image, include both in every call:

```typescript theme={null}
// Good: atomic update
await realtimeClient.set({
  prompt: "Transform into this character",
  image: characterPhoto,
  enhance: true,
});

// Avoid: separate calls can cause flickering
await realtimeClient.setPrompt("Transform into this character");
await realtimeClient.setImage(characterPhoto); // brief mismatch between prompt and image
```

### Debounce rapid prompt changes

If your UI lets users type prompts in a text field, debounce the input to avoid sending dozens of rapid updates:

```typescript theme={null}
let debounceTimer: ReturnType<typeof setTimeout>;

function onPromptInput(text: string) {
  clearTimeout(debounceTimer);
  debounceTimer = setTimeout(() => {
    realtimeClient.setPrompt(text, { enhance: true });
  }, 300); // wait 300ms after the user stops typing
}
```

## Error handling

### Listen for errors

Always register an error handler. Without one, connection failures go unnoticed:

```typescript theme={null}
import type { DecartSDKError } from "@decartai/sdk";

realtimeClient.on("error", (error: DecartSDKError) => {
  console.error(`[${error.code}] ${error.message}`);

  switch (error.code) {
    case "INVALID_API_KEY":
      redirectToLogin();
      break;
    case "WEB_RTC_ERROR":
      showNetworkError();
      break;
    case "MODEL_NOT_FOUND":
      showModelError();
      break;
    default:
      showGenericError(error.message);
  }
});
```

### Auto-reconnect

The SDK automatically reconnects when an unexpected disconnection occurs (e.g., network interruption). It retries with exponential backoff up to 5 times:

1. Connection drops → state moves to `"reconnecting"`
2. SDK retries with increasing delays
3. If reconnection succeeds → state moves back to `"generating"`
4. If all retries fail → state moves to `"disconnected"` and an `error` event fires

You don't need to implement reconnection logic yourself — but you should update your UI to reflect the `reconnecting` state.

### Wrap setup in try/catch

Camera access and connection can both fail. Catch errors early:

```typescript theme={null}
async function startRealtime() {
  try {
    const stream = await navigator.mediaDevices.getUserMedia({
      video: { frameRate: model.fps, width: model.width, height: model.height },
    });
    const realtimeClient = await client.realtime.connect(stream, {
      model,
      onRemoteStream: (s) => { videoElement.srcObject = s; },
    });
    return realtimeClient;
  } catch (error) {
    if (error instanceof DOMException && error.name === "NotAllowedError") {
      showCameraPermissionError();
    } else {
      showConnectionError(error);
    }
  }
}
```

## Cleanup

Always release resources when the session ends or the component unmounts:

```typescript theme={null}
function cleanup() {
  // 1. Disconnect from Decart
  realtimeClient.disconnect();

  // 2. Remove event listeners
  realtimeClient.off("connectionChange", onConnectionChange);
  realtimeClient.off("error", onError);

  // 3. Stop the camera
  stream.getTracks().forEach((track) => track.stop());
}

// Browser: cleanup on page unload
window.addEventListener("beforeunload", cleanup);

// React: cleanup on unmount
useEffect(() => {
  return () => cleanup();
}, []);
```

<Warning>
  Forgetting to call `disconnect()` leaves WebRTC connections open, consuming bandwidth and server resources. Always clean up.
</Warning>

## Client-side authentication

Never expose your permanent API key in client-side code. Use short-lived client tokens instead:

<Steps>
  <Step title="Create a backend endpoint">
    ```typescript theme={null}
    // Your server (Express, Next.js, etc.)
    const token = await client.tokens.create();
    res.json(token);
    ```
  </Step>

  <Step title="Fetch and use the token in the browser">
    ```typescript theme={null}
    const { apiKey } = await fetch("/api/realtime-token", { method: "POST" }).then((r) => r.json());
    const client = createDecartClient({ apiKey }); // short-lived token, not your permanent key
    ```
  </Step>
</Steps>

<Info>
  Client tokens only prevent **new** connections after expiry. Active WebRTC sessions continue working even after the token expires.
</Info>

## Mobile considerations

### React Native / Expo

React Native's WebRTC implementation differs from the browser:

* **Force VP8 codec.** React Native works best with VP8. Use `customizeOffer` to reorder the SDP:

```typescript theme={null}
const realtimeClient = await client.realtime.connect(stream, {
  model,
  onRemoteStream: (s) => setRemoteStream(s),
  customizeOffer: (offer: RTCSessionDescriptionInit) => {
    forceCodecInSDP(offer, "VP8");
  },
});
```

* **Type casting.** React Native's `MediaStream` type differs from `globalThis.MediaStream`. Cast when needed:

```typescript theme={null}
const realtimeClient = await client.realtime.connect(
  stream as unknown as globalThis.MediaStream,
  { ... }
);
```

* **Handle app lifecycle.** Disconnect when backgrounded, reconnect when foregrounded:

```typescript theme={null}
const handleAppState = (nextState: AppStateStatus) => {
  if (nextState === "active") {
    initializeWebRTC({ model, facingMode });
  } else {
    cleanupWebRTC();
  }
};

AppState.addEventListener("change", handleAppState);
```

<Tip>
  See the [Realtime Mobile App](/examples/real-time-mobile-app) walkthrough for a complete Expo integration with model switching and style presets.
</Tip>

### Battery and bandwidth

Realtime WebRTC streaming consumes significant resources on mobile:

* **Disconnect when not visible.** Use app lifecycle events to pause streaming in the background.
* **Match camera constraints to model.** Don't request 4K when the model expects 720p — this wastes encoding power and bandwidth.
* **Use front camera when possible.** Front cameras typically have lower resolution, which reduces encoding overhead.

## Session tracking

Use the `generationTick` event to track session duration and display billing information to users:

```typescript theme={null}
realtimeClient.on("generationTick", ({ seconds }) => {
  document.getElementById("timer").textContent = `${seconds}s`;
});
```

## Quick reference

| Topic          | Recommendation                                         |
| -------------- | ------------------------------------------------------ |
| Camera setup   | Use `model.fps`, `model.width`, `model.height`         |
| Prompt changes | Use `set()` or `setPrompt()` — don't reconnect         |
| Prompt + image | Use `set()` atomically — avoid separate calls          |
| Enhancement    | Keep `enhance: true` unless you need exact control     |
| Error handling | Register `error` and `connectionChange` listeners      |
| Reconnection   | Handled by SDK automatically (5 retries, backoff)      |
| Cleanup        | Always `disconnect()` + stop media tracks              |
| Auth           | Use client tokens in the browser, never your API key   |
| Mobile         | Force VP8, handle app lifecycle, match camera to model |

## Next steps

<CardGroup cols={2}>
  <Card title="Lucy 2.1 Guide" icon="sparkles" href="/models/realtime/lucy-2.1">
    Character transformation with reference images
  </Card>

  <Card title="Reference Image Guide" icon="image" href="/models/realtime/reference-images">
    Best practices for character and style reference images
  </Card>

  <Card title="JavaScript SDK" icon="js" href="/sdks/javascript-realtime">
    Full SDK reference for realtime features
  </Card>

  <Card title="Mobile App" icon="mobile" href="/examples/real-time-mobile-app">
    Complete Expo walkthrough
  </Card>
</CardGroup>