Skip to main content
The Realtime API enables you to transform live video streams with minimal latency using WebRTC. Perfect for building camera effects, video conferencing filters, VR/AR applications, and interactive live streaming.

​
Quick Start

import { createDecartClient, models } from "@decartai/sdk";

const model = models.realtime("mirage");

// Get user's camera stream
const stream = await navigator.mediaDevices.getUserMedia({
  audio: true,
  video: {
    frameRate: model.fps,
    width: model.width,
    height: model.height,
  },
});

// Create client and connect
const client = createDecartClient({
  apiKey: "your-api-key-here",
});

const realtimeClient = await client.realtime.connect(stream, {
  model,
  onRemoteStream: (transformedStream) => {
    videoElement.srcObject = transformedStream;
  },
  initialState: {
    prompt: {
      text: "Anime",
      enrich: true,
    },
  },
});

// Change style on the fly
realtimeClient.setPrompt("Cyberpunk city");

// Disconnect when done
realtimeClient.disconnect();

​
Connecting

​
Getting Camera Access

Request access to the user’s camera using the WebRTC getUserMedia API: 
const stream = await navigator.mediaDevices.getUserMedia({
  audio: true,
  video: {
    frameRate: 25,
    width: 1280,
    height: 704,
  },
});
Use the model’s fps, width, and height properties to ensure optimal performance.

​
Establishing Connection

Connect to the Realtime API with your media stream: 
const realtimeClient = await client.realtime.connect(stream, {
  model: models.realtime("mirage"),
  onRemoteStream: (transformedStream: MediaStream) => {
    // Display the transformed video
    const videoElement = document.getElementById("output-video");
    videoElement.srcObject = transformedStream;
  },
  initialState: {
    prompt: {
      text: "Lego World",
      enrich: true, // Let Decart enhance the prompt (recommended)
    },
    mirror: false, // Set to true for front-facing cameras
  },
});
Parameters:
  • stream (required) - MediaStream from getUserMedia
  • model (required) - Realtime model from models.realtime()
  • onRemoteStream (required) - Callback that receives the transformed video stream
  • initialState.prompt (required) - Initial style prompt
    • text - Style description
    • enrich - Whether to auto-enhance the prompt (default: true)
  • initialState.mirror (optional) - Enable mirror mode for front-facing cameras

​
Managing Prompts

Change the transformation style dynamically without reconnecting: 
// Simple prompt with automatic enhancement
realtimeClient.setPrompt("Anime style");

// Custom detailed prompt without enhancement
realtimeClient.setPrompt(
  "A detailed artistic style with vibrant colors and dramatic lighting",
  { enrich: false }
);
Parameters:
  • prompt (required) - Text description of desired style
  • options.enrich (optional) - Whether to enhance the prompt (default: true)
Prompt enhancement uses Decart’s AI to expand simple prompts for better results. Disable it if you want full control over the exact prompt.

​
Camera Mirroring

Toggle horizontal mirroring, useful for front-facing cameras: 
// Enable mirror mode
realtimeClient.setMirror(true);

// Disable mirror mode
realtimeClient.setMirror(false);
This flips the video horizontally, making front-facing camera views feel more natural to users.

​
Connection State

Monitor and react to connection state changes: 
// Check state synchronously
const isConnected = realtimeClient.isConnected(); // boolean
const state = realtimeClient.getConnectionState(); // "connected" | "connecting" | "disconnected"

// Listen to state changes
realtimeClient.on("connectionChange", (state) => {
  console.log(`Connection state: ${state}`);
  
  if (state === "disconnected") {
    // Handle disconnection
    showReconnectButton();
  } else if (state === "connected") {
    // Handle successful connection
    hideReconnectButton();
  }
});
Use this to update your UI and handle reconnection logic.

​
Error Handling

Handle errors with the error event: 
import type { DecartSDKError } from "@decartai/sdk";

realtimeClient.on("error", (error: DecartSDKError) => {
  console.error("SDK error:", error.code, error.message);

  switch (error.code) {
    case "INVALID_API_KEY":
      showError("Invalid API key. Please check your credentials.");
      break;
    case "WEB_RTC_ERROR":
      showError("Connection error. Please check your network.");
      break;
    case "MODEL_NOT_FOUND":
      showError("Model not found. Please check the model name.");
      break;
    default:
      showError(`Error: ${error.message}`);
  }
});
Error Codes:
  • INVALID_API_KEY - API key is invalid or missing
  • WEB_RTC_ERROR - WebRTC connection failed
  • MODEL_NOT_FOUND - Specified model doesn’t exist
  • INVALID_INPUT - Invalid input parameters

​
Session Management

Access the current session ID: 
const sessionId = realtimeClient.sessionId;
console.log(`Current session: ${sessionId}`);
This can be useful for logging, analytics, or debugging.

​
Cleanup

Always disconnect when done to free up resources: 
// Disconnect from the service
realtimeClient.disconnect();

// Remove event listeners
realtimeClient.off("connectionChange", onConnectionChange);
realtimeClient.off("error", onError);

// Stop the local media stream
stream.getTracks().forEach(track => track.stop());
Failing to disconnect can leave WebRTC connections open and waste resources.

​
Complete Example

Here’s a full application with all features: 
import { createDecartClient, models, type DecartSDKError } from "@decartai/sdk";

async function setupRealtimeVideo() {
  try {
    // Get camera stream with optimal settings
    const model = models.realtime("mirage");
    const stream = await navigator.mediaDevices.getUserMedia({
      audio: true,
      video: {
        frameRate: model.fps,
        width: model.width,
        height: model.height,
      },
    });

    // Display input video
    const inputVideo = document.getElementById("input-video") as HTMLVideoElement;
    inputVideo.srcObject = stream;

    // Create client
    const client = createDecartClient({
      apiKey: process.env.DECART_API_KEY,
    });

    // Connect to Realtime API
    const realtimeClient = await client.realtime.connect(stream, {
      model,
      onRemoteStream: (transformedStream) => {
        const outputVideo = document.getElementById("output-video") as HTMLVideoElement;
        outputVideo.srcObject = transformedStream;
      },
      initialState: {
        prompt: {
          text: "Studio Ghibli animation style",
          enrich: true,
        },
        mirror: true, // Using front camera
      },
    });

    // Handle connection state changes
    realtimeClient.on("connectionChange", (state) => {
      const statusElement = document.getElementById("status");
      statusElement.textContent = `Status: ${state}`;
      statusElement.className = `status-${state}`;
    });

    // Handle errors
    realtimeClient.on("error", (error: DecartSDKError) => {
      console.error("Realtime error:", error);
      const errorElement = document.getElementById("error");
      errorElement.textContent = error.message;
      errorElement.style.display = "block";
    });

    // Allow user to change styles
    const styleInput = document.getElementById("style-input") as HTMLInputElement;
    styleInput.addEventListener("change", (e) => {
      const prompt = (e.target as HTMLInputElement).value;
      realtimeClient.setPrompt(prompt, { enrich: true });
    });

    // Toggle mirror mode
    const mirrorToggle = document.getElementById("mirror-toggle") as HTMLInputElement;
    mirrorToggle.addEventListener("change", (e) => {
      const enabled = (e.target as HTMLInputElement).checked;
      realtimeClient.setMirror(enabled);
    });

    // Cleanup on page unload
    window.addEventListener("beforeunload", () => {
      realtimeClient.disconnect();
      stream.getTracks().forEach(track => track.stop());
    });

    return realtimeClient;
  } catch (error) {
    console.error("Failed to setup realtime video:", error);
    throw error;
  }
}

// Initialize
setupRealtimeVideo();

​
Best Practices

Always use the model’s fps, width, and height properties when calling getUserMedia to ensure optimal performance and compatibility.
const model = models.realtime("mirage");
const stream = await navigator.mediaDevices.getUserMedia({
  video: {
    frameRate: model.fps,
    width: model.width,
    height: model.height,
  },
});
For best results, keep enrich: true (default) to let Decart’s AI enhance your prompts. Only disable it if you need exact prompt control.
Always listen to connectionChange events to update your UI and handle reconnection logic gracefully.
Always call disconnect() and stop media tracks when done to avoid memory leaks and unnecessary resource usage.
Enable mirror mode when using front-facing cameras to match user expectations from selfie/video call experiences.

​
API Reference

​
client.realtime.connect(stream, options)

Connects to the realtime transformation service. Parameters:
  • stream: MediaStream - MediaStream from getUserMedia
  • options.model: ModelDefinition - Realtime model from models.realtime()
  • options.onRemoteStream: (stream: MediaStream) => void - Callback for transformed video stream
  • options.initialState.prompt: { text: string; enrich?: boolean } - Initial transformation prompt
  • options.initialState.mirror?: boolean - Enable mirror mode (default: false)
Returns: Promise<RealtimeClient> - Connected realtime client instance

​
realtimeClient.setPrompt(prompt, options?)

Changes the transformation style. Parameters:
  • prompt: string - Text description of desired style
  • options.enrich?: boolean - Whether to enhance the prompt (default: true)

​
realtimeClient.setMirror(enabled)

Toggles video mirroring. Parameters:
  • enabled: boolean - Whether to enable mirror mode

​
realtimeClient.isConnected()

Check if currently connected. Returns: boolean

​
realtimeClient.getConnectionState()

Get current connection state. Returns: "connected" | "connecting" | "disconnected"

​
realtimeClient.sessionId

The ID of the current realtime inference session. Type: string

​
realtimeClient.disconnect()

Closes the connection and cleans up resources.

​
Events

​
connectionChange

Fired when connection state changes. Callback: (state: "connected" | "connecting" | "disconnected") => void

​
error

Fired when an error occurs. Callback: (error: DecartSDKError) => void

​
Next Steps

Process API

Learn how to generate and transform media with the Process API

Examples

See complete example applications
⌘I