Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.platform.decart.ai/llms.txt

Use this file to discover all available pages before exploring further.

Transform live video streams in realtime. Insert your imagination into any live video feed, turning passive viewing into active creation. The Video Restyling Realtime API takes a video stream and a prompt, then generates a transformed version of the video based on that prompt with minimal latency.

​
Model Specifications

The model expects video input with these specifications:
  • Frame Rate: 25 FPS
  • Resolution: 1280x704 (16:9 aspect ratio)

​
Realtime API

The realtime API uses WebRTC for low-latency video transformation. Clients establish a connection with our inference server, stream video with a prompt, and receive a transformed video stream in return. The connection remains active as long as the input video stream continues, allowing you to change prompts on-the-fly to modify the output video dynamically.

​
Installation

npm install @decartai/sdk

​
Basic Usage

import { createDecartClient, models } from "@decartai/sdk";

const model = models.realtime("lucy-restyle-2");

// Get user's camera stream with model specifications
const stream = await navigator.mediaDevices.getUserMedia({
  audio: true,
  video: {
    frameRate: model.fps,
    width: model.width,
    height: model.height,
  }
});

// Create a client
const client = createDecartClient({
  apiKey: "your-api-key-here"
});

// Connect and transform the video stream
const realtimeClient = await client.realtime.connect(stream, {
  model,
  onRemoteStream: (transformedStream) => {
    // Display the transformed video
    const videoElement = document.querySelector("#video-output");
    videoElement.srcObject = transformedStream;
  }
});

// Change the style on the fly
realtimeClient.setPrompt("Cyberpunk city");

// Disconnect when done
realtimeClient.disconnect();

​
Advanced Features

​
Image Reference

Use a reference image to guide the style transformation. Instead of describing a style with text, you can provide an image that the model will use as a visual reference. 
// Set a reference image to guide the style
const fileInput = document.querySelector('input[type="file"]');
fileInput.addEventListener('change', async (e) => {
  const file = e.target.files[0];
  if (file) {
    await realtimeClient.setImage(file);
  }
});

// Or set from a URL
await realtimeClient.setImage('https://example.com/style-reference.jpg');
Supported formats: JPEG, PNG, WebP
Image reference works great for capturing specific artistic styles, color palettes, or visual aesthetics that are hard to describe in text.

​
Dynamic Prompt Management

// Change prompts without reconnecting
realtimeClient.setPrompt("Studio Ghibli animation style");

​
Connection State Management

import { createDecartClient, type DecartSDKError } from "@decartai/sdk";

const realtimeClient = await client.realtime.connect(stream, {
  model,
  onRemoteStream: (transformedStream) => {
    videoElement.srcObject = transformedStream;
  }
});

// Monitor connection state
realtimeClient.on("connectionChange", (state) => {
  console.log(`Connection: ${state}`); // "connecting" | "connected" | "disconnected"
  
  if (state === "connected") {
    document.getElementById("status").textContent = "Live";
  } else if (state === "disconnected") {
    document.getElementById("status").textContent = "Disconnected";
  }
});

// Handle errors
realtimeClient.on("error", (error: DecartSDKError) => {
  console.error("SDK error:", error.code, error.message);
});

// Check connection synchronously
const isConnected = realtimeClient.isConnected();
const currentState = realtimeClient.getConnectionState();

​
Complete Example with Error Handling

import { createDecartClient, models, type DecartSDKError } from "@decartai/sdk";

async function setupVideoTransform() {
  try {
    const model = models.realtime("lucy-restyle-2");
    
    // Get camera with optimal settings
    const stream = await navigator.mediaDevices.getUserMedia({
      audio: true,
      video: {
        frameRate: model.fps,
        width: model.width,
        height: model.height,
      }
    });

    const client = createDecartClient({
      apiKey: process.env.DECART_API_KEY
    });

    const realtimeClient = await client.realtime.connect(stream, {
      model,
      onRemoteStream: (transformedStream) => {
        const videoElement = document.getElementById("output-video");
        videoElement.srcObject = transformedStream;
      }
    });

    // Set up event handlers
    realtimeClient.on("connectionChange", (state) => {
      updateUIStatus(state);
    });

    realtimeClient.on("error", (error) => {
      console.error("Error:", error);
      showErrorMessage(error.message);
    });

    // Style controls
    document.getElementById("style-input").addEventListener("change", (e) => {
      realtimeClient.setPrompt(e.target.value);
    });

    // Cleanup on page unload
    window.addEventListener("beforeunload", () => {
      realtimeClient.disconnect();
    });

    return realtimeClient;
  } catch (error) {
    console.error("Setup failed:", error);
    throw error;
  }
}

​
WebSocket + WebRTC (Vanilla)

For direct WebSocket communication without the SDK: 
<!DOCTYPE html>
<html>
<head>
  <title>Decart Video Restyling - WebRTC</title>
</head>
<body>
  <!-- Local and transformed video streams -->
  <video id="localVideo" autoplay muted playsinline width="45%"></video>
  <video id="remoteVideo" autoplay playsinline width="45%"></video>
  <br>
  
  <!-- Controls -->
  <button id="startBtn">Start</button>
  <input id="promptInput" placeholder="Enter style prompt...">
  <button id="sendBtn">Send Prompt</button>

  <script>
    // WebSocket connection with API key and model
    const ws = new WebSocket('wss://api3.decart.ai/v1/stream?api_key=YOUR_API_KEY&model=lucy-restyle-2');
    let peerConnection;

    // Handle server messages
    ws.onmessage = async (event) => {
      const message = JSON.parse(event.data);
      
      if (message.type === 'answer' && peerConnection) {
        await peerConnection.setRemoteDescription({
          type: 'answer', 
          sdp: message.sdp
        });
      }
    };

    // Start WebRTC connection
    async function startConnection() {
      if (ws.readyState !== WebSocket.OPEN) return;

      // Create peer connection
      peerConnection = new RTCPeerConnection({
        iceServers: [{ urls: 'stun:stun.l.google.com:19302' }]
      });

      // Send ICE candidates to server
      peerConnection.onicecandidate = (event) => {
        if (event.candidate) {
          ws.send(JSON.stringify({
            type: 'ice-candidate',
            candidate: event.candidate
          }));
        }
      };

      // Receive transformed video stream
      peerConnection.ontrack = (event) => {
        document.getElementById('remoteVideo').srcObject = event.streams[0];
      };

      // Get user camera with optimal settings
      const localStream = await navigator.mediaDevices.getUserMedia({
        video: {
          width: { ideal: 1280 },
          height: { ideal: 704 },
          frameRate: { ideal: 25 }
        },
        audio: false
      });

      // Display local video
      document.getElementById('localVideo').srcObject = localStream;
      
      // Add tracks to peer connection
      localStream.getTracks().forEach(track => {
        peerConnection.addTrack(track, localStream);
      });

      // Create and send offer
      const offer = await peerConnection.createOffer();
      await peerConnection.setLocalDescription(offer);
      
      ws.send(JSON.stringify({
        type: 'offer',
        sdp: offer.sdp
      }));
    }

    // Send style prompt
    function sendPrompt() {
      const promptInput = document.getElementById('promptInput');
      const prompt = promptInput.value.trim();
      
      if (prompt && ws.readyState === WebSocket.OPEN) {
        ws.send(JSON.stringify({
          type: 'prompt',
          prompt: prompt
        }));
      }
    }

    // Event listeners
    document.getElementById('startBtn').onclick = startConnection;
    document.getElementById('sendBtn').onclick = sendPrompt;
    document.getElementById('promptInput').onkeypress = (e) => {
      if (e.key === 'Enter') sendPrompt();
    };
  </script>
</body>
</html>