Transform live video streams in realtime. Insert your imagination into any live video feed, turning passive viewing into active creation.The Video Restyling Realtime API takes a video stream and a prompt, then generates a transformed version of the video based on that prompt with minimal latency.
Model Specifications
The model expects video input with these specifications:
Frame Rate: 25 FPS
Resolution: 1280x704 (16:9 aspect ratio)
Realtime API
The realtime API uses WebRTC for low-latency video transformation. Clients establish a connection with our inference server, stream video with a prompt, and receive a transformed video stream in return.The connection remains active as long as the input video stream continues, allowing you to change prompts on-the-fly to modify the output video dynamically.
Installation
npm install @decartai/sdk
Basic Usage
import { createDecartClient, models } from "@decartai/sdk";const model = models.realtime("lucy-restyle-2");// Get user's camera stream with model specificationsconst stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: { frameRate: model.fps, width: model.width, height: model.height, }});// Create a clientconst client = createDecartClient({ apiKey: "your-api-key-here"});// Connect and transform the video streamconst realtimeClient = await client.realtime.connect(stream, { model, onRemoteStream: (transformedStream) => { // Display the transformed video const videoElement = document.querySelector("#video-output"); videoElement.srcObject = transformedStream; }});// Change the style on the flyrealtimeClient.setPrompt("Cyberpunk city");// Disconnect when donerealtimeClient.disconnect();
Advanced Features
Image Reference
Use a reference image to guide the style transformation. Instead of describing a style with text, you can provide an image that the model will use as a visual reference.
// Set a reference image to guide the styleconst fileInput = document.querySelector('input[type="file"]');fileInput.addEventListener('change', async (e) => { const file = e.target.files[0]; if (file) { await realtimeClient.setImage(file); }});// Or set from a URLawait realtimeClient.setImage('https://example.com/style-reference.jpg');
Supported formats: JPEG, PNG, WebP
Image reference works great for capturing specific artistic styles, color palettes, or visual aesthetics that are hard to describe in text.
Dynamic Prompt Management
// Change prompts without reconnectingrealtimeClient.setPrompt("Studio Ghibli animation style");