> ## Documentation Index
> Fetch the complete documentation index at: https://docs.platform.decart.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Lucy 2.6 Realtime

> Edit any element in a live video stream - add, replace, or remove objects, swap characters, change backgrounds, and apply effects in realtime.

> Transform any video in realtime with AI-powered editing at 720p. Lucy 2.6 is a general editing model - capable across edit type, object, and scene.

> [!NOTE]
>
> Use **Lucy 2.6** as your default realtime editing model. It is available as `lucy-latest` and supports text prompts, reference images, or both in the same integration. [Lucy 2.5](https://docs.platform.decart.ai/models/realtime/lucy-2.5.md) remains available as `lucy-2.5`.

## What's new in Lucy 2.6

* **Reference adherence.** Higher fidelity to the reference image on character, garment, and object edits, including complex or non-clean references.
* **Add and replace.** Added objects are sized and placed accurately relative to the subject and scene, with better prompt and reference adherence. Small, non-central, and non-human objects are replaced in place at the right scale.
* **Character swap.** Sharper output with more detail and more natural placement in the frame. Improved lipsync and realism for both photoreal and animated characters.
* **Garments.** Complex and unusual garments hold their shape and detail throughout the stream.
* **Multi-edit.** Several edit operations can be combined in a single prompt, for example background replacement and character swap together. Each request still takes one reference image.

## Quick start

### Installation

**JavaScript**

```bash theme={null}
npm install @decartai/sdk
```

**Python**

```bash theme={null}
pip install decart
```

**Android**

Add the JitPack repository to your `settings.gradle.kts`:

```kotlin settings.gradle.kts theme={null}
dependencyResolutionManagement {
    repositories {
        google()
        mavenCentral()
        maven { url = uri("https://jitpack.io") }
    }
}
```

Then add the dependency to your app's `build.gradle.kts`:

```kotlin build.gradle.kts theme={null}
dependencies {
    implementation("com.github.DecartAI:decart-android:0.2.0")
}
```

### Connect and start editing

**JavaScript**

```typescript theme={null}
import { createDecartClient, models } from "@decartai/sdk";

const model = models.realtime("lucy-2.6");

const stream = await navigator.mediaDevices.getUserMedia({
  video: {
    frameRate: model.fps,
    width: model.width,
    height: model.height,
  },
});

const client = createDecartClient({
  apiKey: "your-api-key-here",
});

const realtimeClient = await client.realtime.connect(stream, {
  model,
  mirror: "auto",
  onRemoteStream: (transformedStream) => {
    document.getElementById("output").srcObject = transformedStream;
  },
  initialState: {
    prompt: {
      text: "Change the wall's color to light blue, natural consistent paint finish.",
      enhance: true,
    },
  },
});

await realtimeClient.set({
  prompt: "Substitute the character in the video with the person in the reference image.",
  image: characterImage,
  enhance: true,
});
```

**Python**

```python theme={null}
import asyncio
from decart import DecartClient, models
from decart.types import ModelState, Prompt

async def main():
    async with DecartClient(api_key="your-api-key-here") as client:
        model = models.realtime("lucy-2.6")

        realtime = await client.realtime.connect(
            stream=media_stream,
            model=model,
            on_remote_stream=lambda s: display(s),
            initial_state=ModelState(
                prompt=Prompt(
                    text="Change the wall's color to light blue, natural consistent paint finish.",
                    enhance=True,
                ),
            ),
        )

        with open("character.jpg", "rb") as f:
            await realtime.set(
                prompt="Substitute the character in the video with the person in the reference image.",
                image=f,
                enhance=True,
            )

asyncio.run(main())
```

**Android**

```kotlin theme={null}
val model = RealtimeModels.LUCY_2_6

val client = DecartClient(context, DecartClientConfig(apiKey = "your-api-key"))
client.realtime.initialize(eglBase)

val videoSource = client.realtime.createVideoSource(isScreencast = false)!!
val videoTrack = client.realtime.createVideoTrack("camera", videoSource)!!

val enumerator = Camera2Enumerator(context)
val cameraName = enumerator.deviceNames.first { enumerator.isFrontFacing(it) }
val capturer = enumerator.createCapturer(cameraName, null)

capturer.initialize(
    SurfaceTextureHelper.create("CaptureThread", client.realtime.getEglBaseContext()),
    context,
    videoSource.capturerObserver
)
capturer.startCapture(model.width, model.height, model.fps)

client.realtime.connect(
    localVideoTrack = videoTrack,
    options = ConnectOptions(
        model = model,
        onRemoteVideoTrack = { track ->
            remoteRenderer.addSink(track)
        },
        initialPrompt = InitialPrompt(
            text = "Change the wall's color to light blue, natural consistent paint finish.",
            enhance = true,
        ),
    )
)

val characterBase64 = Base64.encodeToString(characterBytes, Base64.NO_WRAP)
client.realtime.setImage(
    imageBase64 = characterBase64,
    prompt = "Substitute the character in the video with the person in the reference image.",
    enhance = true
)
```

> [!TIP]
>
> Pass the prompt and/or reference image in `initialState` so the first frame is already transformed - otherwise viewers briefly see the raw camera feed.

## Model capabilities

Lucy 2.6 is designed as a general editing model - reliable across a wide range of tasks, object types, and input conditions.

| Quality dimension | Description |
| - | - |
| **Reference adherence** | Follows the provided reference image precisely |
| **Task adherence** | Executes the prompt exactly as written |
| **Aesthetic and realism** | Natural results across face, body, clothing, and environment |
| **Motion accuracy** | Tracks facial expressions, lip sync, and body movement |
| **Consistency** | Stays stable across long sessions |

### Edit types

| Edit type | Description |
| - | - |
| **Character swap** | Replace a person or character with a new identity from a reference image |
| **Virtual try-on** | Swap clothing or accessories onto a person |
| **Add** | Insert a new object into the scene |
| **Replace** | Swap an existing object with something different |
| **Remove** | Clean removal of objects with background reconstruction |
| **Background** | Swap the scene environment |
| **Style transformation** | Apply a global visual aesthetic to the entire video |
| **Visual effects** | Add dynamic effects - fire, explosions, smoke, particles |

> [!NOTE]
>
> Virtual try-on has a dedicated optimized model. See the [Lucy - Virtual try-on model](https://docs.platform.decart.ai/models/realtime/virtual-try-on.md) for details.

## Use cases

> **Virtual try-on**
>
> Let customers see clothing and accessories on themselves in realtime across a wide range of garment types and materials.

> **Ad variation**
>
> Generate personalized ad creatives at scale. Swap products, backgrounds, or characters across live or pre-recorded content without reshoots.

> **Social engagement**
>
> Give audiences interactive effects, filters, and transformations that drive participation and sharing.

> **Content monetization**
>
> Embed branded products and sponsored placements directly into live video.

> **Live shopping**
>
> Sell any product, in any video. Give sellers the ability to place any item into live content instantly.

> **Furniture replacement**
>
> Swap furniture and home decor in live video, letting customers visualize products in their own space before buying.

## Updating the reference image

You can change the reference image at any time without reconnecting. Use the `set()` method to atomically replace the session state - include all fields you want to keep:

```typescript JavaScript theme={null}
// Change to a new character
await realtimeClient.set({
  prompt: "Substitute the character in the video with the person in the reference image.",
  image: newCharacterImage,
  enhance: true,
});

// Set a new image only (clears any previous prompt)
await realtimeClient.set({ image: newCharacterImage });

// Set a new prompt only (clears any previous image)
await realtimeClient.set({ prompt: "Add dark sunglasses to the person's face." });

// Clear the reference image (fall back to text-only editing)
await realtimeClient.set({ image: null });
```

```python Python theme={null}
# Change to a new character
with open("new_character.jpg", "rb") as f:
    await realtime.set(
        prompt="Substitute the character in the video with the person in the reference image.",
        image=f,
        enhance=True,
    )

# Set a new image only (clears any previous prompt)
with open("another.jpg", "rb") as f:
    await realtime.set(image=f)

# Set a new prompt only (clears any previous image)
await realtime.set(prompt="Add dark sunglasses to the person's face.")
```

```kotlin Android theme={null}
// Change to a new character
val newCharacterBase64 = Base64.encodeToString(newCharacterBytes, Base64.NO_WRAP)
client.realtime.setImage(
    imageBase64 = newCharacterBase64,
    prompt = "Substitute the character in the video with the person in the reference image.",
    enhance = true
)

// Set a new image only
client.realtime.setImage(imageBase64 = newCharacterBase64)

// Set a new prompt only (text-only editing)
client.realtime.setPrompt("Add dark sunglasses to the person's face.")

// Clear the reference image (fall back to text-only editing)
client.realtime.setImage(imageBase64 = null)
```

> [!TIP]
>
> `set()` replaces the entire state - fields you omit are cleared. Always include every field you want to keep.

### Reference image best practices

> **Use a clear, well-lit portrait**
>
> Consistent, even lighting works best regardless of edit type. For characters, use a front-facing photo. For garments and objects, avoid harsh shadows or reflections that obscure important details.

> **Match the framing**
>
> Match the reference framing to the source video. If the source shows a full-body standing person and you want to swap the character or change clothes, use a full-body reference shot rather than a head-and-shoulders crop.

> **Avoid occlusion**
>
> Make sure the key features of the reference are fully visible. Partially hidden characters, garments, or objects reduce what the model can extract and may affect output quality.

> **Resolution and format**
>
> At least 512×512 pixels recommended. Supported formats: JPEG, PNG, and WebP.

> **For object replacement**
>
> Use a clean product or object shot against a neutral background. The cleaner the separation between subject and background, the more precisely the model can apply it.

> **For virtual try-on**
>
> A front-facing, evenly lit garment shot on a plain background gives the model the clearest signal. Include the full item from collar to hem where possible.

## Text-only editing

Lucy 2.6 works without a reference image too. Use text prompts to add, modify, or remove elements in your live video:

```typescript JavaScript theme={null}
// No reference image needed for text-only edits
await realtimeClient.set({ prompt: "Add a pair of sunglasses to the person's face." });
await realtimeClient.set({ prompt: "Change the background to a sandy beach with clear blue water." });
await realtimeClient.set({ prompt: "Change the person's hair color to bright blonde." });
```

```python Python theme={null}
# No reference image needed for text-only edits
await realtime.set(prompt="Add a pair of sunglasses to the person's face.")
await realtime.set(prompt="Change the background to a sandy beach with clear blue water.")
await realtime.set(prompt="Change the person's hair color to bright blonde.")
```

```kotlin Android theme={null}
// No reference image needed for text-only edits
client.realtime.setPrompt("Add a pair of sunglasses to the person's face.")
client.realtime.setPrompt("Change the background to a sandy beach with clear blue water.")
client.realtime.setPrompt("Change the person's hair color to bright blonde.")
```

## Prompting guide

See the [Lucy 2.6 prompting guide](https://docs.platform.decart.ai/models/realtime/lucy-2.6-prompting.md) for prompt templates, reference-image guidance, and video examples for each edit type, including combined edits.

## Connection lifecycle

Lucy 2.6 shares the same connection lifecycle as all realtime models. See the [JavaScript SDK](https://docs.platform.decart.ai/sdks/javascript-realtime.md#connection-state), [Python SDK](https://docs.platform.decart.ai/sdks/python-realtime.md), or [Android SDK](https://docs.platform.decart.ai/sdks/android-realtime.md#connection-state) for details on:

* Connection states (`connecting`, `connected`, `generating`, `reconnecting`, `disconnected`)
* Auto-reconnect with exponential backoff
* Error handling with `DecartSDKError`
* Session tracking with `generationTick` events
* Session viewing with subscribe tokens

## Complete example

A full application with character switching, connection management, and error handling:

```typescript JavaScript theme={null}
import { createDecartClient, models, type DecartSDKError } from "@decartai/sdk";

async function setupLucy26() {
  const model = models.realtime("lucy-2.6");

  const stream = await navigator.mediaDevices.getUserMedia({
    video: {
      frameRate: model.fps,
      width: model.width,
      height: model.height,
    },
  });

  document.getElementById("input-video").srcObject = stream;

  const client = createDecartClient({
    apiKey: process.env.DECART_API_KEY,
  });

  const realtimeClient = await client.realtime.connect(stream, {
    model,
    onRemoteStream: (transformedStream) => {
      document.getElementById("output-video").srcObject = transformedStream;
    },
  });

  realtimeClient.on("connectionChange", (state) => {
    document.getElementById("status").textContent = state;
  });

  realtimeClient.on("generationTick", ({ seconds }) => {
    document.getElementById("usage").textContent = `${seconds}s`;
  });

  realtimeClient.on("error", (error: DecartSDKError) => {
    console.error("Lucy 2.6 error:", error.code, error.message);
  });

  document.getElementById("character-input").addEventListener("change", async (e) => {
    const file = (e.target as HTMLInputElement).files[0];
    if (file) {
      await realtimeClient.set({
        prompt: "Substitute the character in the video with the person in the reference image.",
        image: file,
        enhance: true,
      });
    }
  });

  window.addEventListener("beforeunload", () => {
    realtimeClient.disconnect();
    stream.getTracks().forEach((track) => track.stop());
  });

  return realtimeClient;
}

setupLucy26();
```

```python Python theme={null}
import asyncio
from decart import DecartClient, models

async def main():
    async with DecartClient(api_key="your-api-key-here") as client:
        model = models.realtime("lucy-2.6")

        realtime = await client.realtime.connect(
            stream=media_stream,
            model=model,
            on_remote_stream=lambda s: display(s),
        )

        @realtime.on("connection_change")
        def on_state(state):
            print(f"Connection: {state}")

        @realtime.on("generation_tick")
        def on_tick(seconds):
            print(f"Usage: {seconds}s")

        @realtime.on("error")
        def on_error(error):
            print(f"Lucy 2.6 error: {error.code} {error.message}")

        with open("character.jpg", "rb") as f:
            await realtime.set(
                prompt="Substitute the character in the video with the person in the reference image.",
                image=f,
                enhance=True,
            )

asyncio.run(main())
```

```kotlin Android theme={null}
import ai.decart.sdk.*
import ai.decart.sdk.realtime.*
import org.webrtc.*

val model = RealtimeModels.LUCY_2_6
val client = DecartClient(context, DecartClientConfig(apiKey = "your-api-key"))
client.realtime.initialize(eglBase)

val videoSource = client.realtime.createVideoSource(false)!!
val videoTrack = client.realtime.createVideoTrack("camera", videoSource)!!
val enumerator = Camera2Enumerator(context)
val capturer = enumerator.createCapturer(
    enumerator.deviceNames.first { enumerator.isFrontFacing(it) }, null
)
capturer.initialize(
    SurfaceTextureHelper.create("CaptureThread", client.realtime.getEglBaseContext()),
    context, videoSource.capturerObserver
)
capturer.startCapture(model.width, model.height, model.fps)

client.realtime.connect(
    localVideoTrack = videoTrack,
    options = ConnectOptions(
        model = model,
        onRemoteVideoTrack = { track -> remoteRenderer.addSink(track) }
    )
)

lifecycleScope.launch {
    client.realtime.connectionState.collect { state -> statusText.text = state.name }
}
lifecycleScope.launch {
    client.realtime.generationTicks.collect { tick -> usageText.text = "${tick.seconds}s" }
}
lifecycleScope.launch {
    client.realtime.errors.collect { error ->
        Log.e("Lucy26", "Error: ${error.code} ${error.message}") }
}

fun onCharacterSelected(imageBytes: ByteArray) {
    val base64 = Base64.encodeToString(imageBytes, Base64.NO_WRAP)
    client.realtime.setImage(
        imageBase64 = base64,
        prompt = "Substitute the character in the video with the person in the reference image.",
        enhance = true
    )
}

capturer.stopCapture()
client.realtime.disconnect()
client.release()
eglBase.release()
```

## Client-side authentication

For browser and mobile apps, use client tokens instead of your permanent API key.

```typescript JavaScript theme={null}
// Backend: generate a short-lived token (expires in 60 s by default)
const token = await client.tokens.create();

// Frontend: let the SDK fetch a fresh token right before every connect and reconnect
const frontendClient = createDecartClient({
  apiKeyProvider: async () => {
    const res = await fetch("/api/realtime-token", { method: "POST" });
    const { apiKey } = await res.json();
    return apiKey;
  },
});
```

```python Python theme={null}
# Backend: generate a short-lived token
token = await client.tokens.create()

# Frontend: connect with the token
frontend_client = DecartClient(api_key=token.api_key)
```

```kotlin Android theme={null}
// Fetch a short-lived token from your backend
val ephemeralKey = fetchTokenFromBackend()

// Connect with the token
val client = DecartClient(context, DecartClientConfig(apiKey = ephemeralKey))
```

See the full pattern for [JavaScript](https://docs.platform.decart.ai/sdks/javascript-realtime.md#client-side-authentication) or [Android](https://docs.platform.decart.ai/sdks/android-realtime.md#client-side-authentication).

Client tokens expire 60 seconds after minting by default, so fetch them per dial with `apiKeyProvider` rather than on page load. See [Token lifetime](https://docs.platform.decart.ai/getting-started/client-tokens.md#token-lifetime).

## Technical specifications

| Property | Value |
| - | - |
| **Model ID** | `lucy-2.6` (also `lucy-latest`) |
| **Resolution** | 1280×720 |
| **Orientation** | Landscape (16:9) and Portrait (9:16) |
| **Transport** | WebRTC |
| **Character reference** | Yes (JPEG, PNG, WebP) |
| **Prompt enhancement** | Yes (default: enabled) |
| **Self-anchoring** | Yes (default: enabled) |
| **Fast mode** | Yes (`speed: "fast"`; 2x rate) |
| **Auto-reconnect** | Yes (exponential backoff, up to 5 retries) |

### Portrait mode (9:16)

On mobile devices (iOS/Android), portrait mode works automatically - the OS maps the front camera to a vertical stream regardless of the constraints you pass. For desktop browsers or external webcams, swap `width` and `height` when calling `getUserMedia`:

```typescript theme={null}
const stream = await navigator.mediaDevices.getUserMedia({
  video: {
    frameRate: { ideal: model.fps },
    width: { ideal: model.height },  // swap: use height as width
    height: { ideal: model.width },  // swap: use width as height
    facingMode: "user",
  },
});
```

### Generation stability

In long sessions, Lucy 2.6 can use its own output as an additional anchor - feeding recent generated frames back as the reference image. This self-anchoring mechanism keeps the output stable over time, preventing drifting. It is enabled by default, so most integrations need to do nothing.

Self-anchoring is a connect-time setting. To turn it off, pass `self_anchor=false` as a connection query parameter - the model then anchors only to the live input. Changing it requires a reconnect; it can't be toggled mid-session with `set()`.

```typescript theme={null}
const realtimeClient = await client.realtime.connect(stream, {
  model,
  onRemoteStream: (transformedStream) => {
    document.getElementById("output").srcObject = transformedStream;
  },
  queryParams: { self_anchor: "false" }, // disable self-anchoring
});
```

> [!NOTE]
>
> Self-anchoring works best when the scene stays consistent. If the stream changes significantly - a different person enters the frame, the camera cuts, or the scene shifts entirely - the model may still anchor to the last edited frame, which no longer matches the new input. In these cases do not use self-anchoring.

### Fast mode

Fast mode (`speed: "fast"`) gives the session lower latency and higher throughput; output quality is unchanged. It is available for `lucy-2.6` / `lucy-latest`, `lucy-2.5`, `lucy-vton-3.6` / `lucy-vton-latest`, and `lucy-vton-3.5`, and is billed at 2x the standard realtime rate for those models. Other models ignore the option. Omit it (the default) for standard mode. See [Pricing](https://docs.platform.decart.ai/getting-started/pricing.md#realtime-models).

Like self-anchoring, fast mode is a connect-time setting: pass `speed: "fast"` when connecting. Changing it requires a reconnect; it can't be toggled mid-session with `set()`. The SDK carries the option across its automatic reconnects.

```typescript JavaScript theme={null}
const realtimeClient = await client.realtime.connect(stream, {
  model,
  speed: "fast", // fast mode
  onRemoteStream: (transformedStream) => {
    document.getElementById("output").srcObject = transformedStream;
  },
});
```

```text Raw WebSocket theme={null}
wss://api.decart.ai/v1/stream?api_key=YOUR_API_KEY&model=lucy-2.6&speed=fast
```

See the fast mode reference for [JavaScript](https://docs.platform.decart.ai/sdks/javascript-realtime.md#fast-mode), [Python](https://docs.platform.decart.ai/sdks/python-realtime.md#fast-mode), [Android](https://docs.platform.decart.ai/sdks/android-realtime.md#fast-mode) and [Swift](https://docs.platform.decart.ai/sdks/swift-realtime.md#fast-mode).

> [!NOTE]
>
> Standard mode is the *absence* of the option — don't send `speed=standard` or an empty value. If you run a [signaling proxy](https://docs.platform.decart.ai/integrations/signaling-proxy-ws.md), forward the end user's `speed` to the upstream URL, or the session lands on the standard tier.

## Next steps

> **[Prompting Guide](https://docs.platform.decart.ai/models/realtime/lucy-2.6-prompting.md)**
>
> Prompt templates, reference guidance, and video examples for Lucy 2.6

> **[JavaScript SDK](https://docs.platform.decart.ai/sdks/javascript-realtime.md)**
>
> Full JavaScript SDK reference for realtime features

> **[Python SDK](https://docs.platform.decart.ai/sdks/python-realtime.md)**
>
> Full Python SDK reference for realtime features

> **[Android SDK](https://docs.platform.decart.ai/sdks/android-realtime.md)**
>
> Full Android SDK reference for realtime features

> **[All Models](https://docs.platform.decart.ai/getting-started/models.md)**
>
> Compare all Decart models side by side


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.