> ## Documentation Index
> Fetch the complete documentation index at: https://docs.platform.decart.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Realtime Mobile App

> Build a realtime AI video app with Expo and the Decart SDK

Transform live camera video with AI on mobile devices. This walkthrough covers the architecture and key integration points of an open-source Expo app that connects to Decart's realtime models — including Lucy 2.1 for character transformation.

<Info>
  This example uses Expo with `react-native-webrtc`. The same `@decartai/sdk` works with any React Native setup.
</Info>

**Source code:** [github.com/DecartAI/decart-example-expo-realtime](https://github.com/DecartAI/decart-example-expo-realtime)

## What you'll build

<CardGroup cols={3}>
  <Card title="Model switching" icon="toggle-on">
    Switch between Lucy Restyle Live, Lucy editing, and — after a quick upgrade — Lucy 2.1 for character transformation.
  </Card>

  <Card title="Style presets" icon="palette">
    Swipe through curated looks: anime characters, tuxedos, superhero costumes, and more. Each preset is a text prompt sent to the model.
  </Card>

  <Card title="Camera controls" icon="camera-rotate">
    Toggle front/back camera, switch view modes (fullscreen, picture-in-picture, split), and see your original and transformed feeds side by side.
  </Card>
</CardGroup>

## Prerequisites

* Node.js 18+
* Expo (`npx expo` — no global install needed)
* A Decart API key from [platform.decart.ai](https://platform.decart.ai)
* A physical iOS or Android device (WebRTC requires real camera hardware)

## Project setup

<Steps>
  <Step title="Clone the repository">
    ```bash theme={null}
    git clone https://github.com/DecartAI/decart-example-expo-realtime.git
    cd decart-example-expo-realtime
    ```
  </Step>

  <Step title="Install dependencies">
    ```bash theme={null}
    npm install
    ```
  </Step>

  <Step title="Add your API key">
    Create a `.env` file in the project root:

    ```bash theme={null}
    EXPO_PUBLIC_DECART_API_KEY=your-api-key-here
    ```
  </Step>

  <Step title="Run on a device">
    ```bash theme={null}
    npx expo run:ios
    # or
    npx expo run:android
    ```
  </Step>
</Steps>

<Warning>
  WebRTC requires a physical device. The iOS Simulator and Android Emulator do not support camera streaming.
</Warning>

## Architecture overview

The app uses Expo Router with three screens:

| Screen      | File                        | Purpose                                                  |
| ----------- | --------------------------- | -------------------------------------------------------- |
| Entry       | `app/index.tsx`             | Checks camera/microphone permissions, routes accordingly |
| Permissions | `app/permission-screen.tsx` | Requests camera and microphone access                    |
| Camera      | `app/camera.tsx`            | Renders the main `<Camera />` component                  |

Key directories:

```
components/camera/
  hooks/useWebRTC.ts      — Core Decart SDK integration
  ui/ModelSelector.tsx     — Model toggle UI
  ui/VideoRenderer.tsx     — Displays local + remote streams
  ui/style-carousel/       — Swipeable style picker

lib/realtime/
  media-streams.ts         — Camera stream setup with model constraints
  webrtc-utils.ts          — VP8 codec forcing for React Native

lib/skins/
  lucy-skin-list.ts        — 21 text prompts for Lucy editing presets
  lucy-restyle-live-skin-list.ts      — Style presets with optional audio
```

## Core integration: the WebRTC hook

The `useWebRTC` hook handles the entire Decart SDK lifecycle — creating a client, capturing the camera, connecting to a realtime model, and managing cleanup.

```typescript theme={null}
import { useCallback, useRef, useState } from "react";
import { createDecartClient, type ModelDefinition, type RealTimeClient } from "@decartai/sdk";
import type { MediaStream } from "react-native-webrtc";
import { getMediaStream } from "@/lib/realtime/media-streams";
import { forceCodecInSDP } from "@/lib/realtime/webrtc-utils";

export function useWebRTC({ model, facingMode }: { model: ModelDefinition; facingMode: string }) {
  const [localMediaStream, setLocalMediaStream] = useState<MediaStream | null>(null);
  const [remoteMediaStream, setRemoteMediaStream] = useState<MediaStream | null>(null);
  const [connectionState, setConnectionState] = useState<string>("disconnected");
  const realtimeClientRef = useRef<RealTimeClient | null>(null);

  const initializeWebRTC = useCallback(
    async ({ model, facingMode }: { model: ModelDefinition; facingMode: string }) => {
      try {
        // 1. Create the Decart client
        const decartClient = createDecartClient({
          apiKey: process.env.EXPO_PUBLIC_DECART_API_KEY as string,
        });

        // 2. Capture camera with model-specific constraints
        const stream = await getMediaStream(facingMode, {
          fps: model.fps,
          width: model.width,
          height: model.height,
        });
        setLocalMediaStream(stream);
        setConnectionState("connecting");

        // 3. Connect to the realtime model
        realtimeClientRef.current = await decartClient.realtime.connect(
          stream as unknown as globalThis.MediaStream,
          {
            model,
            onRemoteStream: (transformedStream) => {
              setRemoteMediaStream(transformedStream as unknown as MediaStream);
              setConnectionState("connected");
            },
            customizeOffer: (offer: RTCSessionDescriptionInit) => {
              forceCodecInSDP(offer, "VP8");
            },
          },
        );
      } catch (error) {
        console.error("Connection failed:", error);
        setConnectionState("failed");
      }
    },
    [],
  );

  const cleanupWebRTC = useCallback(() => {
    realtimeClientRef.current?.disconnect();
    setRemoteMediaStream(null);
    setConnectionState("disconnected");
  }, []);

  return {
    localMediaStream,
    remoteMediaStream,
    connectionState,
    realtimeClientRef,
    initializeWebRTC,
    cleanupWebRTC,
  };
}
```

Three things to note:

1. **Model constraints drive camera setup.** Each model defines its own `fps`, `width`, and `height`. The `getMediaStream` helper requests exactly those dimensions from the device camera.

2. **`customizeOffer` forces VP8.** React Native's WebRTC implementation works best with VP8. The `forceCodecInSDP` utility reorders the SDP to prefer VP8 over H264.

3. **Type casting.** React Native's `MediaStream` type differs from the browser's `globalThis.MediaStream`. The `as unknown as` casts bridge this gap.

## Switching models

Each model offers different creative capabilities:

| Model                 | ID               | What it does                                                         |
| --------------------- | ---------------- | -------------------------------------------------------------------- |
| **Lucy 2.1**          | `lucy-2.1`       | Character transformation and text-guided editing in one model        |
| **Lucy Restyle Live** | `lucy-restyle-2` | Full video restyling — transform scenes into different visual styles |

Switch models by creating a new `ModelDefinition` and reconnecting:

```typescript theme={null}
import { models, type ModelDefinition } from "@decartai/sdk";

// Select a model
const restyleLive = models.realtime("lucy-restyle-2");
const lucy2 = models.realtime("lucy-2.1");

// To switch: disconnect, update state, reconnect
async function switchModel(newModel: ModelDefinition) {
  cleanupWebRTC();
  await new Promise((resolve) => setTimeout(resolve, 100)); // allow cleanup to complete
  await initializeWebRTC({ model: newModel, facingMode });
}
```

<Tip>
  Use Lucy 2.1 as the default for new mobile integrations. It supports both character reference and text-only editing.
</Tip>

## Applying style presets

The app includes curated prompt presets for each model. When a user selects a preset, the prompt is sent to the active model:

```typescript theme={null}
// Apply a style preset to the current model
if (realtimeClientRef.current) {
  realtimeClientRef.current.setPrompt(skin.prompt, {
    enhance: skin.enrich ?? false,
  });
}
```

Example prompts from the preset list:

```typescript theme={null}
const presets = [
  {
    title: "Anime Character",
    prompt:
      "Transform the person into a 2D anime character with smooth cel-shaded lines, " +
      "soft pastel highlights, large expressive eyes, clean contours, even lighting, " +
      "simplified textures, and a bright studio-style background for a polished anime look.",
  },
  {
    title: "Black Tuxedo",
    prompt:
      "Change the outfit to a sharp black tuxedo with satin lapels, crisp white shirt, " +
      "black bow tie, tailored fit, polished shoes, and soft indoor lighting reflecting " +
      "off a marble floor.",
  },
  {
    title: "Sunglasses",
    prompt:
      "Add a pair of dark tinted sunglasses resting naturally on the person's face, " +
      "smooth acetate frames, subtle reflections on the lenses, accurate nose " +
      "placement, and soft shadows across the cheeks.",
  },
];
```

For Lucy 2.1, use `set()` instead of `setPrompt()` to include a reference image:

```typescript theme={null}
// Lucy 2.1: transform into a character using a reference image
await realtimeClientRef.current.set({
  prompt: "Transform into this character",
  image: characterImage, // File, Blob, or URL
  enhance: true,
});
```

## Handling app lifecycle

Mobile apps frequently move between foreground and background. The example handles this by disconnecting when backgrounded and reconnecting when foregrounded:

```typescript theme={null}
import { AppState, type AppStateStatus } from "react-native";

useEffect(() => {
  const handleAppStateChange = (nextAppState: AppStateStatus) => {
    if (appStateRef.current.match(/inactive|background/) && nextAppState === "active") {
      // App returned to foreground — reconnect
      initializeWebRTC({ model, facingMode });
    } else if (appStateRef.current === "active" && nextAppState.match(/inactive|background/)) {
      // App going to background — disconnect to free resources
      cleanupWebRTC();
    }
    appStateRef.current = nextAppState;
  };

  const subscription = AppState.addEventListener("change", handleAppStateChange);
  return () => subscription?.remove();
}, [cleanupWebRTC, initializeWebRTC]);
```

<Tip>
  Always disconnect when backgrounded. WebRTC connections consume battery and network resources even when the app isn't visible.
</Tip>

## Camera controls

**Mirror the front camera.** When switching to the front-facing camera, call `setMirror(true)` so the output matches what the user sees:

```typescript theme={null}
function switchCamera(newFacingMode: "user" | "environment") {
  if (realtimeClientRef.current) {
    realtimeClientRef.current.setMirror(newFacingMode === "user");
  }
}
```

**Switch facing mode.** The camera stream needs to be recaptured when toggling between front and back cameras, since each has different hardware capabilities.

## Adding Lucy 2.1 support

<Note>
  Adding Lucy 2.1 requires two small changes: updating the model selector and using the `set()` method for character reference images.
</Note>

**1. Add Lucy 2.1 to the model selector:**

```typescript theme={null}
import { models } from "@decartai/sdk";

// Add lucy-2.1 alongside the existing models
const handleTabPress = (modelName: "lucy-restyle-2" | "lucy-2.1") => {
  const newModel = models.realtime(modelName);
  onModelChange(newModel);
};
```

**2. Use `set()` for character reference images:**

Lucy 2.1's key feature is character transformation via reference images. Replace `setPrompt()` with `set()` when a reference image is available:

```typescript theme={null}
// For Lucy 2.1 with a reference image
await realtimeClientRef.current.set({
  prompt: "Transform into this character",
  image: selectedImage,
  enhance: true,
});

// For text-only editing (works with all models)
realtimeClientRef.current.setPrompt("Add sunglasses", { enhance: true });
```

## Next steps

<CardGroup cols={2}>
  <Card title="Lucy 2.1 guide" icon="sparkles" href="/models/realtime/lucy-2.1">
    Character transformation, reference images, and the `set()` API in detail.
  </Card>

  <Card title="JavaScript SDK" icon="js" href="/sdks/javascript-realtime">
    Full SDK reference for connection management, events, and client-side auth.
  </Card>

  <Card title="All realtime models" icon="bolt" href="/models/realtime/overview">
    Compare Lucy 2.1 and Lucy Restyle Live side by side.
  </Card>

  <Card title="Source code" icon="github" href="https://github.com/DecartAI/decart-example-expo-realtime">
    Clone the full example app and start building.
  </Card>
</CardGroup>