LiveKit

livekit-plugins-prosodyai connects a LiveKit room to the ProsodyAI speech model. One open connection carries caller audio upstream and model audio, transcript spans, speaker events, and identity events downstream.

Install

python -m pip install "livekit-plugins-prosodyai[duplex]"
export PROSODYAI_API_KEY="psk_..."

Add the realtime model

from livekit.agents import AgentSession
from livekit.plugins import prosodyai

model = prosodyai.RealtimeModel()
session = AgentSession(llm=model)

await session.start(room=ctx.room)

ProsodyAI owns the speech loop. Audio advances its recurrence continuously, including while the model speaks. The same model state owns response timing, speaker lanes, identity, and saved conversation context.

Listen for committed events

The gateway sends typed events through the realtime session:

realtime = model.sessions[-1]

@realtime.on("prosody_transcript")
def on_transcript(event):
    for delta in event.deltas:
        print(event.speaker_id, delta.start_ms, delta.end_ms, delta.text)

@realtime.on("prosody_event")
def on_speaker_event(event):
    print(event.to_dict())

@realtime.on("prosody_identity")
def on_identity(event):
    print(event.speaker_id, event.person_id, event.display_name)

prosody_event carries committed SpeakerChangeEvent, NewSpeakerEvent, and IdentityResolvedEvent values. Event timestamps refer to the source audio position. Transcript spans carry the lane selected by the model.

Direct bridge

Workers with their own track publication loop can use FullDuplexBridge.

from livekit.plugins.prosodyai import (
    FullDuplexBridge,
    FullDuplexBridgeConfig,
    GatewayConnection,
)

connection = GatewayConnection.from_environment()
bridge = FullDuplexBridge(
    FullDuplexBridgeConfig(
        url=connection.url,
        api_key=connection.api_key,
        room_sample_rate=24_000,
    )
)

await bridge.run(
    uplink_pcm16(),
    on_downlink_pcm16=publish_pcm16,
    on_event=handle_gateway_event,
)

The bridge accepts little-endian mono PCM16 and handles the gateway's 24 kHz Opus transport.

Identity

Lane IDs such as speaker_0 belong to one session. A person_id is the pseudonymous id of a profile your organization enrolled, with the person's consent, and it is stable across that organization's sessions. Enrollment and retention are governed by the Voice Authentication Policy. The model can emit an identity event during the conversation as soon as an enrolled profile resolves.