Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.tensorstudio.ai/llms.txt

Use this file to discover all available pages before exploring further.

​
Installation

Install the SDK using npm: 
npm install https://github.com/soketlabs/openai-realtime-api-beta

​
Creating a Client

import { RealtimeClient } from '@openai/realtime-api-beta';

const client = new RealtimeClient({
  apiKey: 'your-api-key',
  // For browser environments, set this to true
  /* Warning: Use only in development or testing — exposes your API key */
  dangerouslyAllowAPIKeyInBrowser: true 
});

// Or use a relay server to protect your API key
const client = new RealtimeClient({
  url: 'your-relay-server-url'
});
Refer to Relay Server Guide to set up and use a relay server.

​
Session Configuration

After creating the client, you can set instructions for your AI agent, configure various session parameters: 
// Configure the session with multiple parameters
client.updateSession({
  // Set system instructions for the AI
  instructions: 'You are a helpful assistant',
  
  // Set voice and language
  voice: 'monica',
  language: 'en',
  
  // Set up Voice Activity Detection
  turn_detection: {
    type: 'server_vad',
    threshold: 0.5,
    prefix_padding_ms: 300,
    silence_duration_ms: 500
  }
});

​
Basic Usage

​
Connect and Send Messages

// Connect to the service
await client.connect();

// Send a text message
client.sendUserMessageContent([{
  type: 'input_text',
  text: 'Hello!'
}]);

// Listen for responses
client.on('conversation.updated', ({ item, delta }) => {
  if (delta?.audio) {
    // `delta.audio` contains raw audio chunks (typically in binary or base64 format).
    // You need to implement the `playAudio` function to handle playback,
    // e.g., using Web Audio API or any audio player of your choice.
    playAudio(delta.audio);
  }
  if (item.formatted.text) {
    // Handle text response
    console.log(item.formatted.text);
  }
});