Quickstart with WebSocket API

​

Introduction

The Soket’s Realtime Speech API, built on OpenAI’s protocol, enables implementation of conversational AI capabilities through WebSockets, a widely supported protocol optimized for real-time server-to-server data transfer. This guide demonstrates how to establish WebSocket connections and interact with Realtime models. WebRTC support for browser and mobile clients will be available in an upcoming release.

​

Access Your API Key

1. Log in to your TensorStudio account.
2. Navigate to the API Keys section under manage.
3. Generate a new API key for your project.

​

Explore the Realtime Speech API

The Realtime Speech API lets you create responsive conversational applications powered by expressive voice-enabled models. It handles both text and audio in real-time, detects when users are speaking, and can execute custom functions - all with minimal latency.

​

Connection details

Connecting via WebSocket requires the following connection information:

Below are several examples of using these connection details to initialize a WebSocket connection to the Realtime API.

import WebSocket from "ws";

const url = "wss://api.soket.ai/v1/realtime?model=pragna-realtime";
const ws = new WebSocket(url, {
  headers: {
    "Authorization": "Bearer " + process.env.TENSOR_STUDIO_API_KEY,
    "OpenAI-Beta": "realtime=v1",
  },
});

ws.on("open", function open() {
  console.log("Connected to server.");
});

ws.on("message", function incoming(message) {
  console.log(JSON.parse(message.data));
});

​

Sending and receiving events

Communication with Realtime models occurs through bidirectional message exchange over the WebSocket interface. The comprehensive documentation of available client and server messages can be found in the OpenAI Realtime API reference. Upon establishing a connection, the interface supports various event types including text messages, audio streams, tool calling, speech interruptions using Voice Activity Detection (VAD), and configuration modifications - enabling robust real-time interactions with the model.

Below, you’ll find examples of how to send and receive events over the WebSocket interface in several programming environments.

// Server-sent events will come in as messages...
ws.on("message", function incoming(message) {
    // Message data payloads will need to be parsed from JSON:
    const serverEvent = JSON.parse(message.data)
    console.log(serverEvent);
});

// To send events, create a JSON-serializeable data structure that
// matches a client-side event (see API reference)
const event = {
    type: "response.create",
    response: {
        modalities: ["audio", "text"],
        instructions: "Give me a haiku about code.",
    }
};
ws.send(JSON.stringify(event));

​

Need Help?

• Community Support: Join the TensorStudio’s Discord Community for tips and discussions.
• Customer Support: Reach us at support@tensorstudio.ai.