# Delete files
Source: https://docs.tensorstudio.ai/api-reference/endpoint/delete-files

DELETE /files
Delete one or more uploaded files by file_id. Removes storage objects, upload records, and related jobs.

Delete one or more uploaded files by `file_id`. Removes storage objects, upload records, and any related transcription jobs.

## Request body

```json theme={"system"}
{
  "file_ids": [
    "1374132b-b5db-4ec3-9cf1-f61f982e89b0",
    "2485243c-c6ec-5fd4-0dg2-g72g093f90c1"
  ]
}
```

## Example

```bash theme={"system"}
curl -X DELETE "https://api.soket.ai/files" \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"file_ids": ["1374132b-b5db-4ec3-9cf1-f61f982e89b0"]}'
```

## Typical response

```json theme={"system"}
{
  "deleted_file_ids": ["1374132b-b5db-4ec3-9cf1-f61f982e89b0"],
  "deleted_count": 1,
  "storage_objects_deleted": 1,
  "jobs_cleaned": 0
}
```

Returns `404` if any `file_id` is not found or not owned by the authenticated user.


# Download batch results
Source: https://docs.tensorstudio.ai/api-reference/endpoint/get-download-batch-batch-id

GET /download/batch/{batch_id}
Stream all results for a batch as a downloadable NDJSON or JSON file.

Downloads all results for a batch as a stream.

## Query params

* `status` (optional)
* `format` (default: `ndjson`): `ndjson` or `json`
* `raw` (default: `false`): return raw per-job records instead of merged per-file results
* `chunks` (optional bool): include chunk payloads in merged output
* `speech_information` (optional bool): include segment-level speech information in merged output

## Example (NDJSON export)

```bash theme={"system"}
curl -L -o "batch_results.ndjson" \
  "https://api.soket.ai/transcribe/download/batch/$BATCH_ID?format=ndjson" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

## Example (JSON export with filter)

```bash theme={"system"}
curl -L -o "batch_completed.json" \
  "https://api.soket.ai/transcribe/download/batch/$BATCH_ID?status=completed&format=json" \
  -H "Authorization: Bearer $JWT_TOKEN"
```


# List files
Source: https://docs.tensorstudio.ai/api-reference/endpoint/get-files

GET /files
List all files uploaded by the authenticated user, flattened across upload batches. Returns newest files first.

List all files uploaded by the authenticated user, flattened across upload batches. Returns newest files first.

## Query params

* `page` (default: `1`)
* `limit` (default: `50`, max: `200`)
* `upload_status` (optional): `pending`, `uploading`, `completed`, `failed`

## Example

```bash theme={"system"}
curl -X GET "https://api.soket.ai/files?page=1&limit=50&upload_status=completed" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

## Typical response

```json theme={"system"}
{
  "page": 1,
  "limit": 50,
  "total_pages": 1,
  "total_files": 2,
  "count": 2,
  "files": [
    {
      "batch_upload_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
      "file_id": "1374132b-b5db-4ec3-9cf1-f61f982e89b0",
      "filename": "audio1.wav",
      "content_type": "audio/wav",
      "size_bytes": 1048576,
      "estimated_duration_seconds": 65.5,
      "upload_status": "completed",
      "gcs_key": "uploads/2026/06/03/1374132b-b5db-4ec3-9cf1-f61f982e89b0.wav"
    }
  ]
}
```

`gcs_key` is only included when `upload_status` is `completed`.


# Get file upload status
Source: https://docs.tensorstudio.ai/api-reference/endpoint/get-files-upload-batch-upload-id

GET /files/upload/{batch_upload_id}
Poll upload progress for a batch_upload_id returned by POST /files/upload. Returns per-file upload_status and gcs_key for completed files.

Poll upload progress for a `batch_upload_id` returned by `POST /files/upload`. Returns per-file `upload_status` and `gcs_key` for completed files.

## Upload statuses

* `pending` — queued for background upload
* `uploading` — upload in progress
* `completed` — ready for batch submission
* `failed` — upload failed

## Example

```bash theme={"system"}
curl -X GET "https://api.soket.ai/files/upload/$BATCH_UPLOAD_ID" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

## Typical response

```json theme={"system"}
{
  "batch_upload_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "status": "completed",
  "total_files": 2,
  "completed": 2,
  "failed": 0,
  "uploading": 0,
  "pending": 0,
  "files": [
    {
      "file_id": "1374132b-b5db-4ec3-9cf1-f61f982e89b0",
      "filename": "audio1.wav",
      "size_bytes": 1048576,
      "estimated_duration_seconds": 65.5,
      "upload_status": "completed",
      "gcs_key": "uploads/2026/06/03/1374132b-b5db-4ec3-9cf1-f61f982e89b0.wav"
    }
  ]
}
```


# Get batch results
Source: https://docs.tensorstudio.ai/api-reference/endpoint/get-results-batch-batch-id

GET /results/batch/{batch_id}
Paginated batch results grouped by file. By default returns merged per-file transcription results.

Paginated batch results endpoint.

## Query params

* `status` (optional): `completed`, `failed`, `retrying`, `processing`
* `page` (default: `1`)
* `limit` (default: `50`, max: `200`)
* `raw` (default: `false`)
  * `false`: merged per-file results
  * `true`: raw per-job records
* `chunks` (optional bool): include chunk-level details inline
* `speech_information` (optional bool): include segment-level speech information (tone, accent, speaker\_id, timestamps, etc.)

## Example (merged, completed only)

```bash theme={"system"}
curl -X GET "https://api.soket.ai/transcribe/results/batch/$BATCH_ID?status=completed&page=1&limit=100" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

## Example (raw job records)

```bash theme={"system"}
curl -X GET "https://api.soket.ai/transcribe/results/batch/$BATCH_ID?raw=true&page=1&limit=100" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

## Typical response shapes

* `raw=false`: `{ batch_id, page, limit, total_pages, total_files, count, files: [...] }`
* `raw=true`: `{ batch_id, page, limit, total_pages, total_jobs, count, jobs: [...] }`
* Missing batch with no status filter: `{ "error": "batch not found" }`


# Get file results
Source: https://docs.tensorstudio.ai/api-reference/endpoint/get-results-file-file-id

GET /results/file/{file_id}
Returns merged transcript and status for one file.

Returns merged transcript and status for one file.

## Query params

* `chunks` (optional bool): include per-chunk details
* `speech_information` (optional bool): include segment-level speech information (tone, accent, speaker\_id, timestamps, etc.)

## Example

```bash theme={"system"}
curl -X GET "https://api.soket.ai/transcribe/results/file/$FILE_ID?chunks=true" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

## Typical response shape

* Found file: `{ file_id, filename, status, result?, errors?, chunk_results? }`
* Not found: `{ "error": "file not found" }`

In development mode, additional counters may be included:
`total_chunks`, `completed_chunks`, and `failed_chunks`.


# Get batch status
Source: https://docs.tensorstudio.ai/api-reference/endpoint/get-status-batch-batch-id

GET /status/batch/{batch_id}
Returns aggregate progress across files and chunk jobs in the batch.

Returns aggregate progress for all files and chunk jobs in a batch.

## Typical response (production)

```json theme={"system"}
{
  "batch_id": "0f8fad5b-d9cb-469f-a165-70867728950e",
  "total_files": 4,
  "files_completed": 2,
  "files_failed": 0,
  "estimated_audio_seconds": 290.0,
  "estimated_completion_seconds": 21.7,
  "status": "processing"
}
```

In development mode, additional counters can be present:
`total_jobs`, `completed_jobs`, `failed_jobs`, `processing_jobs`, `queued_jobs`, and `files_processing`.

## Example

```bash theme={"system"}
curl -X GET "https://api.soket.ai/transcribe/status/batch/$BATCH_ID" \
  -H "Authorization: Bearer $JWT_TOKEN"
```


# Submit batch transcription
Source: https://docs.tensorstudio.ai/api-reference/endpoint/post-batch

POST /batch
Submit a batch for transcription using a completed batch_upload_id, specific file_ids from prior uploads, and/or an explicit sources list with gcs_key or url per item.

Submit a batch of audio for transcription. Accepts a JSON body with:

* `batch_upload_id` (optional): include all completed files from one upload request
* `file_ids` (optional): list of specific `file_id` strings from prior `POST /files/upload` responses
* `sources` (optional): explicit list of GCS keys or URLs

At least one of `batch_upload_id`, `file_ids`, or `sources` must resolve to at least one file. All fields can be combined in a single request.

Each source item must include either `gcs_key` or `url`, and may include an optional `filename`.

Uploaded files referenced by `batch_upload_id` or `file_ids` must have `upload_status: "completed"`. If uploads are still in progress, the API returns `409`.

When a file is selected via `file_ids`, the same `file_id` from the upload response is reused for transcription results.

## Example (from completed upload)

```bash theme={"system"}
curl -X POST "https://api.soket.ai/transcribe/batch" \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "batch_upload_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
  }'
```

## Example (specific file\_ids)

```bash theme={"system"}
curl -X POST "https://api.soket.ai/transcribe/batch" \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "file_ids": [
      "1374132b-b5db-4ec3-9cf1-f61f982e89b0",
      "2485243c-c6ec-5fd4-0dg2-g72g093f90c1"
    ]
  }'
```

## Example (upload batch + specific files + URLs)

```bash theme={"system"}
curl -X POST "https://api.soket.ai/transcribe/batch" \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "batch_upload_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "file_ids": ["3596354d-d7fd-6ge5-1eh3-h83h104g01d2"],
    "sources": [
      {"url": "https://example.com/audio3.wav", "filename": "audio3.wav"}
    ]
  }'
```

## Typical response

```json theme={"system"}
{
  "batch_id": "0f8fad5b-d9cb-469f-a165-70867728950e",
  "total_jobs": 3,
  "estimated_audio_seconds": 290.0,
  "estimated_completion_seconds": 21.7,
  "jobs": [
    {
      "file_id": "1374132b-b5db-4ec3-9cf1-f61f982e89b0",
      "filename": "audio1.wav",
      "batch_id": "0f8fad5b-d9cb-469f-a165-70867728950e",
      "status": "queued"
    }
  ]
}
```


# Upload files
Source: https://docs.tensorstudio.ai/api-reference/endpoint/post-files-upload

POST /files/upload
Accept multipart files, spool them to disk, start background storage uploads, and return immediately with a batch_upload_id. Poll GET /files/upload/{batch_upload_id} until uploads complete before submitting a batch.

Accept one or more audio files as multipart form data. Files are spooled to disk immediately and uploaded to storage in the background. The response returns `202 Accepted` with a `batch_upload_id` you can poll until uploads finish.

## Example

```bash theme={"system"}
curl -X POST "https://api.soket.ai/files/upload" \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -F "files=@/absolute/path/audio1.wav" \
  -F "files=@/absolute/path/audio2.mp3"
```

## Typical response

```json theme={"system"}
{
  "batch_upload_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "status": "accepted",
  "total_files": 2,
  "spool_seconds": 0.42,
  "files": [
    {
      "file_id": "1374132b-b5db-4ec3-9cf1-f61f982e89b0",
      "filename": "audio1.wav",
      "size_bytes": 1048576,
      "estimated_duration_seconds": 65.5,
      "upload_status": "pending"
    }
  ]
}
```

Poll `GET /files/upload/{batch_upload_id}` until all files reach `upload_status: "completed"` before calling `POST /batch`.


# Dhrith ASR API
Source: https://docs.tensorstudio.ai/api-reference/introduction

Upload audio, submit batches, monitor transcription progress, and fetch merged results from the Dhrith ASR API server.

## Welcome

The Dhrith ASR API is a batch transcription service for Indian multilingual speech. Upload audio files or provide URLs, submit batches for transcription, track processing progress, and retrieve merged transcripts at scale.

Use this API to:

* Upload files asynchronously and poll upload status.
* Submit batches from completed uploads and/or explicit GCS keys or URLs.
* Track progress at batch level and fetch merged results per file.

If you are new to the TensorStudio docs flow, start from the [API Keys](https://docs.tensorstudio.ai/apis).

## 1) Dhrith ASR Overview

### Processing flow

1. Client sends `POST /files/upload` with multipart files.
2. Client polls `GET /files/upload/{batch_upload_id}` until uploads complete.
3. Client sends `POST /batch` with `batch_upload_id`, `file_ids`, and/or a `sources` list.
4. Client polls status endpoints or fetches paginated/downloadable results.

### Authentication

Application endpoints require JWT and expect:

`Authorization: Bearer <JWT_TOKEN>`

The token is validated against JWKS. Missing or invalid tokens return `401`.

### IDs you will use

* `batch_upload_id`: groups files from one upload request.
* `batch_id`: groups everything from one transcription submission.
* `file_id`: stable ID for one source file/URL and its merged result.

## 2) Features and Metrics

### Core features

* **Async file upload**: spool-to-disk with background storage upload.
* **Batch input support**: completed uploads, specific uploaded files by `file_id`, explicit GCS keys, and remote audio URLs in one batch request.
* **File management**: list and delete uploaded files.

### Operational metrics exposed by API responses

* Upload metrics:
  * `spool_seconds`, `upload_status` per file
  * `completed`, `failed`, `uploading`, `pending` counters
* Submission estimates:
  * `estimated_audio_seconds`
  * `estimated_completion_seconds`
* Batch progress counters:
  * `total_files`, `files_completed`, `files_failed`, `files_processing`
  * In development mode: `total_jobs`, `completed_jobs`, `failed_jobs`, `processing_jobs`, `queued_jobs`

### Default pagination behavior

* `GET /files` and results endpoints default:
  * `page=1`
  * `limit=50`
  * max `limit=200`

## 3) Endpoints and Usage

File endpoints use base URL `https://api.soket.ai`. Batch transcription, status, results, and download endpoints use `https://api.soket.ai/transcribe`.

## `POST /files/upload`

Upload one or more audio files. Returns `202 Accepted` immediately with a `batch_upload_id`. Background storage upload continues after the response.

### Example

```bash theme={"system"}
curl -X POST "https://api.soket.ai/files/upload" \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -F "files=@/absolute/path/audio1.wav" \
  -F "files=@/absolute/path/audio2.mp3"
```

### Typical response

```json theme={"system"}
{
  "batch_upload_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
  "status": "accepted",
  "total_files": 2,
  "spool_seconds": 0.42,
  "files": [
    {
      "file_id": "1374132b-b5db-4ec3-9cf1-f61f982e89b0",
      "filename": "audio1.wav",
      "size_bytes": 1048576,
      "estimated_duration_seconds": 65.5,
      "upload_status": "pending"
    }
  ]
}
```

## `GET /files/upload/{batch_upload_id}`

Poll upload progress. Returns per-file `upload_status` and `gcs_key` for completed files.

### Example

```bash theme={"system"}
curl -X GET "https://api.soket.ai/files/upload/$BATCH_UPLOAD_ID" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

## `GET /files`

List all files uploaded by the authenticated user.

### Query params

* `page` (default: `1`)
* `limit` (default: `50`, max: `200`)
* `upload_status` (optional): `pending`, `uploading`, `completed`, `failed`

### Example

```bash theme={"system"}
curl -X GET "https://api.soket.ai/files?upload_status=completed" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

## `DELETE /files`

Delete one or more uploaded files by `file_id`.

### Example

```bash theme={"system"}
curl -X DELETE "https://api.soket.ai/files" \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"file_ids": ["1374132b-b5db-4ec3-9cf1-f61f982e89b0"]}'
```

## `POST /batch`

Submit a batch for transcription with a JSON body:

* `batch_upload_id` (optional): include all completed files from one upload
* `file_ids` (optional): list of specific `file_id` strings from prior uploads
* `sources` (optional): list of `{ gcs_key, filename }` or `{ url, filename }` items

At least one of `batch_upload_id`, `file_ids`, or `sources` must resolve to at least one file. All fields can be combined.

### Example (from completed upload)

```bash theme={"system"}
curl -X POST "https://api.soket.ai/transcribe/batch" \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"batch_upload_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"}'
```

### Example (specific file\_ids)

```bash theme={"system"}
curl -X POST "https://api.soket.ai/transcribe/batch" \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"file_ids": ["1374132b-b5db-4ec3-9cf1-f61f982e89b0", "2485243c-c6ec-5fd4-0dg2-g72g093f90c1"]}'
```

### Example (upload batch + URLs)

```bash theme={"system"}
curl -X POST "https://api.soket.ai/transcribe/batch" \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "batch_upload_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "sources": [
      {"url": "https://example.com/audio3.wav", "filename": "audio3.wav"}
    ]
  }'
```

### Typical response

```json theme={"system"}
{
  "batch_id": "0f8fad5b-d9cb-469f-a165-70867728950e",
  "total_jobs": 3,
  "estimated_audio_seconds": 290.0,
  "estimated_completion_seconds": 21.7,
  "jobs": [
    {
      "file_id": "1374132b-b5db-4ec3-9cf1-f61f982e89b0",
      "filename": "audio1.wav",
      "batch_id": "0f8fad5b-d9cb-469f-a165-70867728950e",
      "status": "queued"
    }
  ]
}
```

## `GET /status/batch/{batch_id}`

Returns aggregate progress across files and chunk jobs in the batch.

### Typical response (production)

```json theme={"system"}
{
  "batch_id": "0f8fad5b-d9cb-469f-a165-70867728950e",
  "total_files": 3,
  "files_completed": 2,
  "files_failed": 0,
  "estimated_audio_seconds": 290.0,
  "estimated_completion_seconds": 21.7,
  "status": "processing"
}
```

In development mode, this response also includes job counters:
`total_jobs`, `completed_jobs`, `failed_jobs`, `processing_jobs`, `queued_jobs`, and `files_processing`.

### Example

```bash theme={"system"}
curl -X GET "https://api.soket.ai/transcribe/status/batch/$BATCH_ID" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

## `GET /results/batch/{batch_id}`

Paginated results endpoint.

### Query params

* `status` (optional): `completed`, `failed`, `retrying`, `processing`
* `page` (default: `1`)
* `limit` (default: `50`, max: `200`)
* `raw` (default: `false`)
  * `false`: merged per-file results
  * `true`: raw per-job records
* `chunks` (optional bool): include chunk-level results inline when enabled
* `speech_information` (optional bool): include segment-level speech information (tone, accent, speaker\_id, timestamps, etc.)

### Example (merged, completed-only)

```bash theme={"system"}
curl -X GET "https://api.soket.ai/transcribe/results/batch/$BATCH_ID?status=completed&page=1&limit=100" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

### Example (raw job records)

```bash theme={"system"}
curl -X GET "https://api.soket.ai/transcribe/results/batch/$BATCH_ID?raw=true&page=1&limit=100" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

### Typical response shape

* `raw=false` (default):\
  `{ batch_id, page, limit, total_pages, total_files, count, files: [...] }`
* `raw=true`:\
  `{ batch_id, page, limit, total_pages, total_jobs, count, jobs: [...] }`
* If batch does not exist and no status filter is applied:\
  `{ "error": "batch not found" }`

## `GET /results/file/{file_id}`

Returns merged transcript and status for one file.

### Query params

* `chunks` (optional bool): include per-chunk details
* `speech_information` (optional bool): include segment-level speech information

### Example

```bash theme={"system"}
curl -X GET "https://api.soket.ai/transcribe/results/file/$FILE_ID?chunks=true" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

### Typical response shape

* Found file: `{ file_id, filename, status, result?, errors?, chunk_results? }`
* Not found: `{ "error": "file not found" }`

In development mode, additional counters such as `total_chunks`, `completed_chunks`, and `failed_chunks` are included.

## `GET /download/batch/{batch_id}`

Download all results for a batch as stream.

### Query params

* `status` (optional)
* `format` (default: `ndjson`): `ndjson` or `json`
* `raw` (default: `false`): raw per-job instead of merged per-file
* `chunks` (optional bool): include chunk payloads in merged output
* `speech_information` (optional bool): include segment-level speech information in merged output

### Example (NDJSON export)

```bash theme={"system"}
curl -L -o "batch_results.ndjson" \
  "https://api.soket.ai/transcribe/download/batch/$BATCH_ID?format=ndjson" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

### Example (JSON export with filter)

```bash theme={"system"}
curl -L -o "batch_completed.json" \
  "https://api.soket.ai/transcribe/download/batch/$BATCH_ID?status=completed&format=json" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

## 4) Examples

### Example A: Complete batch lifecycle

```bash theme={"system"}
# 1) Upload files
UPLOAD=$(curl -s -X POST "https://api.soket.ai/files/upload" \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -F "files=@/absolute/path/call_001.wav")

echo "$UPLOAD"
BATCH_UPLOAD_ID=$(echo "$UPLOAD" | python -c 'import sys,json; print(json.load(sys.stdin)["batch_upload_id"])')

# 2) Poll upload status until completed
curl -s -X GET "https://api.soket.ai/files/upload/$BATCH_UPLOAD_ID" \
  -H "Authorization: Bearer $JWT_TOKEN"

# 3) Submit batch
RESP=$(curl -s -X POST "https://api.soket.ai/transcribe/batch" \
  -H "Authorization: Bearer $JWT_TOKEN" \
  -H "Content-Type: application/json" \
  -d "{\"batch_upload_id\": \"$BATCH_UPLOAD_ID\"}")

echo "$RESP"
BATCH_ID=$(echo "$RESP" | python -c 'import sys,json; print(json.load(sys.stdin)["batch_id"])')
FILE_ID=$(echo "$RESP" | python -c 'import sys,json; print(json.load(sys.stdin)["jobs"][0]["file_id"])')

# 4) Poll batch status
curl -s -X GET "https://api.soket.ai/transcribe/status/batch/$BATCH_ID" \
  -H "Authorization: Bearer $JWT_TOKEN"

# 5) Fetch first page of merged file results
curl -s -X GET "https://api.soket.ai/transcribe/results/batch/$BATCH_ID?page=1&limit=50" \
  -H "Authorization: Bearer $JWT_TOKEN"

# 6) Fetch one file result directly
curl -s -X GET "https://api.soket.ai/transcribe/results/file/$FILE_ID" \
  -H "Authorization: Bearer $JWT_TOKEN"

# 7) Download full export
curl -L -o "batch_${BATCH_ID}.ndjson" \
  "https://api.soket.ai/transcribe/download/batch/$BATCH_ID?format=ndjson" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

### Example B: Get one file's merged transcript

```bash theme={"system"}
curl -s -X GET "https://api.soket.ai/transcribe/results/file/$FILE_ID" \
  -H "Authorization: Bearer $JWT_TOKEN"
```

## Error handling quick reference

* `400`: invalid input or batch constraints violated.
* `401`: missing/invalid JWT.
* `404`: upload batch, batch, file, or `file_ids` not found.
* `409`: uploads still in progress or `file_ids` not yet completed when submitting batch.
* `429`: quota exceeded (if quota checks are enabled).
* `503`: queue saturation or operational dependency unavailable.

## Notes for production docs

* Keep JWT examples as environment variable placeholders (`$JWT_TOKEN`), never hardcoded tokens.
* Recommend NDJSON for large exports due to streaming-friendliness.
* For UI polling, use `GET /status/batch/{batch_id}` as the lightweight progress endpoint.
* Poll `GET /files/upload/{batch_upload_id}` before calling `POST /batch`.

## 5) Architecture and Queue Workflow

When a batch is submitted, the system runs a two-stage queue pipeline:

1. Parent file jobs (one per input file/URL) on the file queue.
2. Child chunk jobs (fan-out after chunking) on the chunk queue.

```mermaid theme={"system"}
flowchart TD
    A["Client\nPOST /files/upload"] --> B[API Server]
    B --> C[JWT verification]
    C --> D["Spool files to disk\nbackground storage upload"]
    D --> E["Client polls\nGET /files/upload/{id}"]

    E --> F["Client\nPOST /batch"]
    F --> G["Validate limits\nbatch size/jobs/queue depth"]
    G --> H[Enqueue parent job\nasr_file_queue]
    H --> I[File Worker\nWORKER_MODE=file]

    I --> J["process_audio_job -> process_gcs_file_job"]
    J --> K[Load audio from storage]
    K --> L["Chunking (VAD/split_audio_into_chunks)"]
    L --> M[Upload each chunk to storage]
    M --> N[Enqueue chunk jobs\nasr_job_queue]

    N --> O[Chunk Worker\nWORKER_MODE=chunk]
    O --> P[process_chunk_job]

    P --> Q{"INFERENCE_BATCH_SIZE > 1?"}
    Q -->|Yes| R[Batch gate\ncollect chunk refs]
    R --> S["Gate leader pulls N chunks,\ndownloads in parallel,\nruns parallel ASR inference"]
    Q -->|No| T["Direct chunk ASR inference"]

    S --> U[Chunk result]
    T --> U
    U --> V["Persist chunk status/result"]
    V --> W["Update completion counters\nchunks_done/chunks_failed"]
    W --> X{"All chunks done?"}
    X -->|No| N
    X -->|Yes| Y[Finalize parent job\nmerge chunk transcriptions]
    Y --> Z["Persist parent result/status"]

    Z --> AA["Client polling endpoints\n/status/batch,\n/results, /download"]
```

### Queue map

* `asr_file_queue`: primary queue for parent jobs submitted by `POST /batch`.
* `asr_job_queue`: primary queue for chunk jobs created during parent fan-out.
* `file_retry_queue` (optional): dedicated retry lane for parent jobs. Only active when `SEPARATE_RETRY_QUEUES=true`.
* `job_retry_queue` (optional): dedicated retry lane for chunk jobs. Only active when `SEPARATE_RETRY_QUEUES=true`.

### Retry modes

The system supports two retry modes, controlled by the `SEPARATE_RETRY_QUEUES` environment variable:

**Default (`SEPARATE_RETRY_QUEUES=false`)**: failed jobs are automatically re-enqueued on the same primary queue after configured intervals. Simple and reliable.

**Separate retry queues (`SEPARATE_RETRY_QUEUES=true`)**: Failed jobs are manually re-enqueued onto dedicated retry queues (`file_retry_queue`, `job_retry_queue`) by custom failure callbacks. Workers listen on `[primary, retry]` in that order, so fresh jobs always take priority over retries. Use this when you need retry jobs to have lower scheduling priority than new work.

Related env vars for separate mode:

* `CHUNK_MAX_RETRIES_SEPARATE` (default `3`): max retry attempts before permanent failure.
* `CHUNK_RETRY_INTERVALS_SEPARATE` (default `30,60,120`): delay in seconds between retry attempts.
* `SAVE_AUDIO_DATA` (default `true`): when `true`, raw audio bytes and metadata are saved for debugging and replay; set to `false` to skip audio-data persistence.

### Major processes

* **Upload ingress**: auth, size limits, spool-to-disk, background storage upload.
* **Batch ingress and validation**: auth, limits, back-pressure checks, and job enqueue.
* **Parent stage**: normalize source input, load source audio, chunk, and enqueue child jobs.
* **Chunk stage**: inference through direct ASR calls or batch gate coordination.
* **Persistence and counters**: update job stores and completion counters.
* **Finalize and expose**: merge chunk transcriptions into parent result, then serve via status/results/download endpoints.

### Failure and retry scenarios

* **Transient network or inference errors**:
  * chunk/parent status is marked `retrying`
  * default mode: automatic retries on the same queue with configured intervals
  * separate mode: failure callback re-enqueues onto the retry queue with a delay
  * work remains isolated to failed jobs/chunks
* **Chunk permanently fails after retries**:
  * chunk marked `failed`
  * `_on_chunk_terminal` still advances completion accounting
  * parent finalizes as `partial` if at least one chunk failed
* **Parent fails mid-fanout after some chunks enqueued**:
  * parent marked `failed` to avoid duplicate enqueue on retry
  * already-enqueued child chunks continue to terminal state
* **Worker crash or stale in-flight state**:
  * startup recovery requeues stale in-flight jobs
  * stuck chunk recovery marks orphaned processing chunks as failed
* **Queue pressure**:
  * API may reject new batch submissions with `503` when queue depth limits are exceeded


# API keys
Source: https://docs.tensorstudio.ai/apis

Easily create and manage your API keys for seamless integration.

## Introduction

The Soket's Realtime Speech API, built on OpenAI's protocol, enables implementation of conversational AI capabilities through [WebSockets](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API), a widely supported protocol optimized for real-time server-to-server data transfer. This guide demonstrates how to establish WebSocket connections and interact with Realtime models. WebRTC support for browser and mobile clients will be available in an upcoming release.

## Access Your API Key

1. Log in to your TensorStudio account.
2. Navigate to the [API Keys](https://app.tensorstudio.ai/studio/api-keys) section under manage.
3. Generate a new API key for your project.

<Info>
  **Note:** Keep your API key secure. Avoid sharing it publicly.
</Info>

<Frame>
  <img alt="Hero Light" />
</Frame>

## Explore the Realtime Speech API

* Connect using [Websocket API](/concepts/quickstart_websocket)
* Explore [SDKs](/sdk/overview) to connect to Speech API in your application

## Need Help?

* **Community Support:** Join the TensorStudio's [Discord Community](https://discord.gg/daRQX4yF) for tips and discussions.
* **Customer Support:** Reach us at [support@tensorstudio.ai](mailto:support@tensorstudio.ai).


# WebSocket API
Source: https://docs.tensorstudio.ai/concepts/quickstart_websocket

Get started with Soket AI Realtime Speech API in minutes

## Introduction

The Soket's Realtime Speech API, built on OpenAI's protocol, enables implementation of conversational AI capabilities through [WebSockets](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API), a widely supported protocol optimized for real-time server-to-server data transfer. This guide demonstrates how to establish WebSocket connections and interact with Realtime models. WebRTC support for browser and mobile clients will be available in an upcoming release.

## Access Your API Key

1. Log in to your TensorStudio account.
2. Navigate to the [API Keys](https://app.tensorstudio.ai/studio/api-keys) section under manage.
3. Generate a new API key for your project.

<Info>
  **Note:** Keep your API key secure. Avoid sharing it publicly.
</Info>

<Frame>
  <img alt="Hero Light" />
</Frame>

## Explore the Realtime Speech API

The Realtime Speech API lets you create responsive conversational applications powered by expressive voice-enabled models. It handles both text and audio in real-time, detects when users are speaking, and can execute custom functions - all with minimal latency.

### Connection details

Connecting via WebSocket requires the following connection information:

|                      |                                                                                                                                                                                                                                 |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **URL**              | `wss://api.soket.ai/v1/realtime`                                                                                                                                                                                                |
| **Query Parameters** | `model` <br /><br /> Realtime model ID to connect to, like `pragna-realtime`                                                                                                                                                    |
| **Headers**          | `Authorization: Bearer YOUR_API_KEY` <br /><br /> Substitute `YOUR_API_KEY` with a standard API key on the server, or an ephemeral token on insecure clients (note that WebRTC `coming soon` is recommended for this use case). |

Below are several examples of using these connection details to initialize a WebSocket connection to the Realtime API.

<CodeGroup>
  ```javascript NodeJS theme={"system"}
  import WebSocket from "ws";
  import 'dotenv/config';

  const API_KEY = process.env.TENSOR_STUDIO_API_KEY;

  const url = "wss://api.soket.ai/v1/realtime?model=pragna-realtime";
  const ws = new WebSocket(url, {
    headers: {
      "Authorization": "Bearer " + process.env.TENSOR_STUDIO_API_KEY,
      "OpenAI-Beta": "realtime=v1",
    },
  });

  ws.on("open", function open() {
    console.log("Connected to server.");
  });

  ws.on("message", function incoming(message) {
    console.log(JSON.parse(message.data));
  });
  ```

  ```python Python theme={"system"}
  # example requires websocket-client library:
  # pip install websocket-client

  import os
  import json
  import websocket

  TENSOR_STUDIO_API_KEY = os.environ.get("TENSOR_STUDIO_API_KEY")

  url = "wss://api.soket.ai/v1/realtime?model=pragna-realtime"
  headers = [
      "Authorization: Bearer " + TENSOR_STUDIO_API_KEY,
      "OpenAI-Beta: realtime=v1"
  ]

  def on_open(ws):
      print("Connected to server.")

  def on_message(ws, message):
      data = json.loads(message)
      print("Received event:", json.dumps(data, indent=2))

  ws = websocket.WebSocketApp(
      url,
      header=headers,
      on_open=on_open,
      on_message=on_message,
  )

  ws.run_forever()
  ```

  ```javascript JavaScript theme={"system"}
  const ws = new WebSocket(
    "wss://api.soket.ai/v1/realtime?model=pragna-realtime",
    [
      "realtime",
      // Auth
      "openai-insecure-api-key." + TENSOR_STUDIO_API_KEY
    ]
  );

  ws.on("open", function open() {
    console.log("Connected to server.");
  });

  ws.on("message", function incoming(message) {
    console.log(message.data);
  });
  ```
</CodeGroup>

## Sending and receiving events

Communication with Realtime models occurs through bidirectional message exchange over the WebSocket interface. The comprehensive documentation of available client and server messages can be found in the [OpenAI Realtime API reference](/docs/api-reference/realtime-client-events). Upon establishing a connection, the interface supports various event types including text messages, audio streams, tool calling, speech interruptions using [Voice Activity Detection (VAD)](/vad), and configuration modifications - enabling robust real-time interactions with the model.

Below, you'll find examples of how to send and receive events over the WebSocket interface in several programming environments.

<CodeGroup>
  ```javascript NodeJS / JavaScript theme={"system"}
  // Server-sent events will come in as messages...
  ws.on("message", function incoming(message) {
      // Message data payloads will need to be parsed from JSON:
      const serverEvent = JSON.parse(message.data)
      console.log(serverEvent);
  });

  // To send events, create a JSON-serializeable data structure that
  // matches a client-side event (see API reference)
  const event = {
      type: "response.create",
      response: {
          modalities: ["audio", "text"],
          instructions: "Give me a haiku about code.",
      }
  };
  ws.send(JSON.stringify(event));
  ```

  ```python Python theme={"system"}
  # To send a client event, serialize a dictionary to JSON
  # of the proper event type
  def on_open(ws):
      print("Connected to server.")
      
      event = {
          "type": "response.create",
          "response": {
              "modalities": ["text"],
              "instructions": "Please assist the user."
          }
      }
      ws.send(json.dumps(event))

  # Receiving messages will require parsing message payloads
  # from JSON
  def on_message(ws, message):
      data = json.loads(message)
      print("Received event:", json.dumps(data, indent=2))
  ```
</CodeGroup>

## Need Help?

* **Community Support:** Join the TensorStudio's [Discord Community](https://discord.gg/daRQX4yF) for tips and discussions.
* **Customer Support:** Reach us at [support@tensorstudio.ai](mailto:support@tensorstudio.ai).


# Realtime Console
Source: https://docs.tensorstudio.ai/concepts/realtime_console

Deploy interactive console to integrate with your web app

### 1. Install Realtime Console

First, install the OpenAI Realtime Console globally:

```bash theme={"system"}
git clone https://github.com/soketlabs/realtime-console.git
cd realtime-console
npm install
```

### 2. Set environment variables for realtime-console

Create a `.env` file in realtime-console directory

```bash theme={"system"}
OPENAI_API_KEY=<your-api-key>
REACT_APP_LOCAL_RELAY_SERVER_URL=http://localhost:8081
```

### 3. Start realtime-console

```bash theme={"system"}
cd realtime-console
npm run start
```

This will start realtime-console in `port:3005` by default. \
Change port:  `PORT=<your port>` in `package.json{}` file in realtime-console directory.

<Frame>
  <img alt="Hero Light" />
</Frame>

### 5. Connect and test realtime-console

Select `vad` mode and click `connect` to connect realtime-console to tensorstudio.\
You should be able to see event logs without any errors if connection is established properly after authentication.\
Note: disconnect after testing to avoid unnecessary consumption.

<Frame>
  <img alt="Hero Light" />
</Frame>


# Tool Calling
Source: https://docs.tensorstudio.ai/concepts/tool_calling

This document explains the Tool Calling feature in our Realtime Speech API.

## Overview

Tool Calling enables AI models to execute predefined functions during conversations, allowing them to perform actions like retrieving data, registering complaints, or manipulating application state. This feature bridges the gap between conversational AI and practical functionality.

## Adding Tools

Tools are added to the Realtime client using the `addTool()` method. Each tool requires two components:

1. Tool Definition - Describes the tool's purpose and parameters
2. Tool Implementation - The actual function that executes when the tool is called

### Basic Tool Structure

```typescript theme={"system"}
client.addTool(
  {
    name: 'tool_name',
    description: 'Description of what the tool does',
    parameters: {
      type: 'object',
      properties: {
        // Parameter definitions
      },
      required: ['param1', 'param2'] // List required parameters
    }
  },
  async (params) => {
    // Tool implementation
    return result;
  }
);
```

### Example: Weather Tool

Here's an example of a tool that retrieves weather information for given coordinates:

```typescript theme={"system"}
client.addTool(
  {
    name: 'get_weather',
    description: 'Retrieves the weather for a given lat, lng coordinate pair',
    parameters: {
      type: 'object',
      properties: {
        lat: {
          type: 'number',
          description: 'Latitude'
        },
        lng: {
          type: 'number',
          description: 'Longitude'
        },
        location: {
          type: 'string',
          description: 'Name of the location'
        }
      },
      required: ['lat', 'lng', 'location']
    }
  },
  async ({ lat, lng, location }) => {
    // Implementation
    const result = await fetch(
      `https://api.open-meteo.com/v1/forecast?latitude=${lat}&longitude=${lng}&current=temperature_2m,wind_speed_10m`
    );
    const json = await result.json();
    return json;
  }
);
```

### Example: Complaint Registration

Here's an example of a tool for registering customer complaints:

```typescript theme={"system"}
client.addTool(
  {
    name: 'register_complaint',
    description: 'Register a customer complaint',
    parameters: {
      type: 'object',
      properties: {
        reason: {
          type: 'string',
          description: 'The reason for the complaint'
        },
        issue_tag: {
          type: 'string',
          description: 'The tag for the issue',
          enum: [
            'router_issue',
            'internet_issue',
            'network_slow',
            'bill_not_paid',
            'network_down',
            'plan_change_in_progress'
          ]
        }
      },
      required: ['reason', 'issue_tag']
    }
  },
  async ({ reason, issue_tag }) => {
    // Implementation
    const complaint_number = generateComplaintNumber();
    return { complaint_number };
  }
);
```

## Best Practices

1. **Clear Descriptions**: Provide detailed descriptions for tools and parameters to help the AI understand when and how to use them.

2. **Parameter Validation**: Use the `required` field and parameter types to ensure the AI provides necessary information.

3. **Error Handling**: Implement proper error handling in tool implementations:

```typescript theme={"system"}
async (params) => {
  try {
    // Tool implementation
    return result;
  } catch (error) {
    return {
      error: true,
      message: 'Failed to execute tool: ' + error.message
    };
  }
}
```

4. **State Management**: When tools modify application state, ensure changes are properly reflected in your UI:

```typescript theme={"system"}
client.addTool(
  {
    name: 'update_state',
    description: 'Updates application state',
    parameters: {
      type: 'object',
      properties: {
        key: { type: 'string' },
        value: { type: 'string' }
      },
      required: ['key', 'value']
    }
  },
  async ({ key, value }) => {
    setState(prevState => ({
      ...prevState,
      [key]: value
    }));
    return { success: true };
  }
);
```

## Tool Types

Tools can be categorized based on their functionality:

1. **Data Retrieval Tools**: Fetch external data (like weather information)
2. **State Management Tools**: Modify application state
3. **Action Tools**: Perform specific actions (like registering complaints)
4. **Integration Tools**: Interface with external systems or APIs

## Security Considerations

1. **Input Validation**: Always validate tool parameters before processing
2. **API Key Protection**: Never expose sensitive credentials in tool implementations
3. **Rate Limiting**: Implement rate limiting for tools that access external services
4. **Error Boundaries**: Implement proper error handling to prevent crashes

## Debugging Tools

To debug tool calls, you can monitor events in the conversation:

```typescript theme={"system"}
client.on('realtime.event', (event) => {
  if (event.event.type === 'function_call') {
    console.log('Tool called:', event.event);
  }
});
```

This will help you track when and how tools are being used during conversations.

*Note: This code uses events directly; you can add your tool calls in the session.update event, following the documented format here: [OpenAI Realtime Client Events](https://platform.openai.com/docs/api-reference/realtime-client-events/session).*


# Understanding VAD
Source: https://docs.tensorstudio.ai/concepts/vad

This document explains the Voice Activity Detection (VAD) feature in our Realtime Speech API.

## Overview

Voice Activity Detection (VAD) is a crucial component in speech processing systems. It is a signal processing technique used to detect the presence or absence of human speech in an audio signal. By differentiating between speech and non-speech segments, VAD optimizes the performance of real-time speech applications, including Automatic Speech Recognition (ASR), Voice over IP (VoIP), and conversational AI.

## Key Parameters in VAD

The following parameters are available for configuring VAD in your Real-Time Speech API:

### Turn Detection

* **Configuration**: Turn detection is an optional feature that can be enabled or disabled. Setting it to `null` disables turn detection.
* **Server VAD**: This is the currently supported type of turn detection. It detects the start and end of speech based on audio volume and responds at the end of the user's speech.

### Parameters for Turn Detection

| **Parameter**         | **Description**                                                                                    | **Default Value** | **Range**    |
| --------------------- | -------------------------------------------------------------------------------------------------- | ----------------- | ------------ |
| `type`                | Type of turn detection. Currently, only `server_vad` is supported.                                 | `server_vad`      | N/A          |
| `threshold`           | Activation threshold for VAD (0.0 to 1.0). Higher values require louder audio to activate.         | 0.5               | 0.0 - 1.0    |
| `prefix_padding_ms`   | Amount of audio to include before the VAD-detected speech (in milliseconds).                       | 300ms             | N/A          |
| `silence_duration_ms` | Duration of silence to detect speech stop (in milliseconds). Shorter values improve response time. | 500ms             | N/A          |
| `create_response`     | Whether to automatically generate a response when VAD is enabled.                                  | `true`            | `true/false` |

### Explanation of Parameters

* **`threshold`**: Adjust this value based on the noise level of the environment. A higher threshold is ideal for noisy settings, ensuring the model activates only for significant audio inputs.
* **`prefix_padding_ms`**: Useful for capturing audio context before detected speech starts, providing smoother interactions.
* **`silence_duration_ms`**: This parameter controls the responsiveness of the system. Lower values result in faster responses but may cut off speech during short pauses.
* **`create_response`**: Enabling this ensures the system generates a response as soon as speech is detected and stops, streamlining interaction workflows.

By fine-tuning these parameters, users can optimize VAD performance for various real-time applications, ensuring precise and efficient voice interactions.


# Voices
Source: https://docs.tensorstudio.ai/concepts/voice

Voices available in the Realtime Speech API

## Available Voices

Our API provides a diverse set of voices across different languages, genders, and styles. Below is a comprehensive list of available voices:

| Voice ID  | Gender | Language | Description                          |
| --------- | ------ | -------- | ------------------------------------ |
| `diya`    | Female | Hindi    | Expressive & Cheerful Hindi Narrator |
| `shreyas` | Male   | Hindi    | Energetic Hindi Voice                |
| `vardhan` | Male   | Hindi    | Authoritative and Deep Hindi Voice   |
| `sahil`   | Male   | Hindi    | Energetic Hindi Voice                |
| `ananya`  | Female | Tamil    | Conversational Tamil Voice           |
| `ramaa`   | Male   | Tamil    | Energetic Conversational Tamil       |
| `sakshi`  | Female | Tamil    | Calm & Conversational Voice          |
| `monica`  | Female | English  | Natural Conversations                |
| `naina`   | Female | English  | Sophisticated Indian Girl            |
| `malik`   | Male   | English  | Customer Support (Neutral Accent)    |
| `keshav`  | Male   | English  | Booklet Guy (Raw)                    |
| `abbas`   | Male   | English  | Highly Energetic Voice               |

We are adding new voices, for custom voices let us know.


# Relay Server
Source: https://docs.tensorstudio.ai/deploy/relay_server

Connect to Realtime Speech API through relay server

## Setting up a Relay Server

Connecting directly to tensorstudio through SDK on browser exposes your api-key. If you are running your own relay server, e.g. with the `Realtime Console`, you can instead connect to the relay server URL.

### 1. Install Realtime Console

First, install the OpenAI Realtime Console globally:

```bash theme={"system"}
git clone https://github.com/soketlabs/realtime-console.git
cd realtime-console
npm install
```

### 2. Set environment variables for realtime-console

Create a `.env` file in realtime-console directory

```bash theme={"system"}
OPENAI_API_KEY=<your-api-key>
```

### 3. Start the Relay Server

Run relay server in a new terminal:

```bash theme={"system"}
cd realtime-console
npm run relay
```

The relay server will start running on `ws://localhost:8081` .

## Start using Tensorstudio Speech API through NodeJS SDK

#### Install NodeJS SDK

```bash theme={"system"}
npm install https://github.com/soketlabs/openai-realtime-api-beta
```

#### Using Node.js SDK

```javascript theme={"system"}
import { RealtimeClient } from '@openai/realtime-api-beta';

const client = new RealtimeClient({
  url: 'your-relay-server-url'
});
```

## Production Deployment

For production:

1. Use secure WebSocket (WSS) with SSL/TLS
2. Update the relay server URL to your production domain
3. Add authentication if needed
4. Use environment variables for configuration

Example production configuration:

```javascript theme={"system"}
const client = new RealtimeClient({
  baseURL: 'wss://your-domain.com/relay',
  // Other configuration options
});
```

## Security Considerations

* Never expose your OpenAI API key in client-side code
* Use HTTPS/WSS in production
* Implement rate limiting
* Add authentication to your relay server
* Monitor server usage and implement proper error handling

## Common Issues

1. **Connection Refused**: Make sure the relay server is running
2. **SSL/TLS Errors**: Check certificate configuration for WSS
3. **Authentication Errors**: Verify API key in `.env` file


# Evaluation and Benchmarking
Source: https://docs.tensorstudio.ai/dhrith/evaluation-and-benchmarking

Benchmark methodology, metrics, and comparative evaluation results for Dhrith.

## Evaluation and Benchmarking

To assess Dhrith's performance, we developed a benchmark specifically tailored for India's multilingual, code-mixed speech patterns, reflecting real-world data across Hindi, English, and Hinglish blends. The benchmark includes diverse speech styles - spontaneous dialogue, emotional tone shifts, regional accents, and natural background noise - making it a rigorous testbed for emotion-aware ASR systems.

We evaluated nine leading ASR models on this benchmark, including open and commercial systems such as Gemini 2.5 Flash, Deepgram Nova 2, GPT-4o Mini Transcribe, Sarvam Sarika 2.5, Google Gemma-3n, ElevenLabs Scribe v1, and AI4Bharat Indic Whisper.

All models were tested on identical audio samples with consistent normalization and transcription post-processing. Metrics were computed using our in-house evaluation suite, designed to handle multilingual and emotion-tagged outputs.

| Model                   | WER (%) | CER (%) | NWER (%) | NCER (%) | SER (%) | DIS (%) | ET (%) | CM (%) |
| ----------------------- | ------- | ------- | -------- | -------- | ------- | ------- | ------ | ------ |
| Gemini 2.5 Flash        | 8.57    | 5.69    | 8.01     | 5.39     | 2.35    | 15.10   | 99.63  | 94.31  |
| Soket Dhrith            | 11.19   | 7.54    | 10.70    | 7.31     | 8.73    | 14.78   | 61.34  | 90.44  |
| Deepgram Nova 2         | 15.74   | 9.03    | 15.03    | 8.66     | 0.57    | 14.39   | 0.00   | 80.31  |
| GPT-4o Mini Transcribe  | 42.34   | 36.97   | 41.65    | 36.58    | 9.28    | 12.47   | 0.00   | 62.36  |
| Sarvam Sarika 2.5       | 56.84   | 49.61   | 58.67    | 51.76    | 8.71    | 10.44   | 0.00   | 37.13  |
| Google Gemma-3n-E4B     | 58.05   | 55.15   | 58.21    | 55.16    | 19.15   | 14.52   | 95.69  | 66.01  |
| Google Gemma-3n-E2B     | 59.47   | 56.83   | 59.82    | 57.31    | 19.87   | 14.38   | 83.44  | 66.59  |
| Elevenlabs Sribe v1     | 71.27   | 63.14   | 72.69    | 64.56    | 9.61    | 9.83    | 0.00   | 46.53  |
| Vaani Whisper Large     | 75.23   | 65.53   | 76.84    | 67.40    | 6.16    | 10.09   | 0.00   | 47.19  |
| AI4Bharat Indic Whisper | 80.86   | 68.79   | 82.03    | 69.86    | 8.00    | 8.97    | 0.00   | 39.09  |

## Benchmark Design

Unlike conventional ASR evaluations that focus only on literal accuracy, this benchmark captures the multidimensional nature of Indian speech. It measures both linguistic and expressive performance through the following metrics:

* **WER (Word Error Rate)**: Standard measure of substitution, insertion, and deletion errors at the word level.
* **CER (Character Error Rate)**: Fine-grained equivalent of WER at character level, capturing minor linguistic mismatches.
* **NWER / NCER (No-Noise WER/CER)**: Recomputed after filtering conversational fillers such as "uh-huh," "haan," "achha," "वैसे," etc. This reflects model accuracy on semantically meaningful content rather than natural speech hesitations.
* **SER (Semantic WER)**: A novel metric that calculates error rate of semantic similarity between ground-truth and predicted transcripts using LaBSE sentence embeddings, rewarding semantically equivalent but lexically different outputs.
* **DIS (Disfluency Density)**: Average frequency of verbal fillers (e.g., "oh," "acha," "तो फिर," "हां हां") per transcript, indicating the model's ability to detect and preserve human-like speech patterns.
* **ET (Expression Tagging Density)**: Measures how often the model identifies expressive or paralinguistic tags like `[laughing]`, `[shouting]`, or `[pause]` - essential for emotionally aware systems.
* **CM (Code-Mix Density)**: Evaluates how well the generated transcript mirrors the language-mixing pattern in the ground truth, ensuring linguistic fidelity in bilingual utterances.

<Note>
  Expression tags were removed before post-processing to compute WER/CER with and without noise.
</Note>

## Key Insights

* **Competitive Accuracy**: Dhrith achieves second-best WER and CER across the entire benchmark, surpassing all models except Gemini 2.5 Flash.
* **Multilingual Robustness**: Despite being trained primarily for Hindi-English, Dhrith maintains high NWER and NCER performance, indicating strong resilience to filler noise and dialectal variation.
* **Emotion and Expression Awareness**: With an Expression Tagging (ET) density of 61.34%, Dhrith is the only open Indian ASR model capable of consistently annotating emotional context - far outperforming all others except Gemini.
* **Code-Mix Fidelity**: Dhrith records a Code-Mix Density of 90.44%, demonstrating exceptional sensitivity to India's bilingual conversational flow - a crucial feature for real-world deployment in call centers, virtual assistants, and entertainment domains.
* **Balanced Performance**: While some systems trade linguistic precision for expressivity or vice versa, Dhrith achieves a strong balance between transcription accuracy, emotional depth, and naturalness, setting a new benchmark for Indian multilingual ASR.

All experiments were conducted on our Hindi-English code-mixed evaluation dataset, built from diverse real-world audio. We will soon release this benchmark on HuggingFace, along with evaluation scripts and reference annotations, to encourage transparent and reproducible comparisons across future ASR systems.


# Introduction
Source: https://docs.tensorstudio.ai/dhrith/introduction

Explore our Guides and API Reference to get the most out of TensorStudio by Soket AI Labs

<img alt="Hero Light" />

## Dhrith: Emotionally Intelligent ASR for India's Multilingual Voices

India has always been a land of languages. From one town to another, dialects, accents, and linguistic blends transform effortlessly - often merging Hindi and English in a uniquely expressive rhythm. Capturing this natural flow of multilingual conversation is not just a technical challenge; it is a cultural mission.

Traditional Automatic Speech Recognition (ASR) systems have long focused on what is spoken - the literal words. But communication in India carries far more - how something is said often holds the true meaning. Tone, pace, emotion, and intent all weave together to form the heartbeat of a conversation.

Dhrith, our next-generation ASR model, listens beyond words. It understands emotion, rhythm, and code-switched language - giving transcription not only linguistic precision but emotional depth. Dhrith transforms ordinary speech recognition into an emotionally aware experience, enabling applications that feel human, responsive, and deeply connected to India's multilingual reality.

By combining linguistic understanding with affective cues, Dhrith enriches conversations with context - bridging the emotional gap between humans and machines. This opens vast possibilities: from emotionally aware conversational AI and empathetic call analytics to the foundation of the next generation of Indic Text-to-Speech (TTS) systems.

## More About Dhrith

Dhrith is built and evaluated specifically for India's multilingual, code-mixed speech patterns across Hindi, English, and Hinglish usage. Beyond literal transcription quality, it is designed to preserve emotional and conversational context in ways that traditional ASR systems usually miss.

### What makes Dhrith different

* **Emotion-aware transcription**: captures expressive cues such as tone, pace, and intent.
* **Code-switch robustness**: handles natural Hindi-English switching and regional variations.
* **Context-rich outputs**: balances word-level accuracy with meaningful emotional annotations.

### Benchmarking approach

Our internal benchmark evaluates both linguistic and expressive quality using metrics like:

* `WER` and `CER` for transcription accuracy
* `NWER` and `NCER` for meaningful-content accuracy after filler filtering
* `SER` for semantic faithfulness
* `ET` for expression tagging
* `CM` for code-mix fidelity

### Real-world impact

* **Customer experience**: emotion-aware call analytics and better quality scoring
* **Conversational AI**: more responsive assistants with human-like understanding
* **Media and education**: better multilingual accessibility and transcription quality
* **Research and insights**: deeper analysis of spoken communication signals

### Read the full deep dive

* [Introducing Dhrith: Emotionally Intelligent ASR](https://soket.ai/blogs/dhrith)

## Examples

Below are all representative samples from the Dhrith benchmark article, with full transcriptions.

### Example 1

<Frame>
  <audio>
    <source type="audio/wav" />

    Your browser does not support the audio element.
  </audio>

  [▶ Play audio](/audio/dhrith-sample-1.wav)
</Frame>

| Model                         | Transcript                                                                                                          |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| **Soket Dhrith**              | **Finally verification successful हुआ, success rate 98.7% था। शायद server overload हो गया था। \[resigned tone]**    |
| Sarvam Sarika 2.5             | फाइनली वेरिफिकेशन सक्सेसफुल हुआ, सक्सेस रेट 98.7% था। शायद सर्वर ओवरलोड हो गया था।                                  |
| Deepgram Nova 2               | finally verification successful हुआ success rate ninety eight point seven percent था शायद server overload हो गया था |
| OpenAI GPT-4o-Mini Transcribe | आखिरकार वेरीफिकेशन सक्सेसफुल हुआ, सक्सेस रेट 98.7% था। शायद सर्वर ओवरलोड हो गया था।                                 |

### Example 2

<Frame>
  <audio>
    <source type="audio/wav" />

    Your browser does not support the audio element.
  </audio>

  [▶ Play audio](/audio/dhrith-sample-2.wav)
</Frame>

| Model                         | Transcript                                                                                                                                                                                                                                                                                                                                                     |
| ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Soket Dhrith**              | **भाई, तेरे को पता है तो मैं बात कर रहा था ना कि मेरे को गाड़ी खरीदना है तो मैं शोरूम पे गया था तो गाड़ी का price करीब 2 करोड़ पड़ रहा था। अब जो deal था वो बोल रहा है तुमको अभी ₹50 लाख pay करना है and बाकी जो amount रहेगा तुमको महीने का ₹70000 पड़ेगा and तुम देख सकते हो ऐसे बोल रहा था। तो मैंने सोचा कि अच्छा deal है तो ले ले लेता हूं। \[calm]**     |
| Sarvam Sarika 2.5             | भाई, तेरे को पता है तेरे से मैं बात कर रहा था ना कि मेरे को गाड़ी खरीदना है तो मैं शोरूम पे गया था। तो गाड़ी का प्राइस करीब दो करोड़ पड़ रहा था। अब जो डीलर था वो बोल रहा है तुमको अभी 50 lakhs rupees pay करना है एंड बाकी जो अमाउंट रहेगा तुमको महीने का 70000 रुपया पड़ेगा एंड तुम देख सकते हो ऐसे बोल रहा था तो मैंने सोचा कि अच्छा डील है तो ले लेता हूं। |
| Deepgram Nova 2               | भाई तेरे को पता है तेरे से बात कर रहा था ना की मेरे को गाड़ी खरीदना है तो मैं form पर गया था तो गाड़ी का price करीब दो करोड़ पद रहा था अब जो डीलर था वो बोल रहा है तुमको अभी fifty लाख पे करना है एंड बाकी जो amount रहेगा तुमको महीने का ₹seventy thousand पड़ेगा एंड तुम देख सकते हो ऐसे बोल रहा था तो मैंने अच्छा deal है तो ले ले                          |
| OpenAI GPT-4o-Mini Transcribe | भाई तुझे पता है, मैंने गाड़ी खरीदनी है तो मैं शोरूम पे गया था, तो गाड़ी का प्राइस करीब दो करोड़ पर रहा था, अब जो डीलर था वो बोल रहा है तुम्हें अभी 50,00,000 रुपये पे करना है, और बाकी जो अमौंड रहेगा तुम्हें माइने का 70,000 रुपये पड़ेगा, और तुम देख सकते हो ऐसे बोल रहा था, तो मैंने सोचा कि अच्छा डील है तो ले लेता हूं.                                   |

### Example 3

<Frame>
  <audio>
    <source type="audio/wav" />

    Your browser does not support the audio element.
  </audio>

  [▶ Play audio](/audio/dhrith-sample-3b.wav)
</Frame>

| Model                         | Transcript                                                                                                                                                                             |
| ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Soket Dhrith**              | **PAN card number AFXPK7190K upload करते वक्त file size में issue आ गया था। \[matter-of-fact] शायद compression ratio घट गया होगा। \[hesitates] 1.58:1 पे आ गया था। \[matter-of-fact]** |
| Sarvam Sarika 2.5             | पैन कार्ड नंबर ए एफ एक्स पी के 7190 के अपलोड करते वक्त फाइल साइज में इशू आ गया था। शायद कंप्रेशन रेशियो घट गया होगा। 1.58:1 पे आ गया था।                                               |
| Deepgram Nova 2               | पान card number afxpk seven thousand one hundred ninety के upload करते वक्त file size में issue ए गया था शायद compression ratio घट गया होगा one point five eight is to one पर आ गया था |
| OpenAI GPT-4o-Mini Transcribe | पैन कार्ड नंबर AFXPK7190K अपलोड करते वक्त फाइल साइज में इश्यू आ गया था. शायद कंप्रेशन रेशियो घट गया होगा, 1.58:1 पे आ गया था.                                                          |

### Example 4

<Frame>
  <audio>
    <source type="audio/wav" />

    Your browser does not support the audio element.
  </audio>

  [▶ Play audio](/audio/dhrith-sample-3.wav)
</Frame>

| Model                         | Transcript                                                                                                                             |
| ----------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
| **Soket Dhrith**              | **हम भी वो हैं जो कभी किसी के पीछे नहीं खड़े होते। \[matter-of-fact] जहां खड़े हो जाते हैं, लाइन वहीं से शुरू होती है। \[deliberate]** |
| Sarvam Sarika 2.5             | हम भी वो हैं जो कभी किसी के पीछे नहीं खड़े होते। जहां खड़े हो जाते हैं, लाइन वहीं से शुरू होती                                         |
| Deepgram Nova 2               | हम भी वो हैं जो कभी किसी के पीछे नहीं खड़े होते जहा खड़े हो जाते हैं line वहीं से शुरू होती है                                         |
| OpenAI GPT-4o-Mini Transcribe | हम भी वो हैं जो कभी किसी के पीछे नहीं खड़े होते। जहाँ खड़े हो जाते हैं, लाइन वहीं से शुरू होती है।                                     |

### Example 5

<Frame>
  <audio>
    <source type="audio/wav" />

    Your browser does not support the audio element.
  </audio>

  [▶ Play audio](/audio/dhrith-sample-6.wav)
</Frame>

| Model                         | Transcript                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| ----------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Soket Dhrith**              | **ओह हो! \[excited] Spell की शुरुआत wicket के साथ। Wonder boy Rachin Ravindra के wicket के साथ। Kuldeep Yadav! \[shouting] ये फल मुबारक! खुल गई किस्मत! खुल जा सिम सिम! \[excited] और ये बड़ा wicket चाहिए था भारत को। एक set batsman जो 200 बना चुका है tournament में। उसका जब wicket लिया, डंडे बिखेर दिए, furniture disturb कर दिया। \[dramatic] देखिए team एकदम से इकट्ठी हो गई और Kuldeep! \[shouting] वो जादूगर जादूगर जादूगर जाएगा किसी को समझ नहीं आएगा। \[shouting] गेंद googly थी, अंदर आई, pad पे लगी, दिशा बदली और फिर डंडे पे लगी। \[dramatic] और देखिए इसे कहते हैं अलग जगाना। \[emphasized] इसे कहते हैं गुरु team में विश्वास जगाना है। \[dramatic] इसे कहते हैं प्रज्जवलित मशाल जिसे देख के सब कहें हम जीत सकते हैं भाई। \[shouting] मैं हूं ना कहां Kuldeep ने। \[shouting]** |
| Sarvam Sarika 2.5             | ओ हो हो! स्पेल की शुरुआत विकेट के साथ वंडर बॉय रचिन रविंद्र के विकेट के साथ कुलदीप यादव ये फल मुबारक नहीं किस्मत, खुल जा सिम सिम और ये बड़ा विकेट चाहिए था भारत को। एक सेट बैट्समैन जो 200 बना चुका है इस टूर्नामेंट में, उसका जब विकेट लिया, डंडे बिखेर दिए, फर्नीचर डिस्टर्ब कर दिया, देखिए टीम एकदम से इकट्ठी हो गई है और कुलदीप वो जादूगर जादूगर जादू कर जाएगा किसी को समझ नहीं आएगा। गेंद गूगली थी ये अंदर आई पैड पे लगी दिशा बदली और फिर डंडे पे लगी और देखिए इसे कहते हैं अलख जगाना। इसे कहते हैं गुरु टीम में विश्वास जगाना है। इसे कहते हैं प्रज्वलित मशाल जिसे देख के सब कहें हम जीत सकते हैं भाई। मैं हूं ना कहा कुलदीप ने।                                                                                                                                                            |
| Deepgram Nova 2               | ओ हो हो spell की शुरुआत wicket के साथ wonder boy रचिन रविंद्र के wicket के साथ कुलदीप यादव यह फ़ल मुबारक खुल गई किस्मत खुल जा सिम सिम और यह बड़ा wicket चाहिए था भारत को एक set batsman जो दो सौ बना चुका है इस tournament में उसका जब wicket लिया डंडे बिखेर दिए furniture disturb कर दिया देखिए team एकदम से इकट्ठी हो गई है और कुलदीप वह जादूगर जादूगर जादूगर कर जाएगा किसी को समझ नहीं आएगा गेंद कूकली थी अंदर आई pack पर लगी दिशा बदली और फिर डंडे पर लगी और देखिए इसे कहते हैं अलग जगाना इसे कहते हैं गुरु team में विश्वास जगाना है इसे कहते हैं प्रज्वलित मशाल जिसे देखकर सब कहें हम जीत सकते हैं भई मैं हूं ना कहा कुलदीप ने                                                                                                                                                            |
| OpenAI GPT-4o-Mini Transcribe | ओहोहो! स्पेल की शुरुआत विकेट के साथ, वंडर बॉय रचिन रविंद्र के विकेट के साथ, खुल दी पियादव, ये फल मुबारक, खुल गई किसमत, खुल जा सिमसिम, और ये बड़ा विकेट चाहिए था भारत को, एक सेट बैट्समैन जो 200 बना चुका है इस टॉर्नमेंट में, उसका जब विकेट लिया, डंडे बिखेर दिये, फर्नीचर डिस्टर्ब कर दिया, देखिए टीम एकटम से इकठी हो गई, और कुलदीप, वो जादूगर, जादूगर, जादूगर जाएगा, किसी को समझ नहीं आएगा, गेंद गूगली थी, अंदर आई, पैट पे लगी, दिशा बदली, और फिर डंडे पे लगी, और देखिए इसे कहते हैं अलग जगाना, इसे कहते हैं गुरू टीम में विश्वास जगाना, इसे कहते हैं प्रज्जवलित मशाल, जिसे देखके सब कहें हम जीत सकते हैं भाई, मैं हूँ ना कहा कुलदीप ने,                                                                                                                                                       |

### Example 6

<Frame>
  <audio>
    <source type="audio/wav" />

    Your browser does not support the audio element.
  </audio>

  [▶ Play audio](/audio/dhrith-sample-7.wav)
</Frame>

| Model                         | Transcript                                                                                                |
| ----------------------------- | --------------------------------------------------------------------------------------------------------- |
| **Soket Dhrith**              | **और इस बार तो छक्के के लिए तैयार है। \[excited] ये गेंद गई है एक बार फिर दर्शक दीर्घा में। \[dramatic]** |
| Sarvam Sarika 2.5             | और इस बार तो छक्के के लिए तैयार है। ये गेंद गई है एक बार फिर दर्शक दीर्घा में।                            |
| Deepgram Nova 2               | और इस बार तो छक्के के लिए तैयार है यह गेंद गई है एक बार फिर दर्शक दीर्घा में                              |
| OpenAI GPT-4o-Mini Transcribe | और इस बार तो छक्के के लिए तैयार है।                                                                       |

### Example 7

<Frame>
  <audio>
    <source type="audio/wav" />

    Your browser does not support the audio element.
  </audio>

  [▶ Play audio](/audio/dhrith-sample-8.wav)
</Frame>

| Model             | Transcript                                                                                                          |
| ----------------- | ------------------------------------------------------------------------------------------------------------------- |
| **Soket Dhrith**  | **गब्बर के ताप से तुम्हें एक ही आदमी बचा सकता है \[dramatic] एक ही आदमी \[emphasized] खुद गब्बर \[dramatic tone].** |
| Sarvam Sarika 2.5 | गब्बर के ताप से तुम्हें एक ही आदमी बचा सकता है, एक ही आदमी! खुद गब्बर!                                              |
| Deepgram Nova 2   | गब्बर के ताप से तुम्हें एक कोई आदमी बचा सकता है एक कोई आदमी खुद गब्बर                                               |

For additional samples (including commentary-style speech and numeric/entity-heavy audio), see the full blog:

* [https://soket.ai/blogs/dhrith](https://soket.ai/blogs/dhrith)

## Getting Started

<CardGroup>
  <Card title="Signup" icon="rocket" href="https://app.tensorstudio.ai/signup">
    Create your account to get started with TensorStudio
  </Card>
</CardGroup>

## API keys

* [API Keys](/apis)


# Integrations
Source: https://docs.tensorstudio.ai/integrations/index

Integrate TensorStudio with your favorite tools and platforms

Welcome to TensorStudio Integrations. We provide multiple ways to integrate our AI capabilities into your applications.

Choose from our available SDKs and integration guides:

* [Plivo SDK](https://github.com/soketlabs/plivo-integration): Integrate telephony capabilities into your Voice AI application using Plivo.
* [Soket Labs - Realtime Console](https://github.com/soketlabs/realtime-console): A reference console for testing and managing realtime speech API interactions.
* [Voice Agent in Python](https://github.com/soketlabs/voice-agent-python): A sample voice agent implementation using direct event handling with realtime speech.
* [Python SDK](https://github.com/soketlabs/realtime-sdk-python): Python SDK to connect your application with the Realtime Speech API.
* [NodeJS SDK](https://github.com/soketlabs/openai-realtime-api-beta): Official Node.js SDK for integrating with the Realtime Speech API.
* RAG DB integrations for your documents(coming soon)


# Introduction
Source: https://docs.tensorstudio.ai/introduction

Explore our Guides and API Reference to get the most out of TensorStudio by Soket AI Labs

<img alt="Hero Light" />

## Welcome

Welcome to TensorStudio's documentation! TensorStudio provides a powerful Real-Time Voice Intelligence API that enables you to integrate advanced conversational AI capabilities with human-like speech, automated actions through tool-calling, and interaction analysis into your applications.

Our Voice Intelligence API offers:

* Natural conversational AI with human-like speech patterns and tones
* Seamless integration with external tools for automated task execution
* Detailed interaction summaries and actionable insights
* Simple SDK integration optimized for varied infrastructure

This documentation will help you quickly get started with implementing intelligent voice features in your applications. Whether you're building customer support solutions, process automation systems, or data analytics tools, TensorStudio provides the foundation you need with powerful voice intelligence capabilities.

Need help? Our team is here to support you - reach out at [support@tensorstudio.ai](mailto:support@tensorstudio.ai) or through our dedicated support channels.

## Getting Started

<CardGroup>
  <Card title="Signup" icon="rocket" href="https://app.tensorstudio.ai/signup">
    Create your account to get started with TensorStudio
  </Card>

  <Card title="Playground" icon="play" href="/quickstart_playground">
    Get started with TensorStudio's voice processing capabilities in minutes
  </Card>

  <Card title="Examples" icon="code" href="examples/index">
    Explore our examples to see how our Speech API can be used in real-world applications
  </Card>

  <Card title="SDK Integration" icon="wrench" href="sdk/overview">
    Learn how to integrate TensorStudio into your applications
  </Card>
</CardGroup>


# Quickstart with Playground
Source: https://docs.tensorstudio.ai/quickstart_playground

Experiment with TensorStudio Playground to create smart voice interactions.

## Introduction

The [Realtime Speech API Playground](https://app.tensorstudio.ai/studio/playground) is an interactive environment designed to enable developers, businesses, and hobbyists to experiment with and fine-tune AI-powered speech interactions. It allows users to test and understand the capabilities of the platform in real-time, providing the ability to simulate various conversational scenarios using customizable settings. Whether you're building voice assistants, automated customer support agents, or seamless conversational AI experiences, the Playground equips you with the tools to create, test, and refine your AI-driven speech applications.

<Frame>
  <img alt="Hero Light" />
</Frame>

## Interact with Agent

* Adjust `Playground Settings` to tailor conversations with your AI agent.
* Choose between `Manual` or `VAD` mode, then click `Connect` to start interacting.
* View and review conversation logs in the `Transcriptions` tab.
* Use the `Functions` panel to integrate tool calling and track actions triggered by the AI agent.

*Note: Playground consumes credits in the background.*

## Key Features of the Playground

Tune your playground for testing various scenarios for your AI agent

* **Customizable system Prompts**: Define your AI agent’s behavior and personality by setting a system prompt and an optional default message. This helps guide how the agent understands context and responds to user input.
  * **Custom prompt box**: Input your unique system prompt to define the AI’s context.
  * **Default Input Greeting**: Optionally add a starting message or greeting that initializes the interaction context.
* **Template agent prompts**: Choose from a set of pre-defined tailored prompts for some common scenarios to build your `AI agents`.
  * **Available templates**: Tech support, Real estate, Human Resources, Healthcare
  * **Flexible**: Quickly set up and modify if needed to test scenarios.
* **Model Selection and response settings**: Customize How Your AI Thinks, Sounds, and Responds.
  * **Input Language**: Choose from multiple input languages to match your AI agent’s needs.
  * **Model selection**: Choose from the available models based on your requirements. Optimize your AI’s performance with models tailored for efficiency, speed, or high-quality, use-specific output.
  * **Voice Settings**: Personalize your AI agent’s interaction style with a variety of voice modes.
    * `Voice Type`: Male, female, or neutral.
    * `Language and Accent`: Choose from multiple languages and regional accents to better suit your audience.
    * `Prosodic Features`: Choose a voice that best matches your desired pitch, intonation, and stress patterns for enhanced naturalness.
    * `Tonality`: Select voices with emotional tones suitable for your use case.
    * `Speed`: Opt for voices with appropriate speech rates for clarity and audience suitability.
  * **Temperature Control**: Temperature determines the randomness of the AI’s responses. Customize the variability in your AI agent’s responses to suit your specific needs.
    * **Low Temperature (e.g., 0.2)**: Produces consistent and predictable outputs.
    * **High Temperature (e.g., 0.8)**: Generates more creative and varied responses.
* **VAD settings**: Use `Voice Activity Detection` to adjust how your AI agent detects when someone is speaking and determines the right moment to respond, ensuring smoother, more natural interactions.
  * **VAD mode**: Choose how your AI listens. `Vad` Mode detects speech and responds on its own, while `Manual` Mode uses push-to-talk for more controlled interactions.
  * **Threshold**: Sets the minimum audio level (volume) required to detect speech. A lower threshold makes the agent more sensitive to quiet voices, while a higher threshold helps ignore background noise.
  * **Prefix padding**: Adds a short buffer of audio before speech is detected. This helps capture the beginning of speech more accurately, avoiding cut-off words or syllables.
  * **Silence duration**: Defines how long a period of silence must last before the agent decides the speaker has finished talking. Adjust this to balance responsiveness with avoiding premature interruptions.

## Transcriptions and Functions

Review your audio transcripts and events trigerred by your AI agent through function calling.

For a detailed guide, see [Tool Calling](/concepts/tool_calling).


# TensorStudio SDK
Source: https://docs.tensorstudio.ai/sdk/overview



Welcome to TensorStudio’s SDK Quickstart guide—your gateway to integrating real‑time voice intelligence into your applications effortlessly. With this SDK, you can:

* Seamlessly add human-like conversational AI via voice or text.
* Integrate with diverse infrastructures—from web and mobile apps to backend systems.
* Use [NodeJS SDK](https://github.com/soketlabs/openai-realtime-api-beta) or [Python SDK](https://github.com/soketlabs/realtime-sdk-python) as per your application needs.
* Enable tool‑calling so your agent can perform actions and fetch data autonomously.
* Capture interaction logs and transcripts for analysis, monitoring, and optimization.

In just a few simple steps, you can:

1. **Install the SDK and authenticate.**
2. **Configure core settings** like VAD, voice styles, models, and temperature.
3. **Connect and enable real‑time interaction** or asynchronous flows.
4. **Access transcripts, function‑call events, and analytics.**
5. **Disconnect when testing is complete and review logs.**

Whether you're building customer‑support bots, voice‑enabled assistants, or internal workflows, TensorStudio provides the tools you need to create natural, responsive, and intelligent voice experiences.

**Need help?** Our team is here—reach out to [support@tensorstudio.ai](mailto:support@tensorstudio.ai) or visit the support section in the docs.


# NodeJS SDK
Source: https://docs.tensorstudio.ai/sdk/quickstart_nodejs_sdk

Get started with the Realtime Speech API SDK in minutes

## Installation

Install the SDK using npm:

```bash theme={"system"}
npm install https://github.com/soketlabs/openai-realtime-api-beta
```

## Creating a Client

```typescript theme={"system"}
import { RealtimeClient } from '@openai/realtime-api-beta';

const client = new RealtimeClient({
  apiKey: 'your-api-key',
  // For browser environments, set this to true
  /* Warning: Use only in development or testing — exposes your API key */
  dangerouslyAllowAPIKeyInBrowser: true 
});

// Or use a relay server to protect your API key
const client = new RealtimeClient({
  url: 'your-relay-server-url'
});
```

Refer to [Relay Server Guide](/deploy/relay_server) to set up and use a relay server.

## Session Configuration

After creating the client, you can set instructions for your AI agent, configure various session parameters:

```typescript theme={"system"}
// Configure the session with multiple parameters
client.updateSession({
  // Set system instructions for the AI
  instructions: 'You are a helpful assistant',
  
  // Set voice and language
  voice: 'monica',
  language: 'en',
  
  // Set up Voice Activity Detection
  turn_detection: {
    type: 'server_vad',
    threshold: 0.5,
    prefix_padding_ms: 300,
    silence_duration_ms: 500
  }
});
```

## Basic Usage

### Connect and Send Messages

```typescript theme={"system"}
// Connect to the service
await client.connect();

// Send a text message
client.sendUserMessageContent([{
  type: 'input_text',
  text: 'Hello!'
}]);

// Listen for responses
client.on('conversation.updated', ({ item, delta }) => {
  if (delta?.audio) {
    // `delta.audio` contains raw audio chunks (typically in binary or base64 format).
    // You need to implement the `playAudio` function to handle playback,
    // e.g., using Web Audio API or any audio player of your choice.
    playAudio(delta.audio);
  }
  if (item.formatted.text) {
    // Handle text response
    console.log(item.formatted.text);
  }
});
```

* Explore [Tool Calling](/concepts/tool_calling) for extended functionality
* Learn about [Voice Activity Detection](/concepts/vad) for better speech interaction
* Check out available [Voices](/concepts/voice) for your application


# Python SDK
Source: https://docs.tensorstudio.ai/sdk/quickstart_python_sdk

Get started quickly with the Realtime Python SDK

This guide helps you get started with the **Realtime Python SDK** to send and receive messages via the Realtime API.

## 📦 Install the SDK

Install the package:

```bash theme={"system"}
git clone https://github.com/soketlabs/realtime-sdk-python.git
cd realtime-sdk-python
pip install .
```

## 🚀 Basic Usage

Below is a minimal example to connect, send a message, and receive responses using the Python SDK.

```python theme={"system"}
import asyncio
import pyaudio
import base64
from loguru import logger
from realtime.client import RealtimeClient
import os
from connect import ConversationHandler, AudioManager, audio_recorder
from dotenv import load_dotenv
load_dotenv()

# Initialize the OpenAI client
client = RealtimeClient(
        url="wss://api.soket.ai/v1/realtime",
        api_key=os.getenv("TENSOR_STUDIO_API_KEY"),  # Ensure your API key is set
        debug=True  # Set to True to enable debugging logs
    )
# Initialize the conversation handler and audio manager
audio_manager = AudioManager()
conversation_handler = ConversationHandler(audio_manager)

# Register the event handlers
client.on("conversation.updated", conversation_handler.on_conversation_updated)
client.on("conversation.item.completed", conversation_handler.on_conversation_item_completed)
client.on("error", conversation_handler.on_error)
client.on("conversation.interrupted", conversation_handler.on_conversation_interrupted)

async def main():
    recorder_task = None
    try:
        await client.connect()
        await client.update_session(
            instructions = '''
        You are Soket bot, a helpful assistant. Please respond clearly and concisely.
                ''',
                turn_detection = {
                    "type": "server_vad",
                    "threshold": 0.2,
                    "prefix_padding_ms": 300,
                    "silence_duration_ms": 1000,
                }
        )
        logger.info("Connected to Realtime API.")
        message_content = [
                {
                    "type": "input_text",
                    "text": "तुम्हारा नाम क्या है?"
                }
            ]
        await client.send_user_message_content(content=message_content)
        logger.info("Message sent.")

        # Start audio recorder as a coroutine
        recorder_task = asyncio.create_task(audio_recorder(client))

        # Keep the connection alive to process events
        await asyncio.Event().wait()

    except Exception as e:
        logger.error(f"An error occurred: {e}")
        raise e

    finally:
        # Disconnect from the API
        await client.disconnect()
        logger.info("Disconnected from Realtime API.")

        # Stop and clean up AudioManager
        await audio_manager.shutdown()

        # Cancel recorder_task if it's still running
        if recorder_task and not recorder_task.done():
            recorder_task.cancel()
            try:
                await recorder_task
            except asyncio.CancelledError:
                logger.info("Recorder task cancelled.")

if __name__ == "__main__":
    asyncio.run(main())

```

## 🛠️ More Examples

* See the [connect.py example](https://github.com/soketlabs/realtime-sdk-python/blob/main/connect.py) for a full script.
* Explore advanced features like tool-calling and VAD in the [Realtime SDK Python repo](https://github.com/soketlabs/realtime-sdk-python/tree/main).

## 🔗 Next Steps

* Explore [Tool Calling](/concepts/tool_calling) for extended functionality
* Learn about [Voice Activity Detection](/concepts/vad) for better speech interaction
* Check out available [Voices](/concepts/voice) for your application