​
Overview

Tool Calling enables AI models to execute predefined functions during conversations, allowing them to perform actions like retrieving data, registering complaints, or manipulating application state. This feature bridges the gap between conversational AI and practical functionality.

​
Adding Tools

Tools are added to the Realtime client using the addTool() method. Each tool requires two components:
  1. Tool Definition - Describes the tool’s purpose and parameters
  2. Tool Implementation - The actual function that executes when the tool is called

​
Basic Tool Structure

client.addTool(
  {
    name: 'tool_name',
    description: 'Description of what the tool does',
    parameters: {
      type: 'object',
      properties: {
        // Parameter definitions
      },
      required: ['param1', 'param2'] // List required parameters
    }
  },
  async (params) => {
    // Tool implementation
    return result;
  }
);

​
Example: Weather Tool

Here’s an example of a tool that retrieves weather information for given coordinates: 
client.addTool(
  {
    name: 'get_weather',
    description: 'Retrieves the weather for a given lat, lng coordinate pair',
    parameters: {
      type: 'object',
      properties: {
        lat: {
          type: 'number',
          description: 'Latitude'
        },
        lng: {
          type: 'number',
          description: 'Longitude'
        },
        location: {
          type: 'string',
          description: 'Name of the location'
        }
      },
      required: ['lat', 'lng', 'location']
    }
  },
  async ({ lat, lng, location }) => {
    // Implementation
    const result = await fetch(
      `https://api.open-meteo.com/v1/forecast?latitude=${lat}&longitude=${lng}&current=temperature_2m,wind_speed_10m`
    );
    const json = await result.json();
    return json;
  }
);

​
Example: Complaint Registration

Here’s an example of a tool for registering customer complaints: 
client.addTool(
  {
    name: 'register_complaint',
    description: 'Register a customer complaint',
    parameters: {
      type: 'object',
      properties: {
        reason: {
          type: 'string',
          description: 'The reason for the complaint'
        },
        issue_tag: {
          type: 'string',
          description: 'The tag for the issue',
          enum: [
            'router_issue',
            'internet_issue',
            'network_slow',
            'bill_not_paid',
            'network_down',
            'plan_change_in_progress'
          ]
        }
      },
      required: ['reason', 'issue_tag']
    }
  },
  async ({ reason, issue_tag }) => {
    // Implementation
    const complaint_number = generateComplaintNumber();
    return { complaint_number };
  }
);

​
Best Practices

  1. Clear Descriptions: Provide detailed descriptions for tools and parameters to help the AI understand when and how to use them.
  2. Parameter Validation: Use the required field and parameter types to ensure the AI provides necessary information.
  3. Error Handling: Implement proper error handling in tool implementations: 
async (params) => {
  try {
    // Tool implementation
    return result;
  } catch (error) {
    return {
      error: true,
      message: 'Failed to execute tool: ' + error.message
    };
  }
}
  1. State Management: When tools modify application state, ensure changes are properly reflected in your UI:
client.addTool(
  {
    name: 'update_state',
    description: 'Updates application state',
    parameters: {
      type: 'object',
      properties: {
        key: { type: 'string' },
        value: { type: 'string' }
      },
      required: ['key', 'value']
    }
  },
  async ({ key, value }) => {
    setState(prevState => ({
      ...prevState,
      [key]: value
    }));
    return { success: true };
  }
);

​
Tool Types

Tools can be categorized based on their functionality:
  1. Data Retrieval Tools: Fetch external data (like weather information)
  2. State Management Tools: Modify application state
  3. Action Tools: Perform specific actions (like registering complaints)
  4. Integration Tools: Interface with external systems or APIs

​
Security Considerations

  1. Input Validation: Always validate tool parameters before processing
  2. API Key Protection: Never expose sensitive credentials in tool implementations
  3. Rate Limiting: Implement rate limiting for tools that access external services
  4. Error Boundaries: Implement proper error handling to prevent crashes

​
Debugging Tools

To debug tool calls, you can monitor events in the conversation: 
client.on('realtime.event', (event) => {
  if (event.event.type === 'function_call') {
    console.log('Tool called:', event.event);
  }
});
This will help you track when and how tools are being used during conversations. Note: This code uses events directly; you can add your tool calls in the session.update event, following the documented format here: OpenAI Realtime Client Events.