Overview
Tool Calling enables AI models to execute predefined functions during conversations, allowing them to perform actions like retrieving data, registering complaints, or manipulating application state. This feature bridges the gap between conversational AI and practical functionality.
Tools are added to the Realtime client using the addTool()
method. Each tool requires two components:
- Tool Definition - Describes the tool’s purpose and parameters
- Tool Implementation - The actual function that executes when the tool is called
client.addTool(
{
name: 'tool_name',
description: 'Description of what the tool does',
parameters: {
type: 'object',
properties: {
// Parameter definitions
},
required: ['param1', 'param2'] // List required parameters
}
},
async (params) => {
// Tool implementation
return result;
}
);
Here’s an example of a tool that retrieves weather information for given coordinates:
client.addTool(
{
name: 'get_weather',
description: 'Retrieves the weather for a given lat, lng coordinate pair',
parameters: {
type: 'object',
properties: {
lat: {
type: 'number',
description: 'Latitude'
},
lng: {
type: 'number',
description: 'Longitude'
},
location: {
type: 'string',
description: 'Name of the location'
}
},
required: ['lat', 'lng', 'location']
}
},
async ({ lat, lng, location }) => {
// Implementation
const result = await fetch(
`https://api.open-meteo.com/v1/forecast?latitude=${lat}&longitude=${lng}¤t=temperature_2m,wind_speed_10m`
);
const json = await result.json();
return json;
}
);
Example: Complaint Registration
Here’s an example of a tool for registering customer complaints:
client.addTool(
{
name: 'register_complaint',
description: 'Register a customer complaint',
parameters: {
type: 'object',
properties: {
reason: {
type: 'string',
description: 'The reason for the complaint'
},
issue_tag: {
type: 'string',
description: 'The tag for the issue',
enum: [
'router_issue',
'internet_issue',
'network_slow',
'bill_not_paid',
'network_down',
'plan_change_in_progress'
]
}
},
required: ['reason', 'issue_tag']
}
},
async ({ reason, issue_tag }) => {
// Implementation
const complaint_number = generateComplaintNumber();
return { complaint_number };
}
);
Best Practices
-
Clear Descriptions: Provide detailed descriptions for tools and parameters to help the AI understand when and how to use them.
-
Parameter Validation: Use the required
field and parameter types to ensure the AI provides necessary information.
-
Error Handling: Implement proper error handling in tool implementations:
async (params) => {
try {
// Tool implementation
return result;
} catch (error) {
return {
error: true,
message: 'Failed to execute tool: ' + error.message
};
}
}
- State Management: When tools modify application state, ensure changes are properly reflected in your UI:
client.addTool(
{
name: 'update_state',
description: 'Updates application state',
parameters: {
type: 'object',
properties: {
key: { type: 'string' },
value: { type: 'string' }
},
required: ['key', 'value']
}
},
async ({ key, value }) => {
setState(prevState => ({
...prevState,
[key]: value
}));
return { success: true };
}
);
Tools can be categorized based on their functionality:
- Data Retrieval Tools: Fetch external data (like weather information)
- State Management Tools: Modify application state
- Action Tools: Perform specific actions (like registering complaints)
- Integration Tools: Interface with external systems or APIs
Security Considerations
- Input Validation: Always validate tool parameters before processing
- API Key Protection: Never expose sensitive credentials in tool implementations
- Rate Limiting: Implement rate limiting for tools that access external services
- Error Boundaries: Implement proper error handling to prevent crashes
To debug tool calls, you can monitor events in the conversation:
client.on('realtime.event', (event) => {
if (event.event.type === 'function_call') {
console.log('Tool called:', event.event);
}
});
This will help you track when and how tools are being used during conversations.
Responses are generated using AI and may contain mistakes.