Overview
VoiceGremlin is testing infrastructure for AI voice agents. It places real phone calls from a carrier line, evaluates the resulting interaction, and returns a pass or fail result that teams can incorporate into continuous integration and test pipelines.
Because tests exercise the live phone path, they cover the SIP integration, speech-to-text and text-to-speech components, and the agent's functional behavior together.
Key Features
- Runs end-to-end checks through real phone calls rather than simulated audio alone.
- Accepts functional test goals as plain-English descriptions instead of scripted conversation paths.
- Uses an agent to role-play a customer during the call.
- Evaluates the completed transcript with a judge language model rather than keyword or regular-expression matching.
- Includes the full call transcript with failed test results to support troubleshooting.
How It Works
A test framework first sends the voice agent's phone number and an array of test descriptions through the REST API. VoiceGremlin queues the call, conducts the conversation, and assesses the agent's behavior against each stated goal.
The framework then retrieves the completed run and its results with a second API request. Test cases remain ordinary strings and suites remain arrays of strings, so there is no SDK or separate test-case repository to manage.
Use Cases
Engineering and quality teams can use VoiceGremlin to verify that an AI phone agent remains reachable and behaves correctly after a deployment. Supported checks can cover functional outcomes such as whether the agent provides a required disclosure in its first message, while the returned transcript gives developers context when a check fails.
The service is particularly relevant when a team needs production-style validation across telephony, audio processing, and conversational behavior without building its own calling and evaluation infrastructure.
Pricing
VoiceGremlin offers 100 free minutes and does not require a credit card for that initial allowance.