DeciBench

DeciBench is a testing tool built specifically for voice AI agents. It works like a standard testing framework for code but focuses on voice conversations. The tool runs entirely on your computer without sending any data to the cloud. This ensures that sensitive customer information stays private while you verify that your voice AI works correctly.

Benefits

DeciBench helps teams catch problems before customers experience them. It solves common issues like the AI making up facts or taking too long to respond. The tool automatically checks for security leaks and ensures the system handles interruptions smoothly. It offers three different testing speeds and costs to fit various budgets. You can choose fast basic checks or slower detailed evaluations using advanced AI models. The system also protects your data by removing personal details like phone numbers and credit card numbers before saving results.

Use Cases

Developers use DeciBench to test voice assistants before launching them to the public. Teams can run tests on their own servers to keep data secure. The tool works with many different voice platforms including Twilio and ElevenLabs. It is useful for companies building customer service bots or interactive voice response systems. You can generate test scenarios automatically from your own documentation to ensure the AI answers questions accurately. The dashboard provides clear reports in formats like HTML or JSON to help teams understand test results quickly.

Pricing

The tool offers flexible pricing based on the testing mode you choose. Basic deterministic tests are free and run very fast. Semantic evaluations cost about one cent per call and take a few seconds. The most advanced option that includes RAG capabilities costs three cents per call and takes around five seconds. These costs apply only when using the advanced AI scoring features.

Vibes

Users appreciate the tool for its focus on privacy and security. The zero telemetry feature is a major plus for teams handling sensitive data. Developers like the flexibility of choosing different testing modes without needing to change their setup. The community around the project is active and open, allowing anyone to contribute to the code. The tool is praised for solving the gap between demo performance and real-world reliability.

Additional Information

DeciBench is built by the unforkopensource-org community. It is released under the Apache 2.0 license which allows free use and modification. The project is currently hosted on GitHub and can be installed directly from there. It requires Python 3.11 or higher to run on macOS, Linux, or Windows Subsystem for Linux. The team is working on making it easier to install via standard package managers in the future.