We’ve released two open source projects today, both taken out of the AI Voice Agent that ships in ICTContact. asterisk-ai-voice-agent is a self-hosted Python sidecar that turns an Asterisk call into a spoken conversation with an LLM. asterisk-audiosocket is the protocol layer underneath it, packaged on its own for Node. Both live on GitHub, and you run them on your own hardware with your own API keys.
ICTContact has been answering calls with configurable AI Personas for a while now, and the media path that makes those conversations sound like conversations is the part people kept asking about. So we lifted it out, cleaned it up, and put it under an MIT license.
What the voice agent actually does
You point an Asterisk dialplan at it with a single AudioSocket() line. From there the sidecar takes the raw call audio, works out when the caller has stopped speaking, transcribes what they said, sends it to a language model, synthesises the reply, and pushes that reply back down the same socket as audio.
The pieces you can pick:
- Speech in. OpenAI Whisper or ElevenLabs Scribe. WebRTC voice activity detection decides when a turn has ended, so the agent isn’t waiting on a fixed timer.
- The reasoning. Anthropic Claude, streamed token by token, so synthesis can start before the full answer exists.
- Speech out. Piper runs locally on CPU, or ElevenLabs if you want the cloud voice. Piper means the whole loop can run with no text to speech bill at all.
- Tools. When the model decides to transfer a call, book a callback, look something up in your CRM, or check a do not call list, the sidecar POSTs that decision to a webhook you own. Your code does the work. The model just asks.
Barge-in is in there too. Start talking over the agent and the queued speech gets dropped mid-sentence, the way a person stops when you interrupt them. Without that, an AI caller feels like a recorded message, and people hang up on recorded messages.
The pacing bug that eats your first week
Here’s the part that catches everyone, and the reason we split the protocol layer into its own package.
Your text to speech engine hands you a finished sentence. Say 2.4 seconds of audio, which at 8 kHz slin16 works out to 120 frames of 320 bytes. The obvious thing to do is write all of it to the socket and move on.
Don’t. Asterisk’s app_audiosocket forwards every frame to the channel the instant it arrives. It does not buffer on your behalf. So 120 frames land on the caller’s jitter buffer in a few milliseconds, that buffer holds a small fraction of them, and the rest are thrown away. The caller hears the tail of your sentence and nothing else.
The fix is to meter the audio to the frame clock: one 320 byte frame, then wait until the next 20 millisecond deadline. The subtlety is that the deadline has to be re-clamped on every frame. If synthesis stalls for half a second, a naive writer will burst to catch up and you’re back where you started. Both projects ship this writer already built, so you don’t have to rediscover it.
asterisk-audiosocket on its own
If you’d rather build your agent in Node, or you just want a clean AudioSocket implementation without any AI attached, npm install asterisk-audiosocket gets you the protocol on its own. TypeScript types, no runtime dependencies, the framed slin16 codec, the paced play() writer described above, and barge-in support.
It joins the two clients we published for Asterisk AMI and FreeSWITCH ESL. Those three cover most of what you need to drive a PBX from Node: manager interface, event socket, and now live media. The AMI client is the one you’d reach for to implement a transfer tool from the voice agent.
Why we built it this way
Plenty of hosted voice AI products will happily take your call audio. That’s fine until you look at where the recording goes, what the per-minute cost does at volume, or what happens to your deployment when the vendor changes a model.
This runs inside your stack. Your PBX, your keys, your prompts, your logs. The only traffic leaving the box is the provider calls you explicitly configured, and with Piper doing synthesis locally there’s one fewer of those.
It also follows on from what we’ve been building elsewhere. pbx-mcp gives AI assistants a read-only window into Asterisk and FreeSWITCH, and we’ve written before about AI becoming a default feature rather than an add-on in the open source VoIP stack. The voice agent is the same idea pointed at the live call instead of the console.
Being straight about v0.1.0
This is a first tagged release. The protocol handling is covered by tests, CI imports every module on every push, and the pacing behaviour is the part we’re most confident in because it’s the part we got wrong first. What it hasn’t had yet is a long soak on production traffic. Treat it as something to run in a lab, read, and tell us about, not something to point your main queue at on Monday.
Two practical notes before you start. Ports 9091 and 9092 should stay off the public internet, since neither speaks any authentication. And each concurrent call uses roughly 150 MB during synthesis bursts, so the concurrency cap in the config exists for a reason.
Where to get it
- github.com/ictinnovations/asterisk-ai-voice-agent (Python sidecar)
- github.com/ictinnovations/asterisk-audiosocket and npm (Node library)
Both are MIT licensed. Issues and pull requests are open, and if you get it running on something interesting we’d like to hear about it.
Questions people asked us first
Do I need a specific Asterisk version?
You need one with app_audiosocket, which means Asterisk 16 or newer. It’s a single dialplan line, so there’s no patching and no module to compile yourself.
Can I swap Claude for a different model?
Yes. Claude is what ships and what’s tested, but the LLM sits behind one small module interface. Implementing that interface against another provider, including a local model, is documented in PORTING.md in the repo. The same applies to the speech engines.
How much does a call cost to run?
It depends entirely on which providers you pick. With Piper handling synthesis locally you’re only paying for transcription and the model tokens. If you switch text to speech to ElevenLabs, that becomes the largest line item on a chatty call.
Does this work with FreeSWITCH?
Not directly, because AudioSocket is an Asterisk application. FreeSWITCH has its own media paths, and our ESL client is the starting point there. Porting the agent’s audio layer is on the list, not done.
Is this the same thing as the AI voice agent in ICTContact?
It came out of it, but it isn’t the same thing. ICTContact’s AI Personas are a product feature, with campaign management, reporting, and the multi-tenant pieces built around them. What we’ve published here is the plumbing on its own, for people who want to build something themselves. If you’d rather have the finished product, open a ticket and ask us.
What’s the roadmap?
Wider provider coverage, a longer soak on real traffic, and better numbers on latency per stage so you can see where a slow turn is actually going. Feature requests on GitHub carry real weight this early.