OpenAI just announced support for WebRTC in their Realtime API. Combining it with Cloudflare Calls allows you to build experiences that weren’t possible before. https://blog.cloudflare.com/bring-multimodal-real-time-interaction-to-your-ai-applications-with-cloudflare-calls
Comments
Still some challenges with it tho, especially in Norwegian.
- seems like context overload is a real problem. The output gets garbled pretty quickly even at the lowest temperature setting.
- no file/vector lookup ability
- function call responses don’t automatically trigger the bot to answer, still need voice or manual input to get a response