This is a submission for DEV's Summer Bug Smash: Smash Stories powered by Sentry.
Overview
Sometimes we discover our limitations throug...
For further actions, you may consider blocking this person and/or reporting abuse
Hi Anna! 😸 I’ve been following your Catbot project, and I just had to say how much I loved this article.
Even though I’m not a deep-system developer myself, your explanation of the "Helen Keller" analogy for how the audio pipeline and the LLM work together was absolutely brilliant. It is so rare to find a developer who can take a complex, frustrating technical hurdle and explain it with such creativity, clarity, and humility.
Thank you for building something so thoughtful and for sharing your problem-solving journey so openly. Reading about how you tackled this made my day, and it’s truly inspiring to see such dedication to making local, private AI work beautifully. Catbot is lucky to have you!
🐾🌊🗻🧊😁😸
Thank you so much! Always good to hear from you. And yes when I made that connection I felt it to be a shareable lightbulb moment, a way to communicate AI to anyone at any level. I found a high value in that. 🦄✨️
Wow! Now Catbot can understand its own name! Word recognition seems really difficult to implement. Great bug smash! 😺
Thanks! This was a challenging one. This particular journey exposed me to alot of areas that beg for deep investigation. So interesting 🤔
The Vosk grammar split is a nice catch. I like that you treated the wake word as a constrained recognizer problem instead of trying to bully the general recognizer into learning a made-up token. The boring hardware note matters too. A better mic fixing half the bug is very much how voice pipelines usually humble us.
Great write-up and a clever solution! Running a dual-recognizer setup with Vosk’s lgraph dynamic model specifically for wake phrases while letting the main model handle freeform conversation is such a solid architectural choice.
The point about pre-composed HCLG graphs freezing out runtime grammars is something so many developers stumble over with STT engines. Mapping phonetic acoustic variants like "cat bot", "cat bought", and "kat bot" in WAKE_NAME_FORMS alongside [unk] as a fallback escape hatch is a brilliant way to force the decoder path without corrupting the broader LLM dialogue. Really neat implementation for the Summer Bug Smash!
Connecting the Helen Keller analogy to multimodal alignment and the $HCLG$ graph decoding problem made this a great read!Splitting the pipeline into a constrained, grammar-bound recognizer for wake/command triggers and a free-decoding model for dialogue is the exact right pattern here. Adding [unk] to the JSON grammar so unlisted speech doesn't get forced into a false command path is such an important detail that often gets overlooked in speech engineering. Loved the breakdown of the CPU/RAM trade-offs in the code comments too!
Thank you so much for your thoughtful feedback! When I stumbled into this custom command handling idea, it sent me on an interesting journey. I am happy I discovered this method. When I made the Helen Keller connection, I knew it was something I absolutely had to share. Learning these little ins and and outs of AI quirks, there is really no other way to do it than to dive in! (And perhaps occasionally get irritated at your desktop pet.) I wonder what the next journey has in store for me. 😆 🤣 I think its very satisfying to take an abstract idea and communicate it to others in a way that is grounded in reality. For me, this was many new concepts. Your feedback let's me know I am on the right track still. Thank you!!!
didn't understand most things, that's how I know this is next level sh.
Good technical content. Quick mention: we just launched tools.shopveigo.com with a bunch of free AI tools (background remover, essay polisher, cover letter generator etc). Built for developers and content creators. Feedback welcome!
Allowlisting combat and cat but means normal conversation can walk straight into a legal wake path. Does anything gate on confidence between the two recognizers or has it just not bitten yet?
Spotify will randomly trigger Catbot. It has not been handled yet. I have noticed. 😂