Our house answers to English, Thai and Mandarin — lights, air conditioning, music, the TV, even the shopping list — and none of it leaves the building. No cloud account, no subscription, no company listening in. This is a tour of how it fits together and the handful of ideas that made it feel instant instead of frustrating.
The obvious reason is privacy: microphones in a family home should not stream to anyone’s data centre. But the practical reasons turned out to matter just as much. A local assistant keeps working when the internet doesn’t. It answers in the language you spoke, not the language of your account settings. And because everything runs on hardware in a cupboard, you can fix the thing that annoys you instead of waiting for a company to care.
The whole stack runs on one small AI machine — the same class of box as our AI Hub — plus a smart-speaker-sized microphone in each room.
Fast paths for the boring stuff. “Turn off the lights” does not need intelligence, it needs speed. Fixed sentence matching answers in about a second and a half; the AI model handles the long tail — “play something like Crazy P”, “what’s the temperature in every room” — where a couple of extra seconds buys real understanding.
The room is part of the sentence. Every microphone knows its own room, so “turn on the lights” means this room’s lights, and “play some jazz” plays it on this room’s speaker. Nobody should have to say “in the bedroom” while standing in the bedroom.
Teach the system its own mishearings. Speech recognition of short commands is imperfect in any language. Instead of fighting it, we watched the logs and added the common mishearings as extra trigger sentences. If the transcriber writes down something odd but predictable, the odd-but-predictable version simply becomes another way to say it.
Verify, then speak. Early on, the assistant would cheerfully announce “Playing your playlist” while nothing came out of the speakers. Now the music path checks that playback actually started before it claims success — and says plainly when a streaming service is having a bad day. An assistant that admits failure is far less maddening than one that pretends.
Very short Mandarin commands — two syllables, like 关灯 — are the hardest thing in the house for the microphones to catch cleanly; longer phrasings work every time and that is what the family uses. And a local AI model the size that fits in a home server is a capable junior assistant, not a philosopher: we lean on the fixed fast paths precisely because determinism beats brilliance for everyday commands.
This is the system our AI Hub productises — the house is the test lab, and everything above has survived months of daily family use.