Blue Iris → Home Assistant → a kitchen tablet that talks back

What this is: a build journal for a fully-local home-security setup — taking object-detection alerts from a Blue Iris NVR (backed by a self-hosted CodeProject.AI vision server), routing them through Home Assistant over MQTT, and turning them into spoken announcements and a live camera grid on a wall-mounted kitchen tablet. No cloud and no smart-display — everything stays on the LAN. It's one living post that grows step by step as the build progresses.

Date started: 2026-07-05 Status: MQTT auth done; BI → MQTT → HA person/vehicle/animal sensors working; kitchen tablet onboarding started — dedicated non-admin HA user created and the Companion app registered. Fully Kiosk wired into HA REST-only (Step 8) after an MQTT plan turned out to be over-build. Step 9 done — it talks: the garage-person sensor now wakes the tablet and speaks per detection via Google TTS. Step 10 — it shows (mostly): any object on the three front cams pops a 3-across camera grid on the tablet and announces per-camera ("Person at the front door."). Open issue: the grid tiles are still static, not live (go2rtc dropping to MJPEG — being debugged). Next: get the tiles live, then swap Google TTS for local Piper.

TLDR

Goal: get Blue Iris (my Windows NVR) talking to Home Assistant, then have HA push alerts — with spoken announcements — to a wall-mounted Galaxy Tab A11+ acting as a kitchen monitor.
Kicked it off from a Gemini search asking whether an Echo Show, a Nest Hub, or "some open-source device" was the right kitchen display. Conclusion: none of the smart displays — a plain Android tablet running Home Assistant Companion + Fully Kiosk keeps everything local, which is the whole point of this homelab.
Transport decision: MQTT is the spine for events (BI already has an MQTT client, I already run Mosquitto). The Blue Iris HACS integration is optional, kept for later if I want to arm/disarm BI profiles from HA. Video is a separate channel (RTSP/MJPEG), not MQTT's job.
MQTT auth: the broker currently runs -c /mosquitto-no-auth.conf on purpose, with an open todo to fix it. Adding Blue Iris as a new client was the push to finally enable auth. Doing it step by step with verification at each step rather than one big flip, because the cutover briefly takes Zigbee2MQTT + HA's MQTT offline.
Surprise find: an auth config already existed but was dead — a misnamed mosquitto.config (should be .conf) full of allow_anonymous false settings that the broker was never told to load, because the compose command still pointed at mosquitto-no-auth.conf. The exact "config vs running command mismatch" a past commit had flagged. Renamed it, wired it up, and found the published 9001 websockets port had no matching listener.
Backups first: before touching auth I added a home_automation/backup.sh that snapshots the gitignored runtime data (HA config, MQTT, Zigbee2MQTT coordinator state, ESPHome) into a timestamped tarball — cold by default for consistency, with a --hot no-downtime option, optional offsite scp, and retention.
Second git landmine: zigbee2mqtt/configuration.yaml is tracked in git with the Zigbee network_key sitting in it — a real secret already committed. So the MQTT password went in via a gitignored z2m.env (env override) instead of that file, per the repo's own "secrets via env_file" rule. Flagged the network-key leak as a separate follow-up; didn't touch it mid-cutover.
The flip (Step 4) landed cleanly on the first try — because of the pre-seed ordering, the moment the broker switched to allow_anonymous false, Zigbee2MQTT just reconnected authenticated and kept publishing all ~18 devices. No re-pairing, no lost automations, no scramble. The careful ordering paid for itself.
Blue Iris is wired in (Step 5): BI publishes per-camera alert JSON to MQTT, and the object labels in it — person:86%,bird:82%,... — come from CodeProject.AI, a self-hosted vision server BI hands each triggering frame to. So the detection is local too, no cloud vision API. HA now has smart, confidence-tagged events to build on.
HA sensors built (Step 6): three MQTT binary_sensors — person, vehicle, animal — that parse the CodeProject.AI labels out of the memo, take the highest confidence in each category, and fire above 60%. Adding the extra types was trivial once the person one worked: same template, different label list.
Third landmine, surfaced by a config check: a long-dead panel_iframe: block (a removed HA integration, and a broken double-http:// URL pointing at the wrong port) had been sitting in configuration.yaml, silently failing. Removed it; logged a todo to re-add an "MQTT Explorer" link the modern way (a dashboard Webpage card) if wanted.
Kitchen tablet started (Step 7): made a dedicated, non-admin, local-only HA user for the wall tablet (so a physically-exposed, always-logged-in device can't wield my admin account), and registered the Galaxy Tab A11+ in the HA Companion app as Kitchen Tablet. Fully Kiosk + TTS + the actual announcement are the remaining pieces.
Fully Kiosk wired in (Step 8) — but not before a self-inflicted flop: I designed this as "REST + MQTT" and even provisioned a tablet broker user, on the assumption the HA integration would consume Fully Kiosk's MQTT for instant push. It doesn't — the official integration is REST/polling only, and Fully Kiosk MQTT is a separate channel that just spawns duplicate entities. Once I dropped a battery-"unplugged" alert I didn't need, MQTT bought this project nothing. Killed it, went REST-only, and the two things the announcement needs — a screen switch that wakes the tablet and a media_player — came up clean. Lesson: read the integration's contract before building infra for it.
It talks (Step 9): wired up Google Translate TTS (free, zero-setup, cloud — to prove the chain fast before swapping to local Piper) and wrote the automation: garage-person sensor → wake the tablet screen → volume 0.8 → speak "Person entering driveway.", with a 10s cooldown. One snag — Fully Kiosk threw "wrong url or unsupported sound" because HA's internal URL wasn't tablet-reachable; setting it to http://192.168.4.2:8123 fixed it, and the tablet spoke on the next run.
It shows, too (Step 10) — mostly: generalized to all three front cameras and added a 3-across grid that pops on the tablet on any detection (person/vehicle/animal), driven by a timer.front_alert + a conditional dashboard card — zero HACS. Folded the announcement into one per-camera automation ("Person at the driveway / front door / front yard."). The HA YAML was the easy part; the fight was infra plumbing — Blue Iris serves RTSP on port 81 not 554, and HA→cameras crossed two pfSense VLANs that had to be opened with one firewall rule. Also re-learned that the Step-7 non-admin tablet user (by design) means dashboards are built from admin, not on the tablet. Open issue: the grid tiles currently show a static snapshot, not live video — go2rtc keeps dropping to MJPEG (proved HA can pull the stream over TCP; prime suspect is the cameras' 8 kHz AAC audio breaking WebRTC/MSE). Being solved next.
End state: the project's whole point is working end-to-end — camera → CodeProject.AI → MQTT → HA → a wall tablet that announces whoever's out front (and pops a camera grid; live tiles still being sorted), all local. Left to do: get the grid live, then swap Google TTS → Piper for a fully-local voice.

The goal

I run Blue Iris as my home DVR. I want two things out of it beyond just recording:

Alerts land in Home Assistant so I can build automations on top of camera events (person at the front door, motion in the driveway, etc.).
A dedicated kitchen monitor — a Galaxy Tab A11+ — that shows the cameras and, more importantly, announces alerts out loud so I don't have to be looking at a screen.

I started by asking Gemini whether the kitchen display should be an Alexa Echo Show, a Google/Nest device, or something open-source. That framed the first real decision.

Decisions made (and why)

Not an Echo Show or a Nest Hub

Both are cloud-tethered. This homelab is deliberately local and VPN-only — Home Assistant, MQTT, Zigbee2MQTT and ESPHome all sit on a private network with no cloud dependency, and remote access is a WireGuard tunnel rather than anything internet-facing. Dropping an Amazon or Google display into the middle of the kitchen would drag exactly the cloud dependency I've avoided everywhere else back in. So: a plain Android tablet running the Home Assistant Companion app + Fully Kiosk Browser. HA can push it text-to-speech announcements, wake the screen, and pop up the triggering camera — all on-LAN.

MQTT as the event spine, HACS integration optional

I went back and forth on how Blue Iris should talk to HA. Two candidates:

MQTT — Blue Iris pushes an alert the instant it fires to a topic like BlueIris/FrontDoor/Status; HA subscribes and turns it into a binary_sensor. Real-time, and it runs on the Mosquitto broker I already have.
The Blue Iris HACS integration — a community custom_component that polls BI's JSON API and mirrors camera state as entities. Good for control/status, laggier for instant alerts, and it's a third-party dependency to maintain.

They're not really competitors — they do different jobs. So the call was: MQTT is the spine for events (it drives the "announce it now" behaviour), and the HACS integration is an optional convenience layer I'll add only if I later want to arm/disarm BI profiles from inside HA. Video is neither of these — that's RTSP/MJPEG substreams pulled into HA, handled separately.

Enable MQTT auth — now, but step by step

The broker currently runs mosquitto -c /mosquitto-no-auth.conf. That was a deliberate shortcut with a standing todo to fix it. The real exposure isn't Blue Iris (which only publishes) — it's that anything on the LAN can publish to Zigbee2MQTT and control my Zigbee devices, and read all home state, with no credentials. That's fine right up until something hostile gets on the LAN.

Adding Blue Iris as a new MQTT client was the nudge to finally close that todo. The catch: the cutover to allow_anonymous false briefly knocks HA's MQTT and Zigbee2MQTT offline until each client has its new credentials — so if I fat-finger a password, existing automations go dark. Decision: enable auth, but incrementally, verifying after each step rather than flipping everything at once.

Architecture (target)

Two independent channels:

Events: Blue Iris → MQTT (authenticated Mosquitto) → HA binary_sensors → automations.
Video: Blue Iris RTSP/MJPEG substreams → HA (via go2rtc/WebRTC for smooth playback) → dashboard cards + snapshot-on-alert.

Then the payoff: HA automation fires → TTS announcement on the kitchen tablet + wake screen

show the triggering camera.

Safety net: a backup script first

Before flipping any auth, I wanted a one-command rollback point. The stack's real state isn't in git — the compose.yaml is, but the data it mounts (HA config, MQTT config + persistence, Zigbee2MQTT's coordinator state, ESPHome) is all gitignored and lives only on the server. The scary one is zigbee2mqtt/data/: lose the coordinator state / network key and you re-pair every Zigbee device by hand.

So I wrote home_automation/backup.sh. It tars the four data directories into a timestamped archive:

./backup.sh          # cold: docker compose stop -> tar -> start (consistent snapshot)
./backup.sh --hot    # no downtime, may catch a DB mid-write

Cold is the default because SQLite (HA history) and the Z2M database can be mid-write during a hot copy, and the stack restarts in seconds anyway. It skips any directory that isn't present rather than failing, keeps the newest KEEP archives (default 7), and will scp offsite if OFFSITE is set — because a backup on the same disk isn't really a backup. HA's own Settings → System → Backups is a nice bonus copy, but on a Container install it only covers HA itself, so the tarball is the real net.

Enabling MQTT auth, step by step

I decided to do the cutover as discrete, individually-verified steps rather than one big flip, in this order so existing automations never go dark:

File prep (local) → 2. create passwd users (broker still no-auth) → 3. pre-seed HA + Z2M credentials while anonymous is still allowed → 4. flip the broker to auth + restart →
add Blue Iris. The broker only starts requiring auth in step 4, by which point every existing client already has credentials.

Step 1 — file prep (done)

Poking at the actual files turned up the dead auth config described above. The fixes:

git mv mqtt/config/mosquitto.config mqtt/config/mosquitto.conf — so a mosquitto -c /mosquitto/config/mosquitto.conf command will actually find it.
Added the missing websockets listener so the config matches the port compose already publishes:
```
# MQTT over websockets (compose publishes 9001)
listener 9001
protocol websockets
```
Added home_automation/mqtt/config/passwd to .gitignore. The passwd file holds password hashes, not plaintext, but it's still not something to push to a repo.

No broker change in this step — it's still literally running -c /mosquitto-no-auth.conf. Zero downtime.

Step 2 — create the passwd users (done)

Chose per-client users (ha, z2m, blueiris) over one shared account, so any single client can be rotated or revoked without disturbing the others. The commands exec into the still-running (no-auth) mqtt container and write the passwd file the broker isn't reading yet — so still no downtime:

docker compose exec mqtt mosquitto_passwd -c /mosquitto/config/passwd ha
docker compose exec mqtt mosquitto_passwd    /mosquitto/config/passwd z2m
docker compose exec mqtt mosquitto_passwd    /mosquitto/config/passwd blueiris

Only the first gets -c (create) — -c on a later line would wipe the earlier users.

The gotcha I'd flagged actually fired, right on cue:

Warning: File /mosquitto/config/passwd has world readable permissions.
Future versions will refuse to load this file.

mosquitto_passwd created the file world-readable. Fixed it before Step 4 so the broker loads cleanly — and made sure the mosquitto user still owns it, otherwise locking it to owner-only would lock the broker itself out:

docker compose exec -u root mqtt sh -c "chown mosquitto:mosquitto /mosquitto/config/passwd && chmod 0700 /mosquitto/config/passwd && ls -l /mosquitto/config/passwd"

Operational note: this server needs sudo on every Docker command (sudo docker compose …). Nothing about the plan changes — just a prefix — but it's why the first backup.sh run landed in /root/… before I hardcoded the backup path.

Step 3 — pre-seed HA + Z2M credentials (broker still anonymous)

The whole point of the ordering: give every existing client its credentials before the broker starts demanding them, so nothing drops when I flip. Two different clients, two different mechanisms:

Zigbee2MQTT — turned out its configuration.yaml is git-tracked and holds the network key, so I refused to put the MQTT password there. Instead added an env_file: z2m.env to the z2m service; z2m maps ZIGBEE2MQTT_CONFIG_MQTT_USER / ZIGBEE2MQTT_CONFIG_MQTT_PASSWORD onto its mqtt.* config. Committed a z2m.env.example for documentation; the real z2m.env stays on the server (gitignored). Then docker compose up -d zigbee2mqtt — targeting only z2m so the mqtt broker isn't recreated and stays on its old no-auth config.
Home Assistant — its MQTT is a UI config entry (.storage), not a YAML block, so no file edits: Settings → Devices & Services → MQTT → Reconfigure, add username ha + password.

The honest caveat I keep reminding myself: while the broker is still anonymous it will happily accept these clients even with a wrong password. So "connected" here only proves the plumbing — the passwords don't actually get tested until Step 4. That's fine because the flip is instantly reversible, but it means Step 4 is the one to watch the logs on.

Step 4 — the flip (done)

Two file edits: the compose command: went from mosquitto -c /mosquitto-no-auth.conf to mosquitto -c /mosquitto/config/mosquitto.conf, and I added log_dest stdout to the config so docker compose logs mqtt would actually show output during the cutover (it was only logging to a file). scp'd both up, deleted the stale old-named mosquitto.config on the server, then recreated only the broker:

sudo docker compose up -d mqtt

The verification I actually trust: Zigbee2MQTT reconnecting. With the broker now refusing anonymous clients, z2m connecting at all is the proof its credentials are right:

zigbee2mqtt-1  | [2026-07-05 10:18:31] info: z2m: Connected to MQTT server
zigbee2mqtt-1  | [2026-07-05 10:18:31] info: z2m:mqtt: MQTT publish: topic
                 'zigbee2mqtt/bridge/state', payload '{"state":"online"}'

...followed by every device republishing its state (lights, power plugs, the kitchen door sensor, the laundry water-leak sensor, the grow lights). Anonymous connections now get Connection Refused: not authorised; correct credentials publish fine. The pre-seed ordering meant this was a non-event — exactly the goal.

Step 5 — Blue Iris publishing, powered by CodeProject.AI (done)

Set up BI's global MQTT (server LAN IP:1883, user blueiris) and a per-camera alert action publishing JSON to BlueIris/<cam>/alert. First message across the wire:

BlueIris/GarageWide/alert
{"camera":"GarWid","type":"Motion_A","memo":"person:86%,bird:82%,Bicycle:78%,..."}

The memo is the good part, and it comes from CodeProject.AI (https://codeproject.github.io/codeproject.ai/) — a self-hosted AI server that Blue Iris hands each triggering frame to for object detection. It returns labels with confidence, BI stuffs them into the alert memo, and now they ride MQTT straight into Home Assistant. Two things I like about this:

It's local. CodeProject.AI runs on my own hardware — no Rekognition, no Google Vision, nothing phoning a cloud. That's the same posture as the rest of this homelab, and it's a big reason the tablet was never going to be an Echo Show.
type:Motion_A = motion with AI. HA isn't getting "something moved," it's getting "something moved, and the model is 86% sure it's a person." That confidence is what lets the automations be picky instead of spammy — and because BI already applies its own CodeProject.AI confidence threshold before firing, the memo is pre-filtered to things worth reacting to.

What CodeProject.AI decides for the next step

Since detection is already handled (and handled locally, with confidence), the HA side doesn't need to be clever about vision — it just needs to read the memo. So the next step is a thin HA layer: an MQTT binary_sensor per camera that flips on when the memo contains person, with an off_delay to auto-reset (BI sends an event, not a lasting state). Person is the primary trigger; vehicle/animal are easy to add later by matching different labels. CodeProject.AI's face-recognition and ALPR modules are a tempting future layer (known-person vs stranger, license plates) but out of scope for the first pass.

Step 6 — Home Assistant sensors from the detections (done)

Added three MQTT binary_sensors to configuration.yaml, one per category, all reading the garage cam's BlueIris/GarageWide/alert topic:

person — labels [person]
vehicle — labels [car, truck, bus, motorcycle, motorbike, bicycle]
animal — labels [bird, cat, dog, horse, sheep, cow, bear, deer, raccoon, fox, squirrel]

Each one walks the comma-separated memo, pulls the confidence for any label in its category, keeps the highest, and reports on if it's ≥ 60%. off_delay: 30 auto-resets them because Blue Iris publishes a momentary event, not a lasting state. The nice part: once the person sensor's template worked, vehicle and animal were copy-paste with a different label list — the CodeProject.AI memo does all the real work.

The config-check landmine. Before any of this would load, Check Configuration failed with Integration 'panel_iframe' not found. That block had nothing to do with my changes — panel_iframe was removed from Home Assistant a while back, and the entry had been sitting there silently broken (it even had a double-http:// URL pointing at the Zigbee2MQTT port, not an MQTT explorer that doesn't run here). Removed it and wrote a todo to re-add such a link the modern way (a dashboard Webpage card) if it's ever wanted. Classic case of a "simple" edit forcing you to clean up someone-past-me's mess first.

Step 7 — The kitchen tablet (in progress)

The whole reason for the project. Before installing anything I made a dedicated Home Assistant user for the tablet — non-admin, and set to "can only log in from the local network." A wall-mounted tablet is a physically-exposed device that stays logged in forever; it should never be carrying my admin credentials, and binding it to its own user means I can revoke it wholesale (and its long-lived tokens) if it's ever lost, without touching my account.

Then installed the Home Assistant Companion app on the Galaxy Tab A11+ and signed in as that user — it registered cleanly as Kitchen Tablet (denying location, since a wall tablet doesn't need presence). That gives a notify.mobile_app_kitchen_tablet service and the usual device sensors.

Stopped here for the session. Remaining tablet work:

Fully Kiosk Browser + its HA integration → gives a media_player.<tablet> and a screen-on switch (what an announcement actually needs: wake + speak).
A TTS engine — leaning Google Translate TTS to prove it fast, then Piper for a fully local voice consistent with the rest of the homelab.
The automation: binary_sensor.garage_wide_person → on → wake screen + announce.

Step 8 — The tablet's control surfaces: Fully Kiosk (REST-only, done)

Step 7 onboarded the tablet; Step 8 gives Home Assistant real control of it. An announcement needs two things the Companion app doesn't provide: a way to turn the screen on (a wall tablet is asleep most of the day) and a media_player to speak through. Both come from Fully Kiosk Browser — the kiosk app that pins the tablet to the HA dashboard — through its official HA integration.

Decision — pay for Fully Kiosk Plus. The free version's remote admin is too thin; the integration's screen switch and media_player lean on Plus features. One-time ~€8 per device, and this is the device the whole project exists for — easy yes.

The MQTT flop. I nearly over-built this step. My first plan was "REST and MQTT": HA would drive the tablet over Fully Kiosk's REST admin API, and Fully Kiosk would publish state to my Mosquitto broker so HA got instant push instead of ~30s polling. I talked myself through what MQTT buys with the tablet camera left out — push state, an instant "unplugged" alert — decided it was "cheap now, annoying to retrofit," and went and created a tablet broker user for it (the same careful SIGHUP-reload dance as the Step 4 cutover; the mosquitto_passwd also threw a "file not owned by root" warning, which turned out to be a non-fatal future-version nag — the broker runs as the mosquitto user, so chown root would actually break it, and I left it alone).

Then I read the integration docs before configuring the tablet, and the premise fell apart. The official Fully Kiosk integration is REST/polling only — it does not consume MQTT. Fully Kiosk can publish to MQTT, but on the HA side that's a separate channel: you either enable Fully Kiosk's MQTT auto-discovery (which spawns a second, parallel set of entities beside the REST ones) or hand-write MQTT sensors in YAML. It was never "the same entities, just faster." And the two things the announcement actually needs — the media_player and the screen switch — come from the REST integration regardless of MQTT.

So I asked what push would even buy once I dropped the unplugged alert: nothing this project consumes. MQTT push only speeds up HA learning about tablet-originated state (screen tapped, screensaver started), and I have no automation that triggers on any of that. Commands HA sends (wake, speak) are REST either way and fire instantly. Faster updates of states nothing reads, in exchange for a duplicate entity set to maintain — cost with no benefit. Killed it. Step 8 is REST-only.

The tablet broker user isn't wasted — it sits idle, ready for the day I want a tablet-state-triggered automation and flip on Fully Kiosk's MQTT discovery then. Lesson (re)learned: read an integration's actual contract before provisioning infrastructure for it. The entire MQTT limb got built on an assumption a two-minute docs read would have killed.

The plan, REST-only:

Tablet — activate Plus, point Fully Kiosk's start URL at the HA dashboard signed in as the Kitchen Tablet user, enable Remote Administration + a password (the port-2323 admin password HA authenticates with), set kiosk behavior (stay awake on charge, screen-off timeout, camera/motion off). No MQTT.
Home Assistant — add the Fully Kiosk Browser integration (tablet IP + port 2323 + the admin password). It spins up media_player.kitchen_tablet, the screen switch, and battery/storage/connectivity sensors, plus a TTS service and custom services (launch app, load URL).

Done. Added the integration against 192.168.3.54:2323; it created the device and — the one gotcha — named its entities after the tablet's Fully Kiosk device name ("Blue Iris Alert's Tab A11"), so the announcement output landed as media_player.blue_iris_alerts_s_tab_a11, not the tidy kitchen_tablet I'd assumed in the plan. The screen switch physically wakes and sleeps the tablet from HA — the one proof that actually matters — and the media_player is live. That's the whole control surface the announcement needs, and it came together with zero broker involvement, which is the flop above paying its dividend. The voice (a TTS engine) and the "garage person → wake + speak" automation are Step 9.

Step 9 — Giving it a voice: the announcement (Google TTS)

The payoff. Two pieces: pick a text-to-speech engine, then write the automation that ties the whole pipeline to the tablet.

TTS choice — Google first, Piper next. Both are free. Piper is the on-brand answer — a local neural TTS engine, nothing leaves the LAN, consistent with the whole all-local thesis — but it needs a container + the Wyoming integration stood up. Google Translate TTS is built into HA with zero setup (it is a cloud call, a snippet of text per announcement), so I used it to prove the chain end-to-end first, with a clean swap to Piper as the follow-up. Adding it was one click: Add Integration → Google Translate Text-to-Speech → tts.google_translate_en_com.

The automation (hand-written into the git-tracked automations.yaml, scp'd to the server, reloaded): on binary_sensor.garage_wide_person → on, it wakes the tablet screen (switch.blue_iris_alerts_s_tab_a11_screen), sets volume to 0.8 so a quiet tablet doesn't swallow it, and speaks "Person entering driveway." through media_player.blue_iris_alerts_s_tab_a11. A trailing delay: 10s with mode: single gives a ~10-second cooldown so a flapping sensor doesn't announce on repeat.

The one snag — "wrong url or unsupported sound." First test, the tablet threw exactly that. It's the classic Fully-Kiosk-TTS trap: the error comes from Fully Kiosk, not TTS — the audio rendered fine, but HA had handed the tablet a sound URL it couldn't reach. HA builds the TTS proxy URL from its configured internal URL, which wasn't a tablet-reachable address. Fix: Settings → System → Network → set the internal URL to http://192.168.4.2:8123 (the LAN address the tablet already uses). Re-ran it and the kitchen tablet said, out loud, "Person entering driveway."

That closes the loop the whole project was for: camera → CodeProject.AI → MQTT → HA sensor → spoken announcement on a wall tablet, entirely local. Still to do: a real walk-into-the-driveway trigger test (the manual Run proves the actions; the sensor trigger is already proven from Step 6), then swap Google → Piper for a fully-local voice.

Step 10 — Front-camera grid on detection (working; live video an open issue)

Now that the tablet talks, the next want: when something shows up out front, don't just say it — show it. Any object (person, vehicle, or animal) on any of the three front cameras (GarageWide, FrontDoor, FrontYard) now pops a live 3-across grid of all three on the kitchen tablet, announces what it saw and where, and clears itself 30 seconds after the last alert.

Decisions (a couple were course-corrections):

Display = a conditional dashboard card, not a popup. Weighed browser_mod (a HACS overlay popup, the "standard" way to do camera-on-detection) against a plain conditional Lovelace card gated on a timer. Went conditional card — zero dependencies, fewest moving parts, hardest to break — in keeping with how this project has gone. A single timer.front_alert helper drives it: each alert restarts it (30s), the card shows while it's active.
Streams = WebRTC via HA's built-in go2rtc, main stream — which turned out to be H.264, so no need for the sub-stream I'd been fretting about.
Voice folded Step 9 in, and went per-camera. One automation now handles all three front cams and every object type, speaking the location: GarageWide→"driveway", FrontDoor→"front door", FrontYard→"front yard", prefixed with the object ("Person at the front door."). Debounced ~15s so continuous activity doesn't chatter, while the grid window still extends on every alert.
Sensors generalized to all three front cams — also clearing the old "generalize the sensors to other cameras" backlog item.

The HA build was quick; the plumbing was the saga. The YAML — 6 new sensors, 2 timers, the merged automation — went in and validated on the first try. Then the cameras turned into a proper homelab archaeology dig:

RTSP short names ≠ MQTT topic names. The topic names I'd chosen (GarageWide/FrontYard/ FrontDoor) aren't the Blue Iris camera short names the RTSP path needs — those were GarWid / wwest / FrontDoor, sitting in plain sight in the alert JSON's camera field the whole time.
Port 554 was a red herring. ffprobe timed out on 554 because Blue Iris serves RTSP on its web-server port (81), not 554 — there's no separate "enable RTSP" toggle to hunt for.
The real wall: VLAN segmentation. Even on 81, HA's preview wouldn't load though my Mac streamed it fine — because HA and the cameras live on different pfSense VLANs (WebSrv 192.168.4.x vs CAMERAS 192.168.6.x), and the firewall blocks cross-segment by default. One pass rule (HA host → BI host, TCP), placed above the "block home network" rule, and a curl from HA came back 302 — streams alive. The camera-VLAN isolation is deliberate (those cheap cams get no internet and can reach nothing); I just poked a single inbound hole for HA.
The small stuff: the Generic Camera entities auto-named themselves camera.192_168_6_5 (renamed to camera.driveway / front_door / front_yard); a dashboard "no card type" error from a YAML indent slip (fixed by pasting a clean full config); and — the one I should've seen coming — I couldn't edit the dashboard on the tablet at all, because in Step 7 I made that user non-admin on purpose. That's the security working, not a bug: dashboards get built from my admin account, the tablet only ever views.

End result (mostly there): walk into the driveway → the tablet wakes, says "Person at the driveway," and the 3-camera grid pops for 30 seconds. The detection → announce → grid-pops chain works end to end. The one thing not yet right: the grid tiles show a static snapshot, not live video — the open issue below.

Version-control note: the sensors/timers/automation are tracked YAML (committed). The camera entities and the dashboard live in HA's .storage on purpose — the RTSP URLs carry the streaming password, so keeping them UI-side keeps that secret out of git.

Open issue — the grid tiles aren't live yet. The card pops and shows each camera, but as a still image (it loads after a ~30s stall), not a moving feed. Debugging so far:

Not the source, not the firewall. Proved HA can pull the RTSP straight from the container over TCP (ffmpeg counted 99 frames in 5s). The earlier cross-VLAN block was real and is fixed — HA sits on the WebSrv VLAN, the cameras on their own isolated VLAN, and one pfSense pass rule opened HA→cameras.
Not resolution alone. The main stream is a hefty 2376×1080; a lighter 1184×536 variant (Blue Iris &decode=-1) still came up static.
Current read: go2rtc is falling back to MJPEG (its log spams broken-pipe writes to the browser) because the efficient live paths — WebRTC / MSE, which would push the compressed H.264 straight over :8123 — aren't taking. Prime suspect: the stream's oddball 8 kHz AAC audio breaking MSE. Next test is stripping the audio at Blue Iris; if that's not it, the escalation is the AlexxIT WebRTC/go2rtc HACS integration for full transport/audio control (which means walking back "no HACS," but only for streaming).
Secondary quirk: on an alert the tablet sometimes reloads to the default Overview dashboard instead of staying on ours (waking the screen alone doesn't do it, so it's tied to the grid/stream render). Likely fixed by pinning Fully Kiosk's start URL to the explicit dashboard path and setting that user's default dashboard.

Parking it here — the pipeline and the pop-up work; making the tiles genuinely live is next session's job.

What's next

Fix the live grid tiles (top priority): test stripping the cameras' 8 kHz audio at Blue Iris; if that's not it, adopt the AlexxIT WebRTC/go2rtc HACS integration for full stream control. Also pin the tablet to its dashboard so an alert doesn't bounce it to Overview.
Swap Google TTS → Piper (local, on-brand) via a Wyoming container — no more cloud calls.
Later: face recognition / ALPR from CodeProject.AI; rotate the Zigbee network key out of git.
Backlog: rotate the Zigbee network key out of git-tracked configuration.yaml; re-add the MQTT Explorer link (todo 03); maybe add CodeProject.AI face recognition / ALPR later.

Honest take

The decisions were the hard part, and the useful move was refusing to treat "MQTT vs the integration" as an either/or — splitting events from video from control instead. Once I actually opened the files, the payoff was immediate: there was a fully-written auth config sitting there doing nothing because of a one-character filename mismatch and a stale compose command. Classic homelab archaeology. I'm deliberately not rushing the auth cutover — it's the step most likely to silently break automations I already rely on, so it's going one verified step at a time.

Looking back at the whole session: the theme was that the "real" task (Blue Iris → tablet) kept unearthing pre-existing debt that had to be cleared first — a dead auth config, a network key committed to git, a removed panel_iframe integration silently failing a config check. None of it was the thing I set out to do, but ignoring any of it would have bitten me later. The staged, verify-after-each-step approach to the MQTT cutover was the standout: zero downtime, and when I flipped the broker, Zigbee2MQTT just reconnected like nothing happened. Ended the session with the pipeline fully working end-to-end (camera → AI → MQTT → HA sensor) and the tablet half-onboarded — the fun part, making it talk, is next time.