One sentence to a room microphone, and the right TV wakes, Moonfin comes to the foreground, and the episode you were part-way through resumes from where you left it — playback verified before the assistant says a word. This page is the complete recipe: the architecture, the measured client behaviour that shaped it, and the full sanitised code, ready to adapt to your own house.
Five hops, each with one job. The voice agent chooses a title and a room; everything below it is deterministic.
| Hop | What happens |
|---|---|
| 1 · Voice | A room microphone hears “play Castle on the TV”. The local voice agent picks the tool and fills two fields: title and target. |
| 2 · HA script | script.play_jellyfin — the tool the agent sees. Its field descriptions carry the routing rules (which room maps to which target, how to spell titles), and it relays the script’s result sentence back as the spoken reply. |
| 3 · Shell command | shell_command.play_jellyfin runs the script synchronously — Home Assistant waits, so the agent cannot answer before the truth is known. |
| 4 · The script | play_jellyfin.sh: finds the item (resume → series → movie → fuzzy hints), wakes the TV, launches Moonfin if needed, sends the play command over the Jellyfin Sessions API, and verifies playback before reporting. |
| 5 · Moonfin | The client on each TV. Moonfin supports Jellyfin remote control on both Android TV and tvOS — the reason it is the client of choice for a voice-driven house. |
SupportsRemoteControl: true on both Apple TV
and Android TV — the capability this whole page rides on. It is a
polished, actively developed Jellyfin client, and playback started remotely lands
exactly where a voice assistant needs it: on the TV, resumable, controllable.Home Assistant’s Jellyfin integration creates one
media_player per session, and the entity’s
unique ID embeds the session ID — which changes at every installation and
whenever a client app is reinstalled. Great for a dashboard; wrong shape for a
voice pipeline you want to reproduce in more than one house.
The scripted route talks to the Jellyfin Sessions API directly and
resolves targets by client name + device name — strings that are
stable across reinstalls and identical in every house running the same apps.
The result is a clean split: two scripts that never change, one small config
file that holds everything site-specific. Setting up a new house is: deploy
the scripts, open Moonfin on each TV, run --sessions,
paste the strings.
# jellyfin-targets.conf — per-installation target map
#
# THIS IS THE ONLY FILE THAT SHOULD DIFFER BETWEEN SITES.
#
# Sourced by play_jellyfin.sh and jellyfin-transport.sh. Deliberately contains
# no entity IDs derived from Jellyfin session IDs — those change per install
# and per app reinstall. Targets are resolved from /Sessions by client +
# device name.
#
# To fill this in on a new site:
# 1. Open Moonfin on each device.
# 2. Run: ~/scripts/play_jellyfin.sh --sessions
# 3. Copy the exact CLIENT and DEVICE strings into the case block below.
# 4. For LAUNCH_APP_ID (Android TV) use the app deep link. For tvOS,
# LAUNCH_SOURCE is the app name from the Apple TV entity's source_list.
JELLYFIN_URL="${JELLYFIN_URL:-http://192.168.1.20:8096}" # your server
JELLYFIN_USERNAME="${JELLYFIN_USERNAME:-your-user}" # library user
HA_URL="${HA_URL:-http://192.168.1.10:8123}" # Home Assistant
# Sets the per-target variables. Returns 1 for an unknown target.
jellyfin_target() {
case "$1" in
appletv)
SESSION_CLIENT="Moonfin for tvOS"
SESSION_DEVICE="Apple TV"
LAUNCH_ENTITY="media_player.apple_tv"
LAUNCH_METHOD="appletv_source"
LAUNCH_SOURCE="Moonfin"
LAUNCH_APP_ID=""
LAUNCH_REMOTE=""
# Moonfin tvOS is slow twice over: ~17s to surface NowPlayingItem,
# and ~15s reporting IsPaused with a frozen position while it is
# genuinely playing. Measured. Do not lower without re-measuring.
VERIFY_WINDOW=30
ROOM_LABEL="living room"
;;
chromecast)
SESSION_CLIENT="Moonfin for Android TV"
SESSION_DEVICE="Google Chromecast"
LAUNCH_ENTITY="media_player.bedroom_tv"
LAUNCH_METHOD="androidtv_app"
LAUNCH_SOURCE=""
# Use the app's DEEP LINK, not the bare package name. The Android
# TV remote integration wraps bare packages in market://launch,
# which the Play Store may refuse for sideloaded apps. Moonfin
# registers the "moonfin" scheme; an id containing :// is passed
# through as a direct VIEW intent, Play not involved.
LAUNCH_APP_ID="moonfin://"
# Android TV Remote entity for the SAME device: used to send HOME
# before a relaunch, forcing a real background-to-foreground
# cycle. A repeated launch intent at an already-running Moonfin
# is ignored; the resume event is what makes >=2.5.0 reconnect
# and re-post its remote-control capabilities.
LAUNCH_REMOTE="remote.bedroom_tv"
# Measured: surfaces its session in ~6s and reports state honestly.
VERIFY_WINDOW=16
ROOM_LABEL="bedroom"
;;
*)
return 1
;;
esac
return 0
}
# Every target name this site supports, for usage messages.
JELLYFIN_TARGETS="appletv chromecast"
Every one of these was earned by measuring real clients rather than trusting documentation — and each is written into the script as a comment at the exact line it shaped, so the code carries its own reasoning.
IsPaused: true with a frozen position for the
first ~15 seconds of genuine playback, and a backgrounded Android TV
ticks forward invisibly before pausing itself. A single reading cannot tell
these apart — so the test is two consecutive “playing”
samples: a backgrounded client degrades, a real one holds. Position is
deliberately excluded (tvOS was observed ticking backwards while playing).AMBIGUOUS rather than guessing — playing in the
wrong room is worse than not playing.moonfin:// goes straight to the app as a VIEW intent.
Launching by bare package name routes through the Play Store, which can refuse
sideloaded apps — the deep link works every time.| Client | Session appears in | State reporting | VERIFY_WINDOW |
|---|---|---|---|
| Moonfin for Android TV | ~6 s | honest from the start | 16 s |
| Moonfin for tvOS | ~17 s | reports paused for ~15 s of real playback | 30 s |
Cold start end-to-end — device asleep, app closed — measured at 20–25 seconds worst case: well inside the budget, and the reason the per-target windows are config, not constants.
The complete play_jellyfin.sh, sanitised
only in names and addresses. Pure REST — curl and
python3, no SDK, no venv. Secrets
(JELLYFINAPI, HA_TOKEN) live
in ~/.config/secrets.env; swap the alert line in
fail_loud for your own notifier.
#!/bin/bash
# play_jellyfin.sh - Play Jellyfin content on a Moonfin client via the Jellyfin
# Sessions remote-control API.
#
# Usage:
# play_jellyfin.sh "Castle" appletv
# play_jellyfin.sh "The Rookie" chromecast
# play_jellyfin.sh "John Wick movie" appletv
# play_jellyfin.sh --sessions (list live sessions, for config)
#
# ── Design properties ────────────────────────────────────────────────────────
# - fail-loud: each fatal stop sends a distinct alert (swap in your own
# alerting in fail_loud below)
# - distinct stops: server unreachable vs no match vs no session vs command
# rejected vs never started vs started-then-died
# - honest-synchronous: Home Assistant waits for this script, so the voice
# agent only claims playback after it is VERIFIED; failures surface as its
# spoken reply
# - two-phase verification with a sustain re-check (a session that appears
# and dies in a second is a real failure mode and must NOT verify)
# - global time budget keeps the run inside HA's 60s shell_command kill window
# - search ladder over query variants (STT says "and", libraries use "&")
# - last stdout line is machine-readable: "PLAYING: ..." or "FAILED: ..."
#
# ── What is structurally different from Plex, and matters ────────────────────
# Plex Companion registers a discoverable client on :32500 that answers even
# when the app is backgrounded. Jellyfin has no equivalent: a session exists
# ONLY while the client app is running and connected. There is no port to poll.
#
# Consequences:
# 1. Readiness is "does a matching session appear in /Sessions", not "is a
# port open" — and unlike a port, a Jellyfin session is a truthful signal.
# 2. If the app is not running we must launch it first; there is no fallback,
# so the launch config below is mandatory.
# 3. Cold start is slower, because the app must fully connect to the server
# before its session registers.
#
# Ticks: Jellyfin positions are 100-nanosecond ticks. 1 ms = 10,000 ticks.
set -Eeuo pipefail
SECRETS="$HOME/.config/secrets.env" # holds JELLYFINAPI and HA_TOKEN
LOG="$HOME/logs/play-jellyfin.log"
BUDGET=50 # seconds; HA kills shell_command at 60s, HA-side pre-steps eat ~6s.
# A cold tvOS start needs ~30s of launch-wait AND up to
# VERIFY_WINDOW of verify; 45 could not hold both.
mkdir -p "$HOME/logs"
log() {
echo "[$(date '+%H:%M:%S')] $*" | tee -a "$LOG"
}
# Fail loud: log, alert, terse machine-readable last line, non-zero exit.
fail_loud() {
local reason="$1"
local spoken="${2:-Could not play ${TITLE:-that} from Jellyfin.}"
log "ERROR: $reason"
# Substitute your own alerting (ntfy, Telegram, email...):
"$HOME/scripts/alert-send.sh" "play_jellyfin: ${reason} — title='${TITLE:-?}' target=${TARGET:-?}" \
>/dev/null 2>>"$LOG" || log "WARN: alert also failed"
echo "FAILED: $spoken"
exit 1
}
if [[ ! -f "$SECRETS" ]]; then
log "ERROR: Secrets file not found: $SECRETS"
exit 1
fi
source "$SECRETS"
TITLE="${1:-}"
TARGET="${2:-}"
# Per-site config: server URL, library user, HA URL, and the target map.
# The ONLY file that should differ between installations.
TARGETS_CONF="$HOME/scripts/jellyfin-targets.conf"
[[ -f "$TARGETS_CONF" ]] || { echo "FAILED: jellyfin-targets.conf missing."; exit 1; }
source "$TARGETS_CONF"
: "${JELLYFINAPI:?JELLYFINAPI is missing from secrets.env}"
# Discovery helper: prints every live session so the target map can be filled
# in with the exact Client / DeviceName strings. Open Moonfin on the device
# first, then: play_jellyfin.sh --sessions
if [[ "$TITLE" == "--sessions" ]]; then
curl -sS -f -H "Authorization: MediaBrowser Token=\"${JELLYFINAPI}\", Client=\"MorisonicsVoice\", Device=\"voice-host\", DeviceId=\"morisonics-voice-1\", Version=\"1.0\"" -H "Accept: application/json" \
--max-time 15 "${JELLYFIN_URL}/Sessions" \
| python3 -c '
import json, sys
rows = json.load(sys.stdin)
if not rows:
print("(no sessions — open the app on the device first)"); sys.exit(0)
print("%-28s %-22s %-6s %s" % ("CLIENT", "DEVICE", "REMOTE", "SESSION ID"))
for s in rows:
print("%-28s %-22s %-6s %s" % (
(s.get("Client") or "")[:28],
(s.get("DeviceName") or "")[:22],
"yes" if s.get("SupportsRemoteControl") else "no",
s.get("Id", "")))
'
exit 0
fi
[[ -z "$TITLE" || -z "$TARGET" ]] && {
log "Usage: play_jellyfin.sh <title> <appletv|chromecast>"
log " play_jellyfin.sh --sessions (list live sessions)"
fail_loud "called without title/target (usage error)"
}
: "${HA_TOKEN:?HA_TOKEN is missing from secrets.env}"
LAUNCHED=0
jellyfin_target "$TARGET" || fail_loud "unknown target: $TARGET (known: ${JELLYFIN_TARGETS})"
jf() {
# jf <path-with-query> [curl args...]
local path="$1"; shift
curl -sS -f -H "Authorization: MediaBrowser Token=\"${JELLYFINAPI}\", Client=\"MorisonicsVoice\", Device=\"voice-host\", DeviceId=\"morisonics-voice-1\", Version=\"1.0\"" -H "Accept: application/json" \
--max-time 15 "${JELLYFIN_URL}${path}" "$@" 2>>"$LOG"
}
urlencode() {
python3 -c "import urllib.parse,sys; print(urllib.parse.quote(sys.argv[1]))" "$1"
}
# Set whenever any Jellyfin endpoint answers — distinguishes "no match" from
# "server unreachable".
JF_HTTP_OK=0
# ── Resolve the library user ─────────────────────────────────────────────────
USERS_JSON="$(jf "/Users")" || fail_loud "Jellyfin server unreachable (/Users failed)" \
"The Jellyfin server is not responding."
JF_HTTP_OK=1
USER_ID=$(DATA="$USERS_JSON" NAME="$JELLYFIN_USERNAME" python3 -c '
import json, os, sys
users = json.loads(os.environ["DATA"])
want = os.environ["NAME"].strip().lower()
for u in users:
if (u.get("Name") or "").strip().lower() == want:
print(u.get("Id", "")); sys.exit(0)
# Fall back to the first administrator, then simply the first user.
for u in users:
if (u.get("Policy") or {}).get("IsAdministrator"):
print(u.get("Id", "")); sys.exit(0)
print(users[0].get("Id", "") if users else "")
')
[[ -n "$USER_ID" ]] || fail_loud "could not resolve a Jellyfin user id (looked for '${JELLYFIN_USERNAME}')"
log "User: ${JELLYFIN_USERNAME} (${USER_ID})"
# ── Intent hints ─────────────────────────────────────────────────────────────
LOWER_TITLE="$(echo "$TITLE" | tr '[:upper:]' '[:lower:]')"
PREFER_MOVIE=0
if [[ "$LOWER_TITLE" =~ (^|[[:space:]])(movie|film)([[:space:]]|$) ]]; then
PREFER_MOVIE=1
fi
SEARCH_TITLE=$(python3 -c '
import re, sys
s = sys.argv[1]
s = re.sub(r"\b(tv show|movie|film|series|show|tv)\b", "", s, flags=re.I)
s = " ".join(s.split())
print(s if s else sys.argv[1])
' "$TITLE")
log "Looking for '$TITLE' → search='$SEARCH_TITLE' → $TARGET"
mapfile -t QUERY_VARIANTS < <(python3 -c '
import re, sys
s = sys.argv[1]
out = []
def add(v):
v = " ".join(v.split())
if v and v.lower() not in [o.lower() for o in out]:
out.append(v)
add(s)
add(re.sub(r"\band\b", "&", s, flags=re.I))
add(s.replace("&", " and "))
add(re.sub(r"[^\w\s&]", " ", s))
print("\n".join(out))
' "$SEARCH_TITLE")
ITEM=""
# ── Pickers ──────────────────────────────────────────────────────────────────
# One shared normaliser: lowercase, & ↔ and, leading "the" dropped,
# whitespace collapsed.
PICKER_PRELUDE='
import json, os, re, sys
def norm(s):
s = (s or "").lower().strip()
s = s.replace("&", " and ")
s = re.sub(r"^the\s+", "", s)
return " ".join(s.split())
def rank(items, query, key):
exact = [i for i in items if norm(key(i)) == query]
contains = [i for i in items if query in norm(key(i)) or norm(key(i)) in query]
return exact or contains or items
'
choose_by_type() {
# choose_by_type <json> <query> <Type>
DATA="$1" QUERY="$2" WANT="$3" python3 -c "${PICKER_PRELUDE}"'
try:
data = json.loads(os.environ["DATA"])
except Exception:
sys.exit(0)
query = norm(os.environ["QUERY"])
want = os.environ["WANT"]
items = [i for i in (data.get("Items") or []) if i.get("Type") == want]
if not items:
sys.exit(0)
print(json.dumps(rank(items, query, lambda i: i.get("Name"))[0]))
'
}
choose_hint() {
# choose_hint <json from /Search/Hints> <query>
DATA="$1" QUERY="$2" python3 -c "${PICKER_PRELUDE}"'
try:
data = json.loads(os.environ["DATA"])
except Exception:
sys.exit(0)
query = norm(os.environ["QUERY"])
hints = data.get("SearchHints") or []
if not hints:
sys.exit(0)
movies = [h for h in hints if h.get("Type") == "Movie"]
series = [h for h in hints if h.get("Type") == "Series"]
episodes = [h for h in hints if h.get("Type") == "Episode"]
movie_exact = [h for h in movies if norm(h.get("Name")) == query]
episode_exact = [h for h in episodes if norm(h.get("Name")) == query]
series_exact = [h for h in series if norm(h.get("Name")) == query]
ep_show_exact = [h for h in episodes if norm(h.get("SeriesName")) == query]
movie_part = [h for h in movies if query in norm(h.get("Name")) or norm(h.get("Name")) in query]
series_part = [h for h in series if query in norm(h.get("Name")) or norm(h.get("Name")) in query]
episode_part = [h for h in episodes if query in norm(h.get("Name")) or norm(h.get("Name")) in query]
playable = [h for h in hints if h.get("Type") in ("Movie", "Series", "Episode")]
chosen = (movie_exact or series_exact or ep_show_exact or episode_exact
or movie_part or series_part or episode_part or playable)
if not chosen:
sys.exit(0)
h = chosen[0]
# Normalise a hint into an item-shaped dict.
print(json.dumps({
"Id": h.get("ItemId") or h.get("Id"),
"Name": h.get("Name"),
"Type": h.get("Type"),
"SeriesName": h.get("SeriesName"),
"IndexNumber": h.get("IndexNumber"),
"ParentIndexNumber": h.get("ParentIndexNumber"),
"UserData": {},
}))
'
}
# Episode selection: resume first (most recently played), then first unwatched
# in season/episode order, then the beginning.
choose_episode() {
python3 -c '
import json, sys
try:
data = json.load(sys.stdin)
except Exception as e:
print(f"EPISODE_PICKER_ERROR: {e}", file=sys.stderr); sys.exit(1)
eps = [i for i in (data.get("Items") or []) if i.get("Type") == "Episode"]
if not eps:
print("EPISODE_PICKER_ERROR: no episodes", file=sys.stderr); sys.exit(1)
def iv(v, d=0):
try: return int(v)
except Exception: return d
def ud(e, k, d=0):
return (e.get("UserData") or {}).get(k, d)
def order(e):
return (iv(e.get("ParentIndexNumber"), 9999),
iv(e.get("IndexNumber"), 9999),
str(e.get("Id", "")))
resume = [e for e in eps if iv(ud(e, "PlaybackPositionTicks")) > 0 and not ud(e, "Played", False)]
if resume:
resume.sort(key=lambda e: (str(ud(e, "LastPlayedDate", "")), iv(ud(e, "PlaybackPositionTicks"))),
reverse=True)
print(json.dumps(resume[0])); sys.exit(0)
unwatched = [e for e in eps if not ud(e, "Played", False)]
if unwatched:
unwatched.sort(key=order)
print(json.dumps(unwatched[0])); sys.exit(0)
eps.sort(key=order)
print(json.dumps(eps[0]))
'
}
# ── Search rungs ─────────────────────────────────────────────────────────────
try_resume() {
# Jellyfin's equivalent of on-deck. Non-fatal: this endpoint has moved
# between Jellyfin versions and must never be a hard stop.
local resume
resume=$(jf "/Users/${USER_ID}/Items/Resume?Limit=30&Fields=UserData&MediaTypes=Video") || {
log "WARN: Resume fetch failed — continuing with search"
return 1
}
JF_HTTP_OK=1
ITEM=$(DATA="$resume" QUERY="$SEARCH_TITLE" python3 -c "${PICKER_PRELUDE}"'
try:
data = json.loads(os.environ["DATA"])
except Exception:
sys.exit(0)
query = norm(os.environ["QUERY"])
for i in (data.get("Items") or []):
name = norm(i.get("Name"))
show = norm(i.get("SeriesName") or i.get("Name"))
if query in (name, show) or query in show or show in query or query in name or name in query:
print(json.dumps(i)); sys.exit(0)
' 2>/dev/null || true)
[[ -n "$ITEM" ]]
}
resolve_series_to_episode() {
# $1 = series item JSON. Prefers NextUp, falls back to the full episode list.
local series_item series_id series_name nextup episodes
series_item="$1"
series_id=$(DATA="$series_item" python3 -c 'import json,os; print(json.loads(os.environ["DATA"]).get("Id",""))')
series_name=$(DATA="$series_item" python3 -c 'import json,os; print(json.loads(os.environ["DATA"]).get("Name",""))')
[[ -n "$series_id" ]] || return 1
log "Matched series: $series_name ($series_id)"
# NextUp is Jellyfin's own "what should play next" — cheaper and smarter
# than reimplementing it, but it returns nothing for a never-started series.
nextup=$(jf "/Shows/NextUp?userId=${USER_ID}&seriesId=${series_id}&Fields=UserData&Limit=1") || nextup=""
if [[ -n "$nextup" ]]; then
ITEM=$(DATA="$nextup" python3 -c '
import json, os, sys
items = (json.loads(os.environ["DATA"]).get("Items") or [])
if items:
print(json.dumps(items[0]))
' 2>/dev/null || true)
if [[ -n "$ITEM" ]]; then
log "NextUp supplied the episode"
return 0
fi
fi
episodes=$(jf "/Shows/${series_id}/Episodes?userId=${USER_ID}&Fields=UserData") || {
log "WARN: could not fetch episodes for: $series_name"
return 1
}
ITEM="$(printf "%s" "$episodes" | choose_episode 2>>"$LOG" || true)"
[[ -n "$ITEM" ]]
}
try_items_search() {
# try_items_search <Series|Movie>
local want="$1" search picked
log "Searching ${want}..."
search=$(jf "/Items?userId=${USER_ID}&searchTerm=${ENCODED_TITLE}&IncludeItemTypes=${want}&Recursive=true&Limit=15&Fields=UserData") || {
log "WARN: ${want} search failed"
return 1
}
JF_HTTP_OK=1
picked="$(choose_by_type "$search" "$SEARCH_TITLE" "$want" 2>/dev/null || true)"
[[ -n "$picked" ]] || return 1
if [[ "$want" == "Series" ]]; then
resolve_series_to_episode "$picked"
else
ITEM="$picked"
fi
[[ -n "$ITEM" ]]
}
try_hint_search() {
# /Search/Hints is the fuzzier rung — and the one that matches episode titles.
local search hint hint_type
log "Trying Jellyfin search hints (matches episode titles)..."
search=$(jf "/Search/Hints?searchTerm=${ENCODED_TITLE}&userId=${USER_ID}&includeItemTypes=Movie,Series,Episode&limit=15") || {
log "WARN: hint search failed"
return 1
}
JF_HTTP_OK=1
hint="$(choose_hint "$search" "$SEARCH_TITLE" 2>/dev/null || true)"
[[ -n "$hint" ]] || return 1
hint_type=$(DATA="$hint" python3 -c 'import json,os; print(json.loads(os.environ["DATA"]).get("Type",""))')
if [[ "$hint_type" == "Series" ]]; then
resolve_series_to_episode "$hint"
else
ITEM="$hint"
fi
[[ -n "$ITEM" ]]
}
# ── 1. Resume, unless a movie was explicitly asked for ───────────────────────
if [[ "$PREFER_MOVIE" -eq 0 ]]; then
try_resume || true
fi
# ── 2. Search ladder over query variants ─────────────────────────────────────
# TV-first by default: voice requests like "The Rookie" usually mean the series.
if [[ -z "$ITEM" ]]; then
for VARIANT in "${QUERY_VARIANTS[@]}"; do
SEARCH_TITLE="$VARIANT"
ENCODED_TITLE="$(urlencode "$VARIANT")"
log "Search variant: '$VARIANT'"
if [[ "$PREFER_MOVIE" -eq 1 ]]; then
try_items_search Movie || try_items_search Series || try_hint_search || true
else
try_items_search Series || try_items_search Movie || try_hint_search || true
fi
[[ -n "$ITEM" ]] && break
done
fi
if [[ -z "$ITEM" ]]; then
if [[ "$JF_HTTP_OK" -eq 0 ]]; then
fail_loud "Jellyfin server unreachable (resume and all search endpoints failed)" \
"The Jellyfin server is not responding."
fi
fail_loud "no Jellyfin match for '$TITLE' (tried ${#QUERY_VARIANTS[@]} query variants incl. hints)" \
"Could not find ${TITLE} in the Jellyfin library."
fi
# ── 3. Extract metadata ──────────────────────────────────────────────────────
ITEM_ID=$(DATA="$ITEM" python3 -c 'import json,os; print(json.loads(os.environ["DATA"]).get("Id",""))')
ITEM_TYPE=$(DATA="$ITEM" python3 -c 'import json,os; print(json.loads(os.environ["DATA"]).get("Type",""))')
EP_TITLE=$(DATA="$ITEM" python3 -c 'import json,os; print(json.loads(os.environ["DATA"]).get("Name",""))')
SHOW_TITLE=$(DATA="$ITEM" python3 -c 'import json,os; d=json.loads(os.environ["DATA"]); print(d.get("SeriesName") or d.get("Name",""))')
SEASON=$(DATA="$ITEM" python3 -c 'import json,os; print(json.loads(os.environ["DATA"]).get("ParentIndexNumber") or "")')
EPISODE=$(DATA="$ITEM" python3 -c 'import json,os; print(json.loads(os.environ["DATA"]).get("IndexNumber") or "")')
START_TICKS=$(DATA="$ITEM" python3 -c '
import json, os
d = json.loads(os.environ["DATA"])
ud = d.get("UserData") or {}
print(int(ud.get("PlaybackPositionTicks") or 0))
')
[[ -n "$ITEM_ID" ]] || fail_loud "matched an item with no Id (unexpected API shape)"
if [[ "$ITEM_TYPE" == "Episode" ]]; then
log "Found episode: $SHOW_TITLE S${SEASON}E${EPISODE} / $EP_TITLE (ticks=${START_TICKS})"
else
log "Found ${ITEM_TYPE}: $EP_TITLE (ticks=${START_TICKS})"
fi
# ── 4. Find the target session, launching the app if needed ──────────────────
# A session in /Sessions IS the readiness signal, and it is a truthful one:
# if it is there, the app is connected.
find_session_id() {
local sessions
sessions=$(jf "/Sessions") || return 1
JF_HTTP_OK=1
DATA="$sessions" WANT_CLIENT="$SESSION_CLIENT" WANT_DEVICE="$SESSION_DEVICE" python3 -c '
import json, os, sys
try:
sessions = json.loads(os.environ["DATA"])
except Exception:
sys.exit(1)
wc = os.environ["WANT_CLIENT"].strip().lower()
wd = os.environ["WANT_DEVICE"].strip().lower()
cands = []
for s in sessions:
client = (s.get("Client") or "").strip().lower()
device = (s.get("DeviceName") or "").strip().lower()
if not s.get("SupportsRemoteControl"):
continue
if wc and wc not in client:
continue
if wd and wd not in device:
continue
cands.append((client, device, s.get("Id", "")))
if not cands:
sys.exit(1)
# Prefer an exact client-name match. "Moonfin" is a SUBSTRING of "Moonfin for
# tvOS", so a loose match would silently play to the wrong room. Wrong-room
# actuation is worse than failing: if the candidates cannot be narrowed to
# exactly one, refuse and report them.
exact = [c for c in cands if c[0] == wc] if wc else []
if wd:
exact_dev = [c for c in (exact or cands) if c[1] == wd]
if len(exact_dev) == 1:
print(exact_dev[0][2]); sys.exit(0)
pool = exact or cands
if len(pool) == 1:
print(pool[0][2]); sys.exit(0)
sys.stderr.write("AMBIGUOUS: %d sessions match client=%r device=%r -> %s\n" % (
len(pool), wc, wd, "; ".join("%s / %s" % (c[0], c[1]) for c in pool)))
sys.exit(2)
'
}
# A matching session that refuses remote control is its own failure state and
# must be NAMED, not hidden inside "no session". Observed: after an app
# update, tvOS Moonfin re-registered with SupportsRemoteControl=false until
# the Apple TV itself was restarted — the generic "did not come online" error
# turned a one-line fix into a forty-minute mystery.
find_uncontrollable() {
local sessions
sessions=$(jf "/Sessions") || return 1
DATA="$sessions" WANT_CLIENT="$SESSION_CLIENT" WANT_DEVICE="$SESSION_DEVICE" python3 -c '
import json, os, sys
try:
sessions = json.loads(os.environ["DATA"])
except Exception:
sys.exit(1)
wc = os.environ["WANT_CLIENT"].strip().lower()
wd = os.environ["WANT_DEVICE"].strip().lower()
for s in sessions:
client = (s.get("Client") or "").strip().lower()
device = (s.get("DeviceName") or "").strip().lower()
if wc and wc not in client:
continue
if wd and wd not in device:
continue
if not s.get("SupportsRemoteControl"):
print("%s / %s" % (s.get("Client"), s.get("DeviceName")))
sys.exit(0)
sys.exit(1)
'
}
ha_service() {
# ha_service <domain/service> <json body>
curl -sS -f -X POST "${HA_URL}/api/services/$1" \
-H "Authorization: Bearer ${HA_TOKEN}" \
-H "Content-Type: application/json" \
-d "$2" --max-time 10 >/dev/null 2>>"$LOG"
}
# Prints the entity's source_list — used to explain a failed select_source
# rather than leaving the operator guessing at the app's exact name.
report_sources() {
curl -sS -f "${HA_URL}/api/states/${LAUNCH_ENTITY}" \
-H "Authorization: Bearer ${HA_TOKEN}" --max-time 10 2>>"$LOG" \
| python3 -c '
import json, sys
try:
a = json.load(sys.stdin).get("attributes", {})
except Exception:
sys.exit(0)
srcs = a.get("source_list") or []
print(" available sources: " + (", ".join(map(str, srcs)) if srcs else "(none reported)"))
' 2>>"$LOG" || true
}
launch_app() {
case "$LAUNCH_METHOD" in
androidtv_app)
[[ -n "$LAUNCH_APP_ID" ]] || return 1
ha_service "media_player/play_media" \
"{\"entity_id\": \"${LAUNCH_ENTITY}\", \"media_content_type\": \"app\", \"media_content_id\": \"${LAUNCH_APP_ID}\"}"
;;
appletv_source)
[[ -n "$LAUNCH_SOURCE" ]] || return 1
# The Apple TV sleeps; select_source on a sleeping device does
# nothing. Wake it and then VERIFY it actually woke (entity state
# leaves off/standby) before launching.
ha_service "media_player/turn_on" "{\"entity_id\": \"${LAUNCH_ENTITY}\"}" || true
local _waited=0 _atv_state=""
while (( _waited < 12 )); do
_atv_state=$(curl -sS -f "${HA_URL}/api/states/${LAUNCH_ENTITY}" \
-H "Authorization: Bearer ${HA_TOKEN}" --max-time 5 2>>"$LOG" \
| python3 -c 'import json,sys; print(json.load(sys.stdin).get("state",""))' 2>>"$LOG" || echo "")
if [[ -n "$_atv_state" && "$_atv_state" != "off" && "$_atv_state" != "standby" && "$_atv_state" != "unavailable" ]]; then
break
fi
sleep 2
_waited=$(( _waited + 2 ))
done
if [[ -z "$_atv_state" || "$_atv_state" == "off" || "$_atv_state" == "standby" || "$_atv_state" == "unavailable" ]]; then
log "WARN: ${LAUNCH_ENTITY} still '${_atv_state:-unreachable}' after wake attempts — launching anyway"
else
log "${LAUNCH_ENTITY} awake (state=${_atv_state}) after ${_waited}s"
fi
ha_service "media_player/select_source" \
"{\"entity_id\": \"${LAUNCH_ENTITY}\", \"source\": \"${LAUNCH_SOURCE}\"}" || {
log "WARN: select_source '${LAUNCH_SOURCE}' failed"
report_sources | tee -a "$LOG"
return 1
}
;;
*)
return 1
;;
esac
}
# Is a launch path actually configured for this target?
LAUNCH_CONFIGURED=0
if [[ "$LAUNCH_METHOD" == "androidtv_app" && -n "$LAUNCH_APP_ID" ]]; then
LAUNCH_CONFIGURED=1
fi
if [[ "$LAUNCH_METHOD" == "appletv_source" && -n "$LAUNCH_SOURCE" ]]; then
LAUNCH_CONFIGURED=1
fi
# Wake the display FIRST, before any app-launch attempt. An app-launch command
# sent to a sleeping device is silently dropped — the TV comes up on the
# launcher while a stale background session accepts playback invisibly.
if [[ -n "${LAUNCH_ENTITY:-}" ]]; then
ha_service "media_player/turn_on" "{\"entity_id\": \"${LAUNCH_ENTITY}\"}" \
&& log "Sent turn_on to ${LAUNCH_ENTITY}" \
|| log "WARN: turn_on for ${LAUNCH_ENTITY} failed (continuing)"
sleep 2
fi
SESSION_ID="$(find_session_id || true)"
if [[ -z "$SESSION_ID" ]]; then
if [[ "$LAUNCH_CONFIGURED" -eq 1 ]]; then
log "No ${SESSION_CLIENT} session — launching Moonfin on ${LAUNCH_ENTITY} (${LAUNCH_METHOD})..."
LAUNCHED=1
launch_app || log "WARN: app launch via HA failed — polling anyway"
LAST_LAUNCH=$SECONDS
# Launch-wait must match the TARGET's measured session-registration
# time, not a global constant: a cold tvOS Moonfin can register its
# server session AFTER a too-eager cutoff — walking away from a
# launch that had actually worked.
while (( SECONDS < BUDGET - 12 )); do
SESSION_ID="$(find_session_id || true)"
[[ -n "$SESSION_ID" ]] && break
# Re-sends are ANDROID-ONLY. On tvOS every select_source RESTARTS
# Moonfin and kills the server handshake it was mid-way through.
# Launch once, then wait patiently; check for late success instead.
# 15s between cycles: a cold Moonfin needs ~10-15s to boot AND
# finish its handshake — an 8s cycle interrupts registration
# mid-flight, repeatedly.
if [[ "$LAUNCH_METHOD" == "androidtv_app" ]] && (( SECONDS - LAST_LAUNCH >= 15 )); then
# A repeated launch intent at an already-running Moonfin is
# IGNORED. If a remote entity is configured, send HOME first —
# a real background→foreground cycle fires onAppResumed, which
# is what makes Moonfin ≥2.5.0 reconnect its socket and
# re-post capabilities.
if [[ -n "${LAUNCH_REMOTE:-}" ]]; then
log "Cycling Moonfin: HOME, then relaunch..."
ha_service "remote/send_command" \
"{\"entity_id\": \"${LAUNCH_REMOTE}\", \"command\": \"HOME\"}" \
|| log "WARN: HOME via ${LAUNCH_REMOTE} failed"
sleep 2
else
log "Re-sending app launch..."
fi
launch_app || true
LAST_LAUNCH=$SECONDS
fi
sleep 2
done
else
log "No session and no launch path configured for ${TARGET}"
fi
fi
if [[ -z "$SESSION_ID" ]]; then
if [[ "$LAUNCH_CONFIGURED" -eq 0 ]]; then
fail_loud "no ${SESSION_CLIENT:-configured} session and no launch path for ${TARGET}" \
"Please open Moonfin on the ${TARGET} first."
fi
UNCTRL="$(find_uncontrollable || true)"
if [[ -n "$UNCTRL" ]]; then
fail_loud "session for ${TARGET} exists but SupportsRemoteControl=false (${UNCTRL}) — restart the device (proven remedy)" \
"Moonfin is open on the ${TARGET} but not accepting control. Restarting that TV device fixes this."
fi
# Moonfin that fails its FIRST connect (server still warming after a
# restart) never retries on its own — but a relaunch against a warm server
# registers in ~6s. So on the way out, fire one last HOME+relaunch so the
# app boots fresh against the now-warm server, and tell the user the
# truth: the retry will work.
if [[ "$LAUNCHED" -eq 1 && "$LAUNCH_METHOD" == "androidtv_app" && -n "${LAUNCH_REMOTE:-}" ]]; then
log "Parting shot: HOME + relaunch so the NEXT attempt finds a live session"
ha_service "remote/send_command" \
"{\"entity_id\": \"${LAUNCH_REMOTE}\", \"command\": \"HOME\"}" || true
sleep 2
launch_app || true
fail_loud "no controllable session appeared on ${TARGET} within budget (parting relaunch fired)" \
"The ${TARGET} app is still starting up. Ask me again in half a minute."
fi
fail_loud "no controllable session appeared on ${TARGET} within budget" \
"The ${TARGET} did not come online, so ${TITLE} is not playing."
fi
log "Target session: ${SESSION_ID} (t=${SECONDS}s)"
# A live session does NOT mean the display is on, and it does NOT mean the app
# is on screen — Moonfin keeps its session while backgrounded, so playback
# sent to it progresses invisibly and then stalls. ALWAYS bring the app to the
# foreground; a relaunch can register a NEW session, so re-resolve afterwards.
if [[ "$LAUNCH_CONFIGURED" -eq 1 ]]; then
log "Foregrounding Moonfin on ${LAUNCH_ENTITY} (${LAUNCH_METHOD})..."
launch_app || log "WARN: app foreground failed (continuing)"
sleep 4
NEW_SID="$(find_session_id || true)"
if [[ -n "$NEW_SID" && "$NEW_SID" != "$SESSION_ID" ]]; then
log "Session changed after foregrounding: ${SESSION_ID} -> ${NEW_SID}"
SESSION_ID="$NEW_SID"
fi
fi
# ── 5. Play, verify, sustain-check, single retry ─────────────────────────────
# Exit codes:
# 0 verified 4 command rejected 5 never started 6 started then died
# 7 loaded but never played (backgrounded / display off)
send_play() {
curl -sS -f -X POST \
-H "Authorization: MediaBrowser Token=\"${JELLYFINAPI}\", Client=\"MorisonicsVoice\", Device=\"voice-host\", DeviceId=\"morisonics-voice-1\", Version=\"1.0\"" \
-H "Content-Length: 0" \
--max-time 15 \
"${JELLYFIN_URL}/Sessions/${SESSION_ID}/Playing?playCommand=PlayNow&itemIds=${ITEM_ID}&startPositionTicks=${START_TICKS}" \
>/dev/null 2>>"$LOG"
}
# Prints "<state> <ticks>" when the target session is playing OUR item.
observe() {
local sessions
sessions=$(jf "/Sessions") || return 1
DATA="$sessions" SID="$SESSION_ID" IID="$ITEM_ID" python3 -c '
import json, os, sys
try:
sessions = json.loads(os.environ["DATA"])
except Exception:
sys.exit(1)
sid, iid = os.environ["SID"], os.environ["IID"]
for s in sessions:
if s.get("Id") != sid:
continue
npi = s.get("NowPlayingItem") or {}
if npi.get("Id") != iid:
sys.exit(1)
ps = s.get("PlayState") or {}
state = "paused" if ps.get("IsPaused") else "playing"
print("%s %s" % (state, ps.get("PositionTicks") or 0))
sys.exit(0)
sys.exit(1)
'
}
play_and_verify() {
local attempt="$1" deadline first second
log "Sending play command (attempt ${attempt}, t=${SECONDS}s)..."
# RESTART SAFETY: on a retry, look for an existing session first. A blind
# re-send restarts the player mid-startup — exactly the "started then
# stopped" failure retries used to cause.
if [[ "$attempt" != "1" ]] && observe >/dev/null 2>&1; then
log "note attempt=${attempt}: already playing — skipping resend, verifying instead"
else
send_play || { log "PLAY_ERROR attempt=${attempt}: Playing command rejected"; return 4; }
fi
# ── Verification: poll for TWO CONSECUTIVE "playing" samples ─────────────
#
# Measured on real clients. A single sample cannot tell these apart,
# because at any given instant they look identical:
#
# Android TV, app BACKGROUNDED (really not visible):
# playing 4914130000 -> paused 4933280000 (advances, then pauses)
# Apple TV, genuinely playing on screen:
# paused 2753000000 -> paused 2753000000 (for the first ~15s!)
# ... then later: playing -> playing
#
# tvOS Moonfin reports IsPaused=true with a frozen position for roughly
# the first 15 seconds of real playback — its PlayState lags reality. So
# a snapshot at a fixed moment produced a false NEGATIVE on the Apple TV
# and a false POSITIVE on the Chromecast, in successive versions of this
# check.
#
# What separates them is behaviour over time: a backgrounded client
# DEGRADES (playing -> paused), a real one HOLDS (playing -> playing).
# Position is deliberately NOT part of the test: tvOS was observed ticking
# BACKWARDS while genuinely playing (10000000 -> 9000000).
local cur state ticks consec=0 saw_item=0 misses=0
deadline=$(( SECONDS + (attempt == 1 ? VERIFY_WINDOW : 12) ))
# Never let the verify window sail past the global budget: HA kills the
# shell at 60s, and a truncated run reports NOTHING — worse than an
# honest shorter verify.
(( deadline > BUDGET )) && deadline=$BUDGET
while (( SECONDS < deadline )); do
cur="$(observe 2>/dev/null || true)"
if [[ -z "$cur" ]]; then
# Tolerate a single blank read (transient API hiccup); two in a
# row after the item was seen means the session really dropped it.
if (( saw_item )); then
misses=$(( misses + 1 ))
if (( misses >= 2 )); then
log "PLAY_ERROR attempt=${attempt}: item was loaded but the session dropped it"
return 6
fi
fi
sleep 2
continue
fi
misses=0
saw_item=1
read -r state ticks <<< "$cur"
if [[ "$state" == "playing" ]]; then
consec=$(( consec + 1 ))
if (( consec >= 2 )); then
log "VERIFIED attempt=${attempt}: two consecutive playing samples (${cur}, t=${SECONDS}s)"
return 0
fi
else
consec=0
fi
sleep 2
done
if (( saw_item )); then
log "PLAY_ERROR attempt=${attempt}: item loaded but never reported playing twice in ${VERIFY_WINDOW}s"
return 7
fi
log "PLAY_ERROR attempt=${attempt}: command sent but the item never appeared in the session"
return 5
}
RC=0
play_and_verify 1 || RC=$?
# A retry costs ~14s. Gate on the real remaining budget rather than a fixed
# constant, so widening a verify window for a slow client does not silently
# disable the retry.
if [[ "$RC" -ne 0 ]] && (( SECONDS + 14 < BUDGET )); then
log "WARN: attempt 1 failed (code $RC) — retrying once..."
sleep 2
RC=0
play_and_verify 2 || RC=$?
elif [[ "$RC" -ne 0 ]]; then
log "WARN: attempt 1 failed (code $RC) — no budget left for a retry (t=${SECONDS}s)"
fi
# Late-success grace watch. A launch that was merely SLOW can start playing
# moments after the verify window closes. Observing costs no resend, so spend
# every remaining second watching before declaring failure.
if [[ "$RC" -ne 0 ]]; then
GRACE_CONSEC=0
while (( SECONDS < BUDGET - 1 )); do
CUR="$(observe 2>/dev/null || true)"
if [[ "${CUR%% *}" == "playing" ]]; then
GRACE_CONSEC=$(( GRACE_CONSEC + 1 ))
if (( GRACE_CONSEC >= 2 )); then
log "VERIFIED late (grace watch): playback started after the verify window (${CUR}, t=${SECONDS}s)"
RC=0
break
fi
else
GRACE_CONSEC=0
fi
sleep 2
done
fi
if [[ "$RC" -ne 0 ]]; then
case "$RC" in
4) fail_loud "session ${SESSION_ID} on ${TARGET} rejected the Playing command" \
"The ${TARGET} did not respond, so ${TITLE} is not playing." ;;
5) fail_loud "play command sent but playback never started on ${TARGET}" \
"${TITLE} did not start playing on the ${TARGET}." ;;
6) fail_loud "playback started on ${TARGET} but died within seconds" \
"${TITLE} started and then immediately stopped on the ${TARGET}." ;;
7) fail_loud "item loaded on ${TARGET} but never reported playing twice (backgrounded / display off?)" \
"${TITLE} is loaded on the ${TARGET} but not playing. The screen may be off." ;;
*) fail_loud "playback failed with unexpected code $RC" ;;
esac
fi
log "Playback verified on $TARGET: $SHOW_TITLE / $EP_TITLE (t=${SECONDS}s)"
if [[ "$ITEM_TYPE" == "Episode" ]]; then
echo "PLAYING: $SHOW_TITLE S${SEASON}E${EPISODE} ($EP_TITLE) on the $TARGET"
else
echo "PLAYING: $EP_TITLE on the $TARGET"
fi
Two pieces: a shell_command that runs the
script, and a script entity exposed to Assist whose field descriptions carry the
routing rules. In our testing, rules about how to fill a parameter work best in
that parameter’s own description — the model reads them at exactly the
moment it needs them.
# configuration.yaml — the bridge between the voice agent and the script.
# NOTE: adding a shell_command needs `shell_command.reload` (or a restart)
# before the script that calls it will work.
shell_command:
play_jellyfin: '/config/scripts/play_jellyfin.sh "{{ title }}" "{{ target }}"'
jellyfin_transport: '/config/scripts/jellyfin-transport.sh "{{ command }}" "{{ target }}"'
# The data key is `command`, not `action` — `action` collides with HA's own
# step keyword and the validator flags it.
# scripts.yaml — the tool the voice agent actually sees. The field
# descriptions ARE the interface: they tell the model how to fill the
# parameters, and in our measurements that is where such rules belong.
script:
play_jellyfin:
alias: Play Jellyfin
description: >-
Play a film or TV series from the Jellyfin library on a TV. Waits for
verified playback and returns the result sentence - speak it as-is.
fields:
title:
description: >-
The film or show title, in Latin letters. If the person said a
title in another script, spell it the way the library would
(e.g. a romanised title), because the library is stored in Latin
spelling.
required: true
target:
description: >-
Which TV to play on. appletv = the living room TV. chromecast =
the bedroom TV. If the person named a device or a room, use that.
If they named NEITHER, use the room they are speaking from: a
request from the living room is appletv, a request from the
bedroom is chromecast. Never default to one room for a request
made in the other.
required: true
sequence:
- service: shell_command.play_jellyfin
data:
title: "{{ title }}"
target: "{{ target }}"
response_variable: shell
- stop: "Done"
response_variable: shell
mode: single
jellyfin-transport.sh) sharing the
same config file, with the verification logic inverted: for an already-playing item,
position is the ground truth (pause = position static across two samples,
resume = position advancing, stop = the item disappears), because that is the field
the clients report honestly in that state. Same machine-readable last line, same
budget discipline. And for the highest-frequency phrases — “pause”,
“resume” — a trilingual sentence trigger answers before the
language model is even consulted: fixed phrases deserve the fastest, most
deterministic layer available.secrets.env as JELLYFINAPI, with a long-lived Home Assistant token as HA_TOKEN.play_jellyfin.sh --sessions, and copy the exact CLIENT and DEVICE strings into jellyfin-targets.conf.moonfin:// for Android TV (plus its remote entity), the app name from source_list for tvOS.script.play_jellyfin to Assist, and adjust the room names in the target field description to your rooms.PLAYING:.