haunt-fm

Personal music recommendation engine that captures listening history from Music Assistant, discovers similar music via Last.fm, computes audio embeddings with CLAP, and generates playlists mixing known favorites with new discoveries — played back on house speakers via Apple Music.

How It Works

You play music on any speaker
  → HA automation logs the listen event
  → Last.fm discovers similar tracks (~50 per listened track)
  → iTunes Search API finds 30-second audio previews
  → CLAP model computes 512-dim audio embeddings
  → pgvector stores and indexes embeddings (HNSW cosine similarity)
  → Taste profile = weighted average of listened-track embeddings
  → Recommendations = closest unheard tracks by cosine similarity
  → Playlist mixes known favorites + new discoveries
  → Music Assistant plays it on speakers via Apple Music

Deployment

Runs on the NAS as two Docker containers:

Container	Image	Port	Purpose
haunt-fm	Custom build	8321 → 8000	FastAPI app + embedding worker
haunt-fm-db	pgvector/pgvector:pg17	internal	PostgreSQL + pgvector

# Deploy / rebuild
cd /volume1/homes/antialias/projects/haunt-fm
git pull && docker-compose up -d --build haunt-fm

# Run migrations
docker exec haunt-fm alembic upgrade head

Access:

• Status page: https://recommend.haunt.house
• Health check: http://192.168.86.51:8321/health
• API status: http://192.168.86.51:8321/api/status
• Source: https://git.dev.abaci.one/antialias/haunt-fm

API Endpoints

Method	Path	Purpose
GET	/health	Health check (DB connectivity)
GET	/api/status	Full pipeline status JSON
GET	/	HTML status dashboard
POST	/api/history/webhook	Log a listen event (from HA automation)
GET	/api/history/recent?limit=20&profile=name	Recent listen events (optional profile filter)
POST	/api/admin/discover	Expand listening history via Last.fm
POST	/api/admin/build-taste-profile	Rebuild taste profile from embeddings
GET	/api/recommendations?limit=50&vibe=chill+ambient	Get ranked recommendations (optional vibe)
POST	/api/playlists/generate	Generate and optionally play a playlist
GET	/api/profiles	List all named profiles with stats
POST	/api/profiles	Create a named profile
GET	/api/profiles/{name}	Get profile details + stats
DELETE	/api/profiles/{name}	Delete profile (reassigns events to default)
PUT	/api/profiles/{name}/speakers	Set speaker→profile mappings
GET	/api/profiles/{name}/speakers	List speaker mappings

Usage

Generate and play a playlist

curl -X POST http://192.168.86.51:8321/api/playlists/generate \
  -H "Content-Type: application/json" \
  -d '{
    "total_tracks": 20,
    "known_pct": 30,
    "speaker_entity": "media_player.living_room_speaker_2",
    "auto_play": true
  }'

Generate a vibe-based playlist

curl -X POST http://192.168.86.51:8321/api/playlists/generate \
  -H "Content-Type: application/json" \
  -d '{
    "total_tracks": 15,
    "vibe": "chill ambient lo-fi",
    "speaker_entity": "media_player.living_room_speaker_2",
    "auto_play": true
  }'

Parameters:

• total_tracks — number of tracks in the playlist (default 20)
• known_pct — percentage of known-liked tracks vs new discoveries (default 30)
• speaker_entity — Music Assistant entity ID (must be a _2 suffix entity)
• auto_play — true to immediately play on the speaker
• vibe — text description of the desired mood/vibe (e.g. "chill lo-fi beats", "upbeat party music"). Uses CLAP text embeddings to match tracks in the same vector space as audio.
• alpha — blend factor between taste profile and vibe (default 0.5). 1.0 = pure taste profile, 0.0 = pure vibe match, 0.5 = equal blend. Ignored when no vibe is provided.
• profile — named taste profile to use (default: "default"). Each profile has its own listening history and taste embedding.

Speaker entities

The speaker_entity must be a Music Assistant entity (the _2 suffix ones) for text search to resolve through Apple Music. Raw Cast entities cannot resolve search queries.

Speaker	Entity ID
Living Room speaker	media_player.living_room_speaker_2
Dining Room speaker	media_player.dining_room_speaker_2
basement mini	media_player.basement_mini_2
Kitchen stereo	media_player.kitchen_stereo_2
Study speaker	media_player.study_speaker_2
Butler's Pantry speaker	media_player.butlers_pantry_speaker_2
Master bathroom speaker	media_player.master_bathroom_speaker_2
Kids Room speaker	media_player.kids_room_speaker_2
Guest bedroom speaker 2	media_player.guest_bedroom_speaker_2_2
Garage Wifi	media_player.garage_wifi_2
Whole House	media_player.whole_house_2
downstairs	media_player.downstairs_2
upstairs	media_player.upstairs_2

Named profiles

Named profiles let each household member get personalized recommendations without polluting each other's taste.

# Create a profile
curl -X POST http://192.168.86.51:8321/api/profiles \
  -H "Content-Type: application/json" \
  -d '{"name":"antialias","display_name":"Me"}'

# Map speakers to auto-attribute listens
curl -X PUT http://192.168.86.51:8321/api/profiles/antialias/speakers \
  -H "Content-Type: application/json" \
  -d '{"speakers":["Study speaker","Master bathroom speaker"]}'

# Log a listen event with explicit profile
curl -X POST http://192.168.86.51:8321/api/history/webhook \
  -H "Content-Type: application/json" \
  -d '{"title":"Song","artist":"Artist","profile":"antialias"}'

# Get recommendations for a profile
curl "http://192.168.86.51:8321/api/recommendations?limit=20&profile=antialias"

# Generate playlist for a profile
curl -X POST http://192.168.86.51:8321/api/playlists/generate \
  -H "Content-Type: application/json" \
  -d '{"total_tracks":20,"profile":"antialias","speaker_entity":"media_player.study_speaker_2","auto_play":true}'

# Build taste profile manually
curl -X POST "http://192.168.86.51:8321/api/admin/build-taste-profile?profile=antialias"

All endpoints are backward compatible — omitting profile uses the "default" profile. Events with no profile assignment (including all existing events) belong to "default".

Other operations

# Log a listen event manually
curl -X POST http://192.168.86.51:8321/api/history/webhook \
  -H "Content-Type: application/json" \
  -d '{"title":"Paranoid Android","artist":"Radiohead","album":"OK Computer"}'

# Run Last.fm discovery (expand candidate pool)
curl -X POST http://192.168.86.51:8321/api/admin/discover \
  -H "Content-Type: application/json" \
  -d '{"limit": 50}'

# Rebuild taste profile
curl -X POST http://192.168.86.51:8321/api/admin/build-taste-profile

# Get recommendations (without playing)
curl http://192.168.86.51:8321/api/recommendations?limit=20

# Get vibe-matched recommendations
curl "http://192.168.86.51:8321/api/recommendations?limit=20&vibe=dark+electronic&alpha=0.3"

Pipeline Stages

1. Listening History — HA automation POSTs to webhook when music plays on any Music Assistant speaker. Deduplicates events within 60 seconds.
2. Discovery — Last.fm track.getSimilar expands each listened track to ~50 candidates.
3. Preview Lookup — iTunes Search API finds 30-second AAC preview URLs (rate-limited ~20 req/min).
4. Embedding — Background worker downloads previews, runs CLAP model (laion/larger_clap_music), stores 512-dim vectors in pgvector with HNSW index.
5. Taste Profile — Weighted average of listened-track embeddings (play count * recency decay).
6. Recommendations — pgvector cosine similarity against taste profile, excluding known tracks.
7. Playlist — Mix known-liked + new recommendations, interleave, play via Music Assistant.

Improving Recommendations Over Time

Recommendations improve as the system accumulates more data:

• Listen to music — every track played on any speaker is logged automatically
• Run discovery periodically — POST /api/admin/discover to expand the candidate pool via Last.fm
• Rebuild taste profile — POST /api/admin/build-taste-profile after significant new listening activity
• Embedding worker runs continuously — new candidates are automatically downloaded and embedded

The taste profile is a weighted average of all listened-track embeddings. More diverse listening history = more nuanced recommendations.

Tech Stack

Component	Choice
App framework	FastAPI + SQLAlchemy async + Alembic
Database	PostgreSQL 17 + pgvector (HNSW cosine similarity)
Embedding model	CLAP laion/larger_clap_music (512-dim, PyTorch CPU)
Audio previews	iTunes Search API (free, no auth, 30s AAC)
Discovery	Last.fm track.getSimilar API
Playback	Music Assistant via Home Assistant REST API
Music catalog	Apple Music (via Music Assistant)
Reverse proxy	Traefik (recommend.haunt.house)

Environment Variables

All prefixed with HAUNTFM_. Key ones:

Variable	Purpose
HAUNTFM_DATABASE_URL	PostgreSQL connection string
HAUNTFM_LASTFM_API_KEY	Last.fm API key for discovery
HAUNTFM_HA_URL	Home Assistant URL
HAUNTFM_HA_TOKEN	Home Assistant long-lived access token
HAUNTFM_EMBEDDING_WORKER_ENABLED	Enable/disable background embedding worker
HAUNTFM_EMBEDDING_BATCH_SIZE	Tracks per batch (default 10)
HAUNTFM_EMBEDDING_INTERVAL_SECONDS	Seconds between batch checks (default 30)
HAUNTFM_MODEL_CACHE_DIR	CLAP model cache directory
HAUNTFM_AUDIO_CACHE_DIR	Downloaded preview cache directory

See .env.example for full list.

Integrations

• Home Assistant — automation haunt_fm_log_music_play captures listening history; REST API used for speaker playback
• Music Assistant — resolves text search queries to Apple Music tracks, streams to Cast speakers
• OpenClaw — has a skill doc (skills/haunt-fm/SKILL.md) so you can request playlists via Telegram/iMessage
• Traefik — routes recommend.haunt.house to the service
• Porkbun DNS — CNAME for recommend.haunt.house