Profesor Abelton — AI Copilot for Ableton Live
A standalone desktop AI that turns natural language into Ableton Live actions — built to feel like a friend who actually knows Ableton.
Problem
Ableton Live has a steep learning curve and most learning resources are cold, slow, and abstract — beginners get stuck Googling basics and watching 45-minute tutorials for 30-second questions.
Solution
A local desktop copilot connected to Ableton via an official Control Surface Remote Script. It streams full session state (tracks, clips, devices, tempo) to a dual-LLM engine (Claude with a custom MCP of 35 schema-validated tools, plus Groq for sub-second responses). A hardened command pipeline enforces a 40-action allowlist, parameter sanitization, and a 12-command batch limit. A 5-step First Launch Wizard auto-installs the Remote Script, encrypts API keys (Fernet), and activates a machine-bound Gumroad license.
Architecture
A desktop app (PyInstaller) talks to a local FastAPI + WebSocket engine, which connects to Ableton through an official Control Surface Remote Script. A dual-LLM router sends complex reasoning to Claude — over a custom MCP of 35 schema-validated tools — and quick queries to Groq for sub-second responses. Every requested action passes a hardened command pipeline before it ever reaches Ableton.
Engineering challenges
- Streaming full Ableton session state (tracks, clips, devices, tempo) in real time without blocking the UI.
- Designing 35 schema-validated MCP tools so the LLM can act safely and predictably.
- Hardening the command pipeline: a 40-action allowlist, parameter sanitization, and a 12-command batch limit.
- Making it a real product: Fernet-encrypted API keys, machine-bound Gumroad licensing, and a 5-step first-launch wizard.
Outcome
A commercially launched (v2.0.1) desktop product on Windows and macOS. Static analysis via Bandit (zero medium/high findings) plus a structured security audit before release.
In production
Shipped to paying users in less than 3 months from first line of code — music producers running it inside their day-to-day Ableton Live sessions.
The users are music producers who keep Profesor Abelton open next to Ableton Live while they work. Instead of pausing to search forums or watch tutorials, they ask it in plain language — “add a MIDI track”, “how do I add a device”, “analyse my session” — and it reads the live session state and either answers or performs the action directly in the project. The assistant sits inside the creative loop rather than in a separate tab.
Engineering impact
- Real-time session sync: full Ableton state (tracks, clips, devices, tempo) streams over WebSockets, so answers reflect the project as it is right now — not a stale snapshot.
- Stays usable mid-session: a Groq fast-path answers quick queries in under a second while Claude handles complex reasoning.
- Safe to run in a live project: a 40-action allowlist, parameter sanitization, and a 12-command batch limit stop the AI from damaging a producer's work.
- Resilient in real use: failures from Ableton, the LLM APIs, or the network are caught and surfaced instead of crashing the session.
- Local-first and private: it runs on the user's machine with Fernet-encrypted API keys — nothing about their projects leaves the device except the LLM calls they trigger.
Iteration loop
The product is shaped by how this small group actually uses it. Real sessions surface edge cases demos never do — unusual project layouts, unexpected phrasings, device combinations — and those drive the fixes and new tools in each release. It evolves through live usage feedback, not a roadmap written in isolation.
Tech stack
Static analysis via Bandit (0 medium/high findings) + structured security audit. Supplementary LLM-assisted review checks were also run as part of the audit methodology (GPT-4o: 88/100, Grok: 80/100).