vst-bench

Open Source

First MCP server for AI-driven VST/AU plugin testing. Render audio, inspect UI, and run visual and audio regression, all from natural language.

GitHub→PyPI→
vst-bench screenshot 1

Overview

vst-bench is an open-source MCP server that lets AI agents (Claude, Cursor) automate VST3/AU plugin testing. A Python MCP server communicates with a C++ JUCE-based TestHost over JSON-RPC. AI agents can load plugins, render audio with MIDI, inspect and interact with UI components, run visual and audio regression tests, monitor CPU/memory, and capture screenshots — 31 tools total.

Tech Stack

Python: 3.10+, MCP 1.0+, pydantic 2.0+, numpy, pyobjc-framework-Quartz, websocketsC++: C++17, JUCE 8.0.12, CMake 3.22+, OpenSSLPlatform: macOS 13+ (Windows/Linux on roadmap)Testing: pytest — 47 unit tests across 6 modulesLicense: MIT (Python), JUCE dual-license (TestHost)

Engineering Highlights

  1. 01Dual UI testing strategies — JUCE component tree traversal for native plugins, CGWindowListCreateImage + CGEvent for WebView-based plugins. Automatic fallback for reliability.
  2. 02Plugin-agnostic parameter API — Reads/writes AudioProcessor parameters directly, bypassing UI layer. Works for any VST3/AU regardless of UI framework.
  3. 03Multi-level regression — Visual (SSIM-based image comparison), audio (correlation-based waveform comparison), and performance (CPU/memory profiling) regression in one framework.
  4. 04Reproducible test state — Full plugin state serialization to base64. Save before test, restore after. Deterministic.
  5. 05Synthetic test data generation — Programmatically generates realistic audio (kicks, snares, melodies at configurable BPM/keys) for repeatable testing without external assets.

Stats

1,666 lines Python + 4,264 lines C++
stat_0
31 MCP tools across 10 categories
stat_1
47 unit tests across 6 modules
stat_2
v0.1.0 Alpha
stat_3
7 commits (carefully extracted and decoupled from proprietary codebase)
stat_4