Soundor Driver

Source: packages/driver/.

Soundor Driver works with a Soundor plugin that is already running: it reads the native UI the engine really has (its node tree, bounds, styles and accessibility) and acts on it (clicks, drags, scrolls, keys and text, through the engine's own input dispatch), and captures it as PNG, from Node.js or from the soundor driver commands. Use it to debug a plugin, to look at a control while you work on it, to write scripts that reproduce a UI bug, and from coding agents that need a reliable, machine-readable view of the UI.

It is a client of the DevTools session of soundor dev: it signs in the way the DevTools dashboard does and uses the same protocol. The dashboard and any number of Driver clients can watch one plugin at once.

Soundor Driver is not:

• a test runner: use it from any script or test runner, or none;
• a plugin host or launcher: it attaches to a plugin that soundor dev started (a JUCE Standalone with DevTools);
• parameter automation, nor an audio or MIDI driver: it works with the UI only, never with audio processing;
• a browser automation tool: the Web runtime runs in a browser, which has its own tools.

Quick startCopy Anchor Link

Start the plugin with DevTools, as usual:

pnpm dev

dev prints the path of the session's protected descriptor, as "DevTools session file". Give it to the Driver, here through the environment:

export SOUNDOR_DEVTOOLS_SESSION=/tmp/soundor-devtools-…/….json

Give a node a debugId to find it by:

<Pressable debugId="reset" onPress={reset}>
  <Text>Reset</Text>
</Pressable>

From the command line:

pnpm exec soundor driver targets
pnpm exec soundor driver tree --json
pnpm exec soundor driver inspect reset
pnpm exec soundor driver click reset
pnpm exec soundor driver screenshot ./after-reset.png

From TypeScript, with @soundor/driver:

pnpm add -D @soundor/driver

import { connect } from '@soundor/driver';

const driver = await connect(); // $SOUNDOR_DEVTOOLS_SESSION
try {
  const tree = await driver.getTree();
  console.log(`${tree.nodes.length} nodes`);
  const reset = driver.getByDebugId('reset');
  console.log(await reset.getBounds());
  await reset.click();
  await driver.screenshot({ path: './after-reset.png' });
} finally {
  await driver.disconnect();
}

Finding nodesCopy Anchor Link

Locators find nodes by what the engine knows of them:

• getByDebugId(id): the node whose debugId prop (on every component, and node.debugId in soundor:ui) is id. It is for tools only: users and assistive technology never see it. DevTools shows it too.
• getByRole(role, { name }): a node assistive technology sees with this role, and this accessible name.
• getByText(text): a text or input node showing this text.

A locator is a query, not a node: each call finds the node again, so it follows re-renders and reloads. See Inspecting UI.

How it worksCopy Anchor Link

flowchart LR
    script["Your script / soundor driver"] --> driver["@soundor/driver"]
    dashboard["DevTools dashboard"] --> server
    driver -- "WebSocket /browser (cookie, Origin)" --> server["DevTools server (soundor dev)"]
    server -- "HTTP long poll, bearer token" --> bridge["Runtime bridge (worker threads)"]
    bridge -- "UI thread tick" --> engine["Soundor engine: Inspector, Surface"]

1. The Driver reads the descriptor's browserKey, exchanges it at POST /browser/auth for the session's cookie, and opens the /browser WebSocket with it and the session's own Origin, as the dashboard does.
2. It sends typed, validated requests (UI.getTree, UI.getNode, Input.dispatchPointer, …) for its selected target. The server checks the target's capabilities, then queues the command for that instance, naming the client that sent it: the engine keeps each client's input, highlight and picker apart.
3. The plugin's runtime bridge long-polls the server from worker threads and hands commands to the engine's Inspector on the UI thread, in its 60 Hz tick. The Inspector reads the engine's Surface, or dispatches input through it as a platform's input, and answers; the bridge posts the answer back, and the server relays it to the Driver.

Nothing blocks the UI or audio thread on the network. Node references are pairs of an opaque ID and a runtime generation: a reload starts a new generation, and the references of the old one are refused.

SecurityCopy Anchor Link

DevTools runs only in development: the engine compiles its inspector only in Debug builds with DevTools enabled, which soundor build never makes, and a plugin connects only when soundor dev hands it a session.

• The server listens on 127.0.0.1 only. The Driver refuses any other endpoint, so the key is never sent to another host.
• The Driver signs in with the session's browser key, never with the plugin's runtime token, and the server refuses that token from browsers.
• The descriptor holding the key is private to your user (mode 0600 in a private directory) and removed when the session ends. The Driver refuses a descriptor other users can read.
• Keys never appear in Driver errors, CLI output or URLs. Pass a key to the CLI through the environment, not as an argument.

LimitationsCopy Anchor Link

• It attaches to running plugins only: JUCE Standalones started by soundor dev juce, on macOS, Windows and Linux. Plugins inside a DAW have no DevTools session yet.
• A tree snapshot holds at most 5000 nodes; a locator reports when the node may be beyond a truncated snapshot.
• Input is the engine's: pointer, wheel and keyboard as the plugin's view receives them. Native OS controls, the host's windows and audio are out of reach.

PagesCopy Anchor Link

• Connecting to a running plugin: sessions, targets, timeouts, errors.
• Inspecting UI: the tree, node details and locators.
• Interacting with UI: actions, actionability, pointer and keyboard, cleanup.
• Screenshots: captures of the rendered view, and how they travel.
• CLI reference: soundor driver, its JSON output and exit codes.