Probe

Running ML experiments with coding agents

Coding agents can now write the training script, launch the run, read the curves and decide what to try next. What they can't do on their own is remember. This guide shows a setup where every experiment an agent runs is recorded with the question it was testing, so the results add up instead of disappearing with the session.

1. Give your agent a place to record

Tell your coding agent:

Install Probe on this machine and its coding agents. Read https://research.prbe.ai/install.md and follow it.

It signs you up if you're new, installs Probe for the coding agents you use, and asks you to sign in once in the browser. By default a second agent, the Probe daemon, runs beside your coding agent and records the work as it happens. From then on, /probe on records and reads, /probe read only searches, and /probe off keeps Probe out of a conversation.

2. Wire the training code once

Ask the agent to run /instrument-code on your training script, eval or sweep. It adds a few lines like these:

import probe

probe.init(project="folding", experiment="dockq-sweep", question="does temperature 0.7 win?")
probe.log({"train/loss": 0.42, "eval/dockq": 0.71}, step=42)
probe.finish()

The first run in an experiment states the question it is testing; later runs in the same experiment reuse it. Logging never blocks the training loop. Trainers that already have a tracker, and RL trainers such as Miles, TRL, verl, SkyRL and ART, are covered by the same skill. See the SDK quickstart.

3. Let the agent read before it runs

Before proposing a run, the agent can ask what your team already tried: past experiments, their questions, metrics and notes. With the Probe daemon recording, it asks with probe ask; when your agent records its own work, it uses the Probe MCP server's six read-only tools. This is what stops an agent from repeating a run a teammate ruled out last week. See The Probe daemon and MCP server.

4. Keep the reasoning with the result

5. Scale out to several agents

When several agents run experiments in parallel, they write to the same record. The agents doing the work set each project's headline metric and pick the best experiment, branches are ranked on that metric with their agent-hours and cost, and probe session send messages one of your own running sessions that the Probe daemon serves. See Agent swarms.

What changes for you

Learn more