Interactive Simulator

The interactive simulator and the benchmark runner use the same incident lifecycle, tool registry, evaluator, and cleanup semantics. The simulator only changes how control is exposed: agents call reset() and step() instead of returning a diagnosis from one diagnose() call.

Each environment represents one incident and is single-use. Create a new environment to run the scenario again.

from netopsbench.sdk import NetOpsBench, SubmitDiagnosisAction

with NetOpsBench(workspace=".") as bench:
    scenario = bench.scenarios.load("scenarios/generated/xs/generated_link_down_xs_001.yaml")
    environment = bench.simulators.create(scenario=scenario)
    reset = environment.reset()

    result = environment.step(
        SubmitDiagnosisAction(
            diagnosis={"verdict": "network_healthy"},
        )
    )

reset.observation contains the compact canonical model view. Benchmark agents continue to receive the complete DiagnosticContext.topology and DiagnosticContext.symptoms; the compact view is also available through context.metadata["canonical_observation"].

For multiple independent agents diagnosing one prepared incident, use the advanced API:

from netopsbench.sdk import NetOpsBench

with NetOpsBench(workspace=".") as bench:
    scenario = bench.scenarios.load("scenarios/generated/xs/generated_link_down_xs_001.yaml")
    incident = bench.simulators.prepare(scenario=scenario)
    first = incident.open_session()
    second = incident.open_session()
    try:
        # Each session has independent counters and diagnosis state.
        ...
    finally:
        incident.close()  # fault recovery occurs once

The optional localhost HTTP service accepts opaque case IDs and the same typed tool and diagnosis actions. Install netopsbench[simulator] and run netopsbench-simulator --help for its scenario-loading options.

max_agent_seconds is checked at action boundaries. It prevents a session from accepting additional tool or submit actions after its deadline; it does not attempt to terminate arbitrary user threads while an action is running.

Large warm topologies may keep their native Pingmesh, BGP collector, and Telegraf processes active together. The simulator still rejects an incident when the active topology does not meet readiness or complete-coverage gates. See the observation-validity guidance in Benchmark Methodology.