Write the test on mocks. Prove it on the bench.¶

Waiting for hardware usually means waiting to write the system test. With mocks, the test is written, run, and passing before the boards arrive, and when the bench is ready, the same plan runs there.

That changes what the first day on the bench is for. Instead of fixing payloads and working out the order of operations against a device that is also new, the team arrives with a test it already trusts, and spends the bench time on the question only the hardware can answer.

Start before the hardware arrives¶

A system test is mostly decisions about the product, not about the bench: which calls to make, in what order, and what each answer should be. None of that needs the device in the room. It needs the device's API contract.

In Knilb, a mock profile is built from that contract. Each response is picked from the operations and status codes the contract declares, so the test is written against the same interface the real device will answer on. And because a plan names roles rather than machines, nothing written now is tied to the mock. The role is filled by a mock today and by the device later.

The tutorial walks the whole path: from an empty account to a passing result with a mocked device, and no hardware anywhere.

A passing mocked run you can trust¶

A green run is only useful if it's clear what it proved. Two guarantees make a mocked run easy to read.

A mocked role never reaches the real device, even when no mock response matches; the step fails visibly instead. A rehearsal can run on a machine beside the bench with no risk of touching the hardware.

Every step also records which layer answered it. The result says plainly that a mock answered, so nobody mistakes a rehearsal for a hardware run, when it finishes or when someone reopens it later.

What the green run settles is real work: the steps are right, they run in the right order, and the assertions check what they should. That is the part of system testing that usually waits for the bench, and here it's done first. How mocks work covers the details.

Then the bench¶

When the hardware arrives, unbinding the mock is the whole difference between a rehearsal and a hardware run. The plan stays as it is, and the role is cast to the real device instead.

The bench is also where test equipment joins the test. Equipment that exposes its own API contract is cast into a role beside the device, and it reports what happened at the wire rather than what the device says happened.

The bench story shows that half on a real bench, filmed in real time. A Wi-Fi thermostat is the device under test, chosen because it switches a fan relay and reports the fan's state over its API, and an off-the-shelf networked I/O monitor watches its fan terminal. The test sets a known starting state, turns the fan on, and asks both parts what happened.

With the wire connected, all seven steps pass. Then the sense wire from the fan terminal to the monitor is pulled, and the same test runs again. The thermostat accepts the fan call and reports the fan running, exactly as before. The monitor sees the circuit stay open, and the run fails at "Fan relay closed at the wire", the one step where the software and the wire disagree.

Software alone would have called both runs a pass. The test caught the difference because it asks the wire as well as the device.

One record from the first run to the bench¶

Results read the same way whether a mock or the device answered. Every step is badged with the layer that answered it, a mock in a rehearsal and the device on the bench, so there is no argument afterward about what was actually tested.

When a step fails, the case ends there, and the steps after it are kept as not reached instead of adding noise. Every result also freezes the case and the cast it ran against, so the answer a developer reads is the one the run produced.

Mocks get the test right early, and the bench answers for the device. Both results say exactly what answered, which is what makes either one worth handing to someone else.