Swap the instrument, keep the test¶

Test scripts are often tied to one instrument, so moving to another bench, or to less expensive equipment, means rewriting them. Here the instrument changes and the test doesn't: a second instrument takes the first one's role, and the same test runs and passes with no step changed.

Why a test ends up tied to one rig¶

A bench test usually starts as a script written against the equipment in front of it. The script knows the instrument's address, which driver or library talks to it, and how it formats an answer. That is a reasonable way to get a test running under delivery pressure, and it works until the instrument changes.

Then the script changes with it. A second bench with different equipment needs its own copy. A failed instrument replaced with a newer model means finding every line that assumed the old one. In many teams, one person knows which parts of the script are about the product and which are about the bench, and that knowledge leaves when they do.

The test names a role, not an instrument¶

In Knilb, a plan declares named roles, and each role has a contract: the OpenAPI spec of the interface it answers on. Every step targets a role, never a particular machine, so the same plan runs against different devices without editing a step.

Which device fills each role is decided somewhere else. A deployment binds a participant to each role, and the instrument belongs to the deployment, not to the test. Changing the instrument means changing one binding. The testbed model explains how roles keep the test apart from the bench, and it is the same mechanism that lets a mock fill a role in the first post.

The swap on a real bench¶

The bench is the one from the bench story. A Wi-Fi thermostat is the device under test, and an off-the-shelf networked I/O monitor is wired to its fan terminal, in a role called relay_monitor. On the same fan wire sits a second instrument: a small single-board computer that answers the monitor's API contract exactly as the monitor does.

The test has already passed with the monitor in the role. In the video, the computer is given the monitor's interface specification, the monitor's binding is removed, and Knilb offers the computer for the role, because it keeps the same contract. Nothing is rewired and no step changes. The same test runs again, and it passes.

What this bench shows is specific: two instruments in the same role, one at a time, each answering the same contract. Any instrument that keeps the contract can take the role in the same way.

The result says which instrument answered¶

Two passing runs of the same test raise a fair question: which instrument answered each one? Each result freezes the deployment as it was cast, so it keeps the address of the device in every role. The first result shows the monitor's address in relay_monitor, and the second shows the computer's.

Nobody has to remember which box was on the bench that day. The record says so, and it says the same thing when someone reopens it later.

What a swap is good for¶

A second bench can use less expensive equipment, as long as it answers the same contract. A failed instrument can be replaced without touching the test plan. And because the plan is recorded against roles and contracts rather than against one person's bench, it outlives both the rig and the person who built it.

The bench story shows the full bench, with the video in context beside the bench it extends. The next post puts two instruments in one test.