Write a contract¶
- Find the guarantee in a public test suite. The lab measures claims sandboxes make
about themselves, so a contract starts from another project's test and names it in
source. Candidate suites are incorpus-roadmap.md. - Write it in neutral terms in
contracts/<category>/<id>.toml: the policy, the fixtures, an attack that must not take effect, a control that must keep working, and the effect to assert (Contracts, format). - Make the attack real. Run it with no sandbox: it has to work, or the case is
vacuous. Never weaken an attack to make a sandbox pass. - Keep the control independent of the attack. It runs after the attack, so it must not need anything a working attack changes.
- Say what decides it. When an exit status is the evidence, declare
expect_exitand, if the attack can exit with its blocked status for another reason,blocked_exit. Declarerequiresfor host capabilities andosfor the systems it applies to. -
Write the next corpus version in the same pull request:
python3 runner/corpus.py build 1.1and add a line to
contracts/CHANGES.mdsaying why. The lab tests runcorpus.py checkand fail otherwise (Corpus versions). 7. Run the lab tests (python3 -m unittest discover -s runner/tests): the reference sandboxes check that an allow-all sandbox escapes your attack and a deny-all one breaks your control.
If a guarantee only makes sense in one sandbox's model, say so in the contract. A
sandbox that cannot express a field gets unsupported.