Practice
Assay
The independent evidence practice inside ENGRAM. Assay defines the test before the result, reviews the ledger honestly, and decides whether an AI use has earned its level.
Nothing is promoted on a demo. Nothing rises except on evidence.
What Assay is
In every organisation, AI pilots arrive with impressive demos and optimistic projections. The hard question is whether they work reliably enough to hold more authority.
Assay answers that question by separating the evidence from the enthusiasm. It sets the criteria up front, then reviews the real performance data against those criteria.
The result is a recommendation the board can trust: promote, hold, downgrade or retire. No use moves up the trust ladder without passing through Assay.
Where it sits
Inside the ENGRAM loop
Assay touches three steps of the ENGRAM loop, but its decisive moment is between Score and Promote.
04 Contract
Set the bar
Assay helps write the acceptance contract: what thresholds must be met, with what test data, and what would disqualify the use.
05 Score
Watch the ledger
While the use runs, Assay checks that the ledger captures the right evidence and that exceptions are not hidden or explained away.
06 Promote
Decide on evidence
At review, Assay compares the ledger to the contract and recommends the next level, a hold, a downgrade, or retirement.
Principles
The four principles of Assay
- 01
Define the test first
- 02
The builder does not grade the work
- 03
Evidence, not opinion
- 04
No level is permanent
01
Define the test first
Acceptance criteria are agreed before the system is built or measured.
Assay begins with the acceptance contract. What must be true for this AI use to move up a trust level? Which data sets, failure rates, human-in-the-loop triggers and cost thresholds matter? Once those criteria are written down, they become the bar every later review must meet.
This prevents the common pattern in which success is redefined after the numbers arrive. A demo is not a test. A test is a statement made in advance about what would count as good enough.
What it produces
- Written acceptance criteria for the next trust level
- Test data, thresholds and disqualifying conditions
- Review date and named independent reviewer
The question it answers
What would have to be true for us to say this has earned more autonomy?
02
The builder does not grade the work
Assay is carried out by a reviewer independent of the team that owns the AI use.
Independence is not an accusation of bad faith. It is a structural guard against optimism bias. The people who built a system are the wrong people to certify it, because they know where the weaknesses are and have an incentive to explain them away.
Assay assigns the review to someone with no stake in the outcome. That person reads the contract, reads the ledger, and makes a recommendation. The owner still decides, but the decision is now informed by evidence that has not been filtered by the team seeking promotion.
What it produces
- Named reviewer with no delivery or commercial stake
- Conflict-of-interest declaration
- Review record signed by reviewer and owner
The question it answers
Who in this organisation is qualified to review this and has nothing to gain from the answer?
03
Evidence, not opinion
A promotion decision rests on the ledger, not on how convincing the demo felt.
Demos persuade; ledgers decide. Assay reads the quality ledger, covering accuracy, incidents, coverage, cost, user complaints and overrides, then compares it to the contract. A recommendation without that comparison is only an opinion.
The review also checks that the ledger itself is trustworthy. Are exceptions logged honestly? Are failures defined the same way over time? Is the human effort required to keep the system safe counted as part of the cost? Assay treats the evidence as a system to inspect, not a slide deck to applaud.
What it produces
- Comparison of ledger results against the acceptance contract
- List of unmet thresholds and unexplained exceptions
- Honest cost per outcome, including human oversight
The question it answers
If a regulator asked for the evidence behind this promotion, would we be proud to hand it over?
04
No level is permanent
Assay also reviews held and downgraded uses. Evidence can support more autonomy, the same autonomy, or less.
Trust is not a one-way ratchet. Models drift, data changes, users find new edge cases, and the business context shifts. Assay schedules periodic re-reviews so a promoted use can be held, downgraded or retired when the evidence no longer supports its level.
Retirement is a normal outcome, not a failure. A use that no longer earns its place should be removed from the register cleanly, with its data and dependencies accounted for. The goal is not to keep every use alive; it is to keep the estate honest.
What it produces
- Scheduled re-review date and trigger conditions
- Promote, hold, downgrade or retire recommendation
- Updated register entry with rationale for the decision
The question it answers
What evidence would make us take authority away from this use?
Deliverables
What Assay produces
Each Assay engagement leaves a clear record that can be filed in the governance register and explained to an auditor or regulator.
Assay plan
Thresholds, test data, review dates, required evidence and disqualifying conditions for the AI use under review.
Independent review of the quality ledger
A documented assessment of real performance against the acceptance contract, with exceptions surfaced rather than explained away.
Promote, hold, downgrade or retire recommendation
A clear decision with rationale, including the conditions that would justify a different outcome next time.
Written rationale for the governance register
A record that can be filed, audited and explained to a board, regulator or successor team.
When to use Assay
Four moments that call for independent review
Before promotion
A pilot wants wider deployment. Assay decides whether it has earned the next trust level.
After an incident
Something went wrong. Assay reviews whether the level is still appropriate and what evidence would restore confidence.
At scheduled re-review
A promoted use is due for its periodic check. Assay reads the ledger and confirms, adjusts or removes the level.
During procurement
A vendor claims enterprise-grade AI. Assay defines the test your organisation will apply before any contract is signed.
Bring Assay into your governance
Assay can be introduced as part of an ENGRAM adoption or as a standalone review of an existing AI use that is being considered for wider deployment.