Pilot evaluation
The pilot is measured against acceptance criteria agreed in writing with the Authority. Workstream 3 starts only after the Authority approves the pilot in writing.
Acceptance criteria
| Criterion | What is tested | Threshold |
|---|---|---|
| Citation present | Every answer cites at least one approved source, and every cited source supports the statement it is attached to. | 100% |
| Correct source | The cited source is the current approved document for the question in the evaluation key, and no answer relies on a superseded document. | 95% or higher, and no answer citing a superseded or conflicting source as current |
| Access respected | Users outside a document's identity group never receive content from it (negative tests included). | 100% |
| Retained | Every prompt and response appears in the records log with user, time, sources and retention label within 5 minutes. | 100% |
| Conflicts disclosed | Where approved sources disagree, the answer says so and names both sources instead of choosing one. | 100% |
Runs
| Run | Date and time | Question set | Citation present | Correct source | Access respected | Retained | Conflicts disclosed | Outcome |
|---|---|---|---|---|---|---|---|---|
| EV-20261113-01 | 11/13/2026 2:05 PM | Pilot set v1, 60 questions (8 access negative tests) | 60 / 60 | 60 / 60 | 60 / 60 | 60 / 60 | 2 / 2 | Meets acceptance criteria |
| EV-20261110-01 | 11/10/2026 9:12 AM | Pilot set v1, 60 questions (8 access negative tests) | 60 / 60 | 56 / 60 | 60 / 60 | 60 / 60 | 3 / 3 | Not accepted |
Pilot acceptance report for EV-20261113-01 sent to the Executive Assistant to the President on 11/13/2026. Written approval from the Authority is pending.