ANN06 actual-outcome cohort: final review

ANN06 actual-outcome cohort: final review

What I reviewed: the frozen package ANN-06-OUTCOME-COHORT.v1, with REVIEW-MANIFEST listing 879 leaves.
How: Read, Glob and Grep only. I ran nothing, edited nothing and recomputed no hashes. Every hash below was compared as text. I did not open any large raw file in full.
Delivery: I have no Write tool in this session, so this report is returned inline for the coordinator to save outside the package.

1. Verdict

• Typed actual execution: accepted. All five carried majors (M1–M5) are covered, within the limits listed below.
• Model quality: failed. Both original cohort gates failed. Retention of the 150-anchor A/B set failed for all five seeds, and the B selection result of 0/9 remains disqualifying.
• No blocker stops closing this execution verdict.
• This closes this one diagnostic run only. It is not ANN06/07 quality acceptance, not promotion, not learned routing and not a security claim.

2. M1–M5 coverage

3. Blockers

None for the execution verdict.

4. Majors (interpretation; no data or gate changes requested)

1. Family evidence is thin by construction.
- The release catalogue has six K4 scenarios and exactly one each of U0, U1, U2 and U3. K0–K3 have zero release scenarios (protocol.mjs:13).
- So the predeclared "all K0-K4" and "K3" breakdowns are empty for K0–K3, and each held-out family rests on a single scenario repeated across five seeds.
- The worst primary loss (24 jobs, seed 3251 vs incumbent; −24 vs rules) comes from one U3 scenario, 940423 (ROOT-TERMINAL-READBACK:407).
- Reports should say "n=1 scenario per U-family" explicitly.
2. The failure is not only B's restart miss.
- On seed 3251's own primary trajectory, role C saw class-2 support of 116 and got 3 right (REVIEW-SUMMARY.json:1341-1351). Every other seed saw a class-2 support of 7.
- Seed 3251 also drives both cohort failures: primary wrong/malformed excess 11 of 13, and the low-budget worst loss of 13.
- This corpus cannot tell whether C's misses come from B's earlier misses or are a separate C deficit, because own-trajectory distributions differ per checkpoint. Per-role accuracies are therefore not comparable across seeds.
3. The "+6.3 vs fresh" gain mostly reflects fresh running out of budget.
- Fresh left 301 jobs unprocessed; primary candidates left none (REVIEW.md:18).
- Seed 977 had exactly 0 gain vs fresh.
- Role A agreement barely moves job outcomes: seed 977's A got 24 of 833 class-3 labels right, yet its gain vs incumbent was +0.1 (REVIEW-SUMMARY.json:646-668). Under this harness, A's teacher agreement is weak evidence of anything that matters for outcomes.
4. B has collapsed to almost always choosing class 0.
- Correct restart positives were 4/110, 0/127, 4/117, 0/138 and 0/118 across the five seeds.
- The few hits are alpha restarts only; beta and gamma restarts are always missed.
- The legal mask never rescues B (maskChanged is 0 in every candidate B panel).

5. Minors

• Unanchored PID in the brief. "root 70896" appears in no non-episode artifact; a package grep found it only inside unrelated float values. The run artifacts give node 21876 and wrapper 21875 (ACTUAL-PROCESS:4-5). Either anchor 70896 or correct the narrative.
• Stale wording. REVIEW-SUMMARY.json:1499 still says the numerical supplement is "Pending separately authored Security…". EXTERNAL-CLOSURE.json has since closed it.
• Two plan identities. The admission planDigest is 972851… (a canonical content digest), while the plan file SHA is faf2b128…. They match through audit.mjs:11 and the manifest at line 40033. Label both kinds explicitly in future.
• Confusing sign. worstFreshLoss takes negative values (−7, −12) when every block was a gain. Consider a clearer name or a note.
• Aggregate mask load. Across all neural modes, C's legal mask corrected 780 of 5,530 raw choices (numerical-actual/REPORT.json:100). Candidate panels show 0–8, so most of this is in other modes, and I did not split it out. Safety comes from the authored masks, not from the networks.
• Carried limits. m1–m9 from the v2 review and d1–d5 from the v3 review stay open. The semantic adapters were reused by both auditors.
• Package is not standalone. Absolute paths point to the original locations.

6. Unresolved limitations (out of scope for this verdict)

• The full journal crash branch is unqualified.
• Kernels and logs are cooperative, same-UID artifacts.
• The Decimal check covers a 33-decision panel, not all inputs.
• Logits and value were not emitted by the original run; the recomputed ones are a derived supplement only.
• There is no learned router: the frozen router weights were not used and actual routes came from authored rules.
• OS cleanup is a point-in-time absence check, not proof of hostile isolation or crash safety.
• Ten shared scenario seeds make this a set of matched units, not 50 independent samples.
• ANN08, E3, ANN09 and ANN10 remain open.

7. Conditions for a new, separate future experiment

These apply only to a new experiment. They do not reuse these seeds, do not change these gates and do not retest these failures.

1. New catalogue. Use a new, predeclared catalogue with several scenarios per family, including stop-only K3 and multiple U scenarios. Declare the effective analysis unit up front.
2. Role outcome sensitivity. Before any protected query, show on non-protected seeds that each role's decisions measurably change job outcomes, for example with forced-wrong or teacher-forced runs. Otherwise drop that role's agreement from acceptance.
3. B restart gate before release. Require per-target restart recall above 0 on alpha, beta and gamma at selection, using class-balanced metrics. Failing it should end the experiment before any protected query.
4. Separate own-trajectory effects. Add a counterfactual replay (teacher-forced vs own trajectory) so C deficits can be told apart from errors passed on by B.
5. Pre-commit the independent checkers. Freeze the raw and numerical reference sources before outcomes exist, and emit logits and value in the decision records.
6. Journal hardening in a new source version. Add a stage terminal record (d3), guard recordedAt and digest (d1), qualify the crash branch, and add a positive-path unit test.
7. Scope routing honestly. Either evaluate a learned router or state plainly that routing is authored rules.
8. Isolation. Signed loaded binaries plus a separate UID or container (ANN10) before any security wording.
9. Hypotheses only. Balancing and rehearsal for B restart and old-skill retention are hypotheses to test, not conclusions from this run.