← R³ Orion · working collection

Program report

The surrender-phrase program — effort and lessons

report epoch 1788744000 · one working day, 2026-09-06 · thread orion-phrase-context-1788715237 · agent claude-20260904c

The effort, in order

epochwhatoutcome
1788715237Literature collection: 7 strands, 22 papers with hashespublished, all files collected
1788719000Gold data acquired: Kilicoglu 2,257 human-labeled sentences + 6,174 expansion; authors' rule system; CONSORT-TMthe measurement substrate
1788719400Vocabulary transfer measured on goldP 0.833 / R 0.054 — the dialect gap
1788720000Three adjudication lanes built: lexical SVM, embedding SVM, SLM judgeF1 0.698 / 0.661 / 0.647
1788724000Five-model bake-off, one prompt, one splitQwen3-8B wins, F1 0.785; two collapse
1788722200Ensembles + determinism triplicatepair union F1 0.814; gate P 0.945; zero variance
1788728600Qwen3-8B deployed as the serving judge (192.168.206.10)Mistral off, Qwen3 on
1788730000Article published: The Language of Surrender: Model Selection Observations7 figures, 14 references, live
1788737000Context-in-the-prompt test (owner-directed)NEGATIVE: F1 0.564; +0.0008 as gate
1788738000BAGELS (1.6 GB) + LimitGen acquired; AI-register mining27× recall gain from 120 mined phrases
1788742000Theory canon: 10 papers in-system, 4 fresh demos, 10 Plane cardsORION-22; synthesis LESSON
1788744000Benchmark suite defined (7 gates with cadences); procedure-extraction plan filedworking page v8; ORION-33

The numbers that matter

quantitybeforeafter
vocabulary precision on estate calibration0.230.945 auto-admit gate available (Qwen3∧Gemma)
best measured F1, limitation detectionnone measured0.814 (two-judge union; published reference: 0.800)
recall on AI-register limitation language0.0160.429 (vocabulary + 120 mined phrases)
judge run-to-run varianceunknownzero, three identical confusion matrices
cost of measuring a new modelunknownone config swap + ~4 minutes

What was learned

  1. Surrender language is field-specific; the context grammar is not. One vocabulary scored R 0.82 at home, R 0.054 in clinical text, R 0.016 in AI text. Announcement, mitigation, citation, and attribution transferred everywhere. One grammar, many vocabularies, each with a measured floor.
  2. Diverse failures are the resource. The two judges share 10 of 39 false positives and split their misses; the union out-scored the best single by +0.03 F1 and the agreement gate reached P 0.945. Buying a second model family bought decorrelation, not just capacity.
  3. Generalist beats domain-tuned, measured twice over. Biomedical fine-tunes score below their base models on unseen clinical classification (published), and the bake-off's collapses traced to the serving template, not the weights — the harness is part of the model.
  4. Context is a filter, not evidence. Showing judges their neighbors produced the best recall (0.941) and destroyed precision (0.403) through context bleed. The 2018 rules used structure to pick candidates, not to inform verdicts. Negative result, recorded, worth its run time.
  5. Labels hide in existing structure. Section headers labeled 19,431 sentences free; the authors' heuristic labeled 6,174 more (+0.094 F1); one log-odds pass over those free labels lifted AI-register recall 27×. The estate's own decided spans are the next mine — with the noise direction always stated.
  6. Determinism is cheap and non-optional. Three passes, identical confusion matrices, under 250 s each. Every model swap gets a triplicate; single-run numbers are not numbers.
  7. The canon is operational, not historical. Every deployed stage is one theory paper made to run — Hearst patterns, Salton vectors, Mintz labels, Monroe contrasts, Devlin judges — and every improvement this week came from applying a canon method that was missing. Three remain unapplied: collocation units, topic-stratified labeling, open-then-closed procedure schemas.
  8. Judges never grade judges. Machine labels train; human labels score; the gold is frozen per epoch. The line between a self-improving loop and a self-grading one is exactly this rule.

Where everything lives

artifactlocation
working collection, benchmarks, datasets, instrumentationorion-phrase-context-lit.1788715237.html
public articleronaldreck.com/the-language-of-surrender-model-selection-observations
code, data, per-sentence votes, models/mnt/nvme2/dev/20260904c/orion-phrase-context-lit-1788715237/
PlaneORION-20 (program), ORION-21 (reproduction guide), ORION-22 (theory canon + 9 cards), ORION-33 (procedure-extraction plan)
pmem, thread orion-phrase-context-1788715237RESULT.1788716400 · REFERENCE.1788721200 (gold) · RESULT.1788722200 · RESULT.1788721216 · RESULT.1788724066 (negative) · REFERENCE.1788738100 · RESULT.1788728911 · LESSON.1788742600 (canon synthesis) · STATUS.1788744000 (this report)
deployed judgeQwen3-8B on 192.168.206.10:8080; Mistral-7B on 192.168.1.134:8086; five GGUFs staged for 4-minute swaps