Day 26 建立了三個簡單基準：多數類別、知識不足即拒答的部分規則，以及使用 Ridge 迴歸（Ridge Regression）的結構化序位模型。結構化模型的平均準確率高於多數類別，但這仍不能告訴我們階層知識、門控、混合檢索或重排序各自貢獻了什麼。

假設完整系統比簡單基準多出 5 個百分點，可能有三種完全不同的解釋：每個元件都有一點幫助、只有其中一個元件真正有用，或多個元件互相作用，單獨存在時反而沒有增益。只報完整系統分數，無法區分這三種情況。

消融實驗要做的，就是從同一個完整方法出發，每次只移除或替換一個元件，其餘條件保持一致。下圖是概念示意：每條平行管線都使用相同起點與相同衡量裝置，唯一差異是某一個透明缺口。

上圖表達的是「控制變因」，不是「把系統隨意拆壞」。如果移除階層結構時，同時換了提示詞、候選數與生成模型，結果差異就不能只歸因於階層結構。

本篇會完成兩層產物：

1. 稽核 Day 18 到 Day 25 已有的元件級配對，確認它們各自固定了哪些條件、能回答什麼，以及不能回答什麼。
2. 建立完整方法識別字 P0 與六個單一元件移除版本的病患級折外登錄表，但尚未執行的結果欄一律維持 null；null 代表沒有結果，不是結果等於零。

本篇不讀取病患資料列、參考標籤或逐筆預測，也不呼叫模型或網路。輸出是可重跑的消融「就緒稽核」，不是新的病患分類效果。

本篇會用到的名詞

消融實驗回答的問題，比排行榜更窄

消融實驗（Ablation Study）嘗試回答：「在目前這一套完整配置與資料範圍內，拿掉這個元件後，結果如何改變？」

它不會自動證明：

• 這個元件在所有模型、醫院或資料版本都有效。
• 元件造成了臨床結果改善。
• 元件可以單獨部署。
• 拿掉元件得到的差異完全沒有隨機誤差。

一次只改一件事是必要條件，但不是充分條件。元件可能互相作用，例如只有「階層結構加重排序」同時存在時才有幫助；單一移除實驗測到的是元件在目前完整方法中的條件性貢獻，不是跨情境不變的自然定律。

醫療預測研究還需要把資料、模型、評估與結果完整報告。TRIPOD+AI 強調透明、完整且可重現的報告，也要求模型效能估計附帶不確定性；它是報告指引，不是模型品質認證或開發處方。TRIPOD+AI statement

先定義「只改一件事」

控制變因（Controlled Variable）是一次比較中保持不變的條件。對本系列而言，至少包括：

• 相同 1,267 筆研究紀錄與相同納入政策。
• 相同外層折分派與隨機種子時程。
• 相同韓國急診檢傷與急迫度分級量表（Korean Triage and Acuity Scale, KTAS）專家重新判定級數 KTAS_expert。
• 相同大型語言模型（Large Language Model, LLM）與模型識別字。
• 相同提示詞、輸出格式、知識庫版本與候選數。
• 相同指標、覆蓋率定義與拒答分母。

檢索增強生成（Retrieval-Augmented Generation, RAG）會先取回外部證據，再讓生成模型依證據產生回答。原始 RAG 研究把生成模型的參數內知識與可檢索的外部知識分開，這也是本系列能將查詢表示、知識結構、檢索、重排序與生成拆成消融軸的基礎。Lewis 等人的 RAG 論文

用一個小例子檢查混雜

假設完整方法使用：

dual query + hierarchical knowledge + safety union + hybrid retrieval + reranker

若要測試重排序器，合法配對應是：

完整：dual + hierarchical + safety union + hybrid + reranker on
移除：dual + hierarchical + safety union + hybrid + reranker off

如果「移除版」同時把 hybrid 改成 dense，那麼唯一改變不再是重排序，檢索方法也變了。即使分數下降，也無法判斷是哪一項造成。

差值的方向必須先約定

對越高越好的指標，可以定義元件貢獻：

[
Contribution_j = M_{full} - M_{minus\ j}
]

其中：

• (j) 是被移除的元件。
• (M_{full}) 是完整 P0 在同一批折外紀錄上的指標。
• (M_{minus\ j}) 是只移除元件 (j) 的版本。
• 正值表示完整版本在這個指標上較高。

例如完整方法的平衡準確率是 0.62，移除重排序後是 0.59，差值為 (0.62-0.59=0.03)，也就是 3 個百分點。這仍需要配對信賴區間，不能只憑正值宣稱效果成立。

對嚴重檢傷不足率這種越低越好的指標，要反轉方向：

[
Safety\ Contribution_j = U_{minus\ j} - U_{full}
]

若移除 Safety Union 後為 0.20，完整方法為 0.16，差值是 (0.20-0.16=0.04)。正值表示完整方法的嚴重檢傷不足率較低 4 個百分點。分子、分母與信賴區間會在 Day 28 處理；本篇的未執行 arms 不會先填示意數字。

A 到 G 的真實就緒狀態

Day 08 建立了 A 到 F 的控制比較，另把 G 標成探索性檢查。當時所有欄位都是規劃。Day 27 重新讀取 Day 18 到 Day 26 的公開產物後，得到下圖的就緒矩陣。

上圖最右欄全部是「未就緒」。這不代表既有工程沒有價值，而是提醒我們不要跨越證據範圍：元件級配對可以找流程錯誤與建立假設，卻不能直接替 1,267 筆病患估計分類增益。

公開稽核結果共記錄：

實驗 A：raw、semantic 與 dual 查詢表示

A 唯一改變查詢表示：

• raw：原始數值與主訴。
• semantic：加入有來源支持的數值語意。
• dual：同時保留原始與語意表示。

Day 18 確實對同一筆合成案例、同一九筆平面候選、同一向量模型與同一前三名截點（Top-3）執行三種表示。預期規則在 raw、dual 排第二，在 semantic 排第三。

但分母只有一筆合成案例，而且 Day 18 明確標示為 single_synthetic_case_diagnostic_not_retrieval_benchmark。因此 A 的正確狀態是「工程診斷」，不能說 dual 優於 semantic，也不能用這一筆選正式表示。

實驗 B：flat 與 hierarchical 文件結構

B 唯一改變知識結構：平面（flat）或階層（hierarchical）。Day 22 共用六筆公開知識問題、向量模型、生成模型、提示詞、輸出 JSON 綱要（JSON Schema）、最終 Top-3，並同時關閉門控、稀疏檢索與重排序。

在四筆需要特定證據的問題中：

• Flat 取回 4／4 筆必要證據，六題固定檢查通過 6／6。
• Hierarchical 取回 3／4 筆必要證據，六題固定檢查通過 5／6。

這個差異定位到 Parent Top-2 漏掉「主要考量」父層，是有用的結構失敗案例。但六題是作者撰寫的公開知識問答，不是病患分類資料，所以只能寫成「本次元件級配對觀察」，不能寫成「平面架構整體優於階層架構」。

實驗 C：none、Hard、Soft 與 Safety Union

C 唯一改變門控策略。Day 24 的四條路徑共用八筆作者合成案例、相同候選分數、Base Top-2、合成參考級數與同票規則。

描述性方向計數如下：

Safety Union 在這八筆刻意設計案例保留 8／8 筆 Base，沒有合成檢傷不足，但增加兩筆合成檢傷過度。這些門檻、分數與參考級數都是作者合成，不能轉寫成檢傷不足率、臨床安全改善或正式 KTAS 門檻。

實驗 D1：Dense 與 Hybrid Retrieval

D1 唯一改變檢索方法。稠密檢索（Dense Retrieval）使用向量相似度；混合檢索（Hybrid Retrieval）則融合稠密與稀疏名次。Day 19 共用五筆可計分公開規則 probes、九筆候選、相同截點，並關閉門控、重排序與生成。

五筆容易問題中，Dense 與 Hybrid 的 Recall@1、Recall@3、Recall@5 與 MRR 都是 1.0。這是「測試題太容易，沒有形成差異」的觀察，不是兩種檢索已證明等效，也不能反過來刪掉稀疏檢索。

實驗 D2：相同候選成員，只改重排序

D2 是目前最接近標準單一變因配對的元件實驗。Day 20 對八筆較難公開規則 probes 使用完全相同的九筆候選成員，只比較 Hybrid 原順序與線上重排序後順序。

平均倒數排名（Mean Reciprocal Rank, MRR）從 0.9375 到 1.0，Recall@1 從 0.8125 到 0.9375；Recall@3 在前後都是 1.0。這表示重排序改變了名次，但沒有增加候選成員，也沒有提高 Top-3 必要證據覆蓋。

改善集中在一題，且輸入仍是公開規則 probes，不是病患列。正確結論是「線上服務整合與相同候選重排序不變量通過，八題中觀察到有限名次改變」，不是供應商普遍優越或病患安全改善。

實驗 E：完整 P0 逐一移除六個元件

P0 是本系列完整方法識別字，規劃包含 dual 表示、階層知識、Safety Union、Hybrid Retrieval、重排序與證據約束生成／拒答。

下圖要看的不是分數，而是每條 arm 的 delta。完整方法之外，每一條只改一個參數，所有結果仍標成尚未執行、metrics = null。

上圖把「沒有結果」畫得很明確。六條合法移除如下：

所有實驗版本（arms）必須共用相同研究族群、外層折分、內層選擇政策、生成與向量模型、提示詞、知識庫版本、種子時程與指標。每一筆病患在每個實驗版本下的預測也必須保留，Day 28 才能做配對不確定性估計。

實驗 F 與 G：模型敏感度不是元件選拔賽

F 固定完整管線與證據，只比較同一模型家族的 qwen3.5:0.8b-mlx 和 qwen3.5:4b-mlx。它回答的是效果方向是否依賴模型大小，不是為了看完測試結果再挑較高分的模型。

G 固定同一完整管線、提示詞與已取回證據，探索性比較 qwen3.5:4b-mlx 與 gemma4:e2b。跨模型家族差異只能標成探索性，不能反過來取代事先指定的主要模型。

目前 F、G 都沒有執行資料或病患 OOF 預測，所以公開結果中的 observation 與 metrics 保持空值。

開發 probes、內層 validation 與外層 OOF 的分工

這三種資料角色必須分開，否則消融研究會在不知不覺中對測試結果調參。

第一層：開發 probes

公開規則問題與作者合成案例可以快速檢查：格式能否解析、候選有沒有被錯刪、引用集合是否正確、元件不變量是否成立。它們可以揭露 bug，但不能估計病患效果。

第二層：內層 training／validation

提示詞、門檻、前 k 名截點（Top-k）、融合權重、模型檢查點（model checkpoint）或其他超參數，只能在每個外層訓練折內決定。巢狀交叉驗證（Nested Cross-Validation, nested CV）用內層做選擇、外層估計選擇後流程的表現。

官方機器學習文件指出，若在同一份非巢狀交叉驗證結果上選參數又估計泛化誤差，資訊可能洩漏並產生過度樂觀的分數；nested CV 用分開的內外層避免這個問題。scikit-learn nested versus non-nested cross-validation

第三層：外層 OOF evaluation

折外預測（Out-of-Fold prediction, OOF prediction）只應由外層訓練流程凍結後產生。每個 arm 使用完全相同的外層 validation rows，才能建立逐筆配對。

Day 26 的 B0 與 B2 OOF 結果已被我們看過。這表示相同折外分數不能再被描述成「未碰觸的鎖定測試（locked test）」，也不能拿來反覆挑選 P0 元件。後續若需要選門檻或提示詞，應使用 nested CV；若要真正未碰觸的最終評估，還需要新的外部資料或另行鎖定、從未參與開發的評估集。目前兩者都尚未完成。

Run Registry 要記錄什麼

執行登錄表（Run Registry）不是只有一個 run_id。每一個實驗版本（arm）、重複編號（repeat）與折分（fold）至少要保存：

在尚未執行的階段，登錄表只能保存 arm 設計；不能先填一個「預期 accuracy」。若執行中斷，狀態應記成 failed 並保留錯誤，不可把不完整 folds 當完整結果。

Day 27 新增檔案與分工

本篇使用 JavaScript 物件表示法（JavaScript Object Notation, JSON）保存契約與公開結果：

這支稽核程式只讀取 configs/ 與 results/public/。results/public/ 保存可公開聚合，不含逐筆病患資料；因此本篇不需要、也不會打開 Day 15 的逗號分隔值（Comma-Separated Values, CSV）病患輸入檔或 Day 26 的 6,335 筆本機預測。

先在自己的專案資料夾建立本篇完整檔案

接下來不會要求你前往任何程式碼網站。請在自己的電腦開啟專案資料夾，依下列順序建立檔案；每個程式碼區塊都是該檔案的完整內容，不含省略號。

本篇沿用 Day 08–09 的比較與研究契約、模型鎖，以及 Day 15、18–26 已完成的公開聚合；稽核只讀取設定與公開聚合，不讀取病患資料列、參考標籤或逐筆預測。以下是 Day 27 新增的完整消融登錄設定、experiments 子套件、runner、測試與技術圖片生成程式；公開摘要、完整本機稽核與 run manifest 由執行入口產生，不需要人工填入結果。

先從專案根目錄建立需要的資料夾：

mkdir -p configs/experiments src/triage_rag/experiments scripts scripts/figures/day-27 tests results/public results/runs/day-27

如果指令沒有印出訊息是正常的。可用 test -d 資料夾路徑 && echo "資料夾已建立" 驗證單一資料夾。接著使用你熟悉的文字編輯器新增各檔案，把對應區塊完整貼入後儲存。

檔案 1：建立 configs/experiments/day-27-ablation-registry.json

鎖定 A–G 證據範圍、完整 P0 與六個單一移除版本、資料角色、固定條件、來源雜湊及執行登錄欄位。

請在文字編輯器建立 configs/experiments/day-27-ablation-registry.json，貼入以下完整內容並儲存：

{
  "schema_version": 1,
  "experiment_id": "day-27-locked-ablation-readiness-and-future-oof-registry",
  "scope": "aggregate_component_ablation_readiness_audit_and_future_patient_oof_plan_not_model_selection_or_clinical_evaluation",
  "sources": {
    "day08_contract_path": "configs/experiments/day-08-controlled-comparisons.json",
    "day08_contract_sha256": "dd2f9486712b9abcd22e4b2b71130f6b8bb06d147caa14f3807ff89c25dd2cd0",
    "day09_protocol_path": "configs/experiments/day-09-research-protocol.json",
    "day09_protocol_sha256": "be9954891978cf654cf0031adaea6fa37d4e940d067b3e02293b3199a9cf0e19",
    "model_lock_path": "configs/models/local-ollama-model-lock.json",
    "model_lock_sha256": "0cd15cb4d7e735a19cbe439fe9a9344f717029694a4441764f67e396033cc7f4",
    "reranker_lock_path": "configs/models/day-20-cohere-reranker-lock.json",
    "reranker_lock_sha256": "e9a320c2f10aa2483df431173765b938e906df8ff3ab60a2cb63f28658c26ff8",
    "day15_public_path": "results/public/day-15-data-quality-and-splits.json",
    "day15_public_sha256": "4fa89330a10b4278c9c249ec3360a600ad904fc0c5137c32757d4bfaf706d4ad",
    "day18_public_path": "results/public/day-18-numeric-embedding-smoke.json",
    "day18_public_sha256": "510c2f93cf09c6191b151c934b49abf680c2a778a212813536a80a7a7547ac6d",
    "day19_public_path": "results/public/day-19-retrieval-benchmark.json",
    "day19_public_sha256": "48d8ed74543a70cb5ba32d7b4edb918917804bbca3535bcdcbe332b33271b74d",
    "day20_public_path": "results/public/day-20-cohere-rerank-benchmark.json",
    "day20_public_sha256": "15518089a48d3e0e973fc85c150c3e3338dad4d64fff9fddbc43e58d327145d8",
    "day21_public_path": "results/public/day-21-flat-basic-rag.json",
    "day21_public_sha256": "7d5ca9f3594da7d13da3c323244303ae2d52f7d66e219fa6608e1d18705ec3de",
    "day22_public_path": "results/public/day-22-hierarchical-rag.json",
    "day22_public_sha256": "fa3150c5ce32e6fd16bc9b30306e0f09526b46965179939a4f81afd0d090e82e",
    "day23_public_path": "results/public/day-23-vital-hard-gate.json",
    "day23_public_sha256": "e2533938753e362fedffd6964b1209eed9aa6334ccb945c296541a603a85aa93",
    "day24_public_path": "results/public/day-24-safety-union-gate.json",
    "day24_public_sha256": "561a599b562b31b03faabafc39c074b0209708af3a53c5b6325a18db431674fb",
    "day25_public_path": "results/public/day-25-grounded-output-and-abstention.json",
    "day25_public_sha256": "af3b0245dabded6891bee989d292349c943f60b3e24ed00b2d57089b4a11fa65",
    "day26_contract_path": "configs/baselines/day-26-fair-baselines.json",
    "day26_contract_sha256": "ff6c36653da05e83276bd1f5ee671069afa5283b19a94e7103228de2a2f88dd6",
    "day26_public_path": "results/public/day-26-simple-baselines.json",
    "day26_public_sha256": "76c4a296511474b93e97a708e776e6709193bec9316665e45a9d2cb15022123a"
  },
  "audit_policy": {
    "uses_patient_rows": false,
    "uses_reference_labels": false,
    "reads_only_configs_and_public_aggregate_results": true,
    "cross_task_metric_ranking_forbidden": true,
    "patient_oof_effect_estimates_produced": false,
    "confirmatory_statistics_eligible": false,
    "single_changed_factor_required": true,
    "fixed_conditions_must_be_explicit": true,
    "planned_cells_must_keep_metrics_null": true
  },
  "stage_registry": [
    {
      "id": "A",
      "name": "查詢表示",
      "changed_factor": "query_representation",
      "levels": ["raw", "semantic", "dual"],
      "fixed_conditions": ["same_one_synthetic_case", "same_nine_flat_chunks", "same_dense_embedding_model", "same_top_k"],
      "evidence_source": "day18_public",
      "execution_status": "diagnostic_only",
      "observed_scope": "single_synthetic_case_embedding_and_retrieval_smoke",
      "patient_oof_ready": false,
      "effect_claim_allowed": false
    },
    {
      "id": "B",
      "name": "文件結構",
      "changed_factor": "knowledge_structure",
      "levels": ["flat", "hierarchical"],
      "fixed_conditions": ["same_six_public_knowledge_questions", "same_embedding_model", "same_generator", "same_prompt", "same_schema", "same_final_top_k", "no_gate", "no_reranker"],
      "evidence_source": "day22_public",
      "execution_status": "executed_component_pair",
      "observed_scope": "paired_author_written_public_knowledge_qa",
      "patient_oof_ready": false,
      "effect_claim_allowed": false
    },
    {
      "id": "C",
      "name": "門控方式",
      "changed_factor": "gate_strategy",
      "levels": ["none", "vital_hard", "complaint_soft", "safety_union"],
      "fixed_conditions": ["same_eight_author_written_synthetic_cases", "same_candidate_scores", "same_base_top_2", "same_synthetic_reference_levels", "same_tie_break"],
      "evidence_source": "day24_public",
      "execution_status": "executed_component_pair",
      "observed_scope": "paired_author_written_synthetic_gate_microbenchmark",
      "patient_oof_ready": false,
      "effect_claim_allowed": false
    },
    {
      "id": "D1",
      "name": "檢索方式",
      "changed_factor": "retrieval_method",
      "levels": ["dense", "hybrid"],
      "fixed_conditions": ["same_five_scored_public_rule_probes", "same_nine_flat_candidates", "same_cutoffs", "no_gate", "no_reranker", "no_generation"],
      "evidence_source": "day19_public",
      "execution_status": "executed_component_pair",
      "observed_scope": "author_written_public_rule_probe_benchmark",
      "patient_oof_ready": false,
      "effect_claim_allowed": false
    },
    {
      "id": "D2",
      "name": "重排序",
      "changed_factor": "reranker",
      "levels": ["off", "on"],
      "fixed_conditions": ["same_eight_scored_public_rule_probes", "same_nine_candidate_members", "same_first_stage_hybrid_order", "same_cutoffs", "no_generation"],
      "evidence_source": "day20_public",
      "execution_status": "executed_component_pair",
      "observed_scope": "author_written_public_rule_reranking_benchmark",
      "patient_oof_ready": false,
      "effect_claim_allowed": false
    },
    {
      "id": "E",
      "name": "完整 P0 逐一移除元件",
      "changed_factor": "one_removed_component_per_arm",
      "levels": ["full", "minus_semantic", "minus_hierarchy", "minus_safety_union", "minus_sparse_retrieval", "minus_reranker", "minus_grounding_abstention"],
      "fixed_conditions": ["same_patient_cohort", "same_outer_folds", "same_inner_selection_policy", "same_model", "same_prompt", "same_knowledge_version", "same_metrics"],
      "evidence_source": null,
      "execution_status": "planned_not_executed",
      "observed_scope": "none",
      "patient_oof_ready": false,
      "effect_claim_allowed": false
    },
    {
      "id": "F",
      "name": "生成模型大小敏感度",
      "changed_factor": "generator_model_size",
      "levels": ["qwen3.5:0.8b-mlx", "qwen3.5:4b-mlx"],
      "fixed_conditions": ["same_full_pipeline", "same_patient_cohort", "same_outer_folds", "same_prompt", "same_retrieved_evidence", "same_output_contract"],
      "evidence_source": null,
      "execution_status": "planned_not_executed",
      "observed_scope": "none",
      "patient_oof_ready": false,
      "effect_claim_allowed": false
    },
    {
      "id": "G",
      "name": "跨生成模型家族探索性敏感度",
      "changed_factor": "generator_model_family",
      "levels": ["qwen3.5:4b-mlx", "gemma4:e2b"],
      "fixed_conditions": ["same_full_pipeline", "same_patient_cohort", "same_outer_folds", "same_prompt", "same_retrieved_evidence", "not_used_to_select_confirmatory_model"],
      "evidence_source": null,
      "execution_status": "planned_not_executed",
      "observed_scope": "none",
      "patient_oof_ready": false,
      "effect_claim_allowed": false
    }
  ],
  "future_patient_oof_arms": [
    {
      "arm_id": "E-full-P0",
      "role": "reference_full_pipeline",
      "delta": null,
      "result_status": "not_executed",
      "metrics": null
    },
    {
      "arm_id": "E-minus-semantic",
      "role": "single_component_ablation",
      "delta": {"parameter": "query_representation", "from": "dual", "to": "raw_only"},
      "result_status": "not_executed",
      "metrics": null
    },
    {
      "arm_id": "E-minus-hierarchy",
      "role": "single_component_ablation",
      "delta": {"parameter": "knowledge_structure", "from": "hierarchical", "to": "flat"},
      "result_status": "not_executed",
      "metrics": null
    },
    {
      "arm_id": "E-minus-safety-union",
      "role": "single_component_ablation",
      "delta": {"parameter": "gate_strategy", "from": "safety_union", "to": "none"},
      "result_status": "not_executed",
      "metrics": null
    },
    {
      "arm_id": "E-minus-sparse-retrieval",
      "role": "single_component_ablation",
      "delta": {"parameter": "retrieval_method", "from": "hybrid", "to": "dense"},
      "result_status": "not_executed",
      "metrics": null
    },
    {
      "arm_id": "E-minus-reranker",
      "role": "single_component_ablation",
      "delta": {"parameter": "reranker", "from": "on", "to": "off"},
      "result_status": "not_executed",
      "metrics": null
    },
    {
      "arm_id": "E-minus-grounding-abstention",
      "role": "single_component_ablation",
      "delta": {"parameter": "grounding_and_abstention", "from": "evidence_aware", "to": "format_only"},
      "result_status": "not_executed",
      "metrics": null
    }
  ],
  "future_patient_oof_fixed_conditions": {
    "cohort": "same_1267_records_after_locked_eligibility_policy",
    "split": "same_outer_record_level_folds_for_every_arm",
    "patient_identifier_available": false,
    "patient_level_independence_claim_allowed": false,
    "reference_label": "KTAS_expert",
    "human_reference_not_model_input": "KTAS_RN",
    "generator": "qwen3.5:4b-mlx",
    "embedding": "qwen3-embedding:4b",
    "prompt": "same_version_for_every_arm",
    "knowledge_base": "same_version_for_every_arm",
    "randomness": "same_seed_schedule_and_repeated_runs",
    "primary_metrics": ["balanced_accuracy", "macro_f1", "mean_absolute_error", "severe_undertriage_rate"],
    "coverage_metrics": ["coverage", "abstention_rate", "severe_undertriage_among_answered"],
    "paired_record_predictions_required": true
  },
  "data_role_policy": {
    "development_probes": "public_rule_or_synthetic_cases_may_debug_contracts_but_must_not_estimate_patient_effects",
    "inner_training_validation": "all_parameter_prompt_threshold_and_checkpoint_selection_must_occur_inside_each_outer_training_fold",
    "outer_oof_evaluation": "outer_validation_rows_are_used_once_for_frozen_arm_evaluation_and_never_for_tuning",
    "day15_current_status": "day26_results_have_already_been_observed_so_the_same_oof_scores_must_not_be_treated_as_an_untouched_locked_test_for_new_design_choices",
    "locked_external_test": "not_available",
    "required_remedy": "use_nested_cross_validation_for_new_selection_or_acquire_a_new_external_evaluation_set"
  },
  "run_registry_required_fields": [
    "run_id",
    "stage_id",
    "arm_id",
    "execution_status",
    "scope",
    "changed_parameter",
    "fixed_condition_hash",
    "source_hashes",
    "outer_repeat_id",
    "outer_fold_id",
    "seed",
    "model_ids",
    "prompt_hash",
    "knowledge_hash",
    "prediction_artifact_path",
    "metric_artifact_path",
    "started_at_utc",
    "git_commit"
  ],
  "outputs": {
    "public_summary_path": "results/public/day-27-ablation-readiness.json",
    "run_output_root": "results/runs/day-27",
    "full_result_filename": "day-27-ablation-readiness-audit.json"
  },
  "limitations": [
    "Day 18–25 的既有觀察分屬單一合成表示、公開規則 probes、公開知識問答與作者合成門控案例，不能跨任務共用分母或排名。",
    "B、C、D1、D2 只代表元件級配對已執行，不代表這些元件對 1,267 筆病患的分類效果。",
    "完整 P0、其六個單一元件移除版本、F 與 G 都沒有病患折外預測；所有 metrics 必須維持 null。",
    "Day 26 結果已被觀察，不能再把相同 OOF 結果當未碰觸測試集來選元件、提示詞、門檻或模型。",
    "公開資料沒有病患識別碼，未來即使使用相同 outer folds，也不能宣稱病患層級獨立。",
    "本篇不計算信賴區間、不做確認性推論，也不能據此宣稱臨床安全、有效或可部署。"
  ]
}

儲存後先確認檔名與相對路徑完全一致，再繼續建立下一個檔案。

檔案 2：建立 src/triage_rag/experiments/__init__.py

建立 experiments 子套件入口，公開 Day 27 契約錯誤、契約驗證與就緒報告函式。

請在文字編輯器建立 src/triage_rag/experiments/__init__.py，貼入以下完整內容並儲存：

"""Experiment design and audit helpers."""

from .ablation import (
    AblationContractError,
    build_ablation_readiness_report,
    validate_ablation_contract,
)

__all__ = [
    "AblationContractError",
    "build_ablation_readiness_report",
    "validate_ablation_contract",
]

儲存後先確認檔名與相對路徑完全一致，再繼續建立下一個檔案。

檔案 3：建立 src/triage_rag/experiments/ablation.py

驗證 A–G、單一 delta、空結果、模型鎖與元件級不變量，再建立不跨任務混排的就緒摘要。

請在文字編輯器建立 src/triage_rag/experiments/ablation.py，貼入以下完整內容並儲存：

"""Validate and summarize Day 27 single-factor ablation readiness."""

from __future__ import annotations

from collections import Counter
from typing import Any, Mapping


JsonObject = dict[str, Any]


class AblationContractError(ValueError):
    """Raised when the Day 27 ablation registry stops being comparable."""


EXPECTED_STAGE_IDS = ["A", "B", "C", "D1", "D2", "E", "F", "G"]
EXECUTED_COMPONENT_STAGE_IDS = ["B", "C", "D1", "D2"]
PLANNED_STAGE_IDS = ["E", "F", "G"]


def _require(condition: bool, message: str) -> None:
    if not condition:
        raise AblationContractError(message)


def _stage_map(contract: Mapping[str, Any]) -> dict[str, Mapping[str, Any]]:
    stages = list(contract.get("stage_registry", []))
    ids = [str(stage.get("id")) for stage in stages]
    _require(ids == EXPECTED_STAGE_IDS, f"stage 順序或集合不符：{ids}")
    _require(len(ids) == len(set(ids)), "stage id 必須唯一")
    return {str(stage["id"]): stage for stage in stages}


def validate_ablation_contract(
    contract: Mapping[str, Any], sources: Mapping[str, Mapping[str, Any]]
) -> JsonObject:
    """Validate single-factor deltas, planning status, and source boundaries."""

    _require(contract.get("schema_version") == 1, "只支援 schema_version=1")
    policy = contract.get("audit_policy", {})
    for field in [
        "reads_only_configs_and_public_aggregate_results",
        "cross_task_metric_ranking_forbidden",
        "single_changed_factor_required",
        "fixed_conditions_must_be_explicit",
        "planned_cells_must_keep_metrics_null",
    ]:
        _require(policy.get(field) is True, f"audit_policy.{field} 必須為 true")
    for field in [
        "uses_patient_rows",
        "uses_reference_labels",
        "patient_oof_effect_estimates_produced",
        "confirmatory_statistics_eligible",
    ]:
        _require(policy.get(field) is False, f"audit_policy.{field} 必須為 false")

    stages = _stage_map(contract)
    changed_factors = []
    for stage_id, stage in stages.items():
        changed = str(stage.get("changed_factor", "")).strip()
        _require(bool(changed), f"stage {stage_id} 缺少 changed_factor")
        changed_factors.append(changed)
        levels = list(stage.get("levels", []))
        fixed = list(stage.get("fixed_conditions", []))
        _require(len(levels) >= 2, f"stage {stage_id} 至少需要兩個 levels")
        _require(len(fixed) >= 3, f"stage {stage_id} 必須明列固定條件")
        _require(len(fixed) == len(set(fixed)), f"stage {stage_id} 固定條件重複")
        _require(stage.get("patient_oof_ready") is False, f"stage {stage_id} 不可標成病患 OOF ready")
        _require(stage.get("effect_claim_allowed") is False, f"stage {stage_id} 不可允許效果宣稱")

    _require(stages["A"]["execution_status"] == "diagnostic_only", "A 必須維持 diagnostic_only")
    for stage_id in EXECUTED_COMPONENT_STAGE_IDS:
        _require(
            stages[stage_id]["execution_status"] == "executed_component_pair",
            f"{stage_id} 必須標成 executed_component_pair",
        )
        _require(stages[stage_id]["evidence_source"], f"{stage_id} 缺少 evidence_source")
    for stage_id in PLANNED_STAGE_IDS:
        _require(
            stages[stage_id]["execution_status"] == "planned_not_executed",
            f"{stage_id} 必須維持 planned_not_executed",
        )
        _require(stages[stage_id]["evidence_source"] is None, f"{stage_id} 不可偽造 evidence_source")

    arms = list(contract.get("future_patient_oof_arms", []))
    _require(len(arms) == 7, "E 應有一個 full 與六個單一移除 arms")
    arm_ids = [str(arm.get("arm_id")) for arm in arms]
    _require(len(arm_ids) == len(set(arm_ids)), "future arm id 必須唯一")
    full = arms[0]
    _require(full.get("role") == "reference_full_pipeline" and full.get("delta") is None, "第一個 arm 必須是 full P0")
    delta_parameters: list[str] = []
    for arm in arms:
        _require(arm.get("result_status") == "not_executed", f"{arm.get('arm_id')} 不可填入未執行結果")
        _require(arm.get("metrics") is None, f"{arm.get('arm_id')} metrics 必須為 null")
        if arm.get("role") == "single_component_ablation":
            delta = arm.get("delta")
            _require(isinstance(delta, dict), f"{arm.get('arm_id')} 缺少 delta")
            _require(set(delta) == {"parameter", "from", "to"}, f"{arm.get('arm_id')} delta 必須只描述一個參數")
            _require(delta["from"] != delta["to"], f"{arm.get('arm_id')} from 與 to 不可相同")
            delta_parameters.append(str(delta["parameter"]))
    _require(len(delta_parameters) == 6, "必須有六個單一元件移除 arms")
    _require(len(delta_parameters) == len(set(delta_parameters)), "每個移除 arm 必須對應不同參數")

    day08 = sources["day08"]
    day09 = sources["day09"]
    _require(day08.get("status") == "planning", "Day 08 仍應是 planning")
    day08_ids = [str(stage["id"]) for stage in day08["stages"]]
    _require(day08_ids == ["A", "B", "C", "D1", "D2", "E", "F"], "Day 08 stage 集合漂移")
    _require(day08["exploratory_checks"][0]["id"] == "G", "Day 08 必須保留探索性 G")
    _require(day09.get("status") == "planning", "Day 09 尚未完成確認性鎖定")
    _require(day09["analysis_policy"]["multiplicity_adjustment"] == "to-be-locked-before-test", "Day 09 multiplicity 不得假裝已鎖定")
    _require(day09["analysis_policy"]["safety_margins"] == "to-be-locked-with-clinical-review", "Day 09 safety margin 不得假裝已鎖定")

    model_lock = sources["model_lock"]["models"]
    _require(
        stages["F"]["levels"]
        == [model_lock["small_generator_sensitivity"]["name"], model_lock["primary_generator"]["name"]],
        "F 模型名稱與 model lock 不一致",
    )
    _require(
        stages["G"]["levels"]
        == [model_lock["primary_generator"]["name"], model_lock["exploratory_cross_family_generator"]["name"]],
        "G 模型名稱與 model lock 不一致",
    )

    day26 = sources["day26"]
    _require(day26["interpretation_boundaries"]["confirmatory_inference_eligible"] is False, "Day 26 不可變成確認性結果")
    _require(day26["interpretation_boundaries"]["model_selection_performed"] is False, "Day 26 不可宣稱已選模型")
    day26_registry = {item["id"]: item["status"] for item in day26["comparison_registry"]}
    _require(day26_registry.get("P0") == "component_contracts_only_not_patient_oof_ready", "P0 尚未病患 OOF ready")

    roles = contract["data_role_policy"]
    _require(roles["locked_external_test"] == "not_available", "不得假裝已有外部 locked test")
    _require("nested_cross_validation" in roles["required_remedy"], "後續選擇必須要求 nested cross-validation")
    return {
        "stage_ids_exact": True,
        "single_changed_factor_declared_per_stage": True,
        "future_full_plus_six_single_component_arms": True,
        "all_future_metrics_null": True,
        "day08_and_day09_remain_planning": True,
        "model_names_match_lock": True,
        "p0_patient_oof_ready": False,
        "locked_external_test_available": False,
        "cross_task_ranking_forbidden": True,
    }


def _metric(methods: Mapping[str, Any], method: str, field: str, cutoff: str | None = None) -> float:
    value = methods[method][field]
    if cutoff is not None:
        value = value[cutoff]
    return float(value)


def build_ablation_readiness_report(
    contract: Mapping[str, Any], sources: Mapping[str, Mapping[str, Any]]
) -> JsonObject:
    """Build a deterministic public audit from existing aggregate results."""

    checks = validate_ablation_contract(contract, sources)
    stages = _stage_map(contract)
    day18 = sources["day18"]
    day19 = sources["day19"]
    day20 = sources["day20"]
    day22 = sources["day22"]
    day24 = sources["day24"]

    _require(day18["retrieval_smoke"]["status"] == "single_synthetic_case_diagnostic_not_retrieval_benchmark", "Day 18 不可升級為正式 A 效果")
    _require(day18["checks"]["patient_and_label_data_absent"] is True, "Day 18 邊界漂移")
    _require(day22["checks"]["day21_questions_generation_prompt_and_schema_reused"] is True, "Day 22 未固定問題、prompt 或 schema")
    _require(day22["checks"]["bm25_reranker_and_gate_disabled_for_both"] is True, "Day 22 混入其他變因")
    _require(day24["checks"]["all_methods_share_the_same_scores_and_base_top_2"] is True, "Day 24 分數或 Base 不同")
    _require(day24["checks"]["statistics_eligible"] is False, "Day 24 不可變成統計效果")
    _require(day19["checks"]["same_nine_candidates_for_all_methods"] is True, "Day 19 候選集合不一致")
    _require(day19["checks"]["generation_gate_and_reranker_disabled"] is True, "Day 19 混入生成、門控或重排序")
    _require(day20["checks"]["same_nine_candidates_before_and_after"] is True, "Day 20 前後候選成員不同")
    _require(day20["checks"]["reranker_is_not_a_generation_or_abstention_model"] is True, "Day 20 重排序器角色漂移")

    observations: list[JsonObject] = []
    a_ranks = {
        name: int(value["expected_rule_rank"])
        for name, value in day18["retrieval_smoke"]["results"].items()
    }
    observations.append(
        {
            "stage_id": "A",
            "name": stages["A"]["name"],
            "execution_status": stages["A"]["execution_status"],
            "observed_scope": stages["A"]["observed_scope"],
            "patient_oof_ready": False,
            "effect_claim_allowed": False,
            "observation": {
                "synthetic_case_count": 1,
                "expected_rule_rank": a_ranks,
                "query_pairwise_cosine": day18["query_pairwise_cosine"],
                "interpretation": "只確認三種表示可執行；單一案例名次不是 A 的效果估計。",
            },
        }
    )
    b_aggregate = day22["aggregate"]
    observations.append(
        {
            "stage_id": "B",
            "name": stages["B"]["name"],
            "execution_status": stages["B"]["execution_status"],
            "observed_scope": stages["B"]["observed_scope"],
            "patient_oof_ready": False,
            "effect_claim_allowed": False,
            "observation": {
                "case_count": 6,
                "flat_passed_case_count": b_aggregate["flat"]["passed_case_count"],
                "hierarchical_passed_case_count": b_aggregate["hierarchical"]["passed_case_count"],
                "flat_required_evidence_hit_count": b_aggregate["flat"]["required_evidence_retrieval_hit_count"],
                "hierarchical_required_evidence_hit_count": b_aggregate["hierarchical"]["required_evidence_retrieval_hit_count"],
                "required_evidence_denominator": 4,
                "interpretation": "只描述六筆公開知識題的結構配對，不是病患分類效果。",
            },
        }
    )
    c_aggregate = day24["aggregate"]
    observations.append(
        {
            "stage_id": "C",
            "name": stages["C"]["name"],
            "execution_status": stages["C"]["execution_status"],
            "observed_scope": stages["C"]["observed_scope"],
            "patient_oof_ready": False,
            "effect_claim_allowed": False,
            "observation": {
                "synthetic_case_count": c_aggregate["case_count"],
                "direction_counts": c_aggregate["direction_counts"],
                "base_preserved_by_union_count": c_aggregate["base_preserved_by_union_count"],
                "interpretation": "方向計數只屬於八筆作者合成門控案例，不是安全率。",
            },
        }
    )
    d1_methods = day19["evaluation"]["aggregate_supported_metrics"]
    observations.append(
        {
            "stage_id": "D1",
            "name": stages["D1"]["name"],
            "execution_status": stages["D1"]["execution_status"],
            "observed_scope": stages["D1"]["observed_scope"],
            "patient_oof_ready": False,
            "effect_claim_allowed": False,
            "observation": {
                "scored_probe_count": d1_methods["dense"]["supported_probe_count"],
                "dense": {
                    "recall_at_1": _metric(d1_methods, "dense", "mean_recall_at_k", "1"),
                    "mean_reciprocal_rank": _metric(d1_methods, "dense", "mean_reciprocal_rank"),
                },
                "hybrid": {
                    "recall_at_1": _metric(d1_methods, "hybrid", "mean_recall_at_k", "1"),
                    "mean_reciprocal_rank": _metric(d1_methods, "hybrid", "mean_reciprocal_rank"),
                },
                "interpretation": "五筆容易公開規則 probes 兩者皆滿分，不能據此宣稱等效或選方法。",
            },
        }
    )
    d2_methods = day20["evaluation"]["aggregate_supported_metrics"]
    observations.append(
        {
            "stage_id": "D2",
            "name": stages["D2"]["name"],
            "execution_status": stages["D2"]["execution_status"],
            "observed_scope": stages["D2"]["observed_scope"],
            "patient_oof_ready": False,
            "effect_claim_allowed": False,
            "observation": {
                "scored_probe_count": d2_methods["hybrid_before"]["supported_probe_count"],
                "candidate_membership_unchanged": True,
                "hybrid_before": {
                    "recall_at_1": _metric(d2_methods, "hybrid_before", "mean_recall_at_k", "1"),
                    "recall_at_3": _metric(d2_methods, "hybrid_before", "mean_recall_at_k", "3"),
                    "mean_reciprocal_rank": _metric(d2_methods, "hybrid_before", "mean_reciprocal_rank"),
                    "ndcg_at_3": _metric(d2_methods, "hybrid_before", "mean_ndcg_at_k", "3"),
                },
                "reranker_on": {
                    "recall_at_1": _metric(d2_methods, "cohere_after", "mean_recall_at_k", "1"),
                    "recall_at_3": _metric(d2_methods, "cohere_after", "mean_recall_at_k", "3"),
                    "mean_reciprocal_rank": _metric(d2_methods, "cohere_after", "mean_reciprocal_rank"),
                    "ndcg_at_3": _metric(d2_methods, "cohere_after", "mean_ndcg_at_k", "3"),
                },
                "interpretation": "八筆公開規則 probes 的排序觀察；Top-3 成員覆蓋未增加，不是病患效果。",
            },
        }
    )
    for stage_id in PLANNED_STAGE_IDS:
        observations.append(
            {
                "stage_id": stage_id,
                "name": stages[stage_id]["name"],
                "execution_status": "planned_not_executed",
                "observed_scope": "none",
                "patient_oof_ready": False,
                "effect_claim_allowed": False,
                "observation": None,
            }
        )

    status_counts = Counter(item["execution_status"] for item in observations)
    return {
        "schema_version": 1,
        "experiment_id": contract["experiment_id"],
        "scope": contract["scope"],
        "summary": {
            "stage_count": len(observations),
            "diagnostic_only_count": status_counts["diagnostic_only"],
            "executed_component_pair_count": status_counts["executed_component_pair"],
            "planned_not_executed_count": status_counts["planned_not_executed"],
            "patient_oof_effect_ready_count": 0,
            "future_patient_oof_arm_count": len(contract["future_patient_oof_arms"]),
            "future_patient_oof_metrics_filled_count": 0,
            "cross_task_metric_ranking_forbidden": True,
            "confirmatory_statistics_eligible": False,
        },
        "contract_checks": checks,
        "stage_observations": observations,
        "future_patient_oof_arms": contract["future_patient_oof_arms"],
        "future_patient_oof_fixed_conditions": contract["future_patient_oof_fixed_conditions"],
        "data_role_policy": contract["data_role_policy"],
        "run_registry_required_fields": contract["run_registry_required_fields"],
        "claim_boundary": "既有數字只在各自元件級任務內描述；本篇沒有產生完整 P0 或任何移除版本的病患 OOF 效果。",
        "limitations": contract["limitations"],
    }

儲存後先確認檔名與相對路徑完全一致，再繼續建立下一個檔案。

檔案 4：建立 scripts/run_day27_ablation_audit.py

核對所有上游 SHA-256，只讀設定與公開聚合，寫出公開稽核、完整本機稽核及 run manifest。

請在文字編輯器建立 scripts/run_day27_ablation_audit.py，貼入以下完整內容並儲存：

#!/usr/bin/env python3
"""Run the Day 27 aggregate ablation-readiness audit."""

from __future__ import annotations

import argparse
import platform
import sys
from datetime import datetime, timezone
from pathlib import Path

from triage_rag.experiments.ablation import build_ablation_readiness_report
from triage_rag.reproducibility import (
    canonical_json_bytes,
    file_record,
    git_state,
    load_json,
    sha256_bytes,
    sha256_file,
    write_json,
)


PROJECT_ROOT = Path(__file__).resolve().parents[1]
DEFAULT_CONTRACT = "configs/experiments/day-27-ablation-registry.json"


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description="稽核 Day 27 單一變因消融就緒狀態，不執行病患模型。"
    )
    parser.add_argument("--contract", default=DEFAULT_CONTRACT)
    return parser.parse_args()


def _validate_source_hashes(contract: dict) -> list[str]:
    paths: list[str] = []
    for key, relative_path in contract["sources"].items():
        if not key.endswith("_path"):
            continue
        hash_key = f"{key[:-5]}_sha256"
        if hash_key not in contract["sources"]:
            raise ValueError(f"來源 {key} 缺少 SHA-256")
        path = PROJECT_ROOT / relative_path
        observed = sha256_file(path)
        if observed != contract["sources"][hash_key]:
            raise ValueError(f"{relative_path} SHA-256 不符：{observed}")
        paths.append(relative_path)
    return paths


def _load_sources(contract: dict) -> dict[str, dict]:
    sources = contract["sources"]
    return {
        "day08": load_json(PROJECT_ROOT / sources["day08_contract_path"]),
        "day09": load_json(PROJECT_ROOT / sources["day09_protocol_path"]),
        "model_lock": load_json(PROJECT_ROOT / sources["model_lock_path"]),
        "reranker_lock": load_json(PROJECT_ROOT / sources["reranker_lock_path"]),
        "day15": load_json(PROJECT_ROOT / sources["day15_public_path"]),
        "day18": load_json(PROJECT_ROOT / sources["day18_public_path"]),
        "day19": load_json(PROJECT_ROOT / sources["day19_public_path"]),
        "day20": load_json(PROJECT_ROOT / sources["day20_public_path"]),
        "day21": load_json(PROJECT_ROOT / sources["day21_public_path"]),
        "day22": load_json(PROJECT_ROOT / sources["day22_public_path"]),
        "day23": load_json(PROJECT_ROOT / sources["day23_public_path"]),
        "day24": load_json(PROJECT_ROOT / sources["day24_public_path"]),
        "day25": load_json(PROJECT_ROOT / sources["day25_public_path"]),
        "day26_contract": load_json(PROJECT_ROOT / sources["day26_contract_path"]),
        "day26": load_json(PROJECT_ROOT / sources["day26_public_path"]),
    }


def main() -> int:
    args = parse_args()
    contract = load_json(PROJECT_ROOT / args.contract)
    source_paths = _validate_source_hashes(contract)
    report = build_ablation_readiness_report(contract, _load_sources(contract))
    stable_sha256 = sha256_bytes(canonical_json_bytes(report))

    public_path = PROJECT_ROOT / contract["outputs"]["public_summary_path"]
    write_json(public_path, report)
    started_at = datetime.now(timezone.utc)
    run_id = f"{started_at.strftime('%Y%m%dT%H%M%S%fZ')}-{stable_sha256[:8]}"
    run_directory = PROJECT_ROOT / contract["outputs"]["run_output_root"] / run_id
    run_directory.mkdir(parents=True, exist_ok=False)
    full_path = run_directory / contract["outputs"]["full_result_filename"]
    write_json(full_path, report)

    manifest = {
        "manifest_schema_version": 1,
        "run_id": run_id,
        "experiment_id": contract["experiment_id"],
        "started_at_utc": started_at.isoformat().replace("+00:00", "Z"),
        "command": [sys.executable, *sys.argv],
        "documented_command": [
            "poetry",
            "run",
            "python",
            "scripts/run_day27_ablation_audit.py",
        ],
        "git": git_state(PROJECT_ROOT),
        "runtime": {"python": platform.python_version()},
        "inputs": [
            file_record(PROJECT_ROOT, path) for path in [args.contract, *source_paths]
        ],
        "parameters": {
            "stage_ids": [item["id"] for item in contract["stage_registry"]],
            "patient_rows_read": False,
            "reference_labels_read": False,
            "patient_models_executed": False,
        },
        "outputs": [
            file_record(PROJECT_ROOT, str(public_path.relative_to(PROJECT_ROOT))),
            file_record(PROJECT_ROOT, str(full_path.relative_to(PROJECT_ROOT))),
        ],
        "scope": contract["scope"],
        "privacy": "只讀取設定與可公開聚合結果；不讀取病患列、參考標籤或逐筆預測。",
    }
    write_json(run_directory / "run-manifest.json", manifest)

    summary = report["summary"]
    print("Day 27 消融就緒稽核：完成")
    print(f"stage：{summary['stage_count']} 個")
    print(f"元件級已執行配對：{summary['executed_component_pair_count']} 個")
    print(f"單一案例診斷：{summary['diagnostic_only_count']} 個")
    print(f"尚未執行：{summary['planned_not_executed_count']} 個")
    print(f"病患 OOF 效果 ready：{summary['patient_oof_effect_ready_count']} 個")
    print(f"未來 P0 消融 arms：{summary['future_patient_oof_arm_count']} 個，metrics 皆為 null")
    print(f"公開摘要：{public_path.relative_to(PROJECT_ROOT)}")
    print(f"執行目錄：{run_directory.relative_to(PROJECT_ROOT)}")
    print(f"穩定摘要：{stable_sha256}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

儲存後先確認檔名與相對路徑完全一致，再繼續建立下一個檔案。

檔案 5：建立 tests/test_ablation_registry.py

以十二項測試驗證 stage 順序、單一變因、空結果、來源模型鎖、候選不變與元件級範圍。

請在文字編輯器建立 tests/test_ablation_registry.py，貼入以下完整內容並儲存：

"""Tests for the Day 27 ablation-readiness registry."""

from __future__ import annotations

import copy
import json
import unittest
from pathlib import Path

from triage_rag.experiments.ablation import (
    AblationContractError,
    build_ablation_readiness_report,
    validate_ablation_contract,
)


ROOT = Path(__file__).resolve().parents[1]


def load_json(relative_path: str) -> dict:
    with (ROOT / relative_path).open(encoding="utf-8") as handle:
        return json.load(handle)


class Day27AblationRegistryTests(unittest.TestCase):
    @classmethod
    def setUpClass(cls) -> None:
        cls.contract = load_json("configs/experiments/day-27-ablation-registry.json")
        paths = cls.contract["sources"]
        cls.sources = {
            "day08": load_json(paths["day08_contract_path"]),
            "day09": load_json(paths["day09_protocol_path"]),
            "model_lock": load_json(paths["model_lock_path"]),
            "reranker_lock": load_json(paths["reranker_lock_path"]),
            "day15": load_json(paths["day15_public_path"]),
            "day18": load_json(paths["day18_public_path"]),
            "day19": load_json(paths["day19_public_path"]),
            "day20": load_json(paths["day20_public_path"]),
            "day21": load_json(paths["day21_public_path"]),
            "day22": load_json(paths["day22_public_path"]),
            "day23": load_json(paths["day23_public_path"]),
            "day24": load_json(paths["day24_public_path"]),
            "day25": load_json(paths["day25_public_path"]),
            "day26_contract": load_json(paths["day26_contract_path"]),
            "day26": load_json(paths["day26_public_path"]),
        }

    def test_locked_contract_passes(self) -> None:
        checks = validate_ablation_contract(self.contract, self.sources)
        self.assertTrue(checks["single_changed_factor_declared_per_stage"])
        self.assertFalse(checks["p0_patient_oof_ready"])
        self.assertFalse(checks["locked_external_test_available"])

    def test_stage_order_drift_is_rejected(self) -> None:
        broken = copy.deepcopy(self.contract)
        broken["stage_registry"][0], broken["stage_registry"][1] = (
            broken["stage_registry"][1],
            broken["stage_registry"][0],
        )
        with self.assertRaisesRegex(AblationContractError, "stage 順序"):
            validate_ablation_contract(broken, self.sources)

    def test_missing_fixed_conditions_is_rejected(self) -> None:
        broken = copy.deepcopy(self.contract)
        broken["stage_registry"][1]["fixed_conditions"] = ["same_cases"]
        with self.assertRaisesRegex(AblationContractError, "固定條件"):
            validate_ablation_contract(broken, self.sources)

    def test_planned_stage_cannot_claim_execution(self) -> None:
        broken = copy.deepcopy(self.contract)
        broken["stage_registry"][5]["execution_status"] = "executed_component_pair"
        with self.assertRaisesRegex(AblationContractError, "E 必須維持"):
            validate_ablation_contract(broken, self.sources)

    def test_future_arm_metrics_must_remain_null(self) -> None:
        broken = copy.deepcopy(self.contract)
        broken["future_patient_oof_arms"][1]["metrics"] = {"balanced_accuracy": 0.9}
        with self.assertRaisesRegex(AblationContractError, "metrics 必須為 null"):
            validate_ablation_contract(broken, self.sources)

    def test_future_arm_delta_must_change_one_parameter(self) -> None:
        broken = copy.deepcopy(self.contract)
        broken["future_patient_oof_arms"][1]["delta"]["second_parameter"] = "hierarchy"
        with self.assertRaisesRegex(AblationContractError, "只描述一個參數"):
            validate_ablation_contract(broken, self.sources)

    def test_model_lock_drift_is_rejected(self) -> None:
        broken_sources = copy.deepcopy(self.sources)
        broken_sources["model_lock"]["models"]["primary_generator"]["name"] = "different-model"
        with self.assertRaisesRegex(AblationContractError, "F 模型名稱"):
            validate_ablation_contract(self.contract, broken_sources)

    def test_day22_mixed_factor_drift_is_rejected(self) -> None:
        broken_sources = copy.deepcopy(self.sources)
        broken_sources["day22"]["checks"]["bm25_reranker_and_gate_disabled_for_both"] = False
        with self.assertRaisesRegex(AblationContractError, "Day 22 混入其他變因"):
            build_ablation_readiness_report(self.contract, broken_sources)

    def test_day20_candidate_membership_drift_is_rejected(self) -> None:
        broken_sources = copy.deepcopy(self.sources)
        broken_sources["day20"]["checks"]["same_nine_candidates_before_and_after"] = False
        with self.assertRaisesRegex(AblationContractError, "候選成員不同"):
            build_ablation_readiness_report(self.contract, broken_sources)

    def test_report_counts_are_explicit(self) -> None:
        report = build_ablation_readiness_report(self.contract, self.sources)
        self.assertEqual(report["summary"]["stage_count"], 8)
        self.assertEqual(report["summary"]["diagnostic_only_count"], 1)
        self.assertEqual(report["summary"]["executed_component_pair_count"], 4)
        self.assertEqual(report["summary"]["planned_not_executed_count"], 3)
        self.assertEqual(report["summary"]["patient_oof_effect_ready_count"], 0)
        self.assertEqual(report["summary"]["future_patient_oof_metrics_filled_count"], 0)

    def test_component_observations_match_upstream_aggregates(self) -> None:
        report = build_ablation_readiness_report(self.contract, self.sources)
        observations = {item["stage_id"]: item for item in report["stage_observations"]}
        self.assertEqual(observations["B"]["observation"]["flat_required_evidence_hit_count"], 4)
        self.assertEqual(observations["B"]["observation"]["hierarchical_required_evidence_hit_count"], 3)
        self.assertEqual(observations["C"]["observation"]["direction_counts"]["safety_union"]["undertriage"], 0)
        self.assertEqual(observations["D1"]["observation"]["dense"]["recall_at_1"], 1.0)
        self.assertEqual(observations["D2"]["observation"]["hybrid_before"]["mean_reciprocal_rank"], 0.9375)
        self.assertEqual(observations["D2"]["observation"]["reranker_on"]["mean_reciprocal_rank"], 1.0)

    def test_report_is_deterministic_and_has_no_patient_effects(self) -> None:
        first = build_ablation_readiness_report(self.contract, self.sources)
        second = build_ablation_readiness_report(self.contract, self.sources)
        self.assertEqual(first, second)
        self.assertFalse(first["summary"]["confirmatory_statistics_eligible"])
        self.assertTrue(first["summary"]["cross_task_metric_ranking_forbidden"])
        self.assertTrue(
            all(arm["metrics"] is None for arm in first["future_patient_oof_arms"])
        )


if __name__ == "__main__":
    unittest.main()

儲存後先確認檔名與相對路徑完全一致，再繼續建立下一個檔案。

檔案 6：建立 scripts/figures/day-27/generate_day27_figures.py

使用 Pillow 讀取 Day 27 設定與公開結果，重建三張 1920×1080 繁中技術圖，不手動抄寫狀態或數字。

請在文字編輯器建立 scripts/figures/day-27/generate_day27_figures.py，貼入以下完整內容並儲存：

#!/usr/bin/env python3
"""Generate Day 27 ablation-readiness figures from locked JSON artifacts."""

from __future__ import annotations

import json
from pathlib import Path
from typing import Any

from PIL import Image, ImageDraw, ImageFont


ROOT = Path(__file__).resolve().parents[3]
CONFIG_PATH = ROOT / "configs/experiments/day-27-ablation-registry.json"
RESULT_PATH = ROOT / "results/public/day-27-ablation-readiness.json"
OUTPUT_DIRECTORY = ROOT / "articles/assets/day-27"
FONT_PATH = Path("/System/Library/Fonts/STHeiti Medium.ttc")

WIDTH, HEIGHT = 1920, 1080
NAVY = "#102A43"
BLUE = "#2869AE"
TEAL = "#168C8C"
ORANGE = "#EA8A2F"
RED = "#CC5157"
GRAY = "#66788A"
LIGHT = "#F6F9FC"
WHITE = "#FFFFFF"
INK = "#25384A"
MUTED = "#607386"
GRID = "#CCD9E5"
PALE_BLUE = "#EAF2FA"
PALE_TEAL = "#E7F5F3"
PALE_ORANGE = "#FFF1DF"
PALE_RED = "#FBEAEC"
PALE_GRAY = "#EEF2F5"


def load_json(path: Path) -> dict[str, Any]:
    with path.open(encoding="utf-8") as handle:
        return json.load(handle)


def font(size: int) -> ImageFont.FreeTypeFont:
    return ImageFont.truetype(str(FONT_PATH), size=size)


def canvas() -> tuple[Image.Image, ImageDraw.ImageDraw]:
    image = Image.new("RGB", (WIDTH, HEIGHT), LIGHT)
    return image, ImageDraw.Draw(image)


def header(draw: ImageDraw.ImageDraw, heading: str, subtitle: str) -> None:
    draw.text((90, 60), heading, fill=NAVY, font=font(54))
    draw.text((92, 132), subtitle, fill=MUTED, font=font(28))
    draw.rounded_rectangle((90, 188, 1830,