cyb50-quant/research/chinext50_regime_project/memory/2026-04-09.md

2026-04-09

• User asked to continue OpenSpec proposal work.
• Created OpenSpec change add-real-data-regime-input.
• Completed all required artifacts:
  • proposal.md
  • design.md
  • specs/regime-market-data-contract/spec.md
  • specs/point-in-time-breadth-merge/spec.md
  • specs/pipeline-data-quality-gate/spec.md
  • tasks.md
• Proposal scope: prioritize real data onboarding (P0), data contract, point-in-time merge, and pipeline data quality gate.
• User asked to start implementation immediately after proposal.
• Implemented data contract + quality gate stack:
  • Extended data/io.py with centralized validation, point-in-time sidecar merge, data quality report, and strict/non-strict gate evaluation.
  • Added data_quality defaults in config/regime.yaml.
  • Integrated pre-feature gate checks and data_quality_summary.json output into both pipelines/run_demo.py and pipelines/frozen_hypothesis_validation.py.
  • Added CLI flags: --sidecar-csv, --strict-data, --min-coverage.
  • Added tests: tests/test_data_io.py, tests/test_pipeline_data_quality_gate.py.
  • Updated README with real-data contract and quality-gate usage docs.
• Updated OpenSpec task checklist to 9/9 complete for add-real-data-regime-input.
• Verification notes:
  • Installed missing dependencies: pytest, PyYAML.
  • Ran tests: py -m pytest -q tests/test_data_io.py tests/test_pipeline_data_quality_gate.py tests/test_pipeline_demo.py tests/test_policy.py tests/test_utility.py -> 13 passed.
• Ran pipeline smoke checks successfully:
  • py pipelines/run_demo.py --output-dir outputs/demo_quality_check
  • py pipelines/frozen_hypothesis_validation.py --output-dir outputs/frozen_validation_quality_check
• Verified new quality artifact exists in both outputs: data_quality_summary.json.
• User asked to continue autonomously without confirmation until done.
• Archived completed change: add-real-data-regime-input (synced specs).
• Created and completed next change: strengthen-frozen-hypothesis-walkforward.
• Added frozen walk-forward module backtest/frozen_walkforward.py with:
  • train-window candidate evaluation and deterministic winner selection
  • test-window frozen evaluation without candidate re-selection
  • skipped-window status handling and audit summary aggregation
• Refactored pipelines/frozen_hypothesis_validation.py to use the new module.
• Added frozen validation config and candidate defaults in config/regime.yaml.
• Added tests:
  • tests/test_frozen_walkforward.py
  • tests/test_frozen_validation_pipeline.py
• Updated README with train-select/test-freeze semantics and new CLI options.
• Regression verification:
  • py -m pytest -q tests/test_frozen_walkforward.py tests/test_frozen_validation_pipeline.py tests/test_pipeline_data_quality_gate.py tests/test_data_io.py tests/test_pipeline_demo.py tests/test_policy.py tests/test_utility.py -> 18 passed.
  • smoke:
  • py pipelines/run_demo.py --output-dir outputs/demo_quality_check_v2 passed
  • py pipelines/frozen_hypothesis_validation.py --output-dir outputs/frozen_validation_quality_check_v2 passed
• Archived second change: strengthen-frozen-hypothesis-walkforward (synced specs).
• User asked to continue improving according to GPT Pro suggestions and keep using OpenSpec flow.
• Created and completed change add-event-anchored-diagnostics (proposal/design/specs/tasks + implementation + archive).
• Added event-anchor diagnostics:
  • backtest/events.py now supports transition event log generation and taxonomy labels:
  • crash_onset
  • false_rebound
  • true_repair
  • crowded_unwind
  • fallback state_transition
  • pipelines/run_demo.py now writes event_log.csv in addition to event_summary.csv.
• Added tests:
  • tests/test_event_diagnostics.py
  • updated tests/test_pipeline_data_quality_gate.py to validate event artifact output.
• Verification:
  • py -m pytest -q tests -> 21 passed.
  • smoke:
  • py pipelines/run_demo.py --output-dir outputs/demo_event_diag_check passed
  • py pipelines/frozen_hypothesis_validation.py --output-dir outputs/frozen_validation_event_diag_check passed
• Archived third change: add-event-anchored-diagnostics (synced new specs to main).
• Created and completed change enhance-execution-layer-constraints:
  • Added extreme-day cost multiplier and gap-shock cost in backtest/engine.py.
  • Added ledger diagnostics tracking_difference, tracking_error_20.
  • Added metrics fields tracking_diff_mean, tracking_diff_abs_mean, tracking_error_20_p95.
  • Added config defaults in config/regime.yaml.
  • Added tests tests/test_execution_constraints.py.
  • Archived and synced to main specs.
• Created and completed change extend-breadth-concentration-features:
  • Extended breadth inputs/defaults with top1_contribution_5, top10_contribution_5, sector_concentration_20.
  • Added derived concentration features and integrated them into crowding score.
  • Extended synthetic sample generation with new concentration fields.
  • Added tests tests/test_breadth_concentration_features.py.
  • Archived and synced to main specs.
• Latest regression verification:
  • py -m pytest -q tests -> 25 passed.
  • smoke:
  • py pipelines/run_demo.py --output-dir outputs/demo_breadth_ext_check passed
  • py pipelines/frozen_hypothesis_validation.py --output-dir outputs/frozen_validation_breadth_ext_check passed
• User reminded to reference chinext50_regime_build_handoff_2026-04-08.md; re-read with UTF-8 to avoid previous terminal mojibake and keep it as baseline guidance.
• User requested to continue directly by OpenSpec change flow without extra confirmation.
• Created and completed change build-real-data-pit-dataset:
  • Added PIT builder core in data/pit_builder.py (ordered sidecar merge + strict/non-strict quality evaluation + source metadata summary).
  • Added PIT CLI in pipelines/build_pit_dataset.py and exported API from data/__init__.py.
  • Added tests tests/test_pit_builder_pipeline.py:
  • ordered sidecar merge and base-row preservation
  • sidecar conflict failure
  • strict mode failure writes pit_quality_summary.json and blocks PIT output
  • non-strict success writes PIT output + summary
  • Updated README with PIT build command and output semantics.
• Verification after PIT change:
  • py -m pytest -q tests -> 29 passed.
  • smoke:
  • py pipelines/build_pit_dataset.py --market-csv outputs/pit_smoke/market.csv --sidecar-csv outputs/pit_smoke/sidecar.csv --output-path outputs/pit_smoke/chinext50_pit.csv passed
  • confirmed output artifacts include outputs/pit_smoke/chinext50_pit.csv and outputs/pit_smoke/pit_quality_summary.json.
• OpenSpec completion:
  • openspec.cmd instructions apply --change \"build-real-data-pit-dataset\" --json -> state: all_done (9/9 complete)
  • openspec.cmd archive build-real-data-pit-dataset --yes completed
  • new archive folder: openspec/changes/archive/2026-04-09-build-real-data-pit-dataset/
  • synced specs to main: openspec/specs/pit-dataset-construction/spec.md, openspec/specs/pit-build-artifacts/spec.md
• Created and completed change require-real-pit-input:
  • Added full PIT contract loader in data/io.py: FULL_PIT_REQUIRED_COLUMNS, validate_full_pit_data_contract, load_full_pit_data.
  • Runtime pipelines now require PIT input (--pit-csv, alias --data-csv) and no longer fall back to synthetic data.
  • Added/updated tests for required-arg behavior and full PIT missing-column failure.
  • Updated README runtime commands to PIT-required flow.
  • Archived change and synced new spec openspec/specs/real-pit-runtime-input/spec.md.
• Created and completed change tiered-data-quality-gate:
  • Extended evaluate_data_quality_gate with blocking_columns and per-breach severity (error/warning).
  • Strict mode now blocks only on error-severity breaches; warning-only strict runs continue.
  • Wired severity-aware behavior into run_demo, frozen_hypothesis_validation, build_pit_dataset, and PIT builder.
  • Added data_quality.blocking_columns defaults in config/regime.yaml.
  • Added tests for strict warning-only pass and blocking-column failure.
  • Archived change and synced specs:
  • updated openspec/specs/pipeline-data-quality-gate/spec.md
  • new openspec/specs/tiered-quality-severity/spec.md
• Created and completed change add-real-walkforward-report:
  • Added pipeline pipelines/real_walkforward_report.py.
  • Outputs:
  • data_quality_summary.json
  • frozen_validation_board.csv
  • real_walkforward_summary.json
  • real_walkforward_report.md
  • Added tests tests/test_real_walkforward_report_pipeline.py.
  • Archived change and synced new spec openspec/specs/real-walkforward-report/spec.md.
• Created and completed change add-execution-constraint-calibration:
  • Added pipeline pipelines/calibrate_execution_constraints.py.
  • Grid-searches execution params (extreme_day_cost_multiplier, gap_slippage_factor) and emits:
  • execution_calibration_grid.csv
  • execution_calibration_recommendation.json
  • Added test tests/test_execution_calibration_pipeline.py.
  • Updated README with calibration command.
  • Archived change and synced new spec openspec/specs/execution-constraint-calibration/spec.md.
• Final verification after all above changes:
  • py -m pytest -q tests -> 38 passed.
  • openspec.cmd validate --specs --strict -> 18 passed.
  • openspec.cmd list --json -> no active changes.
• User asked for a concrete data-fetch plan and approved immediate implementation.
• Created and completed change add-real-data-ingestion-pipeline:
  • Added ingestion core module data/ingestion.py with:
  • provider abstraction (csv, akshare)
  • source normalization
  • incremental date-merge utilities (merge_incremental_by_date, write_incremental_dataset)
  • layered publication (raw + staging)
  • final PIT publish via existing build_pit_dataset
  • Added CLI pipeline pipelines/ingest_real_data.py.
  • README updated with ingestion commands for csv and akshare modes.
  • Added tests tests/test_ingestion_pipeline.py:
  • csv provider layered artifact generation
  • incremental duplicate-date merge keeps latest row
  • akshare dependency-missing error path
• Verification after ingestion change:
  • py -m pytest -q tests/test_ingestion_pipeline.py -> 3 passed.
  • py -m pytest -q tests -> 41 passed.
• OpenSpec completion for ingestion change:
  • openspec.cmd instructions apply --change \"add-real-data-ingestion-pipeline\" --json -> state: all_done (10/10)
  • openspec.cmd archive add-real-data-ingestion-pipeline --yes completed
  • synced new spec: openspec/specs/real-data-ingestion/spec.md
  • latest openspec.cmd validate --specs --strict -> 19 passed.
  • latest openspec.cmd list --json -> no active changes.
• User requested Akshare primary + Mairui fallback integration and provided Mairui licence token for runtime use.
• Searched Mairui docs/endpoint patterns and validated history endpoints with live HTTP checks.
• Created and completed change add-mairui-fallback-ingestion:
  • Added Mairui support in data/ingestion.py:
  • mairui provider mode
  • Akshare-to-Mairui fallback in akshare mode when Akshare fetch fails
  • configurable breadth endpoint URL + optional field rename mapping
  • Extended CLI pipelines/ingest_real_data.py with:
  • --mairui-licence
  • --mairui-market-code, --mairui-market-kind
  • --mairui-hs300-code, --mairui-star50-code, --mairui-csi1000-code
  • --mairui-breadth-url, --mairui-breadth-map-json
  • Added test test_ingest_akshare_with_mairui_fallback_when_akshare_missing in tests/test_ingestion_pipeline.py.
  • Updated README with Akshare+Mairui fallback and Mairui provider examples.
• Verification after Mairui fallback change:
  • py -m pytest -q tests/test_ingestion_pipeline.py -> 4 passed.
  • py -m pytest -q tests -> 42 passed.
  • openspec.cmd validate --specs --strict -> 20 passed.
  • openspec.cmd list --json -> no active changes.
• Live smoke:
  • Ran pipelines/ingest_real_data.py with provider mairui against real API and generated:
  • outputs/ingestion_mairui_smoke/pit/chinext50_pit.csv
  • outputs/ingestion_mairui_smoke/pit/pit_quality_summary.json
  • outputs/ingestion_mairui_smoke/ingestion_manifest.json
• User requested full end-to-end run and asked to fix bugs if encountered.
• Executed full strict chain on unified PIT input (outputs/system_e2e_20260409_strict/ingestion/pit/chinext50_pit.csv): 1) pipelines/ingest_real_data.py (provider=akshare, Mairui fallback enabled, strict mode) 2) pipelines/run_demo.py --strict-data 3) pipelines/frozen_hypothesis_validation.py --strict-data 4) pipelines/real_walkforward_report.py --strict-data 5) pipelines/calibrate_execution_constraints.py --strict-data
• Initial strict ingestion attempt with start_date=2018-01-01 failed due star50_close coverage breach; reran with start_date=2020-01-01 (valid history coverage window) and strict chain succeeded.
• Final e2e run results:
  • PIT rows: 1517 (2020-01-02 to 2026-04-09), quality passed with 0 breaches.
  • demo metrics:
  • annual_return: 0.068863
  • sharpe: 0.378405
  • max_drawdown: 0.326678
  • drawdown_improvement_ratio: 0.456891
  • upside_capture: 0.374752
  • utility_total_score: -0.014141
  • frozen summary:
  • window_count: 3
  • processed_window_count: 2
  • skipped_window_count: 1
  • positive_window_ratio: 1.0
  • selected_candidate_distribution: {\"defensive\": 2}
  • real_walkforward comparison:
  • annual_return_delta_vs_baseline: -0.077390
  • drawdown_ratio_vs_baseline: 0.543109
  • utility_delta_vs_baseline: -0.048891
  • execution calibration recommendation:
  • extreme_day_cost_multiplier: 1.0
  • gap_slippage_factor: 0.0
  • calibration_score: -0.332920
• Latest verification after this run:
  • py -m pytest -q tests -> 42 passed.
• User requested packaging code + review questions into a single zip for GPT Pro upload.
• Created bundle:
  • folder: deliverables/gpt_pro_review_bundle_2026-04-09/
  • zip: deliverables/gpt_pro_review_bundle_2026-04-09.zip
• Bundle contents include:
  • core code dirs (config, data, features, model, backtest, pipelines, tests, openspec/specs)
  • recent archived OpenSpec changes (2026-04-09-*)
  • strict e2e result artifacts from outputs/system_e2e_20260409_strict/
  • CONTEXT_FOR_GPT_PRO.md and QUESTIONS_FOR_GPT_PRO.md
• User asked to read chinext50_regime_review_2026-04-09.md, analyze the review, and provide an improvement plan.
• Re-validated the review claims against the current code and live PIT run:
  • breadth_score non-null ratio = 0.0
  • crowding_score non-null ratio = 0.0
  • down_hazard / repair_hazard / rebound_hazard collapse to constant 0.5
  • realized states collapse to chop / trend / risk_off
  • baseline and pro_risk exposure paths are identical
• Confirmed the next implementation priority should be integrity repair rather than threshold retuning:
  • NaN-safe score aggregation
  • low-information / constant-series data-quality gate
  • hazard fail-fast when prerequisite scores are missing
  • exposure quantization refinement or removal
  • walk-forward window rebuild to valid data periods only
• User indicated GPT Pro has already generated more precise repair guidance and may provide it for direct incorporation into the next OpenSpec change.
• User provided chinext50_blocker_checklist_for_codex.md as the latest GPT Pro blocker checklist and required strict B1 -> B6 execution order under OpenSpec flow, with per-blocker file/test/metric reporting and no threshold/policy tuning before B1-B5 completion.
• Created and completed OpenSpec change fix-regime-integrity-blockers with all 18/18 tasks done.
• B1 completed:
  • added NaN-safe composite aggregation and valid-weight/readiness fields in model/scores.py
  • added tests/test_scores_integrity.py
  • real PIT metrics improved to:
  • breadth_score non-null ratio 0.9611
  • crowding_score non-null ratio 0.9216
  • hazards no longer collapse to 0.5
• B2 completed:
  • added explicit warmup and invalid-after-warmup fail-fast in model/state_machine.py
  • removed fillna(0) swallowing in model/policy.py
  • warmup rows on real PIT: 119, all zero exposure with veto_reason=warmup
• B3 completed:
  • added post-feature information gate in features/quality.py
  • wired gate into run_demo and run_strategy_bundle
  • real PIT strict feature gate now blocks on concentration_spread_5 with unique_non_null_count=1, dominant_value_ratio=1.0
• B4 completed:
  • upgraded exposure quantization to configurable mode with default 0.10 ladder
  • real PIT candidate differentiation now spans 836 rows (55.1%) between baseline and pro_risk
• B5 completed:
  • replaced hard-coded walk-forward windows with PIT-driven expanding windows
  • real PIT now yields 5 windows / 5 processed / 0 skipped
• B6 completed:
  • redesigned execution calibration score with tracking tolerance bands
  • reran non-strict calibration artifact under outputs/calibration_b6_non_strict/
• Final verification after B1-B6:
  • py -m pytest -q tests -> 59 passed
  • openspec.cmd validate --specs --strict -> 20 passed
  • openspec.cmd instructions apply --change "fix-regime-integrity-blockers" --json -> state: all_done
• Created and completed OpenSpec change derive-real-breadth-sidecar (proposal/design/specs/tasks implemented, archived).
• Added data/breadth_builder.py with:
  • constituent-derived breadth sidecar generation
  • Akshare stock history primary path with Tencent-line fallback
  • optional Mairui fallback and per-symbol provider diagnostics
  • breadth source integrity evaluation (low-unique/dominant/std + constant spread checks)
• Extended ingestion pipeline with derived breadth path and artifacts:
  • new CLI options: --derive-breadth, --breadth-index-symbol, --breadth-min-active-constituents, --breadth-max-constituents
  • manifest now records breadth_source, breadth_derivation_path, breadth_integrity_path
  • strict mode now blocks on breadth integrity failures before PIT publish.
• Added retry/backoff behavior for Mairui JSON requests in ingestion and breadth builder (429/5xx aware).
• Added tests:
  • tests/test_breadth_builder.py
  • extended tests/test_ingestion_pipeline.py for derived-breadth path and strict placeholder blocking.
• Verification after change:
  • py -m pytest -q tests/test_breadth_builder.py tests/test_ingestion_pipeline.py -> 8 passed
  • py -m pytest -q tests -> 63 passed
  • openspec.cmd validate --specs --strict -> 24 passed
• Real derived-breadth smoke:
  • strict run (outputs/ingestion_derived_smoke_20260409_v8) correctly failed on breadth integrity gate (low-information columns) rather than placeholder spread collapse.
  • non-strict run (outputs/ingestion_derived_smoke_20260409_v9) completed and produced derivation/integrity artifacts.
  • key compare vs old placeholder breadth:
  • spread unique count: 1 -> 22
  • spread std: ~4.34e-19 -> 0.0124
• Continued autonomous execution after user asked 继续.
• Verified cache rollout on outputs/ingestion_derived_smoke_20260409_v12 and found a bug:
  • second run still reported cache.hit_count=0, cache.miss_count=12.
  • root cause: cache range check required exact start/end boundary coverage and did not tolerate non-trading-day boundaries (2020-01-01 start while first trade date is 2020-01-02).
• Fixed cache coverage logic in data/breadth_builder.py:
  • added CACHE_BOUNDARY_TOLERANCE_DAYS=10.
  • _history_covers_range now allows small start/end boundary gaps and still rejects truly out-of-range cache panels.
• Added regression test in tests/test_breadth_builder.py:
  • test_derive_breadth_sidecar_cache_handles_non_trading_boundaries.
• Re-verified:
  • py -m pytest -q tests/test_breadth_builder.py -> 5 passed
  • py -m pytest -q tests/test_ingestion_pipeline.py -> 6 passed
  • py -m pytest -q tests -> 66 passed
  • openspec.cmd validate --specs --strict -> 26 passed
• Cache effectiveness after fix (same v12 command rerun):
  • hit_count: 0 -> 12
  • miss_count: 12 -> 0
  • provider distribution: {\"akshare\":12} -> {\"cache\":12}
  • output quality unchanged: breadth_integrity_summary.json remains passed=true, blocking=false, PIT rows 1517.
• Re-ran full downstream chain on cache-fixed PIT:
  • outputs/system_e2e_derived_v12_cachefix/demo/metrics_summary.json
  • outputs/system_e2e_derived_v12_cachefix/frozen/frozen_validation_summary.json
  • outputs/system_e2e_derived_v12_cachefix/report/real_walkforward_summary.json
  • outputs/system_e2e_derived_v12_cachefix/calibration/execution_calibration_recommendation.json
• Key results (v12 cachefix run):
  • demo: annual_return=0.034844, sharpe=0.308062, max_drawdown=0.263973, drawdown_improvement_ratio=0.561139, utility_total_score=-0.178361
  • frozen: window_count=5, processed=5, skipped=0, positive_window_ratio=0.4, selected_candidate_distribution={\"pro_risk\":4,\"baseline\":1}
  • real walk-forward: drawdown_ratio_vs_baseline=0.438861, utility_delta_vs_baseline=-0.213111 (annual_return_delta_vs_baseline is null in current artifact)
  • calibration recommendation: best params remained extreme_day_cost_multiplier=1.0, gap_slippage_factor=0.0, calibration_score=-0.185203.
• Created new OpenSpec change harden-derived-breadth-production and completed proposal/design/specs/tasks for:
  • full50 derived breadth hardening
  • metadata resilience and diagnostics
  • real walk-forward comparison key compatibility
  • issue-log handoff artifact for GPT Pro escalation
• Code changes in this round:
  • data/breadth_builder.py:
  • added resilient metadata chain (akshare retries + optional Mairui fallback)
  • added metadata diagnostics (metadata_provider_by_symbol, metadata_provider_counts, metadata_errors)
  • added metadata cache (_meta_cache.json) with meta_cache.hit/miss summary metrics
  • adjusted low-std integrity rule to avoid false positives when uniqueness is high and dominant-ratio is low
  • pipelines/real_walkforward_report.py:
  • added compatibility keys in comparison block:
    • annual_return_delta_vs_baseline
    • max_drawdown_delta_vs_baseline
  • tests updated:
  • tests/test_breadth_builder.py
  • tests/test_real_walkforward_report_pipeline.py
• Validation:
  • targeted: py -m pytest -q tests/test_breadth_builder.py tests/test_real_walkforward_report_pipeline.py tests/test_ingestion_pipeline.py -> 15 passed
  • full: py -m pytest -q tests -> 68 passed
  • specs: openspec.cmd validate --specs --strict -> 26 passed
• Full50 strict ingestion:
  • output: outputs/ingestion_derived_full50_20260409_v2/
  • PIT publish succeeded with row_count=1517, breadth integrity passed=true, blocking=false
  • cache metrics after second rerun:
  • history cache: hit=47, miss=3
  • metadata cache: hit=50, miss=0
  • runtime improved from ~1199.9s (first run) to ~26.9s (second rerun)
• Remaining blocker recorded for GPT Pro:
  • industry metadata remains degraded (industry_unknown_ratio=1.0, sector_concentration_mode=weight_hhi_proxy)
  • persisted issue log:
  • deliverables/gpt_pro_blockers_harden_derived_breadth_2026-04-09.md
  • representative upstream error:
  • Akshare RemoteDisconnected
  • Mairui HTTP 429 Too Many Requests
• End-to-end run on full50 PIT succeeded:
  • output root: outputs/system_e2e_derived_full50_v2/
  • demo metrics:
  • annual_return=0.067253
  • sharpe=0.519444
  • max_drawdown=0.287127
  • utility_total_score=-0.110408
  • report comparison now includes both canonical and alias keys with equal values for annual-return/drawdown deltas.
• OpenSpec completion status for this round:
  • openspec.cmd instructions apply --change harden-derived-breadth-production --json -> state: all_done (10/10)
  • openspec.cmd archive harden-derived-breadth-production --yes completed
  • archive folder: openspec/changes/archive/2026-04-09-harden-derived-breadth-production/
  • synced specs:
  • openspec/specs/constituent-derived-breadth/spec.md (updated)
  • openspec/specs/real-walkforward-report/spec.md (updated)
  • openspec/specs/implementation-issue-log/spec.md (new)
  • post-archive validation:
  • openspec.cmd validate --specs --strict -> 27 passed
  • openspec.cmd list --json -> no active changes.
• User requested packaging code + blocker questions for GPT Pro.
• Created review bundle folder:
  • deliverables/gpt_pro_bundle_harden_derived_2026-04-09/
• Added context question file:
  • deliverables/gpt_pro_bundle_harden_derived_2026-04-09/QUESTIONS_FOR_GPT_PRO.md
• Created zip artifact:
  • deliverables/gpt_pro_bundle_harden_derived_2026-04-09.zip (includes code changes, specs, archived change artifacts, blocker log, and key output summaries).
• 2026-04-09: Implemented and completed OpenSpec change uild-snapshot-first-index-metadata-and-weight-layer from handoff chinext50_harden_derived_breadth_direction_handoff_2026-04-09.md; added snapshot-weight forward-fill alignment, semantic-gate strict blocking ingestion test, stale-meta-cache/provider-semantics/pre-entry-mask/time-varying-membership tests; regression now 75 passed and OpenSpec state is all_done.
• 2026-04-09: Archived OpenSpec change uild-snapshot-first-index-metadata-and-weight-layer as openspec/changes/archive/2026-04-09-build-snapshot-first-index-metadata-and-weight-layer/; synced specs (readth-semantic-gate, index-metadata-snapshot-layer, updates to constituent-derived-breadth and eal-data-ingestion); strict spec validation now 29 passed and openspec list --json shows no active changes.
• 2026-04-09: Completed and archived OpenSpec change ecalibrate-walkforward-economics-after-semantic-hardening. Added hard-constraint-first + multi-objective train candidate selection with deterministic fallback in acktest/frozen_walkforward.py, selection diagnostics propagated to frozen/report outputs, updated default policy/candidate calibration, and added tests for hard-pass/fallback/summary diagnostics. Verification passed (77 tests, 29 strict specs). Real report rerun at outputs/real_walkforward_recalibrated_20260409_v2/ shows diversified candidate selection (defensive:1, baseline:1, balanced_capture:2, pro_risk:1), hard-pass window ratio .8, upside capture .2849 (vs .2593 prior), utility delta vs baseline improved to -0.1345 (vs -0.1452 prior).