open-swe/evals
Adam Moussa 0d21fb8c47
fix: align reviewer eval with published findings (#1713)
* fix: make reviewer eval reflect published findings

Serialize and deduplicate finding persistence, align review calibration around the final six-finding publication, and make judge matching order-independent and auditable.

* fix: honor reviewer eval limits

Forward configured caps into publication snapshots and keep recall-at-cap bounded for diagnostic all-findings runs.

(cherry picked from commit 71e3b8183882bcc42e318f3f220c291617ebcb67)
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
2026-07-16 17:36:18 -04:00
..
reviewer fix: align reviewer eval with published findings (#1713) 2026-07-16 17:36:18 -04:00