open-swe/evals
Adam Moussa 032d3889e4
fix: align reviewer eval with published findings (upstream #1713) (#201)
* fix: align reviewer eval with published findings (#1713)

* fix: make reviewer eval reflect published findings

Serialize and deduplicate finding persistence, align review calibration around the final six-finding publication, and make judge matching order-independent and auditable.

* fix: honor reviewer eval limits

Forward configured caps into publication snapshots and keep recall-at-cap bounded for diagnostic all-findings runs.

(cherry picked from commit 71e3b8183882bcc42e318f3f220c291617ebcb67)
Co-authored-by: Johannes du Plessis <johannes@langchain.dev>

* Empty commit to trigger CI

---------

Co-authored-by: Johannes du Plessis <johannes@langchain.dev>
2026-07-17 12:10:54 -04:00
..
reviewer fix: align reviewer eval with published findings (upstream #1713) (#201) 2026-07-17 12:10:54 -04:00