Continuously Refreshed Tasks Fight Contamination

Observed now: SWE-bench-Live uses an automatically updating, multi-language, multi-OS software task set designed to keep evaluations current and reduce contamination. PaperBench adds a decomposed end-to-end replication benchmark with substantial remaining headroom. These examples support refreshed, harder-to-game evaluations without claiming that temporal splits, canaries, and exposure maps are universal.