Contribute
SRE-Bench grows through two kinds of contributions: programs that become new benchmark targets, and anti-analysis tooling that hardens them. Both are reviewed before anything lands.
Programs
We are looking for private programs that can become new reverse engineering targets. A program qualifies when it meets all three conditions.
- Private, and older than 2024. The code has never been public, and its last commit predates January 1, 2024. Models have read most of public GitHub; a program they have never seen is what keeps the benchmark contamination-free.
- Builds to a native binary. The program compiles to a standalone executable, ideally in C, C++, Rust, or Go, so it can go through our compilation, obfuscation, and packing pipeline.
- Deterministic, multi-score verification. The program's behaviour can be checked automatically, and one binary supports several independently graded tasks: a file format to decode, a protocol to speak, a hidden behaviour to trigger. Grading has to give the same answer on every run.
Programs between roughly 5,000 and 30,000 lines with real functionality work best. Single-file utilities are too small to hide anything, and anything already on the public web is disqualified however good it is.
Obfuscators and packers
The protections in v0 are in-house. We want to add tooling from the community: obfuscators, packers, virtualizers, anti-debugging layers, or anything else that makes a binary harder to analyse.
- Supports C, C++, Rust, or Go. At least one of them, ideally all four. Source-level, IR-level, and binary-level approaches are all welcome.
- Reproducible. Given the same input and configuration, the tool produces a binary we can rebuild and re-grade.
- Named presets. A documented configuration we can cite as a column on the Tasks page.
How submissions are reviewed
Every submission is evaluated internally before we decide whether to accept it. For programs, we build the binaries, write the grading tasks, and run the current models to confirm the target is neither trivial nor broken. For protections, we verify that the protected binaries still pass their graders and measure how much harder the tasks become. We try to reply to every submission, and accepted contributions are credited on the Contributors page.