Fire hydrant recognition specialist
I want to find all vulnerabilities on the internet. That means I need to make machines that are really good at generating exploit candidates, and give them a way to check if a candidate maps to a vulnerability. The latter is more difficult, as LLMs are good at generating candidates but bad at verification.
A vulnerability is a behavior reachable under the implementation that should be unreachable under the intended specification. To verify one, you need the intended specification. For most vulnerability classes, this specification is application-specific and never written down. There is no general oracle.
For certain vulnerability classes, it's easy to build universal, deterministic oracles because the intended specification is clear. From a SQL injection perspective, the intended behavior of an application is not to let end users run arbitrary SQL queries. This generalizes well to all web applications. Memory corruptions and cross-site scripting are similarly generalizable.
For access control or business logic vulnerabilities, however, it's much harder to define what the intended behavior is.
All questions concerning access control, such as whether a particular object access is permitted, depend on the application's semantics, and by Rice's theorem you cannot recover these from the implementation alone in general.
Many think that LLMs are good solutions to the verification problem. Unfortunately, this is not the case as they will always yield a nontrivial amount of false positives.
Suppose your verifier has a 1% false positive rate. You test 50,000 candidate inputs against an application with 5 real vulnerabilities. The prior P(vulnerability | input) is on the order of 10-4.
You end up with 5 true positives and 500 false positives, which gives you a precision of 1%. Hence, you need 0 false positives.
Whoever solves verification finds all vulnerabilities on the internet.