Benchmark

= Main =  {| style="padding: 0;margin:0;margin-top:10px;text-align:left;" |-
 * valign="top" style="border-right: 1px dotted gray;padding-right:25px;" |

OWASP WebGoat Benchmark
The OWASP WebGoat Benchmark is a Java test suite designed to verify the speed and accuracy of vulnerability detection tools. The initial version is intended to support Static Analysis Security Testing Tools (SAST) and Interactive Analysis Security Testing Tools (IAST). A future release (this year hopefully) will support Dynamic Analysis Security Testing Tools (DAST), like OWASP ZAP. The goal is that this test application is fully runnable and all the vulnerabilities are actually exploitable so its a fair test for any kind of vulnerability detection tool.

Future versions could support other languages, but one step at a time!

Project Philosophy
Security tools (SAST, DAST, and IAST) are amazing when they find a complex vulnerability in your code. But they can drive everyone crazy with complexity, false alarms, and missed vulnerabilities. We are on a quest to see just how good these tools are at discovering and properly diagnosing security problems in applications. So the WebGoat Benchmark test suite is designed to test the abilities of these tools and hopefully help them improve.

Approach
For this Java test suite:


 * Every test case is a servlet or JSP
 * Every test case is either a true vulnerability or a false positive for a single issue
 * Metadata for each test case, including expected result is contained in a matching XML file
 * The overall score for a tool is the average of the scores across all the test categories

There are two types of tests in the current benchmark: 1) Tests written by hand. 2) Tests that are generated from code snippets that include 3 parts a) A source of taint (e.g., a web parameter)    b) Propagation or data flow c) A dangerous sink

There are currently over 150 test cases written by hand, and over 20,000 generated tests.

Test Coverage
For the test suite, we plan to determine, does the tool:

- Simple and complex data flow? - Simple and complex control flow? - Popular frameworks? - Inversion of control? - Reflection? Class loading? Annotations?
 * Detect the vulnerability?
 * Not detect false alarms?
 * Find HTTP request and response problems?
 * Handle scenarios like:

Future enhancements could cover:
 * Does the tool find flaws in libraries?
 * Does the tool find flaws spanning custom code and libraries?
 * Does tool handle web services? REST, XML, GWT, etc…
 * Does tool work with different app servers? Java platforms?

Code Repo
The code for this project is hosted at the OWASP Git repository. Along with the code comes a Maven pom.xml file so you can download all the dependencies and build the entire project with ease using Maven.

Licensing
The OWASP WebGoat Benchmark is free to use under the TBD license.

Mailing List
OWASP WebGoat Benchmark Mailing List

Project Leaders
Dave Wichers [mailto:dave.wichers@owasp.org @]

Related Projects

 * WebGoat
 * NSA's Juliet for Java
 * WAVESEP


 * valign="top" style="padding-left:25px;width:200px;" |

Quick Download

 * TBD

News and Events

 * [Apr 2015] Initial Release

Classifications

 * }

= Roadmap =

2015 Roadmap
* OWASP's ZAP * Findbugs and plugins for it like FindSecurityBugs * Commercial SAST, DAST, and IAST tools
 * [June 2015] TBD
 * Analysis tool integration: So you can automatically run tools against the benchmark. We want to build test harnesses for tools like:
 * FUTURE: Expand to include attack test cases to verify whether defenses (WAF, IDS/IPS, RASP) can identify and protect against them