Review Webserver Metafiles for Information Leakage (OTG-INFO-003)

This is a draft of a section of the new Testing Guide v3

Brief Summary
This section describes how to traverse the content of the application under test using web spiders/robots/crawlers.

Description of the Issue
Web spiders/robots/crawlers retrieve a web page and then recursively traverse hyperlinks to retrieve further web content. While their accepted behavior is specified by the web server and its web pages they may accidentally or intentionally retrieve web content not intended to be stored or published.

Black Box testing and example
Testing for Topic X vulnerabilities: ... Result Expected: ...

Gray Box testing and example
Testing for Topic X vulnerabilities: ... Result Expected: ...