Difference between revisions of "Testing for Web Application Fingerprint (OWASP-IG-004)"

From OWASP
Jump to: navigation, search
(Malformed requests test)
Line 3: Line 3:
  
 
== Brief Summary ==
 
== Brief Summary ==
Web server fingerprinting is a critical task for the penetration tester. Knowing the version and type of a running web server allows testers to determine known vulnerabilities and the appropriate exploits to use during testing.
+
Web server fingerprinting is a critical task for the Penetration tester. Knowing the version and type of a running web server allows testers to determine known vulnerabilities and the appropriate exploits to use during testing.
  
 
== Description of the Issue ==
 
== Description of the Issue ==
Line 9: Line 9:
  
 
== Black Box testing and example ==
 
== Black Box testing and example ==
The simplest and most basic form of identifying a web server is to look at the Server field in the HTTP response header. For our experiments we use netcat.  
+
The simplest and most basic form of identifying a Web server is to look at the Server field in the HTTP response header. For our experiments we use netcat.  
 
Consider the following HTTP Request-Response:  
 
Consider the following HTTP Request-Response:  
 
<pre>
 
<pre>
Line 66: Line 66:
 
Accept-ranges: bytes  
 
Accept-ranges: bytes  
 
Connection: close  
 
Connection: close  
</pre>
 
From a '''SunONE 6.1''' server:
 
<pre>
 
HTTP/1.1 200 OK
 
Server: Sun-ONE-Web-Server/6.1
 
Date: Tue, 16 Jan 2007 14:53:45 GMT
 
Content-length: 1186
 
Content-type: text/html
 
Date: Tue, 16 Jan 2007 14:50:31 GMT
 
Last-Modified: Wed, 10 Jan 2007 09:58:26 GMT
 
Accept-Ranges: bytes
 
Connection: close
 
 
</pre>
 
</pre>
 
However, this testing methodology is not so good. There are several techniques that allow a web site to obfuscate or to modify the server banner string.
 
However, this testing methodology is not so good. There are several techniques that allow a web site to obfuscate or to modify the server banner string.
Line 93: Line 81:
  
 
== Protocol behaviour ==  
 
== Protocol behaviour ==  
Refined techniques of testing take into consideration various characteristics of the several web servers available on the market. We will list some methodologies that allow us to deduce the type of web server in use.
+
Refined techniques of testing take in consideration various characteristics of the several web servers available on the market. We will list some methodologies that allow us to deduce the type of web server in use.
  
 
=== HTTP header field ordering ===  
 
=== HTTP header field ordering ===  
Line 142: Line 130:
 
Connection: close  
 
Connection: close  
 
</pre>
 
</pre>
Response from '''SunONE 6.1'''
+
We can notice that the ordering of the ''Date'' field and the ''Server'' field differs between Apache, Netscape Enterprise and IIS.
<pre>
+
$ nc sunone.example.com 80
+
HEAD / HTTP/1.0
+
 
+
HTTP/1.1 200 OK
+
Server: Sun-ONE-Web-Server/6.1
+
Date: Tue, 16 Jan 2007 15:23:37 GMT
+
Content-length: 0
+
Content-type: text/html
+
Date: Tue, 16 Jan 2007 15:20:26 GMT
+
Last-Modified: Wed, 10 Jan 2007 09:58:26 GMT
+
Connection: close
+
</pre>
+
We can notice that the ordering of the ''Date'' field and the ''Server'' field differs between Apache, Netscape Enterprise, Sun ONE and IIS.
+
  
 
=== Malformed requests test ===  
 
=== Malformed requests test ===  
Line 201: Line 175:
 
Connection: close  
 
Connection: close  
 
</pre>
 
</pre>
Response from '''SunONE 6.1'''
 
<pre>
 
$ nc sunone.example.com 80
 
HEAD / HTTP/3.0
 
  
HTTP/1.1 400 Bad request
 
Server: Sun-ONE-Web-Server/6.1
 
Date: Tue, 16 Jan 2007 15:25:00 GMT
 
Content-length: 0
 
Content-type: text/html
 
Connection: close
 
</pre>
 
 
We notice that every server answers in a different way. The answer also differs in the version of the server. An analogous issue comes if we create requests with a non-existant protocol. Consider the following responses:  
 
We notice that every server answers in a different way. The answer also differs in the version of the server. An analogous issue comes if we create requests with a non-existant protocol. Consider the following responses:  
  
Line 249: Line 212:
 
<BODY><H1>Bad request</H1>  
 
<BODY><H1>Bad request</H1>  
 
Your browser sent to query this server could not understand.  
 
Your browser sent to query this server could not understand.  
</BODY></HTML>
+
</BODY></HTML>  
</pre>
+
Response from '''SunONE 6.1'''
+
<pre>
+
$ nc sunone.example.com 80
+
GET / JUNK/1.0
+
 
+
<HTML><HEAD><TITLE>Bad request</TITLE></HEAD>
+
<BODY><H1>Bad request</H1>
+
Your browser sent a query this server could not understand.
+
</BODY></HTML>
+
</pre>
+
 
+
<H1>Bad request</H1>
+
Your browser sent to query this server could not understand.
+
</BODY></HTML>
+
 
</pre>
 
</pre>
  
 
== Automated Testing ==  
 
== Automated Testing ==  
As can be appreciated from above, a number of tests are needed in order to accurately determine the type and version of web server in use. There are tools available which can automate these tests. One such example is "''httprint''" which compares the web server response with a signature dictionary and then makes a judgement with associated confidence rating.<br>
+
The tests to carry out testing can be several. A tool that automates these tests is "''httprint''" that allows one, through a signature dictionary, to recognize the type and the version of the web server in use.<br>
 
An example of such tool is shown below:<br><br>
 
An example of such tool is shown below:<br><br>
  
 
[[Image:httprint.jpg]]
 
[[Image:httprint.jpg]]
  
<H1>Bad request</H1> Your browser sent to query this server could not understand.
 
</BODY></HTML>
 
  
== Automated Testing ==  
+
== References ==
As can be appreciated from above, a number of tests are needed in order to accurately determine the type and version of web server in use. There are tools available which can automate these tests. One such example is "''httprint''" which compares the web server response with a signature dictionary and then makes a judgement with associated confidence rating.<br>
+
'''Whitepapers'''<br>
An example of such tool is shown below:<br><br>
+
* Saumil Shah: "An Introduction to HTTP fingerprinting" - http://net-square.com/httprint/httprint_paper.html
 +
'''Tools'''<br>
 +
* httprint - http://net-square.com/httprint/index.shtml
  
[[Image:httprint.jpg]]
+
 
 +
{{Category:OWASP Testing Project}}

Revision as of 18:08, 30 January 2007

[Up]
OWASP Testing Guide v2 Table of Contents


Brief Summary

Web server fingerprinting is a critical task for the Penetration tester. Knowing the version and type of a running web server allows testers to determine known vulnerabilities and the appropriate exploits to use during testing.

Description of the Issue

There are several different vendors and versions of web servers on the market today. Knowing the type of web server that you are testing significantly helps in the testing process, and will also change the course of the test. This information can be derived by sending the web server specific commands and analyzing the output, as each version of web server software may respond differently to these commands. By knowing how each type of web server responds to specific commands and keeping this information in a web server fingerprint database, a penetration tester can send these commands to the web server, analyze the respsonse, and compare it to the database of known signatures. Please note that it usually takes several different commands to accurately identify the web server, as different versions may react similarly to the same command. Rarely, however, do different versions react the same to all HTTP commands. So, by sending several different commands, you increase the accuracy of your guess.

Black Box testing and example

The simplest and most basic form of identifying a Web server is to look at the Server field in the HTTP response header. For our experiments we use netcat. Consider the following HTTP Request-Response:

$ nc 202.41.76.251 80
HEAD / HTTP/1.0

HTTP/1.1 200 OK
Date: Mon, 16 Jun 2003 02:53:29 GMT
Server: Apache/1.3.3 (Unix)  (Red Hat/Linux)
Last-Modified: Wed, 07 Oct 1998 11:18:14 GMT
ETag: "1813-49b-361b4df6"
Accept-Ranges: bytes
Content-Length: 1179
Connection: close
Content-Type: text/html

$

from the Server field we understand that the server is Apache, version 1.3.3, running on Linux operating system. Three examples of the HTTP response headers are shown below:

From an Apache 1.3.23 server:

HTTP/1.1 200 OK 
Date: Sun, 15 Jun 2003 17:10: 49 GMT 
Server: Apache/1.3.23 
Last-Modified: Thu, 27 Feb 2003 03:48: 19 GMT 
ETag: 32417-c4-3e5d8a83 
Accept-Ranges: bytes 
Content-Length: 196 
Connection: close 
Content-Type: text/HTML 

From a Microsoft IIS 5.0 server:

HTTP/1.1 200 OK 
Server: Microsoft-IIS/5.0 
Expires: Yours, 17 Jun 2003 01:41: 33 GMT 
Date: Mon, 16 Jun 2003 01:41: 33 GMT 
Content-Type: text/HTML 
Accept-Ranges: bytes 
Last-Modified: Wed, 28 May 2003 15:32: 21 GMT 
ETag: b0aac0542e25c31: 89d 
Content-Length: 7369 

From a Netscape Enterprise 4.1 server:

HTTP/1.1 200 OK 
Server: Netscape-Enterprise/4.1 
Date: Mon, 16 Jun 2003 06:19: 04 GMT 
Content-type: text/HTML 
Last-modified: Wed, 31 Jul 2002 15:37: 56 GMT 
Content-length: 57 
Accept-ranges: bytes 
Connection: close 

However, this testing methodology is not so good. There are several techniques that allow a web site to obfuscate or to modify the server banner string. For example we could obtain the following answer:

403 HTTP/1.1 
Forbidden Date: Mon, 16 Jun 2003 02:41: 27 GMT 
Server: Unknown-Webserver/1.0 
Connection: close 
Content-Type: text/HTML; 
charset=iso-8859-1 

In this case the server field of that response is obfuscated: we cannot know what type of web server is running.

Protocol behaviour

Refined techniques of testing take in consideration various characteristics of the several web servers available on the market. We will list some methodologies that allow us to deduce the type of web server in use.

HTTP header field ordering

The first method consists of observing the ordering of the several headers in the response. Every web server has an inner ordering of the header. We consider the following answers as an example:

Response from Apache 1.3.23

$ nc apache.example.com 80 
HEAD / HTTP/1.0 

HTTP/1.1 200 OK 
Date: Sun, 15 Jun 2003 17:10: 49 GMT 
Server: Apache/1.3.23 
Last-Modified: Thu, 27 Feb 2003 03:48: 19 GMT 
ETag: 32417-c4-3e5d8a83 
Accept-Ranges: bytes 
Content-Length: 196 
Connection: close 
Content-Type: text/HTML 

Response from IIS 5.0

$ nc iis.example.com 80 
HEAD / HTTP/1.0 

HTTP/1.1 200 OK 
Server: Microsoft-IIS/5.0 
Content-Location: http://iis.example.com/Default.htm 
Date: Fri, 01 Jan 1999 20:13: 52 GMT 
Content-Type: text/HTML 
Accept-Ranges: bytes 
Last-Modified: Fri, 01 Jan 1999 20:13: 52 GMT 
ETag: W/e0d362a4c335be1: ae1 
Content-Length: 133 

Response from Netscape Enterprise 4.1

$ nc netscape.example.com 80 
HEAD / HTTP/1.0 

HTTP/1.1 200 OK 
Server: Netscape-Enterprise/4.1 
Date: Mon, 16 Jun 2003 06:01: 40 GMT 
Content-type: text/HTML 
Last-modified: Wed, 31 Jul 2002 15:37: 56 GMT 
Content-length: 57 
Accept-ranges: bytes 
Connection: close 

We can notice that the ordering of the Date field and the Server field differs between Apache, Netscape Enterprise and IIS.

Malformed requests test

Another useful test to execute involves sending malformed requests or requests of nonexistent pages to the server. We consider the following HTTP response:

Response from Apache 1.3.23

$ nc apache.example.com 80 
GET / HTTP/3.0 

HTTP/1.1 400 Bad Request 
Date: Sun, 15 Jun 2003 17:12: 37 GMT 
Server: Apache/1.3.23 
Connection: close 
Transfer: chunked 
Content-Type: text/HTML; charset=iso-8859-1 

Response from IIS 5.0

$ nc iis.example.com 80 
GET / HTTP/3.0 

HTTP/1.1 200 OK 
Server: Microsoft-IIS/5.0 
Content-Location: http://iis.example.com/Default.htm 
Date: Fri, 01 Jan 1999 20:14: 02 GMT 
Content-Type: text/HTML 
Accept-Ranges: bytes 
Last-Modified: Fri, 01 Jan 1999 20:14: 02 GMT 
ETag: W/e0d362a4c335be1: ae1 
Content-Length: 133 

Response from Netscape Enterprise 4.1

$ nc netscape.example.com 80 
GET / HTTP/3.0 

HTTP/1.1 505 HTTP Version Not Supported 
Server: Netscape-Enterprise/4.1 
Date: Mon, 16 Jun 2003 06:04: 04 GMT 
Content-length: 140 
Content-type: text/HTML 
Connection: close 

We notice that every server answers in a different way. The answer also differs in the version of the server. An analogous issue comes if we create requests with a non-existant protocol. Consider the following responses:

Response from Apache 1.3.23

$ nc apache.example.com 80 
GET / JUNK/1.0 

HTTP/1.1 200 OK 
Date: Sun, 15 Jun 2003 17:17: 47 GMT 
Server: Apache/1.3.23 
Last-Modified: Thu, 27 Feb 2003 03:48: 19 GMT 
ETag: 32417-c4-3e5d8a83 
Accept-Ranges: bytes 
Content-Length: 196 
Connection: close 
Content-Type: text/HTML 

Response from IIS 5.0

$ nc iis.example.com 80 
GET / JUNK/1.0 

HTTP/1.1 400 Bad Request 
Server: Microsoft-IIS/5.0 
Date: Fri, 01 Jan 1999 20:14: 34 GMT 
Content-Type: text/HTML 
Content-Length: 87 

Response from Netscape Enterprise 4.1

$ nc netscape.example.com 80 
GET / JUNK/1.0 

<HTML><HEAD><TITLE>Bad request</TITLE></HEAD> 
<BODY><H1>Bad request</H1> 
Your browser sent to query this server could not understand. 
</BODY></HTML> 

Automated Testing

The tests to carry out testing can be several. A tool that automates these tests is "httprint" that allows one, through a signature dictionary, to recognize the type and the version of the web server in use.
An example of such tool is shown below:

Httprint.jpg


References

Whitepapers

Tools


This project is part of the OWASP Breakers community.
Feel free to browse other projects within the Defenders, Builders, and Breakers communities.
OWASP Books logo.png This project has produced a book that can be downloaded or purchased.
Feel free to browse the full catalog of available OWASP books.


[edit]

OWASP Testing Guide v3

16th December 2008: OWASP Testing Guide v3 is finished!

  • You can download the Guide in PDF here
  • Download the presentation here
  • Browse the Testing Guide v3 on the wiki here

'NEW: OWASP projects and resources you can use TODAY'
16th April 2010 in London, OWASP leaders deliver a course focused on the main OWASP Projects.
Matteo Meucci will deliver a training course on the OWASP Testing Guide v3.
More information here

Video @ FOSDEM 09: here

Citations:

http://www.owasp.org/index.php/Testing_Guide_Quotes

Overview

This project's goal is to create a "best practices" web application penetration testing framework which users can implement in their own organizations and a "low level" web application penetration testing guide that describes how to find certain issues.

Version 3 of the Testing Guide was released in December 2008 after going through a major upgrade through the OWASP Summer of Code 2008.

History Behind Project The OWASP Testing guide originated in 2003 with Dan Cuthbert as one of the original editors. It was handed over to Eoin Keary in 2005 and moved onto the new OWASP wiki when it came online. Being in a wiki is easier for people to contribute and has made updating much easier. Matteo Meucci took on the Testing guide after Eoin and shepherded it through the version 2 and version 3 updates, which have been significant improvements.

OWASP Testing Guide v3

Testing Guide v3: plan (archive)

26th April 2008: Version 3 of the Testing Guide started under OWASP Summer of Code 2008.

6th November 2008: Completed draft created and previewed at OWASP EU Summit 2008 in Portugal.

Final stable release in December 2008

OWASP Testing Guide v2

10th February 2007: The OWASP Testing Guide v2 is now published Matteo Meucci (as part of his AoC project) has just published the latest version of Testing guide which:

OWASP Testing Guide v2 in Spanish: Now you can get a complete translation in Ms Doc format

For comments or questions, please join the OWASP Testing mailing list, read our archive and share your ideas. Alternatively you can contact Eoin Keary or Matteo Meucci directly.

Here you can find:

We hope you find the information in the OWASP Testing project useful. Please contribute back to the project by sending your comments, questions, and suggestions to the OWASP Testing mailing list. Thanks!

To join the OWASP Testing mailing list or view the archives, please visit the subscription page.

Thanks to the translators all around the world you can download the guide in the following languages:

  • Japanese in PDF format here (this is a 1st draft, final release coming soon).

PROJECT INFO
What does this OWASP project offer you?
RELEASE(S) INFO
What releases are available for this project?
what is this project?
Name: OWASP Testing Project (home page)
Purpose:
  • The OWASP Testing Guide includes a "best practice" penetration testing framework which users can implement in their own organizations and a "low level" penetration testing guide that describes techniques for testing most common web application and web service security issues.
License: Creative Commons Attribution Share Alike 3.0
who is working on this project?
Project Leader(s):
how can you learn more?
Project Pamphlet: Not Yet Created
Project Presentation: View
Mailing list: Mailing List Archives
Project Roadmap: View
Main links:
Key Contacts
  • Contact the GPC to report a problem or concern about this project or to update information.
current release
Testing Guide V 4.0 - 15th February 2013

The new project is available here - (no download available)

Release description:
  • Review all the control numbers to adhere to the OWASP Common numbering,
  • Review all the sections in v3,
  • Create a more readable guide, eliminating some sections that are not really useful,
  • Insert new testing techniques: HTTP Verb tampering, HTTP Parameter Pollutions, etc.,
  • Rationalize some sections as Session Management Testing,
  • Create a new section: Client side security and Firefox extensions testing.
Rating: Yellow button.JPG Not Reviewed - Assessment Details
last reviewed release
Testing Guide V 3.0 - December 2008 - (download)
Release description: The OWASP Testing Guide v3 is a 349 page book; we have split the set of active tests in 9 sub-categories for a total of 66 controls to test during the Web Application Testing activity.
Rating: Greenlight.pngGreenlight.pngGreenlight.png Stable Release - Assessment Details


other releases