Difference between revisions of "Enumerate Applications on Webserver (OTG-INFO-004)"

From OWASP
Jump to: navigation, search
(Description of the Issue)
(Black Box testing and example)
Line 23: Line 23:
  
 
== Black Box testing and example ==
 
== Black Box testing and example ==
'''Testing for Topic X vulnerabilities:''' <br>
+
'''Web application discovery''' <br>
...<br>
+
Web application discovery is a process aimed at identifying web applications on given infrastructure. The latter is usually specified as a set of IP addresses (maybe a net block), but may consist also of a set of DNS symbolic names, or a mix of the two.<br>
'''Result Expected:'''<br>
+
This information is handed out prior to the execution of an assessment, be it a classic-style penetration test or an application-focused assessment. In both cases, unless the rules of engagement specify otherwise (e.g., “test only the application located at the URL http://www.example.com/”), the assessment should strive to be the most comprehensive in scope, i.e. it should, first of all, identify all the applications accessible through the given target. In the following we will examine a few techniques that can be employed to achieve this goal.
...<br><br>
+
<br><br>
 +
There are two factors influencing how many applications are related to a given DNS name (or an IP address).
 +
<br><br>
 +
'''1. Different base URL''' <br>
 +
The obvious entry point for a web application is ''www.example.com'', i.e. with this shorthand notation we think of the web application originating at http://www.example.com/ (the same applies for https). However, though this is the most common situation, there is nothing forcing the application to start at “/”.<br>
 +
For example, the same symbolic name may be associated to three web applications such as <br>
 +
http://www.example.com/url1 <br>
 +
http://www.example.com/url2 <br>
 +
http://www.example.com/url3 <br>
 +
In this case the URL http://www.example.com/ would not be associated to a meaningful page, and the three applications would be “hidden” unless we explicitly know how to reach them, i.e. we know ''url1'', ''url2'' or ''url3''. There is usually no need to publish web applications in this way, unless you don’t want them to be accessible in a standard way, and you are prepared to inform your users about their exact location. This doesn’t mean that these applications are secret, but that their existence and location is not explicitly advertised.
 +
<br><br>
 +
'''2. Non-standard ports''' <br>
 +
While web applications usually live on port 80 (http) and 443 (https), there is nothing magic about these port numbers. In fact, web applications may be associated with arbitrary TCP ports, and can be referenced by specifying the port number as follows: http[s]://www.example.com:port/. For example, http://www.example.com:20000/.
 +
<br><br>
 +
There is another factor affecting how many web applications are related to a given IP address.
 +
<br><br>
 +
'''3. Virtual hosts''' <br>
 +
DNS allows to associate a single IP address to one or more symbolic names. For example, the IP address ''192.168.1.100'' might be associated to DNS names ''www.example.com, helpdesk.example.com, webmail.example.com'' (actually, it is not necessary that all the names belong to the same DNS domain). This 1-to-N relationship may be reflected to serve different content by using so called virtual hosts. The information specifying the virtual host we are referring to is embedded in the HTTP 1.1 ''Host:'' header [1].
 +
<br>
 +
We would not suspect the existence of other web applications in addition to the obvious ''www.example.com'', unless we know of ''helpdesk.example.com'' and ''webmail.example.com''.
 +
<br><br>
 +
'''Approaches to address issue 1 - non-standard URLs'''
 +
<br>
 +
There is no way to full-proof ascertain the existence of non-standard-named web applications. Being non-standard, there is no magic recipe handing them out. However, we may employ a couple of criteria which may aid in their quest.
 +
<br>
 +
Our first hope is that these application might be referenced by other web pages; as such, there is a chance that they have been spidered and indexed by web search engines. If we suspect the existence of such “hidden” applications on ''www.example.com'' we could, for example, do a bit of googling using the ''site'' operator and examining the result of a query for “site: www.example.com”. Among the returned URLs there could be one pointing to such a non-obvious application.
 +
<br>
 +
Another option is to probe for URLs which might be likely candidates for non-published applications. For example, a web mail front end might be accessible from https://www.example.com/webmail, while this URL could not be referenced anywhere (after all, employees would know where the webmail application is located, while there is no reason to tell this information to outsiders by publishing it onto the corporate web site). The same holds for administrative interfaces, which may be published at standard URLs (for example: a Tomcat administrative interface), and yet not being referenced anywhere. So, doing a bit of dictionary-style searching (or “intelligent guessing”) could yield back some results. Vulnerability scanners may help with this respect.
 +
<br><br>
 +
'''Approaches to address issue 2 - non-standard ports'''
 +
<br>
 +
Existence of web applications on non-standard ports is easy to check. A port scanner such as nmap [2] is capable of performing service recognition by means of the -sV option, and will identify http[s] services on arbitrary ports. What is required is a full scan of the whole 64k TCP port address space.
 +
<br>
 +
For example, the following command will look up, with a TCP connect scan, all open ports on IP ''192.168.1.100'' and will try to determine what services are bound to them (only ''essential'' switches are shown – nmap features a broad set of options, whose discussion is out of scope). <br>
 +
nmap –P0 –sT –sV –p1-65535 192.168.1.100
 +
<br>
 +
It is sufficient to examine the output and looking for http or https (or SSL-wrapped services).
 +
<br>
 +
The same task may be performed by vulnerability scanners – but first check that your scanner of choice is able to identify http[s] services running on non-standard ports. For example, Nessus [3] is capable of identifying them on arbitrary ports (provided you instruct it to scan all the ports), and will provide – with respect to nmap – a number of tests on known web server vulnerabilities, as well as on the SSL configuration of https services.
 +
<br><br>
 +
'''Approaches to address issue 3 - virtual hosts'''
 +
<br>
 +
There is a number of techniques which may be used to identify DNS names associated to a given IP address ''x.y.z.t''.
 +
<br>
 +
<u>DNS zone transfers</u> <br>
 +
This technique is nowadays of limited usage, given the fact that zone transfers are largely not honored by DNS servers, but it is worth a try. <br>
 +
First of all, we must determine the name server serving ''x.y.z.t''. Actually, matters get complicated because there could be many such servers. Maybe you know a name server because it was given as part of the target to be assessed (let it be ''dns.example.com''). You may query ''dns.example.com'' by means of tools such as ''nslookup'' or ''dig'' and, if you are lucky, you may get back a list of the DNS entries for domain ''example.com''. This will include the obvious ''www.example.com'' and the not-so-obvious ''helpdesk.example.com'' and ''webmail.example.com'' (and possibly others). Check all names returned by the zone transfer and consider all of those which are related to the target being evaluated. <br>
 +
If you don’t know a name server, you may look up ''whois'' information about the given IP address, and try to contact the name servers listed by whois, and request a zone transfer as said before.
 +
<br><br>
 +
<u>DNS inverse queries</u> <br>
 +
This process is similar to the previous but relies on inverse (PTR) DNS records. Rather than requesting a zone transfer, try setting the record type to PTR and issue a query on the given IP address. If you are lucky, you may get back a DNS name entry. This technique relies on the existence of IP-to-symbolic name maps, which is not granted.
 +
<br><br>
 +
<u>Web-based DNS searches</u> <br>
 +
This kind of search is akin to DNS zone transfer, but relies on web-based services which allow to perform name-based searched on DNS. One such a service is the ''Netcraft Search DNS'' service, available at http://searchdns.netcraft.com/?host. You may query for a list of names belonging to your domain of choice, such as ''example.com''. Then you will check whether the names you obtained are pertaining to the target you are examining.
 +
<br><br>
 +
<u>Reverse-IP services</u> <br>
 +
Reverse-IP services are similar to DNS inverse queries, with the difference that you query a web-based application instead of a name server. There is a number of such services available. Since they tend to return partial (and often different) results, it is better to use multiple services to obtain a more comprehensive analysis.
 +
<br><br>
 +
''Domain tools reverse IP'': http://www.domaintools.com/reverse-ip/ <br>
 +
(requires free membership) <br>
 +
<br>
 +
''MSN search'': http://search.msn.com <br>
 +
syntax: "ip:x.x.x.x" (without the quotes) <br>
 +
<br>
 +
''Webhosting info'': http://whois.webhosting.info/ <br>
 +
syntax: http://whois.webhosting.info/x.x.x.x <br>
 +
<br>
 +
''DNSstuff'': http://www.dnsstuff.com/ <br>
 +
(multiple services available) <br>
 +
<br>
 +
http://net-square.com/msnpawn/index.shtml <br>
 +
(multiple queries on  domains and IP addresses, requires installation) <br>
 +
<br>
 +
''tomDNS'': http://www.tomdns.net/ <br>
 +
(some services are still private at the time of writing) <br>
 +
<br>
 +
''SEOlogs.com'': http://www.seologs.com/ip-domains.html <br>
 +
(reverse ip/domain lookup) <br>
 +
<br> <br>
 +
<u>Googling</u> <br>
 +
After you have gathered the most information you can with the previous techniques, you can rely on search engines to possibly refine and increment your analysis. This may yield evidence of additional symbolic names belonging to your target, or applications accessible via non-obvious URLs.
 +
<br>
 +
 
 
== Gray Box testing and example ==  
 
== Gray Box testing and example ==  
 
'''Testing for Topic X vulnerabilities:'''<br>
 
'''Testing for Topic X vulnerabilities:'''<br>

Revision as of 03:44, 6 November 2006

[Up]
OWASP Testing Guide v2 Table of Contents

Contents


Brief Summary

A common step for testing vulnerabilities in a Web presence is to find out which particular applications are hosted on a Web Server.
Many different applications, in fact, have known vulnerabilities and known attack strategies than can be exploited in order to gain remote control and/or data exploitation.
In addition to this many applications are often hosted on a particular web server without reference from the main website: this is true for internal and/or extranet website which could be misconfigured or not updated due to the perception they're used only "internally".
In addition to this many application use common path for administrative interfaces which can be used to guess or bruteforce administrative passwords.

Description of the Issue


With the proliferation of virtual web servers, the traditional 1:1-type relationship between an IP address and a web server is loosing much of its original significance. It is not uncommon to have multiple web sites / applications whose symbolic names resolve to the same IP address (and this scenario is not limited to hosting environments, but applies to ordinary corporate environments as well).
Sometimes you, as a security professional, are given a set of IP addresses (or maybe just one) as a target to test. No other knowledge. It is arguable that this setting is more akin to a pentest-type engagement, but in any case it is expected that such an assignment would test all web applications accessible through this target (and possibly other things...). The problem is, the given IP address hosts an http service on port 80, but if you access it specifying the IP address (which is all you know) it reports "No web server configured at this address" or a similar message. But that system could "hide" a bunch of web applications, associated to unrelated symbolic (DNS) names. Obviously the extent of your analysis is deeply affected by the fact that you test the applications, or you do not - because you don't notice them, or you notice only SOME of them. Sometimes the target specification is richer – maybe you are handed out a list of IP addresses and their corresponding symbolic names. Nevertheless, this list might convey partial information, i.e. it could omit some symbolic names – and the client might not even being aware of that! (this is more likely to happen in large organizations).
Other issues affecting the scope of the assessment are represented by web applications published at non-obvious URLs (e.g., http://www.example.com/some-strange-URL), which are not referenced elsewhere. This may happen either by error (due to misconfigurations), or intentionally (for example, unadvertised administrative interfaces).
To address these issues it is necessary to perform a web application discovery.

Black Box testing and example

Web application discovery
Web application discovery is a process aimed at identifying web applications on given infrastructure. The latter is usually specified as a set of IP addresses (maybe a net block), but may consist also of a set of DNS symbolic names, or a mix of the two.
This information is handed out prior to the execution of an assessment, be it a classic-style penetration test or an application-focused assessment. In both cases, unless the rules of engagement specify otherwise (e.g., “test only the application located at the URL http://www.example.com/”), the assessment should strive to be the most comprehensive in scope, i.e. it should, first of all, identify all the applications accessible through the given target. In the following we will examine a few techniques that can be employed to achieve this goal.

There are two factors influencing how many applications are related to a given DNS name (or an IP address).

1. Different base URL
The obvious entry point for a web application is www.example.com, i.e. with this shorthand notation we think of the web application originating at http://www.example.com/ (the same applies for https). However, though this is the most common situation, there is nothing forcing the application to start at “/”.
For example, the same symbolic name may be associated to three web applications such as
http://www.example.com/url1
http://www.example.com/url2
http://www.example.com/url3
In this case the URL http://www.example.com/ would not be associated to a meaningful page, and the three applications would be “hidden” unless we explicitly know how to reach them, i.e. we know url1, url2 or url3. There is usually no need to publish web applications in this way, unless you don’t want them to be accessible in a standard way, and you are prepared to inform your users about their exact location. This doesn’t mean that these applications are secret, but that their existence and location is not explicitly advertised.

2. Non-standard ports
While web applications usually live on port 80 (http) and 443 (https), there is nothing magic about these port numbers. In fact, web applications may be associated with arbitrary TCP ports, and can be referenced by specifying the port number as follows: http[s]://www.example.com:port/. For example, http://www.example.com:20000/.

There is another factor affecting how many web applications are related to a given IP address.

3. Virtual hosts
DNS allows to associate a single IP address to one or more symbolic names. For example, the IP address 192.168.1.100 might be associated to DNS names www.example.com, helpdesk.example.com, webmail.example.com (actually, it is not necessary that all the names belong to the same DNS domain). This 1-to-N relationship may be reflected to serve different content by using so called virtual hosts. The information specifying the virtual host we are referring to is embedded in the HTTP 1.1 Host: header [1].
We would not suspect the existence of other web applications in addition to the obvious www.example.com, unless we know of helpdesk.example.com and webmail.example.com.

Approaches to address issue 1 - non-standard URLs
There is no way to full-proof ascertain the existence of non-standard-named web applications. Being non-standard, there is no magic recipe handing them out. However, we may employ a couple of criteria which may aid in their quest.
Our first hope is that these application might be referenced by other web pages; as such, there is a chance that they have been spidered and indexed by web search engines. If we suspect the existence of such “hidden” applications on www.example.com we could, for example, do a bit of googling using the site operator and examining the result of a query for “site: www.example.com”. Among the returned URLs there could be one pointing to such a non-obvious application.
Another option is to probe for URLs which might be likely candidates for non-published applications. For example, a web mail front end might be accessible from https://www.example.com/webmail, while this URL could not be referenced anywhere (after all, employees would know where the webmail application is located, while there is no reason to tell this information to outsiders by publishing it onto the corporate web site). The same holds for administrative interfaces, which may be published at standard URLs (for example: a Tomcat administrative interface), and yet not being referenced anywhere. So, doing a bit of dictionary-style searching (or “intelligent guessing”) could yield back some results. Vulnerability scanners may help with this respect.

Approaches to address issue 2 - non-standard ports
Existence of web applications on non-standard ports is easy to check. A port scanner such as nmap [2] is capable of performing service recognition by means of the -sV option, and will identify http[s] services on arbitrary ports. What is required is a full scan of the whole 64k TCP port address space.
For example, the following command will look up, with a TCP connect scan, all open ports on IP 192.168.1.100 and will try to determine what services are bound to them (only essential switches are shown – nmap features a broad set of options, whose discussion is out of scope).
nmap –P0 –sT –sV –p1-65535 192.168.1.100
It is sufficient to examine the output and looking for http or https (or SSL-wrapped services).
The same task may be performed by vulnerability scanners – but first check that your scanner of choice is able to identify http[s] services running on non-standard ports. For example, Nessus [3] is capable of identifying them on arbitrary ports (provided you instruct it to scan all the ports), and will provide – with respect to nmap – a number of tests on known web server vulnerabilities, as well as on the SSL configuration of https services.

Approaches to address issue 3 - virtual hosts
There is a number of techniques which may be used to identify DNS names associated to a given IP address x.y.z.t.
DNS zone transfers
This technique is nowadays of limited usage, given the fact that zone transfers are largely not honored by DNS servers, but it is worth a try.
First of all, we must determine the name server serving x.y.z.t. Actually, matters get complicated because there could be many such servers. Maybe you know a name server because it was given as part of the target to be assessed (let it be dns.example.com). You may query dns.example.com by means of tools such as nslookup or dig and, if you are lucky, you may get back a list of the DNS entries for domain example.com. This will include the obvious www.example.com and the not-so-obvious helpdesk.example.com and webmail.example.com (and possibly others). Check all names returned by the zone transfer and consider all of those which are related to the target being evaluated.
If you don’t know a name server, you may look up whois information about the given IP address, and try to contact the name servers listed by whois, and request a zone transfer as said before.

DNS inverse queries
This process is similar to the previous but relies on inverse (PTR) DNS records. Rather than requesting a zone transfer, try setting the record type to PTR and issue a query on the given IP address. If you are lucky, you may get back a DNS name entry. This technique relies on the existence of IP-to-symbolic name maps, which is not granted.

Web-based DNS searches
This kind of search is akin to DNS zone transfer, but relies on web-based services which allow to perform name-based searched on DNS. One such a service is the Netcraft Search DNS service, available at http://searchdns.netcraft.com/?host. You may query for a list of names belonging to your domain of choice, such as example.com. Then you will check whether the names you obtained are pertaining to the target you are examining.

Reverse-IP services
Reverse-IP services are similar to DNS inverse queries, with the difference that you query a web-based application instead of a name server. There is a number of such services available. Since they tend to return partial (and often different) results, it is better to use multiple services to obtain a more comprehensive analysis.

Domain tools reverse IP: http://www.domaintools.com/reverse-ip/
(requires free membership)

MSN search: http://search.msn.com
syntax: "ip:x.x.x.x" (without the quotes)


Webhosting info: http://whois.webhosting.info/
syntax: http://whois.webhosting.info/x.x.x.x


DNSstuff: http://www.dnsstuff.com/
(multiple services available)


http://net-square.com/msnpawn/index.shtml
(multiple queries on domains and IP addresses, requires installation)


tomDNS: http://www.tomdns.net/
(some services are still private at the time of writing)


SEOlogs.com: http://www.seologs.com/ip-domains.html
(reverse ip/domain lookup)



Googling
After you have gathered the most information you can with the previous techniques, you can rely on search engines to possibly refine and increment your analysis. This may yield evidence of additional symbolic names belonging to your target, or applications accessible via non-obvious URLs.

Gray Box testing and example

Testing for Topic X vulnerabilities:
...
Result Expected:
...

References

Whitepapers
...
Tools
...


OWASP Testing Guide v2

Here is the OWASP Testing Guide v2 Table of Contents