Difference between revisions of "Category:OWASP Proxy"

From OWASP
Jump to: navigation, search
(Introduce the OWASP Proxy)
 
(Update code explaining how to extend the library)
Line 29: Line 29:
 
As mentioned above, one objective is correctness. By this I mean correctly handling whatever the major browsers send to it, and successfully retrieving whichever resource was requested. Failure to do so will be addressed as soon as possible.
 
As mentioned above, one objective is correctness. By this I mean correctly handling whatever the major browsers send to it, and successfully retrieving whichever resource was requested. Failure to do so will be addressed as soon as possible.
  
Other than that, there is no intention to add major new features to the library above those required to fulfill its purpose as a Lstener and a HTTP client implementation.
+
Other than that, there is no intention to add major new features to the library above those required to fulfill its purpose as a Listener and a HTTP client implementation.
  
One feature which is under serious consideration is the ability to proxy upstream requests via a SOCKS proxy, for example, as implemented by OpenSSH.
+
==Extending the OWASP Proxy==
  
==Extensibility==
+
The basic classes are Listener and SocksListener, which can be customised to add your own functionality. If you just create an instance of Listener (or SocksListener), it will accept connections on the specified port, and relay them to the requested destination.
  
The library attempts to provide the necessary extension points to allow developers access to the major lifecycle events of a request and a response. Here are the major methods that can be overridden:
+
    Listener l = new Listener(8008); // listens to localhost by default
 +
    Listener sl = new SocksListener(1080);
  
        /**
+
or
        * Override this method to control SSL support.
+
        * Return null to disable SSL CONNECT support
+
        *
+
        * @param host
+
        *            the host that the client wishes to CONNECT to
+
        * @param port
+
        *            the port that the client wishes to CONNECT to
+
        * @return an SSLSocketFactory generated from the relevant server key
+
        *        material, or null to disable CONNECT support
+
        */
+
        protected SSLSocketFactory getSocketFactory(String host, int port) {
+
                return null;
+
        }
+
  
        /**
+
    Listener l = new Listener(InetAddress.getByAddress(new byte[] { 127, 0,
        * Override this method to configure your HttpClient.
+
                    0, 1 }), 8008);
        * For example, configure it to use an upstream proxy
+
    Listener sl = new SocksListener(InetAddress.getByAddress(new byte[] { 127, 0,
        *
+
                    0, 1 }), 1080);
        * @return a preconfigured HttpClient
+
        */
+
        protected HttpClient createHttpClient() {
+
                return new HttpClient();
+
        }
+
  
A developer wishing to implement an in-browser interface to their application can override this method. If the request refers to your UI, simply return an appropriate Response here, and that Response will be returned to the client. Returning null will allow the (possibly modified) Request to be sent on to the server.
+
The first thing a developer might want to do is make some changes to the request or response. This is done by means of a ProxyMonitor. This is an abstract class that can be extended to provide the specific functionality required. I'll show it here as an interface, just to reduce verbosity.
  
        /**
+
    Response requestReceived(Request request)
        * Called when a request is received by the proxy. Changes can be made to
+
        throws MessageFormatException;
        * the Request object to alter what will be sent to the server.
+
   
        *
+
    Response errorReadingRequest(Request request, Exception e)
        * @param request
+
        throws MessageFormatException;
        *            the Request received from the client
+
   
        * @return a custom Response to be sent directly back to the client without
+
    boolean responseHeaderReceived(Conversation conversation)
        *        making any request to a server, or null to forward the Request
+
        throws MessageFormatException;
        * @throws MessageFormatException
+
   
        *            if the request cannot be parsed
+
    void responseContentReceived(Conversation conversation, boolean streamed)
        */
+
         throws MessageFormatException;
         protected Response requestReceived(Request request)
+
   
                        throws MessageFormatException {
+
    Response errorFetchingResponseHeader(Request request, Exception e)
                return null;
+
        throws MessageFormatException;
         }
+
   
 +
    Response errorFetchingResponseContent(Conversation conversation, Exception e)
 +
        throws MessageFormatException;
 +
   
 +
    void wroteResponseToBrowser(Conversation conversation)
 +
         throws MessageFormatException;
 +
   
 +
    void errorWritingResponseToBrowser(Conversation conversation,
 +
        Exception e) throws MessageFormatException;
  
        /**
+
These methods are called at various stages of a request's lifecycle, if appropriate. Obviously error methods won't be called if there are no errors! Each method that returns a Response object can be customized to return a custom Response to the browser. e.g. overriding requestReceived(Request) can allow a developer to provide a browser-based interface to their program, by returning Response objects for specific requests, e.g. to a host called "proxy":
        * Called when an error is encountered while reading the request from the
+
        * client.
+
        *
+
        * @param request
+
        * @param e
+
        * @return a customized Response to be sent to the browser, or null to send
+
        *        the default error message
+
        * @throws MessageFormatException
+
        *            if the request couldn't be parsed
+
        */
+
        protected Response errorReadingRequest(Request request, Exception e)
+
                        throws MessageFormatException {
+
                return null;
+
        }
+
  
        /**
+
    public Response requestReceived(Request request)
        * Called when the Response headers have been read from the server. The
+
         throws MessageFormatException {
        * response content (if any) will not yet have been read. Analysis can be
+
        if (request.getHost().equals("proxy")) {
        * performed based on the headers to determine whether to intercept the
+
            return handleRequest(request);
        * complete response at a later stage. If you wish to intercept the complete
+
        * response message at a later stage, return false from this method to
+
        * disable streaming of the response content, otherwise the response would
+
        * already have been written to the browser when responseContentReceived is
+
        * called.
+
        *
+
        * Note: If you modify the response headers in this method, be very careful
+
        * not to affect the retrieval of the response content. For example,
+
        * deleting a "Transfer-Encoding: chunked" header would be a bad idea!
+
        *
+
        * @param conversation
+
        * @return true to stream the response to the client as it is being read
+
        *         from the server, or false to delay writing the response to the
+
        *        client until after responseContentReceived is called
+
        * @throws MessageFormatException
+
        *            if either the request or response couldn't be parsed
+
        */
+
        protected boolean responseHeaderReceived(Conversation conversation)
+
                        throws MessageFormatException {
+
                return true;
+
 
         }
 
         }
 +
        return super.requestReceived(request);
 +
    }
  
        /**
+
The library provides a LoggingProxyMonitor class, which simply logs the Requests and Responses that pass through. It also prints out any exceptions that may occur (not shown).
        * Called after the Response content has been received from the server. If
+
        * streamed is false, the response can be modified here, and the modified
+
        * version will be written to the client.
+
        *
+
        * @param conversation
+
        * @param streamed
+
        *            true if the response has already been written to the client
+
        * @throws MessageFormatException
+
        *            if either the request or response couldn't be parsed
+
        */
+
        protected void responseContentReceived(Conversation conversation,
+
                        boolean streamed) throws MessageFormatException {
+
        }
+
  
        /**
+
    class LoggingProxyMonitor extends ProxyMonitor {
        * Called in the event of an error occurring while reading the response
+
   
        * header from the client
+
        @Override
        *
+
         public void wroteResponseToBrowser(Conversation conversation)
        * @param request
+
                throws MessageFormatException {
        * @param e
+
             try {
        * @return a custom Response to be sent to the client, or null to use the
+
                int resp = conversation.getResponse().getMessage().length;
        *         default
+
                long time = conversation.getResponseBodyTime()
        * @throws MessageFormatException
+
                        - conversation.getRequestTime();
        *             if either the request or response couldn't be parsed
+
   
        */
+
                Request request = conversation.getRequest();
        protected Response errorFetchingResponseHeader(Request request, Exception e)
+
                StringBuilder buff = new StringBuilder();
                         throws MessageFormatException {
+
                buff.append(request.getMethod()).append(" ");
                 return null;
+
                buff.append(request.isSsl() ? "ssl " : "");
 +
                buff.append(request.getHost()).append(":")
 +
                         .append(request.getPort());
 +
                 buff.append(request.getResource()).append(" ");
 +
                buff.append(conversation.getResponse().getStatus()).append(" - ")
 +
                        .append(resp);
 +
                buff.append(" bytes in ").append(time).append("(").append(
 +
                        resp / (time * 1000));
 +
                buff.append(" bps)");
 +
                System.out.println(buff.toString());
 +
            } catch (MessageFormatException mfe) {
 +
            }
 
         }
 
         }
 +
    }
  
        /**
+
One thing that a developer might want to do is teach the proxy how to use an upstream proxy to reach the target. OWASP Proxy uses the standard Java ProxySelector mechanism for this:
        * Called in the event of an error occurring while reading the response
+
        * content from the client
+
        *
+
        * @param conversation
+
        * @param e
+
        * @return a custom Response to be sent to the client, or null to use the
+
        *        default
+
        * @throws MessageFormatException
+
        *            if either the request or response couldn't be parsed
+
        */
+
        protected Response errorFetchingResponseContent(Conversation conversation,
+
                        Exception e) throws MessageFormatException {
+
                return null;
+
        }
+
  
         protected void wroteResponseToBrowser(Conversation conversation)
+
    final Proxy upstream = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(host, port);
                        throws MessageFormatException {
+
   
 +
    final ProxySelector ps = new ProxySelector() {
 +
   
 +
         @Override
 +
        public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
 +
            System.err.println("Proxy connection failed! "
 +
                    + ioe.getLocalizedMessage());
 
         }
 
         }
 +
   
 +
        @Override
 +
        public List<Proxy> select(URI uri) {
 +
            return Arrays.asList(upstream);
 +
        }
 +
    };
  
        protected void errorWritingResponseToBrowser(Conversation conversation,
+
Having created the ProxySelector, we need to make use of it. OWASP Proxy has its own Http Client implementation, that specifically makes NO unnecessary changes to the requests or responses. Some libraries try to make sure that the requests it sends are well-formed, for instance, but for a security tool, sometimes it is interesting to send malformed requests.
                        Exception e) throws MessageFormatException {
+
 
 +
So, we can tell the Listener how to configure its HttpClient implementations, by providing a customised HttpClientFactory:
 +
 
 +
    HttpClientFactory hcf = new DefaultHttpClientFactory() {
 +
   
 +
        @Override
 +
        public HttpClient createHttpClient() {
 +
            HttpClient client = super.createHttpClient();
 +
            client.setProxySelector(ps);
 +
            return client;
 
         }
 
         }
 +
    };
 +
    listener.setHttpClientFactory(hcf);
 +
 +
By default, OWASP Proxy does not intercept SSL connections (HTTP CONNECT verb), and if it encounters an SSL encrypted channel, it simply refuses to continue. Developers can enable SSL interception by providing a CertificateProvider implementation.
 +
 +
This is what the interface looks like:
 +
 +
    public interface CertificateProvider {
 +
   
 +
        SSLSocketFactory getSocketFactory(String host, int port) throws IOException;
 +
   
 +
    }
 +
 +
A simple implementation is provided in DefaultCertificateProvider that makes use of a single certificate for all connection attempts. A certificate location and passwords can be provided; alternatively, OWASP Proxy's default certificate can be used.
 +
 +
    listener.setCertificateProvider(new DefaultCertificateProvider());
  
 +
Developers may wish to make use of something like the CyberVillains CA library to automatically generate certificates on the fly for each site visited.
  
 
==Project Contributors==
 
==Project Contributors==

Revision as of 15:45, 5 January 2009

Welcome to the OWASP Proxy

The OWASP Proxy aims to provide a high quality intercepting proxy library which can be used by developers who require this functionality in their own programs, rather than having to develop it all from scratch.

The library is developed in Java, making it most attractive to Java developers obviously, but also accessible to Python (Jython) and Ruby (JRuby) developers as well.

Contents

Overview

One of the priorities of this project is to allow developers to do whatever they choose, without enforcing RFC compliance. This is important for a security testing library, as often the most interesting behavior manifests outside the RFCs! Keep in mind that a lot of the safety nets that exist in libraries that enforce RFC compliance do not exist in this library, and that as the developer, you need to be prepared to deal with the consequences!

Another priority is to accurately deliver whatever is specified by the client, and similarly, to accurately reflect whatever is returned by the server, rather than coloured by the parsing and normalisation performed by the library.

Download

At the moment there is no packaged version of this library. Development is done in a git repository, located here.

Interested parties can download a snapshot of the code at any point using the snapshot link next to each revision, or clone the repository:

 $ git clone http://dawes.za.net/rogan/owasp-proxy/owasp-proxy.git/

Implementation details

In order to achieve byte for byte accuracy with what was sent by the client, and received from the server, OWASP Proxy does the bare minimum of message parsing. The basic storage of an HTTP message is as an array of byte (a byte for byte copy of what was read from the network), rather than parsed out into convenient pieces. The library does provide convenience methods for accessing interesting parts of the message, such as headers, content, etc, but the message itself is always stored as a large byte[].

The Request and Response objects that you may deal with also do not decode the message bodies for you. If the message was sent using chunked encoding, the message body will show the individual chunks that were sent. Of course, again, there are also classes which allow you to obtain the actual entity body, with appropriate decoding.

Future development

As mentioned above, one objective is correctness. By this I mean correctly handling whatever the major browsers send to it, and successfully retrieving whichever resource was requested. Failure to do so will be addressed as soon as possible.

Other than that, there is no intention to add major new features to the library above those required to fulfill its purpose as a Listener and a HTTP client implementation.

Extending the OWASP Proxy

The basic classes are Listener and SocksListener, which can be customised to add your own functionality. If you just create an instance of Listener (or SocksListener), it will accept connections on the specified port, and relay them to the requested destination.

   Listener l = new Listener(8008); // listens to localhost by default
   Listener sl = new SocksListener(1080);

or

   Listener l = new Listener(InetAddress.getByAddress(new byte[] { 127, 0,
                   0, 1 }), 8008);
   Listener sl = new SocksListener(InetAddress.getByAddress(new byte[] { 127, 0,
                   0, 1 }), 1080);

The first thing a developer might want to do is make some changes to the request or response. This is done by means of a ProxyMonitor. This is an abstract class that can be extended to provide the specific functionality required. I'll show it here as an interface, just to reduce verbosity.

   Response requestReceived(Request request)
       throws MessageFormatException;
   
   Response errorReadingRequest(Request request, Exception e)
       throws MessageFormatException;
   
   boolean responseHeaderReceived(Conversation conversation)
       throws MessageFormatException;
   
   void responseContentReceived(Conversation conversation, boolean streamed)
       throws MessageFormatException;
   
   Response errorFetchingResponseHeader(Request request, Exception e)
       throws MessageFormatException;
   
   Response errorFetchingResponseContent(Conversation conversation, Exception e)
       throws MessageFormatException;
   
   void wroteResponseToBrowser(Conversation conversation)
       throws MessageFormatException;
   
   void errorWritingResponseToBrowser(Conversation conversation,
       Exception e) throws MessageFormatException;

These methods are called at various stages of a request's lifecycle, if appropriate. Obviously error methods won't be called if there are no errors! Each method that returns a Response object can be customized to return a custom Response to the browser. e.g. overriding requestReceived(Request) can allow a developer to provide a browser-based interface to their program, by returning Response objects for specific requests, e.g. to a host called "proxy":

   public Response requestReceived(Request request)
       throws MessageFormatException {
       if (request.getHost().equals("proxy")) {
           return handleRequest(request);
       }
       return super.requestReceived(request);
   }

The library provides a LoggingProxyMonitor class, which simply logs the Requests and Responses that pass through. It also prints out any exceptions that may occur (not shown).

   class LoggingProxyMonitor extends ProxyMonitor {
   
       @Override
       public void wroteResponseToBrowser(Conversation conversation)
               throws MessageFormatException {
           try {
               int resp = conversation.getResponse().getMessage().length;
               long time = conversation.getResponseBodyTime()
                       - conversation.getRequestTime();
   
               Request request = conversation.getRequest();
               StringBuilder buff = new StringBuilder();
               buff.append(request.getMethod()).append(" ");
               buff.append(request.isSsl() ? "ssl " : "");
               buff.append(request.getHost()).append(":")
                       .append(request.getPort());
               buff.append(request.getResource()).append(" ");
               buff.append(conversation.getResponse().getStatus()).append(" - ")
                       .append(resp);
               buff.append(" bytes in ").append(time).append("(").append(
                       resp / (time * 1000));
               buff.append(" bps)");
               System.out.println(buff.toString());
           } catch (MessageFormatException mfe) {
           }
       }
   }

One thing that a developer might want to do is teach the proxy how to use an upstream proxy to reach the target. OWASP Proxy uses the standard Java ProxySelector mechanism for this:

   final Proxy upstream = new Proxy(Proxy.Type.HTTP, new InetSocketAddress(host, port);
   
   final ProxySelector ps = new ProxySelector() {
   
       @Override
       public void connectFailed(URI uri, SocketAddress sa, IOException ioe) {
           System.err.println("Proxy connection failed! "
                   + ioe.getLocalizedMessage());
       }
   
       @Override
       public List<Proxy> select(URI uri) {
           return Arrays.asList(upstream);
       }
   };

Having created the ProxySelector, we need to make use of it. OWASP Proxy has its own Http Client implementation, that specifically makes NO unnecessary changes to the requests or responses. Some libraries try to make sure that the requests it sends are well-formed, for instance, but for a security tool, sometimes it is interesting to send malformed requests.

So, we can tell the Listener how to configure its HttpClient implementations, by providing a customised HttpClientFactory:

   HttpClientFactory hcf = new DefaultHttpClientFactory() {
   
       @Override
       public HttpClient createHttpClient() {
           HttpClient client = super.createHttpClient();
           client.setProxySelector(ps);
           return client;
       }
   };
   listener.setHttpClientFactory(hcf);

By default, OWASP Proxy does not intercept SSL connections (HTTP CONNECT verb), and if it encounters an SSL encrypted channel, it simply refuses to continue. Developers can enable SSL interception by providing a CertificateProvider implementation.

This is what the interface looks like:

   public interface CertificateProvider {
   
       SSLSocketFactory getSocketFactory(String host, int port) throws IOException;
   
   }

A simple implementation is provided in DefaultCertificateProvider that makes use of a single certificate for all connection attempts. A certificate location and passwords can be provided; alternatively, OWASP Proxy's default certificate can be used.

   listener.setCertificateProvider(new DefaultCertificateProvider());

Developers may wish to make use of something like the CyberVillains CA library to automatically generate certificates on the fly for each site visited.

Project Contributors

The OWASP Proxy project is run by Rogan Dawes of Corsaire Security. He can be contacted at rogan AT dawes.za.net

This category currently contains no pages or media.