Difference between revisions of "OWASP RFP-Criteria"

From OWASP
Jump to: navigation, search
m
Line 1: Line 1:
4/11/2010 - Purpose of this project is to simply provide an objective list and a aggregate set of questions from companies to utilize when they issue a RFPs for web application security (Blackbox Testing).  The discussions for this project can be found [http://www.owasp.org/index.php/Category_talk:OWASP_RFP-Criteria here] and to get more people involved [http://twitter.com/home/?status=Help+Wanted+RFP+Project+at+OWASP+http://bit.ly/98hNFk TWEET IT]
+
4/11/2010 - Purpose of this project is to simply provide an objective list and a aggregate set of questions from companies to utilize when they issue a RFPs for web application security verification.  The discussions for this project can be found [http://www.owasp.org/index.php/Category_talk:OWASP_RFP-Criteria here] and to get more people involved [http://twitter.com/home/?status=Help+Wanted+RFP+Project+at+OWASP+http://bit.ly/98hNFk TWEET IT]
  
 
April 11th = Request for OWASP Volunteers RFC open<br>
 
April 11th = Request for OWASP Volunteers RFC open<br>
Line 6: Line 6:
  
  
==== QUESTIONS ====
+
=SUGGESTED APPLICATION SECURITY VERIFICATION RFP QUESTIONS=
  
<b>Company Background</b>
+
==Company Background==
  
 
1. Brief overview of products and/or services offered?
 
1. Brief overview of products and/or services offered?
Line 14: Line 14:
 
2. How many years has your company been in business?
 
2. How many years has your company been in business?
  
3. Other relevant background information?
+
3. Describe your experience with applications of a similar size, scope, complexity, and vertical as the applications to be verified.
  
<b>Product/Service Implementation and Assessment Methodology</b>
+
4. Describe your experience with the languages, frameworks, libraries, and other technologies that comprise the applications to be verified.
Example: [http://www.owasp.org/index.php/Category:OWASP_Testing_Project OWASP Testing Guide]
+
  
1. Describe the implementation process for your product/service - is software or hardware required?  Vendor training?  Consulting?  Any additional personnel costs on customer side?  How many personnel are needed?  What are their skill sets and expereince levels. --[[User:Walter Houser|Walter Houser]] 20:14, 16 April 2010 (UTC)
+
5. Describe your level of involvement in the application security community, in organizations such as OWASP and WASC.
+
2. Do you have a training and support program --[[User:Walter Houser|Walter Houser]] 20:14, 16 April 2010 (UTC) for your product or service?  Is it required?  If so, what is the typical amount of time and cost associated with training/education?
+
  
3. Approximately how long does it take to implement your product/service?
+
6. Other relevant background information?
  
4. What is the most challenging element associated with installing your product/service?
 
  
5. Describe the steps required for your product or service to assess a website?
 
  
6. What criteria are used to perform assessments?
+
==Application Security Verification Methodology==
  
7. Specifically, how does your solution scale for multiple websites?
+
1. Describe your methodology for all the verification techniques to be used:
  
8. How do you ensure you won't affect the performance of a publicly facing website during the testing process?
+
a) dynamic vulnerability scanning
 +
b) static analysis
 +
c) manual penetration testing
 +
d) manual code review
 +
e) threat modeling
  
9. Are you able to identify business logic vulnerabilities?  If so, how?
+
2. If multiple techniques are used, how will they be used together in the verification effort?
  
10. How do you limit (or eliminate) the reporting of false positives?
 
  
11. How can you ensure you will find all existing vulnerabilities?
+
==Security Coverage==
  
12. How do you identify new vulnerabilities and test for them?
+
1. Describe the vulnerability and security control coverage provided by your verification efforts with reference to the OWASP ASVS,[http://projects.webappsec.org/Threat-Classification WASC 24] Broad Classes of Attacks, and the [http://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project OWASP Top 10]
  
13. How are you able to uncover and test for new attack techniques that can be used to exploit known classes of vulnerabilities?
+
2. Describe the different levels of rigor that you offer for the verification effort. What are the differences in security coverage between these levels?
  
14. How do developers know if they have successfully re-mediated a vulnerability?
+
3. Are you currently able to test for Cross-Site Request Forgery (CSRF) and HTTP Response Splitting?
  
15. Does your product or service allow for on-demand / ad hoc testing?
+
4. How can you ensure you will find all existing vulnerabilities?
  
16. Does your product/service offer multiple navigation options?
+
5. What criteria are used to perform assessments?
  
17. Do you only scan "in the blind" or do you also review a website using log in credentials.
 
  
18. How many custom tests are available and how do you determine the number of custom tests will (or can) be run on a given website?
 
  
19. Does your product or service allow for automated scheduling and testing, with no human intervention on the part of the customer?
+
==Application Coverage==
 +
 
 +
1. How does your product/service baseline an application?
 +
 
 +
2. How do you tune your product/service to verify an application most effectively?
 +
 
 +
3. What methods do you use to ensure coverage of the entire application?
 +
 
 +
4. How do you verify with a customer that you are providing thorough coverage of the targeted application?
 +
 
 +
 
 +
 
 +
==Risk Evaluation==
 +
 
 +
1. Describe your process for determining the specific likelihood and business impact of vulnerabilities you discover.
 +
 
 +
2. How do you limit (or eliminate) the reporting of false positives?
 +
 
 +
3. Describe your approach for combining similar risks so that they can be easily understood and remediated.
 +
 
 +
 
 +
==??==
 +
 
 +
1. Describe the implementation process for your product/service - is software or hardware required?  Vendor training?  Consulting?  Any additional personnel costs on customer side?  How many personnel are needed?  What are their skill sets and expereince levels.  --[[User:Walter Houser|Walter Houser]] 20:14, 16 April 2010 (UTC)
 +
 +
2. Do you have a training and support program --[[User:Walter Houser|Walter Houser]] 20:14, 16 April 2010 (UTC) for your product or service?  Is it required?  If so, what is the typical amount of time and cost associated with training/education?
 +
 
 +
3. Approximately how long does it take to implement your product/service?
 +
 
 +
4. What is the most challenging element associated with installing your product/service?
 +
 
 +
5. Describe the steps required for your product or service to assess a website?
 +
 
 +
7. Specifically, how does your solution scale for multiple websites?
 +
 
 +
8. How do you ensure you won't affect the performance of a publicly facing website during the testing process?
 +
 
 +
9. How do developers know if they have successfully remediated a vulnerability?
  
20. How do you continuously improve the quality of your of product/service?  How frequently do customers receive updates/enhancements?
+
10. Does your product or service allow for on-demand / ad hoc testing?
  
21. Describe any recent innovations your company has introduced to lower costs or improve service for your customers.
 
  
22. How does your product/service compare to competitive offerings?
 
  
23. Does your product/service integrate with web application firewalls (WAFs)?  Which ones and how does it work?
 
  
24. Does your product/service integrate with bug tracking systems?  SIEMs? 
+
==Reporting Interface==
  
25. Do You provide the Coverage of the [http://projects.webappsec.org/Threat-Classification WASC 24] Broad Classes of Attacks and the [http://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project OWASP Top 10]
+
1. Describe exactly how risks will be written up, including:
  
*Brute Force
+
a) title
*Insufficient Authentication
+
b) location (URL and line of code)
*Weak Password Recovery Validation
+
c) specific description
*Credential / Session Prediction
+
d) risk likelihood, business impact, and severity
*Insufficient Authorization
+
e) code snippets
*Insufficient Session Expiration
+
f) specific remediation recommendations
*Session Fixation
+
*Content Spoofing
+
*Cross-site Scripting
+
*Buffer Overflow
+
*Format String Attack
+
*LDAP Injection
+
*OS Commanding
+
*SQL Injection
+
*SSI Injection
+
*XPath Injection
+
*Directory Indexing
+
*Information Leakage
+
*Path Traversal
+
*Predictable Resource Location
+
*Abuse of Functionality
+
*Denial of Service
+
*Insufficient Anti-automation
+
*Insufficient Process Validation
+
  
 +
2. Describe your reporting interface using criteria such as ease of use, clarity, comprehensiveness, how reporting components are organized, etc.
  
26. Are you currently able to test for Cross-Site Request
+
3. How does your product or service provide timely updates on any new web application risks identified?  Are alerts delivered to authorized personnel?  If so, how are they delivered and under what conditions?
Forgery (CSRF) and HTTP Response Splitting?
+
  
 +
4. Do you provide historical trending reports that track open/closed risks and the ongoing remediation process?
  
<b>Web Application Exploration Methodology</b>
+
5. Can assessment reports be generated to reflect the risk status of individual web applications, as well as the security health of all web applications?
  
 +
6. Can your reports be tailored and adapted for viewing by various levels of management, internal/external auditors, security specialists, etc.?
  
1. How does your product/service obtain a baseline of a website?
+
7. Does your solution provide an API so that risks can be exported into other applications, such as CRM apps, bug tracking systems, SIEMs?  Are there any canned scripts or standard integrations that exist?  With which applications?
  
2. How do you tune your product/service to scan a website most effectively?
+
8. Do your reports contain recommendations for application developers?
  
3. What methods do you use to discover links on a website?
+
9. How do you provide timely and reliable reporting of risks for ongoing visibility, measurement, and management?
  
4. How do you verify with a customer that you are thoroughly scanning all of the links on a given website.
+
10. How does your product or service store and protect risk information?
  
5. How do you test new technologies (e.g. new versions of Flash) for vulnerabilities?  
+
11. How frequently do you provide enhancements to your reporting interface?  What is the process?
  
  
<b>Reporting Interface</b>
 
  
 +
==Innovation==
  
1. Describe your reporting interface using criteria such as ease of use, clarity, comprehensiveness, how reporting components are organized, etc.  
+
1. Describe any recent innovations your company has introduced to lower costs or improve service for your customers.
  
2. Is the reporting interface web-based?
+
2. How do you identify new vulnerabilities and test for them?
  
3. How does your product or service provide timely updates on any new web application vulnerabilities identified?  Are alerts delivered to authorized personnel?  If so, how are they delivered and under what conditions?
+
3. How are you able to uncover and test for new attack techniques that can be used to exploit known classes of risks?
  
4. Do you provide historical trending reports that track open/closed vulnerabilities and the ongoing remediation process?
+
4. How do you test new technologies (e.g. new versions of Flash) for risks?  
Can assessment reports be generated to reflect the vulnerability status of individual web applications, as well as the security health of all web applications?  
+
  
5. Can your reports be tailored and adapted for viewing by various levels of management, internal/external auditors, security specialists, etc.?
 
  
6. In your reporting interface, do you assign severity levels to vulnerabilities found?  What criteria is used?
 
  
7. Does your solution provide an API so that reports can be exported into other applications, such as CRM apps, bug tracking systems, SIEMs?  Are there any canned scripts or standard integrations that exist?  With which applications?
+
==Integration==
  
8. Do your reports contain recommendations for application developers so that they can improve coding techniques?
+
1. How does your product/service compare to competitive offerings?
  
9. How do you provide timely and reliable reporting of vulnerabilities for ongoing visibility, measurement, and management?
+
2. Does your product/service integrate with web application firewalls (WAFs)?  Which ones and how does it work?
  
10. How much effort is required on the part of the customer to generate timely and accurate information using your reporting interface?
+
3. Do you offer remediation support to software development groups?
  
11. How does your product or service store scanning and vulnerability report data?
 
  
12. How frequently do you provide enhancements to your reporting interface?  What is the process?
 
  
  

Revision as of 00:06, 9 June 2010

4/11/2010 - Purpose of this project is to simply provide an objective list and a aggregate set of questions from companies to utilize when they issue a RFPs for web application security verification. The discussions for this project can be found here and to get more people involved TWEET IT

April 11th = Request for OWASP Volunteers RFC open
June 15th = Publish Version 1.0 - RC1 July 4th = Publish Version 1.0


SUGGESTED APPLICATION SECURITY VERIFICATION RFP QUESTIONS

Company Background

1. Brief overview of products and/or services offered?

2. How many years has your company been in business?

3. Describe your experience with applications of a similar size, scope, complexity, and vertical as the applications to be verified.

4. Describe your experience with the languages, frameworks, libraries, and other technologies that comprise the applications to be verified.

5. Describe your level of involvement in the application security community, in organizations such as OWASP and WASC.

6. Other relevant background information?


Application Security Verification Methodology

1. Describe your methodology for all the verification techniques to be used:

a) dynamic vulnerability scanning b) static analysis c) manual penetration testing d) manual code review e) threat modeling

2. If multiple techniques are used, how will they be used together in the verification effort?


Security Coverage

1. Describe the vulnerability and security control coverage provided by your verification efforts with reference to the OWASP ASVS,WASC 24 Broad Classes of Attacks, and the OWASP Top 10

2. Describe the different levels of rigor that you offer for the verification effort. What are the differences in security coverage between these levels?

3. Are you currently able to test for Cross-Site Request Forgery (CSRF) and HTTP Response Splitting?

4. How can you ensure you will find all existing vulnerabilities?

5. What criteria are used to perform assessments?


Application Coverage

1. How does your product/service baseline an application?

2. How do you tune your product/service to verify an application most effectively?

3. What methods do you use to ensure coverage of the entire application?

4. How do you verify with a customer that you are providing thorough coverage of the targeted application?


Risk Evaluation

1. Describe your process for determining the specific likelihood and business impact of vulnerabilities you discover.

2. How do you limit (or eliminate) the reporting of false positives?

3. Describe your approach for combining similar risks so that they can be easily understood and remediated.


??

1. Describe the implementation process for your product/service - is software or hardware required? Vendor training? Consulting? Any additional personnel costs on customer side? How many personnel are needed? What are their skill sets and expereince levels. --Walter Houser 20:14, 16 April 2010 (UTC)

2. Do you have a training and support program --Walter Houser 20:14, 16 April 2010 (UTC) for your product or service? Is it required? If so, what is the typical amount of time and cost associated with training/education?

3. Approximately how long does it take to implement your product/service?

4. What is the most challenging element associated with installing your product/service?

5. Describe the steps required for your product or service to assess a website?

7. Specifically, how does your solution scale for multiple websites?

8. How do you ensure you won't affect the performance of a publicly facing website during the testing process?

9. How do developers know if they have successfully remediated a vulnerability?

10. Does your product or service allow for on-demand / ad hoc testing?



Reporting Interface

1. Describe exactly how risks will be written up, including:

a) title b) location (URL and line of code) c) specific description d) risk likelihood, business impact, and severity e) code snippets f) specific remediation recommendations

2. Describe your reporting interface using criteria such as ease of use, clarity, comprehensiveness, how reporting components are organized, etc.

3. How does your product or service provide timely updates on any new web application risks identified? Are alerts delivered to authorized personnel? If so, how are they delivered and under what conditions?

4. Do you provide historical trending reports that track open/closed risks and the ongoing remediation process?

5. Can assessment reports be generated to reflect the risk status of individual web applications, as well as the security health of all web applications?

6. Can your reports be tailored and adapted for viewing by various levels of management, internal/external auditors, security specialists, etc.?

7. Does your solution provide an API so that risks can be exported into other applications, such as CRM apps, bug tracking systems, SIEMs? Are there any canned scripts or standard integrations that exist? With which applications?

8. Do your reports contain recommendations for application developers?

9. How do you provide timely and reliable reporting of risks for ongoing visibility, measurement, and management?

10. How does your product or service store and protect risk information?

11. How frequently do you provide enhancements to your reporting interface? What is the process?


Innovation

1. Describe any recent innovations your company has introduced to lower costs or improve service for your customers.

2. How do you identify new vulnerabilities and test for them?

3. How are you able to uncover and test for new attack techniques that can be used to exploit known classes of risks?

4. How do you test new technologies (e.g. new versions of Flash) for risks?


Integration

1. How does your product/service compare to competitive offerings?

2. Does your product/service integrate with web application firewalls (WAFs)? Which ones and how does it work?

3. Do you offer remediation support to software development groups?



Benefits Provided


1. How do you make the remediation process more efficient?

2. Can you help us save costs by not having to pay outside consultants?

3. Can your product or service help us to avoid heavy reliance on internal resources to perform assessments?

4. Does your solution meet PCI 6.6 standards?

5. Can you deliver accurate results and diminish/eliminate false positives, thus making the VA process more efficient?

6. How are you able to assess a site comprehensively - (i.e.) identify all existing vulnerabilities to minimize the possibility of exploitation?

7. Are you able to demonstrate a positive ROI and increased benefits to management? How?

8. Are you able to influence secure coding techniques / reduce time spent debugging? How?

9. Explain why we would realize a competitive advantage by doing business with your company?


Customer Support


1. Can you describe the process in place whereby customers interact with your internal service and support? What are the escalation procedures?

2. Do you provide tracking of all trouble tickets that have been opened and are they tracked through resolution?

3. Do you offer any kind of Service Level Agreement?


Pricing/Licensing Options


1. Explain your product licensing or service fee options?

2. What is the % for ongoing maintenance / support costs?

3. Do you charge separately for additional seats or user IDs?

4. What are the terms associated with your product or service? Can you include a sample Software License Agreement or Master Services Agreement template?

5. Are there other costs we should be made aware of? If consulting or training costs are involved, how do you charge for these services?

6. Are there in-house personnel costs that need to be factored in? If so, can you estimate how many hours should be allocated to internal resources? (as a reference point, use the amount of personnel time it would take to perform one website assessment as an example).


Customer References

Ref 1

Ref 2

Ref 3

Project Details

PROJECT INFO
What does this OWASP project offer you?
RELEASE(S) INFO
What releases are available for this project?
what is this project?
Name: OWASP RFP-Criteria (home page)
Purpose: Purpose of this project is to simply provide an objective list and a aggregate set of questions from companies to utilize when they issue a RFPs for web application security.
License: N/A
who is working on this project?
Project Leader(s):
how can you learn more?
Project Pamphlet: Not Yet Created
Project Presentation:
Mailing list: Mailing List Archives
Project Roadmap: View
Key Contacts
  • Contact the GPC to report a problem or concern about this project or to update information.
current release
Not Yet Published
last reviewed release
Not Yet Reviewed


other releases