Difference between revisions of "Source Code Analysis Tools"

From OWASP
Jump to: navigation, search
(Corrected name/link for Coverity's static analysis tool)
(Added puma scan professional edition to the commercial tools list.)
 
(48 intermediate revisions by 23 users not shown)
Line 1: Line 1:
Source Code Analysis tools are designed to analyze source code and/or compiled version of code in order to help find security flaws. Ideally, such tools would automatically find security flaws with a high degree of confidence that what is found is indeed a flaw. However, this is beyond the state of the art for many types of application security flaws. Thus, such tools frequently serve as aids for an analyst to help them zero in on security relevant portions of code so they can find flaws more efficiently, rather than a tool that simply finds flaws automatically.
+
Source code analysis tools, also referred to as Static Application Security Testing (SAST) Tools, are designed to analyze source code and/or compiled versions of code to help find security flaws. Ideally, such tools would automatically find security flaws with such a high degree of confidence that what's found is indeed a flaw. However, this is beyond the state of the art for many types of application security flaws. Thus, such tools frequently serve as aids for an analyst to help them zero in on security relevant portions of code so they can find flaws more efficiently, rather than a tool that just automatically finds flaws. If you are interested in the effectiveness of SAST tools, check out the OWASP [[Benchmark]] project, which is scientifically measuring the effectiveness of all types of vulnerability detection tools, including SAST.
  
Some tools are starting to move into the IDE. For the types of problems that can be detected during the software development phase itself, this is a powerful phase within the development lifecycle to employ such tools, as it provides immediate feedback to the developer on issues they might be introducing into the code during code development itself. This immediate feedback is very useful as compared to finding vulnerabilities much later in the development cycle.
+
Some tools are starting to move into the IDE. For the types of problems that can be detected during the software development phase itself, this is a powerful phase within the development life cycle to employ such tools, as it provides immediate feedback to the developer on issues they might be introducing into the code during code development itself. This immediate feedback is very useful, especially when compared to finding vulnerabilities much later in the development cycle.
  
==Strengths and Weaknesses of such tools==
+
== Strengths and Weaknesses ==
  
 
=== Strengths ===
 
=== Strengths ===
* Scales Well (Can be run on lots of software, and can be repeatedly (like in nightly builds))
+
 
* For things that such tools can automatically find with high confidence, such as buffer overflows, SQL Injection Flaws, etc. they are great.
+
* Scales well -- can be run on lots of software, and can be run repeatedly (as with nightly builds or continuous integration)
* Output is good for developers - it highlights the precise source files and line numbers that are affected
+
* Useful for things that such tools can automatically find with high confidence, such as buffer overflows, SQL Injection Flaws, and so forth
 +
* Output is good for developers -- highlights the precise source files, line numbers, and even subsections of lines that are affected
  
 
=== Weaknesses ===
 
=== Weaknesses ===
* Many types of security vulnerabilities are very difficult to find automatically, such as authentication problems, access control issues, insecure use of cryptography, etc. The current state of the art only allows such tools to automatically find a relatively small percentage of application security flaws. Tools of this type are getting better, however.
+
 
 +
* Many types of security vulnerabilities are very difficult to find automatically, such as authentication problems, access control issues, insecure use of cryptography, etc. The current state of the art only allows such tools to automatically find a relatively small percentage of application security flaws. However, tools of this type are getting better.
 
* High numbers of false positives.
 
* High numbers of false positives.
 
* Frequently can't find configuration issues, since they are not represented in the code.
 
* Frequently can't find configuration issues, since they are not represented in the code.
Line 19: Line 21:
 
==Important Selection Criteria==
 
==Important Selection Criteria==
  
* Requirement: Must support your language, but not usually a key factor once it does.
+
* Requirement: Must support your programming language, but not usually a key factor once it does.
 
+
* Types of vulnerabilities it can detect (out of the [[OWASP Top Ten]]?) (plus more?)
* Types of Vulnerabilities it can detect (Out of the OWASP Top Ten?) (plus more?)
+
* How accurate is it? False Positive/False Negative rates?
 +
** Does the tool have an OWASP [[Benchmark]] score?
 +
* Does it understand the libraries/frameworks you use?
 
* Does it require a fully buildable set of source?
 
* Does it require a fully buildable set of source?
 
* Can it run against binaries instead of source?
 
* Can it run against binaries instead of source?
 
* Can it be integrated into the developer's IDE?
 
* Can it be integrated into the developer's IDE?
 +
* How hard is it to setup/use?
 +
* Can it be run continuously and automatically?
 
* License cost for the tool. (Some are sold per user, per org, per app, per line of code analyzed. Consulting licenses are frequently different than end user licenses.)
 
* License cost for the tool. (Some are sold per user, per org, per app, per line of code analyzed. Consulting licenses are frequently different than end user licenses.)
  
 
==OWASP Tools Of This Type==
 
==OWASP Tools Of This Type==
 +
 +
* [[OWASP SonarQube Project]]
 
* [http://www.owasp.org/index.php/Category:OWASP_Orizon_Project OWASP Orizon Project]
 
* [http://www.owasp.org/index.php/Category:OWASP_Orizon_Project OWASP Orizon Project]
 
* [[OWASP_LAPSE_Project | OWASP LAPSE Project]]
 
* [[OWASP_LAPSE_Project | OWASP LAPSE Project]]
 
* [[OWASP O2 Platform]]
 
* [[OWASP O2 Platform]]
 +
* [[OWASP WAP-Web Application Protection]]
 +
 +
==Disclaimer==
 +
 +
Disclaimer: <b>The tools listed in the tables below are presented in alphabetical order. <i>OWASP does not endorse any of the vendors or tools by listing them in the table below.</i> We have made every effort to provide this information as accurately as possible. If you are the vendor of a tool below and think that this information is incomplete or incorrect, please send an e-mail to our mailing list and we will make every effort to correct this information.</b>
  
 
==Open Source or Free Tools Of This Type==
 
==Open Source or Free Tools Of This Type==
  
* [http://www.stachliu.com/resources/tools/google-hacking-diggity-project/attack-tools/ Google CodeSearchDiggity] - Utilizes Google Code Search to identifies vulnerabilities in open source code projects hosted by Google Code, MS CodePlex, SourceForge, Github, and more. The tool comes with over 130 default searches that identify SQL injection, cross-site scripting (XSS), insecure remote and local file includes, hard-coded passwords, and much more.  ''Essentially, Google CodeSearchDiggity provides a source code security analysis of nearly every single open source code project in existence – simultaneously.''  
+
* [https://wiki.openstack.org/wiki/Security/Projects/Bandit Bandit] - bandit is a comprehensive source vulnerability scanner for Python
* [http://findbugs.sourceforge.net/ FindBugs] - Find Bugs (including some security flaws) in Java Programs
+
* [http://brakemanscanner.org/ Brakeman] - Brakeman is an open source vulnerability scanner specifically designed for Ruby on Rails applications
* [http://msdn.microsoft.com/en-us/library/bb429476(VS.80).aspx FxCop] (Microsoft) - FxCop is an application that analyzes managed code assemblies (code that targets the .NET Framework common language runtime) and reports information about the assemblies, such as possible design, localization, performance, and security improvements.
+
* [http://rubygems.org/gems/codesake-dawn Codesake Dawn] - Codesake Dawn is an open source security source code analyzer designed for Sinatra, Padrino for Ruby on Rails applications. It also works on non-web applications written in Ruby
 +
* [http://findbugs.sourceforge.net/ FindBugs] - Find Bugs (including a few security flaws) in Java programs
 +
* [https://find-sec-bugs.github.io/ FindSecBugs] - A security specific plugin for FingBugs that significantly improves FindBug's ability to find security vulnerabilities in Java programs
 +
* [http://www.dwheeler.com/flawfinder/ Flawfinder] Flawfinder - Scans C and C++
 +
* [https://www.bishopfox.com/resources/tools/google-hacking-diggity/attack-tools/ Google CodeSearchDiggity] - Uses Google Code Search to identifies vulnerabilities in open source code projects hosted by Google Code, MS CodePlex, SourceForge, Github, and more. The tool comes with over 130 default searches that identify SQL injection, cross-site scripting (XSS), insecure remote and local file includes, hard-coded passwords, and much more.  ''Essentially, Google CodeSearchDiggity provides a source code security analysis of nearly every single open source code project in existence – simultaneously.''  
 
* [http://pmd.sourceforge.net/ PMD] - PMD scans Java source code and looks for potential code problems (this is a code quality tool that does not focus on security issues)
 
* [http://pmd.sourceforge.net/ PMD] - PMD scans Java source code and looks for potential code problems (this is a code quality tool that does not focus on security issues)
* [http://msdn.microsoft.com/en-us/library/ms933794.aspx PreFast] (Microsoft) - PREfast is a static analysis tool that identifies defects in C/C++ programs
+
* [http://msdn.microsoft.com/en-us/library/ms933794.aspx PreFast] (Microsoft) - PREfast is a static analysis tool that identifies defects in C/C++ programs. Last update 2006.
* [https://www.fortify.com/ssa-elements/threat-intelligence/rats.html RATS] (Fortify) - Scans C, C++, Perl, PHP and Python source code for security problems like buffer overflows and TOCTOU (Time Of Check, Time Of Use) race conditions
+
* [https://pumascan.com/ Puma Scan] - Puma Scan is a .NET C# open source static source code analyzer that runs as an IDE plugin for Visual Studio and via MSBuild in CI pipelines.
* [https://www.owasp.org/index.php/Category:OWASP_SWAAT_Project OWASP SWAAT Project] - Simplistic Beta Tool - Languages: Java, JSP, ASP .Net, and PHP
+
* [https://dotnet-security-guard.github.io/ .NET Security Guard] - Roslyn analyzers that aim to help security audits on .NET applications. It will find SQL injections, LDAP injections, XXE, cryptography weakness, XSS and more.
* [http://www.dwheeler.com/flawfinder/ Flawfinder] Flawfinder - Scans C and C++
+
* [http://sourceforge.net/projects/rips-scanner/ RIPS] - RIPS is a static source code analyzer for vulnerabilities in PHP web applications. Please see notes on the sourceforge.net site.
* [http://sourceforge.net/projects/rips-scanner/ RIPS] - RIPS is a static source code analyzer for vulnerabilities in PHP web applications
+
* [https://github.com/FloeDesignTechnologies/phpcs-security-audit phpcs-security-audit] - phpcs-security-audit is a set of PHP_CodeSniffer rules that finds flaws or weaknesses related to security in PHP and its popular CMS or frameworks.  It currently has core PHP rules as well as Drupal 7 specific rules.
* [http://brakemanscanner.org/ Brakeman] - Brakeman is an open source vulnerability scanner specifically designed for Ruby on Rails applications
+
* [http://www.sonarqube.org/ SonarQube] - Scans source code for more than 20 languages for Bugs, Vulnerabilities, and Code Smells. SonarQube IDE plugins for Eclipse, Visual Studio, and IntelliJ provided by [http://www.sonarlint.org/ SonarLint].
* [http://rubygems.org/gems/codesake-dawn Codesake Dawn] - Codesake Dawn is an open source security source code analyzer designed for Sinatra, Padrino and Ruby on Rails applications. It can work also for non web application wrote in Ruby programming language
+
* [http://sourceforge.net/projects/visualcodegrepp/ VisualCodeGrepper (VCG)] - Scans C/C++, C#, VB, PHP, Java, and PL/SQL for security issues and for comments which may indicate defective code. The config files can be used to carry out additional checks for banned functions or functions which commonly cause security issues.
* [http://sourceforge.net/projects/visualcodegrepp/ VCG] - Scans C/C++, Java, C# and PL/SQL for security issues and for comments which may indicate defective code. The config files can be used to carry out additional checks for banned functions or functions which commonly cause security issues.
+
* [http://www.xanitizer.net Xanitizer] - Scans Java for security vulnerabilities, mainly via taint analysis. The tool comes with a number of predefined vulnerability detectors which can additionally be extended by the user.
 
 
==Commercial Tools from OWASP Members Of This Type==
 
  
These vendors have decided to support OWASP by becoming [[Membership|members]]. OWASP appreciates the support from these organizations, but cannot endorse any commercial products or services.
+
==Commercial Tools Of This Type==
  
* [http://www-01.ibm.com/software/rational/products/appscan/source/ IBM Security AppScan Source Edition] (formerly Ounce)
+
* [http://www-01.ibm.com/software/rational/products/appscan/source/ AppScan Source] (IBM)
* [http://www.armorize.com/codesecure/ Static Source Code Analysis with CodeSecure™] (Armorize Technologies)
+
* [http://www.blueclosure.com BlueClosure BC Detect] (BlueClosure)
* [http://www.checkmarx.com/technology/static-code-analysis-sca/ Static Code Analysis] (Checkmarx)
+
* [https://buguroo.com/products/bugblast-next-gen-appsec-platform/bugscout-sca bugScout] (Buguroo Offensive Security) Latest generation source code analysis tool bugScout detects source code vulnerabilities and makes possible an accurate management of the life cycles due to its easy use.
* [http://www.coverity.com/products/security-advisor.html Security Advisor] (Coverity)
+
* [http://www.castsoftware.com/solutions/application-security/cwe#SupportedSecurityStandards CAST AIP] (CAST) Performs static and architectural analysis to check for: SQL Injection, Cross Site Scripting (XSS), Input Validation,  Insecure Cryptographic Storage, Information Leakage and Improper Error Handling, Data Access, API Abuse, Encapsulation on over 30 languages.
* [https://www.fortify.com/products/hpfssc/source-code-analyzer.html Source Code Analysis] (HP/Fortify)
+
* [https://www.codacy.com/ Codacy] is free for open source projects, and integrates with tools such as Brakeman, Bandit, FindBugs, and a number of others. It offers security patterns for languages such as Python, Ruby, Scala, Java, Javascript and more.
 +
* [http://www.contrastsecurity.com/ Contrast from Contrast Security] Contrast performs code security without actually doing static analysis. Contrast does Interactive Application Security Testing (IAST), correlating runtime code & data analysis. It provides code level results without actually relying on static analysis.
 +
* [http://www.coverity.com/products/code-advisor/ Coverity Code Advisor] (Synopsys)
 +
* [https://www.checkmarx.com/technology/static-code-analysis-sca/ CxSAST] (Checkmarx)
 +
* [http://www8.hp.com/us/en/software-solutions/static-code-analysis-sast/ Fortify] (HP)
 +
* [http://www.juliasoft.com/solutions Julia] - SaaS Java static analysis (JuliaSoft)
 +
* [http://www.klocwork.com/capabilities/static-code-analysis KlocWork] (KlocWork)
 +
* [https://www.kiuwan.com/code-analysis/ Kiuwan] - SaaS Software Quality & Security Analysis (an [http://www.optimyth.com Optimyth] company)
 
* [http://www.parasoft.com/jsp/capabilities/static_analysis.jsp?itemId=547 Parasoft Test] (Parasoft)
 
* [http://www.parasoft.com/jsp/capabilities/static_analysis.jsp?itemId=547 Parasoft Test] (Parasoft)
* [http://www.veracode.com/ Veracode] (Veracode)
+
* [http://www.viva64.com/en/ PVS-Studio] (PVS-Studio) For C/C++, C#
 
+
* [https://pumascanpro.com/ Puma Scan Professional]  - Puma Scan Professional is a .NET C# static source code analyzer that runs as an IDE plugin for Visual Studio and via MSBuild in CI pipelines.
==Other Well Known Commercial Tools Of This Type==
+
* [https://www.whitehatsec.com/products/static-application-security-testing/ Sentinel Source] (Whitehat)
 
+
* [https://www.synopsys.com/software-integrity/products/interactive-application-security-testing.html Seeker] (Synopsys) Seeker performs code security without actually doing static analysis. Seeker does Interactive Application Security Testing (IAST), correlating runtime code & data analysis with simulated attacks. It provides code level results without actually relying on static analysis.
* [http://www.klocwork.com/products/insight.asp Insight] (KlocWork)
 
 
* [http://www.sourcepatrol.co.uk/ Source Patrol] (Pentest)
 
* [http://www.sourcepatrol.co.uk/ Source Patrol] (Pentest)
 +
* [http://www.veracode.com/products/binary-static-analysis-sast Veracode Static Analysis] (Veracode)
  
==More Info==
+
==More info==
  
* TODO: add comments from: http://lists.owasp.org/pipermail/owasp-dotnet/2006-August/000002.html
 
 
* [[Appendix_A:_Testing_Tools | Appendix A: Testing Tools]]
 
* [[Appendix_A:_Testing_Tools | Appendix A: Testing Tools]]
* [http://samate.nist.gov/index.php/Source_Code_Security_Analyzers NIST's list of Source Code Security Analysis Tools]
+
* [http://samate.nist.gov/index.php/Source_Code_Security_Analyzers.html NIST's list of Source Code Security Analysis Tools]
 +
* [[:Category:Vulnerability_Scanning_Tools | DAST Tools]] - Similar info on Dynamic Application Security Testing (DAST) Tools
  
 
[[Category:OWASP .NET Project]]
 
[[Category:OWASP .NET Project]]
 
+
[[Category:SAMM-CR-2]]
 
 
 
__NOTOC__
 
__NOTOC__

Latest revision as of 13:03, 29 September 2017

Source code analysis tools, also referred to as Static Application Security Testing (SAST) Tools, are designed to analyze source code and/or compiled versions of code to help find security flaws. Ideally, such tools would automatically find security flaws with such a high degree of confidence that what's found is indeed a flaw. However, this is beyond the state of the art for many types of application security flaws. Thus, such tools frequently serve as aids for an analyst to help them zero in on security relevant portions of code so they can find flaws more efficiently, rather than a tool that just automatically finds flaws. If you are interested in the effectiveness of SAST tools, check out the OWASP Benchmark project, which is scientifically measuring the effectiveness of all types of vulnerability detection tools, including SAST.

Some tools are starting to move into the IDE. For the types of problems that can be detected during the software development phase itself, this is a powerful phase within the development life cycle to employ such tools, as it provides immediate feedback to the developer on issues they might be introducing into the code during code development itself. This immediate feedback is very useful, especially when compared to finding vulnerabilities much later in the development cycle.

Strengths and Weaknesses

Strengths

  • Scales well -- can be run on lots of software, and can be run repeatedly (as with nightly builds or continuous integration)
  • Useful for things that such tools can automatically find with high confidence, such as buffer overflows, SQL Injection Flaws, and so forth
  • Output is good for developers -- highlights the precise source files, line numbers, and even subsections of lines that are affected

Weaknesses

  • Many types of security vulnerabilities are very difficult to find automatically, such as authentication problems, access control issues, insecure use of cryptography, etc. The current state of the art only allows such tools to automatically find a relatively small percentage of application security flaws. However, tools of this type are getting better.
  • High numbers of false positives.
  • Frequently can't find configuration issues, since they are not represented in the code.
  • Difficult to 'prove' that an identified security issue is an actual vulnerability.
  • Many of these tools have difficulty analyzing code that can't be compiled. Analysts frequently can't compile code because they don't have the right libraries, all the compilation instructions, all the code, etc.

Important Selection Criteria

  • Requirement: Must support your programming language, but not usually a key factor once it does.
  • Types of vulnerabilities it can detect (out of the OWASP Top Ten?) (plus more?)
  • How accurate is it? False Positive/False Negative rates?
    • Does the tool have an OWASP Benchmark score?
  • Does it understand the libraries/frameworks you use?
  • Does it require a fully buildable set of source?
  • Can it run against binaries instead of source?
  • Can it be integrated into the developer's IDE?
  • How hard is it to setup/use?
  • Can it be run continuously and automatically?
  • License cost for the tool. (Some are sold per user, per org, per app, per line of code analyzed. Consulting licenses are frequently different than end user licenses.)

OWASP Tools Of This Type

Disclaimer

Disclaimer: The tools listed in the tables below are presented in alphabetical order. OWASP does not endorse any of the vendors or tools by listing them in the table below. We have made every effort to provide this information as accurately as possible. If you are the vendor of a tool below and think that this information is incomplete or incorrect, please send an e-mail to our mailing list and we will make every effort to correct this information.

Open Source or Free Tools Of This Type

  • Bandit - bandit is a comprehensive source vulnerability scanner for Python
  • Brakeman - Brakeman is an open source vulnerability scanner specifically designed for Ruby on Rails applications
  • Codesake Dawn - Codesake Dawn is an open source security source code analyzer designed for Sinatra, Padrino for Ruby on Rails applications. It also works on non-web applications written in Ruby
  • FindBugs - Find Bugs (including a few security flaws) in Java programs
  • FindSecBugs - A security specific plugin for FingBugs that significantly improves FindBug's ability to find security vulnerabilities in Java programs
  • Flawfinder Flawfinder - Scans C and C++
  • Google CodeSearchDiggity - Uses Google Code Search to identifies vulnerabilities in open source code projects hosted by Google Code, MS CodePlex, SourceForge, Github, and more. The tool comes with over 130 default searches that identify SQL injection, cross-site scripting (XSS), insecure remote and local file includes, hard-coded passwords, and much more. Essentially, Google CodeSearchDiggity provides a source code security analysis of nearly every single open source code project in existence – simultaneously.
  • PMD - PMD scans Java source code and looks for potential code problems (this is a code quality tool that does not focus on security issues)
  • PreFast (Microsoft) - PREfast is a static analysis tool that identifies defects in C/C++ programs. Last update 2006.
  • Puma Scan - Puma Scan is a .NET C# open source static source code analyzer that runs as an IDE plugin for Visual Studio and via MSBuild in CI pipelines.
  • .NET Security Guard - Roslyn analyzers that aim to help security audits on .NET applications. It will find SQL injections, LDAP injections, XXE, cryptography weakness, XSS and more.
  • RIPS - RIPS is a static source code analyzer for vulnerabilities in PHP web applications. Please see notes on the sourceforge.net site.
  • phpcs-security-audit - phpcs-security-audit is a set of PHP_CodeSniffer rules that finds flaws or weaknesses related to security in PHP and its popular CMS or frameworks. It currently has core PHP rules as well as Drupal 7 specific rules.
  • SonarQube - Scans source code for more than 20 languages for Bugs, Vulnerabilities, and Code Smells. SonarQube IDE plugins for Eclipse, Visual Studio, and IntelliJ provided by SonarLint.
  • VisualCodeGrepper (VCG) - Scans C/C++, C#, VB, PHP, Java, and PL/SQL for security issues and for comments which may indicate defective code. The config files can be used to carry out additional checks for banned functions or functions which commonly cause security issues.
  • Xanitizer - Scans Java for security vulnerabilities, mainly via taint analysis. The tool comes with a number of predefined vulnerability detectors which can additionally be extended by the user.

Commercial Tools Of This Type

  • AppScan Source (IBM)
  • BlueClosure BC Detect (BlueClosure)
  • bugScout (Buguroo Offensive Security) Latest generation source code analysis tool bugScout detects source code vulnerabilities and makes possible an accurate management of the life cycles due to its easy use.
  • CAST AIP (CAST) Performs static and architectural analysis to check for: SQL Injection, Cross Site Scripting (XSS), Input Validation,  Insecure Cryptographic Storage, Information Leakage and Improper Error Handling, Data Access, API Abuse, Encapsulation on over 30 languages.
  • Codacy is free for open source projects, and integrates with tools such as Brakeman, Bandit, FindBugs, and a number of others. It offers security patterns for languages such as Python, Ruby, Scala, Java, Javascript and more.
  • Contrast from Contrast Security Contrast performs code security without actually doing static analysis. Contrast does Interactive Application Security Testing (IAST), correlating runtime code & data analysis. It provides code level results without actually relying on static analysis.
  • Coverity Code Advisor (Synopsys)
  • CxSAST (Checkmarx)
  • Fortify (HP)
  • Julia - SaaS Java static analysis (JuliaSoft)
  • KlocWork (KlocWork)
  • Kiuwan - SaaS Software Quality & Security Analysis (an Optimyth company)
  • Parasoft Test (Parasoft)
  • PVS-Studio (PVS-Studio) For C/C++, C#
  • Puma Scan Professional - Puma Scan Professional is a .NET C# static source code analyzer that runs as an IDE plugin for Visual Studio and via MSBuild in CI pipelines.
  • Sentinel Source (Whitehat)
  • Seeker (Synopsys) Seeker performs code security without actually doing static analysis. Seeker does Interactive Application Security Testing (IAST), correlating runtime code & data analysis with simulated attacks. It provides code level results without actually relying on static analysis.
  • Source Patrol (Pentest)
  • Veracode Static Analysis (Veracode)

More info