OWASP ModSec CRS Paranoia Mode

Abstract
This is a page about the development of a paranoia mode aka bringing back the rules that used to yield a high number of false positives. This little project is aimed at inclusion into the 3.0.0 release of the OWASP ModSecurity Core Rules, where some rules have been removed in order to reduce the number of false positives with vanilla installations.

FIXME: Detailed description

Back to the OWASP ModSecurity Core Rules Set.

Sub-Project Infos

 * Status: active (January 2016)
 * Schedule: Pull request in January February March 2016 (Almost there!)
 * Who: Christian Folini (dune73), Noël Zindel (zino), Franziska Bühler (franziskabuehler), Manuel Leos (Spartan), Walter Hop (lifeforms)
 * Documentation: Here on the OWASP Wiki / Mechanics Proposal
 * Discussion / Archive: owasp-modsecurity-core-rule-set@lists.owasp.org / archive: http://lists.owasp.org/pipermail/owasp-modsecurity-core-rule-set/
 * Github Link v3.0.0-rc1 (our base): https://github.com/SpiderLabs/owasp-modsecurity-crs/tree/v3.0.0-rc1
 * Github Link paranoia-mode: https://github.com/dune73/owasp-modsecurity-crs/tree/paranoia-mode
 * Final Pull Request #1: Add paranoia mode mechanics: https://github.com/SpiderLabs/owasp-modsecurity-crs/pull/292 MERGED
 * Final Pull Request #2: Move first rules to paranoia mode: https://github.com/SpiderLabs/owasp-modsecurity-crs/pull/300 MERGED
 * Final Pull Request #3: Add 2.2.X rules to paranoia mode: https://github.com/SpiderLabs/owasp-modsecurity-crs/pull/308 MERGED
 * Final Pull Request #4: Add stricter siblings: FIXME

Open Tasks
Please define state as follows: new, assigned, waiting, closed. When a task is closed, it is moved to the seperate closed tasks table below.

Paranoia Mode Candidates
The 3.0.0-rc1 has all rules renumbered. Existing numbering was fairly crazy and the new numbering follows the numbering scheme of the rules files (-> 9<2-digit-rulefile><3-digit-id>) A mapping table exists [IdNumbering.csv] We need to make sure, we do not mess things up, so let's add both IDs to the table, the old one and the new one.

Please set status as follows : confirmed,candidate, cloning-confirmed,cloning-candidate, unsure, dropped.
 * 'cloning-confirmed', 'cloning-candidates' are rules, that could be cloned into an even stricter variant with a stricter limit in a higher paranoia setting.
 * If dropped, please provide reasoning in the remarks.

Rules from 2.2.X, missing in 3.0.0-rc1
It looks as if only the base_rules made it into 3.0.0. In fact there are a few rule ids know from the optional and experimental rule folders in 2.2.X, but it is more likely, these are new 3.0.0 rules reusing old rule ids as the rules (regexes and msg) do not match at all.

When trying to generate the list below, be aware that the rule ids have been renumbered between 3.0.0-dev and 3.0.0-rc1. IdNumbering.csv in your friend.

Stricter siblings for existing rules
Stricter Siblings are rules that are present in the CRS but could be accompanied by a stricter clone in Paranoia Mode. Adjustments can differ from rule to rule but include higher anomaly ratings or stricter triggers (e.g. regex counters). To prevent masses of false positives, rules can come with additional filters (chained rules) for common use-cases. These can either be included into Paranoia Mode or simply serve as a recommendation.

Note: To avoid a cluttered project main-page, rule proposals are documented in their respective sub-page. When adding new proposals, make sure adding the rules original (2.2.x) ID, a quick description of what changes were made, and, if applicable, which additional filters were added. For every proposed rule change, we will create a Github issue; after discussion a pull request will be created. For brainstorming about CRS rules, see our OWASP ModSecurity rule evaluation framework.

Possible siblings:

981173 : SQL Injection Character Anomaly Usage 981172 : SQL Injection Character Anomaly Usage 981049 : Potential Denial of Service (DoS) 970003 : SQL Error Leakage 960901 : Invalid character in request 958980 : PHP Injection Attack: Variables Found 958979 : PHP Injection Attack: Configuration Directive Found 958977 : PHP Injection Attack: Function Name Found 950907 : Remote Command Execution (RCE) Attempt 950001 : SQL Injection Attack

How to Perform Pull Request to CRS v3.0.0-rc1
By Walter Hop

This example assumes that the Github username is 'lifeforms', replace with your own username.

'''Step 1. Create a Github fork of the original repository'''
 * a. Browse to CRS: https://github.com/SpiderLabs/owasp-modsecurity-crs
 * b. Click "Fork" button in the top right

'''Step 2. Download your Github fork, and checkout the correct branch '''

'''Step 3. Add the original repository as upstream, to integrate new changes easily'''

'''Step 4. Make sure your forked repo is up to date with changes in the original repository'''

You need to re-do these steps if somebody else made changes to upstream in the meantime!

'''Step 5. Create a branch for every separate issue you'd like to fix.'''

'''Step 6. Make local changes and commit them'''

'''Step 7. Push your local changes to your fork at Github'''

'''Step 8. Create the pull request'''
 * a. Browse to your own fork: https://github.com/lifeforms/owasp-modsecurity-crs
 * b. Click "New pull request" button
 * c. In the "base fork", ensure that the correct branch is selected: v3.0.0-rc1
 * d. In the "head fork", pick "myfeature" (if you used branches) or v3.0.0-rc1 (if you didn't)
 * e. Review the content of the pull request (should only contain your commits)
 * f. Press "Create pull request" button

Optional: Updating your pull request

After opening the pull request, people will review it, and probably will make some suggestions for changes before the PR can be accepted. You can keep pushing to your branch, and it will be reflected in the PR.

If in the meantime the code in upstream has drifted, you should re-do step 4 above.

Then, just change some files, commit and push.

See also:


 * https://help.github.com/articles/fork-a-repo/
 * https://help.github.com/articles/syncing-a-fork/
 * https://help.github.com/articles/using-pull-requests/

Testing someone else's pull request
To test a PR locally, use the following commands to create a branch for it.

In this example we check out PR #427, creating a branch chaim-headers locally (you can use any name):

git fetch upstream pull/427/head:chaim-headers git checkout chaim-headers

Purging a file completely from Git
Delete the file from the repo: git rm file git commit git push

Download BFG Repo-Cleaner

Make a NEW mirror: git clone --mirror git@github.com:SpiderLabs/owasp-modsecurity-crs.git

Purge the file: java -jar ~/Downloads/bfg-*.jar --delete-files example.png git reflog expire --expire=now --all && git gc --prune=now --aggressive git push -f

E-mail all users to REMOVE their copies of the repositories, and do a FRESH CLONE of the repo. If a user pulls again, the divergent histories will be merged, and there is a big risk that the dirty history (containing the purged file) will return again when they push later.

See also: BFG Repo-Cleaner