Project Reviews Guideline

=Main=



{| style="padding: 0;margin:0;margin-top:10px;text-align:left;" |-
 * valign="top" style="border-right: 1px dotted gray;padding-right:25px;" |

Purpose
Project Reviews is a process within OWASP to help evaluate the health and quality of OWASP projects. The evaluation is based on a defined criteria which attempts to find out the progress and at which stage of development are the projects.

This is the original plan https://www.owasp.org/index.php/Proposal_Project_Review_QA_Approach

Background
Projects are divided in 3 main categories:
 * Code
 * Tools
 * Documentation

These are the 3 main development classifications
 * Incubators
 * LAB
 * Flagship

Quality of a Code/Tool projects
This kind of evaluation requires more work. It is necessary to download, install and smoke test the project.

The criteria to evaluate the minimum quality of a project is very simple:

For Code and Tools
For projects holding Flagship status, we closely monitor their health every 6 months on the following, among other key indicators:
 * Can the project be built correctly?
 * Does the project has any activity(commits) in the last 6 months?
 * Does the project had any releases in the last 6 months?
 * Has the project leaders updated his wiki or website to reflect latest releases?

For Documentation
For this part, we are working on the development of an adequate assessment criteria The following is a draft of the new process proposal:


 * valign="top" style="padding-left:25px;width:200px;border-right: 1px dotted gray;padding-right:25px;" |

Presentation
https://soundcloud.com/owasp-podcast/owasp-project-reviews-with-johanna-curiel

Project Review Team
Johanna Curiel is not leading this task.


 * Timo Goosen
 * Munir Njiru

Support staff: Claudia.Aviles-Casanovas

Thank you to our season reviewers such as:
 * Jason Johnson
 * Rejah Rehim
 * Abbas Naderi
 * Carlos Allendes
 * Azeddine Islam Mennouchi

Thank you to Norman Yue who helped us acquired the JIRA for project reviews

Openhub
About the Black Duck Open Hub

The Black Duck Open Hub (formerly Ohloh.net) is an online community and public directory of free and open source software (FOSS), offering analytics and search services for discovering, evaluating, tracking, and comparing open source code and projects. Open Hub Code Search is free code search engine indexing over 21,000,000,000 lines of open source code from projects on the Black Duck Open Hub.

Use Openhub to have an overview of OWASP code and tools activity levels


 * 


 * valign="top" style="padding-left:25px;width:200px;" |

Email List
Project Email List https://groups.google.com/a/owasp.org/forum/?hl=en#!forum/projects-task-force

Classifications

 * }

=Basic Instructions=

We need regular or season reviewers to help us evaluate projects.

This is how you can help us evaluate the health of a project
 * Check the criteria that applies for each project (see tabs for criteria)

Follow the criteria with these instructions:
 * Visit the projects inventory page: https://www.owasp.org/index.php/Category:OWASP_Project#tab=Project_Inventory
 * Select the project you want to review or the one assigned to you
 * Click on the project wiki page you want to review
 * Check the history /last updated date and by who
 * Check the amount of views on the bottom of the page
 * Read the content and click all hyperlinks available, are there any broken?
 * Check and read the mailing list and the subjects discussed
 * Visit the repository's project (like Github) and check the wiki, issues, branches
 * Check if the repository is the same as the one in openhub
 * Create a new Google sheet
 * Do some google search on the project and the project leader. Make annotations and provide hyperlinks of information found
 * Fill in the sheet. Try to add to your report also some print screens, hyperlinks, in other words how did you came to the conclusions and findings in the comments sections. It is very important that you can sustain the findings provided in your report sheet
 * share the sheet through the Project task force mailing list

=Code Criteria=


 * Does the project have a publicly accessible bug tracking system established, and source code repository?
 * Does the project include online documention built into the tool?
 * Does the project include build scripts that facilitate building the application from source?
 * Does this project have an easy to use installer (Goal: Fully automated installer) (or stand alone executable version)?
 * Is the tool/deliverable user friendly and easy to use?
 * Smoke test some basic functionalities: Does the project do what they claim? (This might required more a simple smoke test)

Use this Google sheet(rename with Project name you will be reviewing) to fill in your findings: https://docs.google.com/spreadsheets/d/1upIyG0L-P-myUM6EPg0aJmCTDvJrdqaVdnjdNBME9is/edit?usp=sharing

= Tool Criteria =
 * Does the project have a publicly accessible bug tracking system established, and source code repository?
 * Does the project include online documentation built into the library?
 * Does the project include build scripts that facilitate building/adding to the application from source?
 * Does this project have an easy to use installer (Goal: Fully automated installer) (or stand alone executable version)?
 * Is the library/deliverable user friendly and easy to use?
 * Smoke test some basic functionalities: Does the project do what they claim? (This might required more a simple smoke test)

Use this Google sheet(rename with Project name you will be reviewing) to fill in your findings: https://docs.google.com/spreadsheets/d/1upIyG0L-P-myUM6EPg0aJmCTDvJrdqaVdnjdNBME9is/edit?usp=sharing

= Health Criteria = For general health criteria we use the following:

Does it meet quality expectations?
Check the Project wiki page description and introduction sections
 * Does the project have a relevant project summary that can be found on the OWASP Project wiki page?

Check a tab call Roadmap, see if there any sore of planning or projection on releases or deliverables
 * Does the project have a relevant project Roadmap that can be found on the OWASP Project wiki page?

The best place to check this is the repository issues and wiki page of the projects. Does the project have one? Is it easy to find? How many issues are open/closed?
 * Does the project have a good track record of resolving issues and answering questions from project consumers?

Does it follow OWASP Project best practices?
The project wiki page should contain a description with the type of license provided
 * Does the project use an appropriate Community Friendly License?

Does the project have a release version?
 * Are project deliverables, information, and releases readily available and accessible to the public?

This is very difficult to asses but try using Google search and finding information about the leaders of the project
 * Do the project leaders and contributors perform their duties in accordance to applicable laws?

Does it support the OWASP mission and objectives?
This is very difficult to asses but try using Google search and finding information about the leaders of the project. The project mailing lists history is a good source for this information Check for things like Logos in their wiki page or repository, mentioning of commercial activities, logos of vendors in their presentation
 * Do the project leaders and contributors treat everyone with respect and dignity?
 * Is the project vendor neutral?

again, difficult to asses , but try researching through google and find if in any form the leaders are commercial exploiting directly the project by charging users in any form Does the project have one accepted OWASP reviewed deliverable on record within the new project’s infrastructure? Check previous reviews of projects here: https://docs.google.com/a/owasp.org/spreadsheets/d/15NzgmnxKNtexRDs70rBUi1NHhjQiviBdYUa_kDvd3i4/edit?usp=sharing
 * Is the project free and open and not-for-profit?

Use this Google sheet(rename with Project name you will be reviewing) to fill in your findings: https://docs.google.com/spreadsheets/d/1upIyG0L-P-myUM6EPg0aJmCTDvJrdqaVdnjdNBME9is/edit?usp=sharing

= Documentation Criteria = This is a basic check up we do on Documents:
 * Does the project have a publicly accessible bug tracking system established, and source code repository?
 * Is the document in a format which can be converted to an OWASP book?
 * Does the project release/deliverable have a table of contents that links all the wiki content together?
 * Is the project release/deliverable available for download on the OWASP Project wiki page?
 * Has all release/deliverable content been reviewed by a technical editor to ensure that English grammar is correct, understandable, and the content flows well?

For this part, we are working on the development of an adequate assessment criteria The following is a draft of the new process proposal:

=Determining Flagship status= The OWASP Flagship designation is given to projects that have demonstrated superior maturity, established quality, and strategic value to OWASP and application security as a whole. Eligible projects are selected from the OWASP Labs project pool. This selection process generally ensures that there is only one project of each type covering any particular security space. OWASP Flagship projects represent projects that are not only mature, but are also projects that OWASP as an organization provides direct support to maintaining. The core mission of OWASP is to make application security visible and so as an organization, OWASP has a vested interest in the success of its Flagship projects. Since Flagship projects have such high visibility, these projects are expected to uphold the most stringent requirements of all OWASP Projects.

The project must filled all criteria to become flagship. It is essential for a flagship project to have:
 * Considerable number of users and contributors
 * Considerable number of commits and improvements in a time span of at least two years
 * A unique approach or proposition in application security
 * Exposure through security conferences
 * Use and acceptance by the community
 * Being used as reference in books and other resources

=Determining LAB status= OWASP Labs projects represent projects that have produced a deliverable of significant value. Leaders of OWASP Labs projects are expected to stand behind the quality of their projects as these projects have matured to the point where they are accepted by a significant portion of the OWASP community. While these projects are typically not production ready, the OWASP community expects that an OWASP Labs project leader is producing releases that are ready for mainstream usage. OWASP Labs Projects are meant to be the collection of established projects that have gained community support and acclaim by undergoing the project review process.

The project must filled all criteria to become LAB. It is essential for a LAB project to have:
 * A version number with a clear release
 * Github and issue tracking system
 * Stable build and release
 * Instructions on how to use and build the project properly even if not fully automated

=Want a review of your project?= Contact Claudia.Aviles-Casanovas[at]owasp.org for requesting more information regarding the process

Fill the following form http://www.tfaforms.com/264413