Difference between revisions of "Assessing Project Health"

From OWASP
Jump to: navigation, search
m (Assessing Project Sites moved to Assessing Project Health: Changed to reflect latest version of the criteria v2)
 
(8 intermediate revisions by 3 users not shown)
Line 6: Line 6:
  
  
=== Assessing Project Sites ===
+
=== Project Health Level ===
Project sites themselves have a much simpler assessment criteria than releases.  Project sites are categorized into three levelsThe best method to illustrate the difference is to explain the progress of an example project through these categories:
+
Projects almost have a life of their own - beyond the releases they makeRegrettably the life, and health, of a project cannot be easily categorized by a single metricMultiple measures in combination make up a project's health.  Before discussing the method used to rate project health, lets first cover the levels of health. 
  
* A security professional has an idea to address an issue in application security and proposes a new project to the Global Projects Committee (GPC).
+
* Level 0 - a project that exists or is just beginningIt is either a project with no releases or all releases are no more then Alpha quality.
* The GPC agrees with the proposal, gathers some initial data from the security professional and creates a new project pageThe site has reached level 0.
+
* Level 1 - a project that has a release at Beta quality.  It is a project with a release that has been reviewed by at least one project leader since Beta quality level is the minimum.
* The security professional, now the project lead, works on the project and creates a release which reaches Alpha quality.
+
* Level 2 - a project with at least some of the ratings of health. In general, it should have roughly half.  It is difficult to be precise since the exact number and type of these ratings has not yet been determined.  As soon as those are solidified, the number for level 2 will be specified.
* The project lead continues to work on the project, it gets reviewed and reaches Beta quality.  The project has reached level 1.
+
* Level 3 - a project which has all the ratings of health.  This level represents the most healthy state for an OWASP project.
* The project lead continues to work on the project release and reaches a Quality release. Additional metrics are collected (the exact nature and method of collection is to be determined)After reaching a to be specified metric, the site reaches level 2.
+
  
 +
Notes on Project Health Levels:
 +
# The site will be reviewed based on the Project wiki page minimal contents below during any level change to ensure minimal project information is present on the project site.  See below for details.
 +
# Maintenance of the project wiki page can be handled by either the Project Lead or the Project Maintainer if the project has one.
 +
# The ratings of health will be determined shortly.  Some general information is below.  Various logistic and practical aspects need to be determined. --[[User:Mtesauro|Mtesauro]] 17:44, 4 May 2009 (UTC)
  
=== Project Site Levels ===
 
Project sites fall into three discrete levels:
 
  
* Level 0 - a project that is just beginning.  It is either a project with no releases or all releases no more then Alpha quality.
+
=== Project Health Ratings ===
* Level 1 - a project that has reviewed releases.  It is a project that has been reviewed by at least one project leader and has at least 1 release at Beta quality level.
+
* Level 2 - Specifics for level 2 sites have not been determined yet.
+
  
Notes on Project Site Levels:
+
The exact meta-data used to determine project health has not yet been fully determined. Some of the sources of data for the health rating are listed below.  Some, all or none of these may be used in the final versionThis list is to server as a public brainstorming exercise:
# The site will be reviewed based on the Project Site Criteria below during any level change to ensure minimal project information is present.
+
# Maintenance of the project site can be handled by either the Project Lead or the Project Maintainer if the project has one.
+
# The Level 2 specification will be determined shortlyVarious logistic and practical aspects need to be determined. --[[User:Mtesauro|Mtesauro]] 14:57, 28 April 2009 (UTC)
+
  
=== Project Site Criteria ===
+
* Number of releases
 +
* Size of the project's community
 +
* Industry participation
 +
* Usability
 +
* Number of participants
 +
* Number of "stars" - the idea here is to have a rating system similar to what Amazon uses for books, etc.
 +
* Creation and quality of training/tutorial material for the project.  The Global Education Committee is a key driver of this potential metric. 
  
The following questions will be answered by the project lead or project maintainer and be reviewed by the Global Projects Committee:
+
It is likely and should be expected for this list to change between now and its final version.  Note also that the exact method of gather the above data is not yet specified.
  
* Does the project site...  
+
=== Project Wiki Page Minimal Content ===
 +
 
 +
The following questions will be answered by the project lead or project maintainer and be reviewed by the Global Projects Committee whenever a project changes health levels:
 +
 
 +
* Does the project wiki page...  
 
# have an up to date project template with current project information?
 
# have an up to date project template with current project information?
# have a conference style presentation that describes the tool in at least 3 slides?
+
# [http://globalprojectscommittee.wordpress.com/2009/07/27/what-is-the-3x-slide-presentation-thing/ have a conference style presentation that describes the tool/document in at least 3 slides?]
# have a one sheet overview document about the project?
+
# [http://globalprojectscommittee.wordpress.com/2009/07/21/what-is-this-project-flyerpamphlet-thing/ have a one sheet overview document about the project?]
# have a link to a working mail list?
+
# [https://lists.owasp.org/mailman/listinfo have a link to a working mail list?]
 
# have a statement of the application security issue the project addresses?
 
# have a statement of the application security issue the project addresses?
 
# have a project roadmap?
 
# have a project roadmap?
 +
# project leaders and main contributors have a wiki account (with its user page containing contact details about the user and if possible his CV)
  
 
For OWASP project wiki pages, please see the Project Wiki Pages section of the [http://www.owasp.org/index.php/Guidelines_for_OWASP_Projects Guidelines for OWASP Projects] for additional suggestions/recommendations.
 
For OWASP project wiki pages, please see the Project Wiki Pages section of the [http://www.owasp.org/index.php/Guidelines_for_OWASP_Projects Guidelines for OWASP Projects] for additional suggestions/recommendations.
 +
 +
=== Example of the Process ===
 +
 +
As a example of this process, the story below demonstrates a project from inception to optimal health:
 +
* A security professional has an idea to address an issue in application security and proposes a new project to the Global Projects Committee (GPC).
 +
* The GPC agrees with the proposal, gathers some initial data from the security professional and creates a new project page.  The project starts at health level 0.
 +
* The security professional, now the project lead, works on the project and creates a release which reaches Alpha quality.
 +
* The project lead continues to work on the project, it gets reviewed and reaches Beta quality.  The project has reached level 1. The project is generating health metrics.
 +
* The project lead continues to work on the project release and reaches a Stable release.
 +
* Additional metrics are collected (the exact nature and method of collection is to be determined).  After reaching approximately half of the project health ratings, the project moves to level 2.  Note this could have happened before creating a stable release.
 +
* The project lead starts a new sub-project which is Alpha quality.  The new release does not negatively impact the projects health.  Likely, it will help it grow and become more healthy.
 +
* The project grows in activity and prominence to the point that it reaches all the project health ratings.  The project is now a level 3 project.
 +
 +
=== Star Rating System ===
 +
 +
{| style="width:90%" border="0" align="center"
 +
| colspan="7" align="center" style="background:#white; color:black" |
 +
|-
 +
| style="width:20%; background:#white" align="center" | Not Reviewed   
 +
| style="width:20%; background:#white" align="center" | Level 0
 +
| style="width:20%; background:#white" align="center" | level 1
 +
| style="width:20%; background:#white" align="center" | Level 2
 +
| style="width:20%; background:#white" align="center" | Level 3
 +
|-
 +
| style="width:20%; background:#white" align="center" | [[Image:Yellow button.JPG|25px]]
 +
| style="width:20%; background:#white" align="center" | [[Image:Greenlight.png|25px]]
 +
| style="width:20%; background:#white" align="center" | [[Image:Greenlight.png|25px]][[Image:Greenlight.png|25px]]
 +
| style="width:20%; background:#white" align="center" | [[Image:Greenlight.png|25px]][[Image:Greenlight.png|25px]][[Image:Greenlight.png|25px]]
 +
| style="width:20%; background:#white" align="center" | [[Image:Greenlight.png|25px]][[Image:Greenlight.png|25px]][[Image:Greenlight.png|25px]][[Image:Greenlight.png|25px]]
 +
|}
 +
 +
  
 
=== Archiving Project Sites ===
 
=== Archiving Project Sites ===
Line 46: Line 85:
 
The exact criteria for archiving project sites has not yet been determined.  However, the Global Projects Committee sees that an archive of projects that are kept for historical purposes will be needed.  This page or subsequent pages will determine the situation under which project pages are archived.
 
The exact criteria for archiving project sites has not yet been determined.  However, the Global Projects Committee sees that an archive of projects that are kept for historical purposes will be needed.  This page or subsequent pages will determine the situation under which project pages are archived.
  
=== Pre-existing project sites ===
 
  
The Global Projects Committee realizes that there are many current project sites which pre-existed the above assessment criteria.  Those project sites will be reviewed and classified in the near future.  The exact timing and methodology for addressing existing sites has not yet been determined.  The Global Projects Committee will first fully specify the new framework before working on mapping existing projects into the new framework.
+
=== Pre-existing OWASP projects ===
 +
 
 +
The Global Projects Committee realizes that there are many current project sites which pre-existed the above assessment of project health.  Those project sites will be migrated into the new criteria in the near future.  The exact timing and methodology for addressing existing sites has not yet been determined.  The Global Projects Committee will first fully specify the new framework before working on mapping existing projects into the new framework.
 +
 
 +
=== Methodology Follow - Up ===
 +
 
 +
As part of the transition to the new project details template, [[:Assessing Project Health - Check-List|this page is a follow up list for issues to check ]].

Latest revision as of 11:22, 25 September 2009


This is a DRAFT page still under review by the Global Projects Committee

This page is maintained by the Global Projects Committee to help assist Project Leaders with information about successfully running an OWASP Project. It will be updated from time to time, and changes will be discussed and announced on the OWASP-Leaders list.


Contents

Project Health Level

Projects almost have a life of their own - beyond the releases they make. Regrettably the life, and health, of a project cannot be easily categorized by a single metric. Multiple measures in combination make up a project's health. Before discussing the method used to rate project health, lets first cover the levels of health.

  • Level 0 - a project that exists or is just beginning. It is either a project with no releases or all releases are no more then Alpha quality.
  • Level 1 - a project that has a release at Beta quality. It is a project with a release that has been reviewed by at least one project leader since Beta quality level is the minimum.
  • Level 2 - a project with at least some of the ratings of health. In general, it should have roughly half. It is difficult to be precise since the exact number and type of these ratings has not yet been determined. As soon as those are solidified, the number for level 2 will be specified.
  • Level 3 - a project which has all the ratings of health. This level represents the most healthy state for an OWASP project.

Notes on Project Health Levels:

  1. The site will be reviewed based on the Project wiki page minimal contents below during any level change to ensure minimal project information is present on the project site. See below for details.
  2. Maintenance of the project wiki page can be handled by either the Project Lead or the Project Maintainer if the project has one.
  3. The ratings of health will be determined shortly. Some general information is below. Various logistic and practical aspects need to be determined. --Mtesauro 17:44, 4 May 2009 (UTC)


Project Health Ratings

The exact meta-data used to determine project health has not yet been fully determined. Some of the sources of data for the health rating are listed below. Some, all or none of these may be used in the final version. This list is to server as a public brainstorming exercise:

  • Number of releases
  • Size of the project's community
  • Industry participation
  • Usability
  • Number of participants
  • Number of "stars" - the idea here is to have a rating system similar to what Amazon uses for books, etc.
  • Creation and quality of training/tutorial material for the project. The Global Education Committee is a key driver of this potential metric.

It is likely and should be expected for this list to change between now and its final version. Note also that the exact method of gather the above data is not yet specified.

Project Wiki Page Minimal Content

The following questions will be answered by the project lead or project maintainer and be reviewed by the Global Projects Committee whenever a project changes health levels:

  • Does the project wiki page...
  1. have an up to date project template with current project information?
  2. have a conference style presentation that describes the tool/document in at least 3 slides?
  3. have a one sheet overview document about the project?
  4. have a link to a working mail list?
  5. have a statement of the application security issue the project addresses?
  6. have a project roadmap?
  7. project leaders and main contributors have a wiki account (with its user page containing contact details about the user and if possible his CV)

For OWASP project wiki pages, please see the Project Wiki Pages section of the Guidelines for OWASP Projects for additional suggestions/recommendations.

Example of the Process

As a example of this process, the story below demonstrates a project from inception to optimal health:

  • A security professional has an idea to address an issue in application security and proposes a new project to the Global Projects Committee (GPC).
  • The GPC agrees with the proposal, gathers some initial data from the security professional and creates a new project page. The project starts at health level 0.
  • The security professional, now the project lead, works on the project and creates a release which reaches Alpha quality.
  • The project lead continues to work on the project, it gets reviewed and reaches Beta quality. The project has reached level 1. The project is generating health metrics.
  • The project lead continues to work on the project release and reaches a Stable release.
  • Additional metrics are collected (the exact nature and method of collection is to be determined). After reaching approximately half of the project health ratings, the project moves to level 2. Note this could have happened before creating a stable release.
  • The project lead starts a new sub-project which is Alpha quality. The new release does not negatively impact the projects health. Likely, it will help it grow and become more healthy.
  • The project grows in activity and prominence to the point that it reaches all the project health ratings. The project is now a level 3 project.

Star Rating System

Not Reviewed Level 0 level 1 Level 2 Level 3
Yellow button.JPG Greenlight.png Greenlight.pngGreenlight.png Greenlight.pngGreenlight.pngGreenlight.png Greenlight.pngGreenlight.pngGreenlight.pngGreenlight.png


Archiving Project Sites

The exact criteria for archiving project sites has not yet been determined. However, the Global Projects Committee sees that an archive of projects that are kept for historical purposes will be needed. This page or subsequent pages will determine the situation under which project pages are archived.


Pre-existing OWASP projects

The Global Projects Committee realizes that there are many current project sites which pre-existed the above assessment of project health. Those project sites will be migrated into the new criteria in the near future. The exact timing and methodology for addressing existing sites has not yet been determined. The Global Projects Committee will first fully specify the new framework before working on mapping existing projects into the new framework.

Methodology Follow - Up

As part of the transition to the new project details template, this page is a follow up list for issues to check .