I found that our current documentation is ambiguous for several reasons:
reporters of security issue do knows about CVSS while our current matrix is really internal
our matrix is currently very ambiguous for plenty of cases (e.g. how do you assess severity of a RCE right now?)
we do use CVSS when creating the advisories now
So my proposal is the following:
Always compute CVSS for new security ticket and indicate the CVSS computation in a dedicated field in the jira ticket
Provide an indicative mapping between the CVSS score and the Jira priority: e.g. Minor: 0.1-3.9, Major: 4.0-6.9, Critical: 7.0-8.9, Blocker: 9.0-10.0
Indicate that this mapping can be modulated on case by case basis, taking into account at least the rights scheme of XWiki since CVSS only has 3 values for “Privilege required”
Stop using attacker_* and attack_* labels
Add a dedicated jira field for the CWE classification, that could replace the attack_* semantic labels
+1 for relying more on CVSS and replacing our current matrix, with one exception:
I think we should still use the attacker_* labels as CVSS provides no information if a vulnerability can be exploited with comment, view or account right for example which might make a big difference in practice when deciding if a vulnerability affects an instance.
Further, we should decide if we want to map attacks requiring view right to “Privileges Required (PR)” “none” or “low”, considering the possibility of a private wiki. Attacks that are possible through XWiki syntax/vulnerable macros can also be executed using the HTML Converter with just view rights and there this could thus make a difference.
Also in general, I think we should provide some guidance how to select CVSS categories for the specific case of XWiki to make sure we consistently classify issues and also provide examples how to classify commonly-occurring issues like XSS.
Regarding CWE classifications, while this sounds nice, we should make sure that we always use the same format (just the identifier, or also the title?). It would be super nice if we had an integration in Jira where we could just search and get the correct values suggested similar to how this is implemented in GitHub for the advisories but I guess that’s too much work?
+1 too I agree with Michael’s comments, see below.
I agree that it would be nice to have security information in XWiki language (ie based on xwiki rights).
Definitely. I know that would help me a lot as a dev when reporting security issues.
The proposal is missing details regarding existing issues. I guess the idea is to keep them as is (and thus to keep the existing labels and since we’d add 2 custom fields, they’d have these fields empty)?
Thanks for moving us forward towards standardization!
So my proposal here was just to not enforce its usage: in some cases it’s not easy to decide which attacker it is (e.g. if the attack requires an admin to click on a link, is it still attacker_admin?). Now I agree that we should encourage its usage when it makes sense and it’s not ambiguous.
Indeed. We probably should have another proposal just for assessing that.
Yes I was thinking about providing a dedicated CVSS documentation page in xwiki.org with:
a CVSS calculator (I guess we can find some snippets to include to not have to reimplement it ourselves)
explanations over each category / choices and examples in XWiki world
On the long term I’d also like very much the integration. But since we’re not even sure how long we’ll keep Jira I wouldn’t put energy in that right now. My proposal was just about using the CWE ID.
So for existing security issues, I propose to systematically add the CVSS information / CWE information for the ones already closed and for which we created advisories (so basically copy/pasting what’s in the advisory). Then for the labels I wouldn’t change anything.