Reconsidering severity levels for security issues

Development
#1

Hello everyone,

some time ago we decided to rely on CVSS for severity of security issues. However, I have the impression that the resulting priority of issues doesn’t match user expectations. For example, a vulnerability that would give read-only access to all pages of a private wiki without user interaction is CVSS 7.5 which is just a critical severity according to the agreed upon mapping. For such issues, we say that we’ll handle them depending on other priorities and give no guarantee that we’ll fix them in a timely manner. That doesn’t make any sense from a user’s point of view.

I propose to change the security policy to state instead the following:

We mark security issues as blocker issues in Jira and will do our best to fix them within 90 days, unless

  • the attacker needs at least script right to exploit the issue
  • the impact is low (e.g., minor performance impact, data leak that doesn’t concern actual page contents)
  • the issue is hard to exploit (e.g., an admin needs to perform an action that seems unlikely)

In these cases, the security issue can be marked as “Critical” or “Major” in Jira.

The idea of the first two criteria is to match the CVSS criteria “Privileges Required” and the impact for “Confidentiality”/“Integrity”/“Availability”. The third point is less straightforward, and primarily targets issues that require some sort of social engineering. In general, I’ve also tried to take inspiration from the deprecated severity matrix.

I’m not sure if we should have further guidelines for choosing Critical vs. Major, I would leave this to the reporter and/or committer who handles the issue - as far as I know we also don’t have any criteria for other bugs.

Any opinions or other ideas?

#2

Sounds good, +1.

#3

Hi,

thanks for the proposal. I understand the need here, but I’m not a big fan of using something as vague as “the issue is hard to exploit” as it’s really subject to interpretation: the idea of using CVSS and of providing guidelines for filling it was to try as much as possible to have something clear.

But maybe we can try to rephrase your proposal using CVSS.

It’s Privilege Required are High in CVSS

Might be kept like that

Attack complexity is High ? We don’t have proper guidelines for defining it but at least we’d ensure to have a match between the CVSS and the jira priority.

wdyt?

#4

For the first two, yes, I intentionally formulated them to be basically the same as the CVSS scores, we can make this more obvious if you want.

Attack complexity high is defined as follows:

The successful attack depends on the evasion or circumvention of security-enhancing techniques in place that would otherwise hinder the attack. These include: Evasion of exploit mitigation techniques. The attacker must have additional methods available to bypass security measures in place. For example, circumvention of address space randomization (ASLR) or data execution prevention (DEP) must be performed for the attack to be successful. Obtaining target-specific secrets. The attacker must gather some target-specific secret before the attack can be successful. A secret is any piece of information that cannot be obtained through any amount of reconnaissance. To obtain the secret the attacker must perform additional attacks or break otherwise secure measures (e.g. knowledge of a secret key may be needed to break a crypto channel). This operation must be performed for each attacked target.

To me, this matches only a small part of what I consider “hard to exploit”. Also, by referring to attack complexity, you’re not making the definition any clearer. Ultimately, the severity of a security vulnerability is always up to interpretation (e.g., it is also up to interpretation at which point the impact is low or high). My goal with the last point is mostly to have a way to acknowledge a reported security vulnerability like XWIKI-20323 that requires unlikely user interaction without committing to fixing it within 90 days. This is for those kinds of security vulnerabilities that do not fall into a nice category like XSS, therefore I think it is simply not possible to provide clear guidelines here.

If you want to make it a bit clearer, maybe we could have two items:

  • the attack complexity is high (e.g., the successful attack depends on the evasion or circumvention of security-enhancing techniques, or the access to secrets)
  • the attack requires user interaction, and this user interaction seems unlikely to happen

The first point refers to the attack complexity as defined in CVSS while the second point is basically a second level of “requires user interaction” as defined in CVSS.

#5

Ok, just to be sure it would be a “or” between each item? Even between those? e.g. an issue which has high complexity but doesn’t require user interaction would be blocker?

#6

I’m not sure that I understand your interpretation of “or”. My intention is that a vulnerability is a blocker unless one of these conditions applies. So your example of an issue which has high complexity isn’t a blocker regardless of the other conditions. I would say that this is an “or” between each item, but this doesn’t match your example.

Thinking about it again, I would actually suggest making this more flexible in the sense that we say that if one of these conditions applies, we normally don’t consider it a blocker, leaving us the flexibility to still consider it a blocker in case it seems sensible to do so. For example, in the case of a regression, we would consider it a blocker regardless of the severity.

1 Like
#7

Yeah I made a mistake when writing it… But ok we’re on the same page.

+1 what we say here is mostly indicative and should work in most cases: we can rediscuss severity at any point in the process if we think there’s a need for a specific vulnerability.

#8

I’m not sure I understand the consequences of this proposal. Does it mean we won’t be able to compute a CVSS score anymore? Or that we’re changing the rules for computing the CVSS score and that we won’t be able to use a standard calculator such as NVD - CVSS v3 Calculator ?

Basically are you proposing to completely change the whole section named “Severity” at https://dev.xwiki.org/xwiki/bin/view/Community/SecurityPolicy/#HSeverity ?

Also, I think In these cases, the security issue can be marked as “Critical” or “Major” in Jira. is not good enough. We had something very clear and saying that it’s up to the committer to decide if it’s critical or major (what about minor?) is a bit too vague for me.

What we had looks much better to me:

0.1 - 3.9: Minor
4.0 - 6.9: Major
7.0 - 8.9: Critical
9.0 - 10.0: Blocker

I feel it’s maybe ok to revise some criteria and change some computation (but we need to provide a new calculator).

Also, isn’t there already some way with CVSS to have some latitude when computing scores? Isn’t there some parameter for subjective part already? Couldn’t we say we use CVSS and make an exception instead, allowing us to continue using a standard CVSS calculator and then adjust the result based on some well-defined criteria proper to XWiki? (ie compute a score in 2 passes).

Thanks