in the discussion about severity levels for security issues we came across the fact that we don’t treat all blocker issues equally. Originally, the idea was that you cannot release when there are open blocker issues (for this reason, there is an explicit check in the release plan if there are blocker issues).
With this vote, I propose to restore the original meaning of the “Blocker” priority and to use the “Critical” severity for important, non-blocking issues. To be more precise, I propose the following new meaning:
Blocker issues must be closed before releasing a version that is affected by the blocker.
Critical issues should be handled as soon as possible, if they haven’t been fixed before the next roadmap, they must be fixed as part of it.
As part of implementing this vote, all existing critical issues need to be reviewed if they are that important, otherwise, they’ll be changed to “major”.
I’ll open separate votes to change how we classify security issues and regressions to reduce the number of blocker issues to ensure that this new procedure can actually be implemented.
Regardless of that, the idea is that we can use the “Critical” priority to mark issues that are so important that they should be fixed in the next roadmap the latest. Just to make this clear, this is not a “wishlist” feature, setting a critical priority should be an exception and definitely not a way for users to prioritize what they would like to see implemented/fixed. Instead of fixing an issue, we can of course always decide to decrease the priority of an issue if it isn’t important enough.
This vote is open until June 16, 12:00. Thank you very much for your votes!
-1 because of the terminology and the consequences. I feel it’s more important to continue doing time-boxing than to delay the release because of blocker issues. For one, because the current version in progress can contain blocker issues that have been fixed already (or very important issues) and they will be delayed for the users because one or several other Blocker issues are blocking the release. In other words, I don’t want a Blocker to block the release until it’s fixed and this lasts, say, 15 days.
I’d be +1 to say that we need to prioritize Blocker issues for the current release as much as possible by assigning them to developers right away (ie there should be no blocker issues without a dev assigned). I’d also be +1 to allow delaying the release by 1 to 2 days at max for giving the time to fix a blocker issue. I’d also agree to, exceptionally allow delaying to 3 days when we’re releasing a RC by doing a proposal/vote and in this case, we’d skip the RC release and go straight to the final release.
More generally, I’m +1:
to say that we must try to fix Blocker issues in the current version being prepared
to say that we must try to fix Critical issues in the coming releases.
I’m -0 to -1 to only allow a max of 2 releases for fixing Critical issues. I’d prefer 3 releases (ie 3 months). That would also align us with the “Critical” security issues which have 3 months to be fixed. One rationale is that it’s hard to know in advance when we’ll have time to work on Critical issues as plans change often and 3 months would give us a bit more margin, and I don’t feel that it’s too much.
Some questions remain:
Should a Critical issue be relabelled as Blocker when only 1 release remains for it?
Consequently what is the rule re assigning devs to critical issues. IMO we should do a best effort to assign them a dev ASAP (even if the issue is only going to be fixed in 3 months), and apply the Blocker rule when only 1 month remains (ie always set a developer when preparing the roadmap).
My calculation of 90 days on security issues is based on this proposal, my idea wasn’t to treat critical security issues differently. The 90 days consist of:
Up to 30 days for roadmap 1 in which we wouldn’t need to work on the issue yet.
30 days for the next roadmap 2 in which we would need to develop a fix.
Up to 30 days for releasing all branches as at the end of a roadmap, we don’t release all branches, only master. I know that the idea is to have LTS releases as frequently as every 14 days but in reality I think it’s more like every 30 days.
This calculation is based on worst-case assumptions, but the idea is really to have a maximum of 90 days in the worst case as reporters usually threaten to release details of the security vulnerability if no fix is available within 90 days. If we allowed 90 days for Critical issues, it would mean that we might need 120 days for releasing security fixes which would be too long.
My understanding was (but I didn’t search carefully) that the current definition of Blocker is that we cannot release when there is a blocker. At least my intent wasn’t to change the definition of a blocker issue with this vote, my assumption was that this is basically the current definition of a blocker issue, sorry for not making this clear. Could you point me to our current definition of a blocker issue?
I tried to keep the rules for “Critical” issues as simple as possible without the need to do any complex calculations based on when the issue was opened. The only thing we would need to do with my proposed definition is to put them on the roadmap whenever we prepare one. Unless we decide to mark more issues as “Critical”, we would only mark issues as “Critical” that were blockers so far. Thus, I believe it is justified to put them on the next roadmap and not delay them further. To me, a three-month period would seem harder to implement and keep track of.
Of course, as it is now, there can be exceptions, we can still decide to release with open blocker issues as we do now (the idea would mainly be to make it a real exception and not the norm), and while a committer who is assigned to an issue on a roadmap is supposed to fix that issue during that time period, this might not actually happen.
Until now the time-boxing aspect has always taken precedence over a Blocker issue, except when we can fix that issue by delaying only by 1-2 days (we accept a small delay to time-boxing), and exceptionally by 3 days and in this case, we usually send a proposal to skip the RC (or agree on the xwiki chat). Example: Skiping 11.10RC1 release).
Ok thanks for the precision. There are still consequences so let’s be more clear Hence my proposals to clarify.
I don’t think we’ve written one. I’m sure it was discussed on the chat + on the old mailing list but I haven’t found a place on xwiki.org.
That’s a definition of a Blocker, not of a Critical
We define the next roadmap only at the end of the month (when the RC is released). We can use “Critical” to indicate “Candidate for a Blocker in the next roadmap”. I still feel that there are critical issues for which we will need more than the next release to fix because of lack of time. And marking them as major would loose the information that we need to work on them ASAP.
I have the impression we want the same thing, we just don’t agree on the wording. Let me try again formulating this with other words:
A blocker issue is an issue that is so important that it should ideally be addressed before the next release, but there can be cases where we release nevertheless, in particular if the issue is discovered directly before the release, and it’s not, e.g., the final release after an important regression was introduced in the RC. Blocker issues should be issues that are so important that we delay some items of the current roadmap to handle them.
A critical issue is an issue on which we should work ASAP, but that can also wait for the next roadmap. It should be planned with priority for the next roadmap if we don’t find the time to handle it immediately, but of course there can be cases where it just doesn’t fit, and we might decide that it can also wait another month.
In practice, the main change I expect is that when preparing a roadmap we’ll look at all Critical issues and include as many of them as possible in the roadmap. Of course, as I’ve mentioned above, we need to clean up the issues that are currently marked as Critical first before we can realistically start this.
I hope that this clarifies a bit how I meant this. If not, I suggest having a meeting to clarify the formulation as I’m not sure that we’ll reach a conclusion here.
When we decide that we can’t include a Critical issue in the roadmap (because there’s nobody available to work on it or it’s less important than other issues), then I guess you’re saying that we should move it to be a Major issue. IMO it should continue to be a Critical issue but at some point, if it never makes it to the roadmap we should downgrade it to Major (I would do this only if we fail to plan it in a roadmap for 3 consecutive roadmaps).
When we plan a Critical issue for the roadmap, does it become a Blocker issue (ie we change the priority)? Or is it possible to keep it being Critical and thus be moved to the next roadmap if we can’t fix it in time?
My idea wasn’t that there should be any priority change when planning or not planning a Critical issue (we also don’t change the priority of any issues we plan or not plan currently afaik). For issues that don’t fit the roadmap, I wouldn’t define any specific rules but decide on a case-by-case basis as not every “Critical” issue is the same. We could decide that it was a mistake to mark the issue as “Critical” and mark it as “Major”, or we could keep it as “Critical” and just move it to the next roadmap.
At least my idea is that at least for now, the only “Critical” issues are issues that we classify as “Blocker” issues according to old rules. The suggestion to mark them as “Critical” is to make clear that they shouldn’t block a release. Therefore, I expect that “Critical” issues are still very important issues and that we can agree on handling them with priority over other roadmap work. Thus, I hope that not including a “Critical” issue in the roadmap and not completing a “Critical” issue will be an exception and not something normal for which we need to define rules.
Yes, I think we want the same, or almost the same, it’s primarily the wording. My main point regarding what you said above was that it sounded to me as if you were saying that it wouldn’t be a priority to assign a Critical issue to a developer ASAP but only after one or two months. And that’s what I see differently, in my opinion we should make it a priority to assign a developer for the very next roadmap, and if we have the capacity, handle it even in the current roadmap. Not planning/completing a Critical issue should be the exception, but still, I wouldn’t delay a release because of a critical issue.
Basically, we check all Critical issues when planning a roadmap, and it should be an exception that we don’t plan all of them. If we should notice that we frequently can’t plan all Critical issues, we should re-evaluate why we marked those issues as Critical.
I think that’s the part I’m not comfortable with. I’d like us to have a list of priority issues we want to tackle ASAP (but it’s far from sure that we’ll tackle them in 1, 2, 3 or even 6 upcoming releases). Moving those to “Major” doesn’t make sense to me as they’ll be mixed with everything else and we’ll forget that they are more important than others.
I didn’t intend to suggest moving any issues to “Major” that cannot be planned. Issues that cannot be planned will just stay as “Critical” as long as we agree that they should be planned ASAP.
Our current definitions of the different priority levels are in Jira:
Blocker: Blocks development and/or testing work, production could not run. Critical: Crashes, loss of data, severe memory leak. Major: Major loss of function. Minor: Minor loss of function, or other problem where easy workaround is present. Trivial: Cosmetic problem like misspelt words or misaligned text.
Based on that, I would suggest the following new definitions for Blocker, Critical and Major (the others could stay the same):
Blocker: Blocks development and/or testing work, recent regression, or very important security vulnerability that needs to be fixed urgently (e.g., because it is actively exploited). Blocker issues are fixed as soon as possible and may take priority over issues that have been planned in the current roadmap. An open blocker issue can delay releases when releasing without a fix wouldn’t be reasonable (e.g., when an important regression is introduced in a release candidate, the final release could be delayed until a fix is available). Critical: Crashes, loss of data, (older) regression, important security vulnerability (see the XWiki security policy for details). Critical issues are considered with priority when planning a roadmap. Major: Major loss of function, security vulnerability with low impact (see the XWiki security policy for details). Default priority, also for new features, improvements and tasks.
I’ve tried to keep the current definitions where they apply to XWiki and added how we use Blocker, Critical and Major issues for security issues and regressions. For Blocker and Critical issues, I’ve also tried to formulate how we consider them in/instead of the roadmap.
Do you have any feedback or suggestions how these formulations could be improved?
