QA Manager special project role

Development
1

Hi devs,

I’d like to propose that we formally add a new special project role to our Development Practices:
the QA Manager role.

Motivation

I’ve been doing this role informally since the very beginning of the project but it’s never been written down as an official role. I think it’s worth making it explicit, for a few reasons:

  • It makes our quality practices visible and documented, like our other roles.
  • It clarifies that quality is owned by someone and is a first-class concern, not just something that happens as a side effect.
  • It makes the role transferable: whoever holds it is written down, and anyone who wants to take it over can do so.

To be clear about the scope and avoid confusion with the existing
Build Manager role:

  • The Build Manager role is tactical — a weekly rotation whose priority is to get the CI build back to green ASAP whenever it fails.
  • The QA Manager role is strategic — it’s about the long-term quality of the codebase and of our testing/quality practices (test strategy, coverage goals, quality tooling, moving from manual to automated testing, etc.).

The two are complementary: the QA Manager defines and drives the quality practices; the Build Manager rotation keeps the build healthy day to day within those practices.

Proposed role definition

I propose adding the following to the Special Project Roles page:

QA Manager Role

Role definition:

  • Globally responsible for the quality of the XWiki code base and for the quality of our testing and quality practices.
  • In charge and responsible for the Testing strategy and its associated dev documentation (test kinds, naming conventions, testing frameworks — unit/Surefire, integration/Failsafe, functional/Docker-based).
  • Proposes rules and best practices to improve the quality of the code, and ensures they’re documented on dev.xwiki.org.
  • Drives the effort to move from manual testing to automated testing (e.g. converting old functional tests to Docker-based tests, adding missing tests).
  • Owns the strategy for improving test coverage over time (e.g. the per-module JaCoCo instruction-ratio ratchet) and pushes coverage upward.
  • Leads the fight against flickering functional tests: maintaining the process to detect, track, quarantine and fix them.
  • Responsible for the automated quality checks run in the build and on CI (Checkstyle, Revapi, Enforcer, the quality profile) — proposing new checks and keeping the existing ones meaningful and green.
  • Monitors code quality tooling such as SonarCloud/SonarQube and drives the reduction of reported issues.
  • Regularly proposes quality- and testing-related topics for the Roadmap and gets time allocated to work on them.
  • Responsible for the manual QA testing strategy too.

Process:

  • When the Roadmap process starts, discuss taking some quality-/testing-related topics.
  • Review the state of the code quality, test coverage and flickering tests regularly, and raise issues/proposals to the community when practices need to evolve.

Current manager:

I’m happy to keep doing this role, and equally happy to hand it over to anyone who’s interested — now or in the future.

PS: As with the other special roles, it’s not because we have a person assigned that the other devs shouldn’t work on the topics. The “manager” is only there to ensure there’s at least someone thinking about the topic and making it progress.

WDYT?

Thanks,
Vincent

2

lgtm, thanks!
+1

3

+1

4

+1

Thanks,
Marius