About that topic, the proposal you made here is a general vision with long term goal, but I think there’s still some short term things to do that could deeply improve the way we work:
- Be able to detect automatically in the CI the flicker tests that are recorded on Jira: we lost this ability with docker tests, so it now takes us a huge amount of time to check if tests are flickers or not. Time we could spend fixing them.
- Be able to run a specific CI configuration for docker: we have several configurations that we run on the CI, but each time we want to reproduce a specific one we have to check again all the parameters and set them properly one by one. We should have a generic parameter that we could use for that.
- Separate docker builds and standard builds from the CI, we already talked a lot about that, lately we found a technical solution on a suggestion from @tmortagne AFAIR about using git modules, so I think we should now implement it.
Those proposal are not about fixing the flicker indeed, but helping working with them and having more time to fix them.