as part of my BFD yesterday I found a few tickets related to email notifications and bug with them on Outlock. It appeared that those bugs couldn’t be reproduced anymore with more recent versions when @ilie.andriuta tested them so they’ve been closed.
Now we still have from time to time tickets related to emails, and to my knowledge we don’t have a clear support strategy for email clients. So maybe we should clarify this situation: for example, personally I don’t have any Microsoft account and I cannot test with Outlook: I always test mail features with thunderbird on my computer.
I honestly which direction we want to take there, but it might be interesting to document a bit more what we’re actually testing.
The mail tests we have are using Greenmail and the tests will just assert the content of the mail returned by the mail client. It would be hard to test that the mail client renders it correctly. We would need some kind of image regression testing for this.
Definitely, but I think what @surli is after is more what do we do with bugs related to how a mail renders in a specific mail clients. I don’t think it’s realistic to have the kind of testing that we have for supported browsers, as you mentioned, but there are still things we can say:
we just commit to fix reported bugs for a given list of clients (to be decided if we have some regular manual tests on those, I think @ilie.andriuta use Gmail in some tests)
we don’t commit to fix reported bugs for any client
As for what about the clients for which we don’t commit to fix bugs:
a) close as won’t fix issue related to any other mail client
b) keep open issues related to any other mail client and people can provide pull requests for those
Either ways, it’s IMO good to have a page stating it along with the other support pages.
I feel like we should at least commit to fix bugs on a few clients with a very high marked share but which are easy for any of the dev to test on (works with Gmail but maybe not so much with Outlook).
I’m really not a fan of a) (which, from what I understand, is more or less what @surli was hoping for when he started this thread. To me, there is no reason to close them, they are just clearly not a priority.
That we currently don’t have tests for checking how mails are displayed in various clients and that manual tests are done from time to time in an ad-hoc manner
Not sure what “Apple” means, whether it’s iOS mail client or mac’s mail client, or both maybe.
As you say the hard part is testing on the various platforms, but amongst the xwiki dev team members we have access to mac, windows and linux so it should be doable.
I don’t think we can say we won’t commit to fix issues for apple mail when it’s the #1 mail client used. Same for outlook.
For other mail clients, if we don’t close the issues, then we need at least a way to know which are issues that the xwiki core dev team won’t work on so that we can filter them out when we need (it could also be explained that a sponsoring could be done to work on those). And it would be interesting to review our issue list and clean it up to mark issues we don’t plan to work on.
