Clean docker-compose with 14.10.1: “Unknown macro: notifications.” in profile page

Help / Discuss
#3

We need to improve the documentation I think. At xwiki-docker/README.md at master · xwiki/xwiki-docker · GitHub it says:

  • Then run a container for the database and make sure it’s configured to use an UTF8 encoding.

  • Then the examples for MySQL and MariaDB use utf8mb4 but don’t explcitly mentioned it’s needed
  • The example for PostgreSQL uses utf8

The doc at MySQL Installation (XWiki.org) also doesn’t mandate utf8mb4.

My understanding is that XWiki should work fine with both utf8 and utf8mb4 (but if you use utf8 then you won’t have emojis).

So maybe we have broken support for utf8 by mistake?

@tmortagne any idea?

Thanks

#4

Hey vmassol,

Looking at the bug I don’t entirely think your understanding of ‘Xwiki working fine with utf8’ is true. Yes, for a plain installation without default skin it works (i’ve tested that as well) but as I can’t install one of the default extensions, the ‘My Activity Stream’, without having utf8mb4 I would say Xwiki doesn’t work fine without utf8mb4. If your understanding was true it should install without problems but, like you said, not support emojis which would be perfectly fine for me. (I’m sorry vmassol! I’ve read over your last sentence! Keeping this paragraph in here just as a reminder for myself)

The problem is that not having utf8mb4 (for mysql at least) breaks the default skin as the “My Activity Stream” just doesn’t install with a clean (docker) install and also post-installing the plugin just doesn’t work as it installs about 7 translations of the notifications macro but not the notifications macro itself, which it probably also does when installing the default skin!

In my eyes the whole installation of the default skin should fail when it has db-query errors, ESPECIALLY if it’s trying to insert or change something in there! It doesn’t do that at the moment, it installs the skin, throws a few small exceptions inline but after says it’s A-OK, which it clearly isn’t.

The same goes with post-installing the notifications plugin (or any plugin for that matter) as soon as the minimum requirement can’t be installed (i.e. XWiki.Notifications.Code.Macro.NotificationsMacro) it shouldn’t install the translations for it and it should actually just delete the installed translations for it and throw up an error about WHY it failed (which it doesn’t say in the frontend when installing extensions)

As I’m having troubles with xwiki on another system (CPU load suddenly surges to 100% and keeps it there, which makes xwiki get stackoverflows/out of memory errors), I’m now afraid it maybe has something to do with some upgrade I did where I didn’t see the inline error which made this mess.

To be a littlebit more specific about this:
I upgraded a 13.x.x (which worked perfeclty on utf8) to 14.10.0 which looked like it worked flawlessly it upgraded all the plugins… even the Notifications plugin, I thought, but it looks like it didn’t upgrade it because of my utf8 DB so it now, probably, uses some werid mixture of 13.x.x Notifications pages mixed in with 14.10.0 translations on it. It does seem to work, but it makes me think if more plugins have this problem at upgrading what kind of issues could arrise from it (like surging CPU loads and such as there is some compatibility issues, race conditions or unlimited threads spawning because of mismatches).

#5

That’s the current theory, yes, but looks like someone recently added to one of the XWiki Standard pages a character which is not supported by utf8mb3. We need to see what this character is exactly and if we can/want to fix that or not. If we drop support for utf8mb3 then we need to introduce an automatic migration from utf8mb3 to utf8mb4.

#6

Hey tmortagne,

I’ve seen the error come by multiple times when doing a fresh install on utf8mb3, so I’m not sure if it’s just one page.

#7

My point is that it’s very recent and very few, since there are plenty of instances which are still using utf8mb3.

Anyway, the first thing to do is identifying them and either fix them if that’s possible (and IMO we really should for the LTS branch). Would be great if you could create a BUG issue on Loading... with the info you had gathered so far.

#8

My point is that it’s very recent and very few, since there are plenty of instances which are still using utf8mb3.

Agreed!

I don’t have any of the info I’ve gathered except for what’s up here. But I’ll retrace the steps and make a bug report with that info. It’ll take a bit longer though, my apologies.

#9

No problem. Thanks a lot for your efforts !

#10

I’ve made a bug issue: Loading...

Marking this thread as solved now.

#11

Thanks, I was expecting only a few errors but the log you provided is actually full of those and given the name of the documents involved I think I know what is the cause: it seems to be related to the recent huge Japanese contribution.

I was really not expecting utf8mb3 to be so bad that it did not even support (some ?) Japanese characters…

#12

tbf I wouldn’t mind to leave most of the translations out of the package (if at all possible, mabe a feature request is in order for that?).

What stings the most, in my eyes, with this is that if you’re not paying attention when the xwiki tries to install the default skin you won’t notice the problems as it doesn’t mention anywhere that something has failed (except for the logs, which can be scrolled upwards and not noticable). It’ll even tell you everything is installed and you’re ok to move on… which just gives the wrong signals.

#13

… haha! I told you from the start:

:grinning_face_with_smiling_eyes:

#14

Well, this is true for Japanese translations if you don’t need them. Macros pages don’t contain translations, so there is no reason to have missing macros. There are different problems here.

#15

I was just stating as a user the installation sends out conflicting messages:
image
“Oh look, XWiki installed the default flavor just fine, no errors, it even has ‘installed’ in green”
image
^scroll, scroll, scroll^ “Oh maybe it did have errors, why the f didn’t it tell me it had errors”

I’d be more happy with something like this:
image
Where it shows it had errors installing and maybe even make a new tab with errors where it can show ONLY the errors from the install log and if you want to continue with those errors it just ask ‘are you sure?’

The fun thing is, you could even make it work with “only translation errors, so it’s ok to continue”:
image
and you could click ‘continue’ without having a second question.

This is just more about UX than anything…

#16

Sure, I agree. Now it might not always be easy to differentiate between the types of errors, especially when it’s the exact same cause, like here since right now at this level it’s all just wiki pages. What is easy to do at least is indicating that there was errors in the log (without any information on how blocker they are). Note that the best would probably be to create a new IMPROVEMENT (or IDEA) jira issue with your design ideas to make it more visible.

#17

Agreed! I was looking at it from a user perspective, not from a programmings point.
I’ll make a few new tickets!

#18

Thanks !

#19

Hmm, strangely, I don’t seem to reproduce with MySQL server 8.0.31.

I tried to import https://maven.xwiki.org/releases/org/xwiki/platform/xwiki-platform-notifications-macro-ui/14.10.1/xwiki-platform-notifications-macro-ui-14.10.1.xar in a utf8mb3_bin MySQL database and it went well.

What is strange to me in your first message is the fact that apparently 3 translations could not be imported in your case and not just Japanese, as if it was failing to save any utf8 characters and not just 4 bytes ones.

Are you sure you were not using something like latin1 ? You don’t seem to give many details about that actually, so I just assumed it was utf8 (so utf8mb3)

#20

That’s… highly interesting…

I went back and checked and you’re entirely right! I never switched it to utf8mb3, I immediately switched it to utf8mb4 which obviously worked!
image
I assumed the default was utf8 which it obviously isn’t.

I’m such a dumbass, I’ll be over there in the corner now… drowning in my own tears…

#21

OK thanks for the confirmation, I’m going to close Loading... and Loading... as invalid, but we’ll keep Loading... as it’s still theoretically possible from what I could see (even if those characters seems to be rare enough to not appear in our current translation contributions).

#22

maybe an improvement ticket that checks if you’ve set the encoding correct on setup? …for dumbasses like me? Funny thing is; we have been using the latin encoding for 2+ years with xwiki without any (known) issues… that’s also quite a feature I guess, haha!