Unable to edit a page - "Failed to load the page."

Help / Discuss
,
#1

Hello community,

In my company’s xWiki 12.10 instance we’re facing issues with the modification of a single page. When trying to edit it, the following appears at the center bottom of the screen:

image

However, deleting or removing the page works flawlessly. The page’s name is:

Firewall v2 - Excessive Firewall/ACL Connections Accepted From Single Internal Host

I am aware that Tomcat, the server that we are using, has some issues with slashes and backslashes, but hear me out: The configuration for that has already been mitigated. When running ps -aux | grep tomcat on the server, I get the following:

tomcat   25771  6.9 43.7 4951696 1698024 ?     Sl   17:09   5:31 /usr/lib/jvm/jre/bin/java -Djava.util.logging.config.file=/opt/tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.awt.headless=true -Djava.security.egd=file:/dev/./urandom -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Addresses=true -Djdk.tls.ephemeralDHKeySize=2048 -Djava.protocol.handler.pkgs=org.apache.catalina.webresources -Dorg.apache.catalina.security.SecurityListener.UMASK=0027 -Xms1024M -Xmx2048M -server -XX:+UseParallelGC -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true -Dignore.endorsed.dirs= -classpath /opt/tomcat/bin/bootstrap.jar:/opt/tomcat/bin/tomcat-juli.jar -Dcatalina.base=/opt/tomcat -Dcatalina.home=/opt/tomcat -Djava.io.tmpdir=/opt/tomcat/temp org.apache.catalina.startup.Bootstrap start

Specifically:

 -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true

Furthermore, we also have the following page on our instance (note the difference in the name) which can be edited without any issues:

Firewall v2 - Excessive Firewall/ACL Connections Accepted From Single Host

When previewing the console log when clicking “Edit” on the problematic page, it brings up the following:

jquery.js:9175 GET https://our-domain/xwiki/rest/wikis/xwiki/spaces/Dashboard/spaces/%D0%A2%D0%B5%D1%85%D0%BD%D0%B8%D1%87%D0%B5%D1%81%D0%BA%D0%B8%20%D0%BE%D1%82%D0%B4%D0%B5%D0%BB/spaces/%D0%A6%D0%B5%D0%BD%D1%82%D1%8A%D1%80%20%D0%B7%D0%B0%20%D0%BD%D0%B0%D0%B1%D0%BB%D1%8E%D0%B4%D0%B5%D0%BD%D0%B8%D0%B5%20%D0%B8%20%D0%BE%D0%BF%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D0%B8%20-%20SOC/spaces/Technologies/spaces/Demisto%20XSOAR/spaces/%D0%9E%D1%81%D0%BD%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5%20%D0%B7%D0%B0%20%D1%80%D0%B0%D0%B7%D1%81%D0%BB%D0%B5%D0%B4%D0%B2%D0%B0%D0%BD%D0%B5/spaces/Firewall%20v2%20-%20Excessive%20Firewall%2FACL%20Connections%20Accepted%20From%20Single%20Internal%20Host/pages/WebHome?timestamp=1637598790282 500

Monitoring catalina.out only produces the following two log entries, but this happens for each and every single page in xWiki and we don’t have any issues editing the others:

2021-11-22 18:34:55,175 [https-jsse-nio-9443-exec-7 - https://our-domain/xwiki/bin/get/TourCode/TourJson?xpage=plain&outputSyntax=plain&tourDoc=Dashboard.%D0%A2%D0%B5%D1%85%D0%BD%D0%B8%D1%87%D0%B5%D1%81%D0%BA%D0%B8+%D0%BE%D1%82%D0%B4%D0%B5%D0%BB.%D0%A6%D0%B5%D0%BD%D1%82%D1%8A%D1%80+%D0%B7%D0%B0+%D0%BD%D0%B0%D0%B1%D0%BB%D1%8E%D0%B4%D0%B5%D0%BD%D0%B8%D0%B5+%D0%B8+%D0%BE%D0%BF%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D0%B8+-+SOC.Technologies.Demisto+XSOAR.%D0%9E%D1%81%D0%BD%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5+%D0%B7%D0%B0+%D1%80%D0%B0%D0%B7%D1%81%D0%BB%D0%B5%D0%B4%D0%B2%D0%B0%D0%BD%D0%B5.Firewall+v2+-+Excessive+Firewall%2FACL+Connections+Accepted+From+Single+Internal+Host.WebHome] WARNo.h.e.j.s.SqlExceptionHelper - SQL Error: 1267, SQLState: HY000
2021-11-22 18:34:55,175 [https-jsse-nio-9443-exec-7 - https://our-domain/xwiki/bin/get/TourCode/TourJson?xpage=plain&outputSyntax=plain&tourDoc=Dashboard.%D0%A2%D0%B5%D1%85%D0%BD%D0%B8%D1%87%D0%B5%D1%81%D0%BA%D0%B8+%D0%BE%D1%82%D0%B4%D0%B5%D0%BB.%D0%A6%D0%B5%D0%BD%D1%82%D1%8A%D1%80+%D0%B7%D0%B0+%D0%BD%D0%B0%D0%B1%D0%BB%D1%8E%D0%B4%D0%B5%D0%BD%D0%B8%D0%B5+%D0%B8+%D0%BE%D0%BF%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D0%B8+-+SOC.Technologies.Demisto+XSOAR.%D0%9E%D1%81%D0%BD%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5+%D0%B7%D0%B0+%D1%80%D0%B0%D0%B7%D1%81%D0%BB%D0%B5%D0%B4%D0%B2%D0%B0%D0%BD%D0%B5.Firewall+v2+-+Excessive+Firewall%2FACL+Connections+Accepted+From+Single+Internal+Host.WebHome] ERROR o.h.e.j.s.SqlExceptionHelper - (conn=719) Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8mb4_general_ci,COERCIBLE) for operation '='

The following comes from the Tomcat access log:

my-ip - - [22/Nov/2021:18:38:11 +0200] "GET /xwiki/bin/get/Dashboard/%D0%A2%D0%B5%D1%85%D0%BD%D0%B8%D1%87%D0%B5%D1%81%D0%BA%D0%B8%20%D0%BE%D1%82%D0%B4%D0%B5%D0%BB/%D0%A6%D0%B5%D0%BD%D1%82%D1%8A%D1%80%20%D0%B7%D0%B0%20%D0%BD%D0%B0%D0%B1%D0%BB%D1%8E%D0%B4%D0%B5%D0%BD%D0%B8%D0%B5%20%D0%B8%20%D0%BE%D0%BF%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D0%B8%20-%20SOC/Technologies/Demisto%20XSOAR/%D0%9E%D1%81%D0%BD%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5%20%D0%B7%D0%B0%20%D1%80%D0%B0%D0%B7%D1%81%D0%BB%D0%B5%D0%B4%D0%B2%D0%B0%D0%BD%D0%B5/Firewall%20v2%20-%20Excessive%20Firewall%2FACL%20Connections%20Accepted%20From%20Single%20Internal%20Host/?sheet=XWiki.InplaceEditing&action=lock&lockAction=edit&language=en&outputSyntax=plain&timestamp=1637599092315 HTTP/1.1" 302 -
my-ip - - [22/Nov/2021:18:38:11 +0200] "GET /xwiki/bin/lock/Dashboard/%D0%A2%D0%B5%D1%85%D0%BD%D0%B8%D1%87%D0%B5%D1%81%D0%BA%D0%B8%20%D0%BE%D1%82%D0%B4%D0%B5%D0%BB/%D0%A6%D0%B5%D0%BD%D1%82%D1%8A%D1%80%20%D0%B7%D0%B0%20%D0%BD%D0%B0%D0%B1%D0%BB%D1%8E%D0%B4%D0%B5%D0%BD%D0%B8%D0%B5%20%D0%B8%20%D0%BE%D0%BF%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D0%B8%20-%20SOC/Technologies/Demisto%20XSOAR/%D0%9E%D1%81%D0%BD%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5%20%D0%B7%D0%B0%20%D1%80%D0%B0%D0%B7%D1%81%D0%BB%D0%B5%D0%B4%D0%B2%D0%B0%D0%BD%D0%B5/Firewall%20v2%20-%20Excessive%20Firewall%2FACL%20Connections%20Accepted%20From%20Single%20Internal%20Host/WebHome?ajax=1&action=edit&language=en HTTP/1.1" 204 -
my-ip - - [22/Nov/2021:18:38:11 +0200] "GET /xwiki/rest/wikis/xwiki/spaces/Dashboard/spaces/%D0%A2%D0%B5%D1%85%D0%BD%D0%B8%D1%87%D0%B5%D1%81%D0%BA%D0%B8%20%D0%BE%D1%82%D0%B4%D0%B5%D0%BB/spaces/%D0%A6%D0%B5%D0%BD%D1%82%D1%8A%D1%80%20%D0%B7%D0%B0%20%D0%BD%D0%B0%D0%B1%D0%BB%D1%8E%D0%B4%D0%B5%D0%BD%D0%B8%D0%B5%20%D0%B8%20%D0%BE%D0%BF%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D0%B8%20-%20SOC/spaces/Technologies/spaces/Demisto%20XSOAR/spaces/%D0%9E%D1%81%D0%BD%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5%20%D0%B7%D0%B0%20%D1%80%D0%B0%D0%B7%D1%81%D0%BB%D0%B5%D0%B4%D0%B2%D0%B0%D0%BD%D0%B5/spaces/Firewall%20v2%20-%20Excessive%20Firewall%2FACL%20Connections%20Accepted%20From%20Single%20Internal%20Host/pages/WebHome?timestamp=1637599092715 HTTP/1.1" 500 45

For comparison, trying to modify produces the following 4 entries to the access log (+ a dozen more for fonts, stylesheets, etc.):

my-ip - - [22/Nov/2021:18:43:55 +0200] "GET /xwiki/bin/get/Dashboard/%D0%A2%D0%B5%D1%85%D0%BD%D0%B8%D1%87%D0%B5%D1%81%D0%BA%D0%B8%20%D0%BE%D1%82%D0%B4%D0%B5%D0%BB/%D0%A6%D0%B5%D0%BD%D1%82%D1%8A%D1%80%20%D0%B7%D0%B0%20%D0%BD%D0%B0%D0%B1%D0%BB%D1%8E%D0%B4%D0%B5%D0%BD%D0%B8%D0%B5%20%D0%B8%20%D0%BE%D0%BF%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D0%B8%20-%20SOC/Technologies/Demisto%20XSOAR/%D0%9E%D1%81%D0%BD%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5%20%D0%B7%D0%B0%20%D1%80%D0%B0%D0%B7%D1%81%D0%BB%D0%B5%D0%B4%D0%B2%D0%B0%D0%BD%D0%B5/dddd/?sheet=XWiki.InplaceEditing&action=lock&lockAction=edit&language=en&outputSyntax=plain&timestamp=1637599436150 HTTP/1.1" 302 -
my-ip - - [22/Nov/2021:18:43:55 +0200] "GET /xwiki/bin/lock/Dashboard/%D0%A2%D0%B5%D1%85%D0%BD%D0%B8%D1%87%D0%B5%D1%81%D0%BA%D0%B8%20%D0%BE%D1%82%D0%B4%D0%B5%D0%BB/%D0%A6%D0%B5%D0%BD%D1%82%D1%8A%D1%80%20%D0%B7%D0%B0%20%D0%BD%D0%B0%D0%B1%D0%BB%D1%8E%D0%B4%D0%B5%D0%BD%D0%B8%D0%B5%20%D0%B8%20%D0%BE%D0%BF%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D0%B8%20-%20SOC/Technologies/Demisto%20XSOAR/%D0%9E%D1%81%D0%BD%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5%20%D0%B7%D0%B0%20%D1%80%D0%B0%D0%B7%D1%81%D0%BB%D0%B5%D0%B4%D0%B2%D0%B0%D0%BD%D0%B5/dddd/WebHome?ajax=1&action=edit&language=en HTTP/1.1" 204 -
my-ip - - [22/Nov/2021:18:43:55 +0200] "GET /xwiki/rest/wikis/xwiki/spaces/Dashboard/spaces/%D0%A2%D0%B5%D1%85%D0%BD%D0%B8%D1%87%D0%B5%D1%81%D0%BA%D0%B8%20%D0%BE%D1%82%D0%B4%D0%B5%D0%BB/spaces/%D0%A6%D0%B5%D0%BD%D1%82%D1%8A%D1%80%20%D0%B7%D0%B0%20%D0%BD%D0%B0%D0%B1%D0%BB%D1%8E%D0%B4%D0%B5%D0%BD%D0%B8%D0%B5%20%D0%B8%20%D0%BE%D0%BF%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D0%B8%20-%20SOC/spaces/Technologies/spaces/Demisto%20XSOAR/spaces/%D0%9E%D1%81%D0%BD%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5%20%D0%B7%D0%B0%20%D1%80%D0%B0%D0%B7%D1%81%D0%BB%D0%B5%D0%B4%D0%B2%D0%B0%D0%BD%D0%B5/spaces/dddd/pages/WebHome?timestamp=1637599436208 HTTP/1.1" 200 10741
my-ip - - [22/Nov/2021:18:43:55 +0200] "GET /xwiki/bin/view/Dashboard/%D0%A2%D0%B5%D1%85%D0%BD%D0%B8%D1%87%D0%B5%D1%81%D0%BA%D0%B8%20%D0%BE%D1%82%D0%B4%D0%B5%D0%BB/%D0%A6%D0%B5%D0%BD%D1%82%D1%8A%D1%80%20%D0%B7%D0%B0%20%D0%BD%D0%B0%D0%B1%D0%BB%D1%8E%D0%B4%D0%B5%D0%BD%D0%B8%D0%B5%20%D0%B8%20%D0%BE%D0%BF%D0%B5%D1%80%D0%B0%D1%86%D0%B8%D0%B8%20-%20SOC/Technologies/Demisto%20XSOAR/%D0%9E%D1%81%D0%BD%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5%20%D0%B7%D0%B0%20%D1%80%D0%B0%D0%B7%D1%81%D0%BB%D0%B5%D0%B4%D0%B2%D0%B0%D0%BD%D0%B5/dddd/?xpage=get&outputTitle=true&outputSyntax=annotatedxhtml&language=en&timestamp=1637599436247 HTTP/1.1" 200 1448

One thing to note is that when this issue initially came up, the option for backslash wasn’t added to Tomcat so I did that and it initially worked fine - then it broke again.

I’ve tried the following:

  • Restarting Tomcat (duh :slight_smile:)
  • Restarting the server (duuuh :slight_smile:)
  • Deleting the page and creating it again
  • Deleting the page, removing it from recycle bin and creating it again
    – One thing to note here is that when creating the page, I can put any content I want in it - but then it can’t be edited again.

Now all of this is really strange and any help in resolving the issue would be greatly appreciated. Thank you!

#2

Your MySQL database seems to be in a pretty weird state where you have some tables in latin1_swedish_ci (which is completely wrong) and others in utf8mb4_general_ci (which is half right since it should be utf8mb4_bin) as if someone created the database with the wrong collation, tried to fix that by migrating it to a slightly less wrong collation but only did half of it.

#3

This is surely not the cause of your problem, but if anyone else is searching for this in the future - I had the same error and it was caused by too little hard disk space (no pages could be edited and no attachments uploaded).

1 Like
#4

That someone would be me - I tried to fix that in the past but had to move to other things and haven’t looked back. I just tried to do that again and realized why I couldn’t do that in the past - as per this article, for utf8mb4 we need MySQL 5.7 or later. Our installation is running on CentOS with MariaDB whose version is 5.5.68, the latest available from the default repository. From MariaDB’s release notes for this version:

In most respects MariaDB 5.5 will work exactly as MySQL 5.5: all commands, interfaces, libraries and APIs that exist in MySQL also exist in MariaDB.

This means that we need to first resolve the version issue before being able to convert the collation and character sets but I’m going to look into that separately.

Do you think this is the issue here? As mentioned, the same error shows up when loading any page on the xWiki installation but none of the other pages have the issue described in the original post.

As for the recommendation proposed by @dahin, this is not the case here but hopefully someone will find it helpful in the future.

Thank you for your help!

#5

I would definitely recommend upgrading your MariaDB to at least 10.3 and then migrate to utf8mb4_bin. And yes, that’s the reason why you could not migrate most of the tables to utf8mb4.

Another possibility is to migrate to utf8_bin but it’s not very future-proof, and it means you will have missing characters like emojis (what MySQL calls utf8 is actually “UTF8 characters which don’t take more than 3 bytes”, in other words utf8mb3), not upgrading MariaDB also means that you won’t benefit from the new size limit for page names and other short strings (you will stay on 255 max characters instead of 768).

#6

Thank you for that, what I was rather more interested in though was if this would resolve the issue with being unable to edit this specific page.

I’ll see how we can go about this and hopefully there’ll be some development by the end of the week and if no further ideas arise, I’ll let you know if upgrading the database fixed this problem.

Thank you again for your help.

#7

The fix for “mix of collations” error is to not have a mix of collations anymore so migrating all the tables to the same one will fix it, yes. By the way, using _bin instead of _general_ci collation will also fix a few issues you just did not notice yet.

#8

There must be some misunderstanding. The main issue that I am facing is the inability to modify a single specific page after its creation, not the conflict of collations in the database. The same error messages appear when loading any page on the wiki - when loading the page, not when editing it - but all other pages are editable regardless of the message in the Catalina log.

I am therefore skeptical that the collation / character set conflict is the main issue here but this is a good start nonetheless.

P.S. Found out that I used “delete” instead of “edit” in the original post, that’s resolved now.

#9

Well, I really doubt you can save a page which is affected by this collation inconsistency.

It’s also the only real error you pasted, so I can hardly tell you more. What error do you get exactly when you try to save ? You should take a look at the log when you have “Failed to load the page”.

#10

I understand your doubts, as a technical person I would doubt the same if I were you. However, it’s in my best interest to provide correct information to you and, unfortunately, there are no other logs in the catalina.out log - I checked all Tomcat logs and, unless there are some xWiki-specific logs, I am not sure what else c an I refer to.

I just noticed that when saving other pages, there are again collation-related errors in the catalina.out log, but other than that the pages are saved fine.

Please see this gif:

Animation

#11

Clicking save execute an AJAX request in which there might be more details on why that request failed. You should try to get the content of that request in your browser Network dev tool.

The error might not directly be related to the page save, a lot can happen in reaction to a save and not all pages are necessarily affected by this problem (might depend on the objects they contain).

#12

Good idea. This is the request / response:

Request (URL Decoded):

GET https://our-domain:9443/xwiki/rest/wikis/xwiki/spaces/Dashboard/spaces/Технически отдел/spaces/Център за наблюдение и операции - SOC/spaces/Technologies/spaces/Demisto XSOAR/spaces/Основание за разследване/spaces/Firewall v2 - Excessive Firewall/ACL Connections Accepted From Single Internal Host/pages/WebHome?timestamp=1637675032967 HTTP/1.1

Response:

HTTP/1.1 500
Content-Script-Type: text/javascript
Date: Tue, 23 Nov 2021 13:43:52 GMT
Accept-Ranges: bytes
Server: Restlet-Framework/2.3.12
XWiki-User: xwiki:XWiki.myUser
XWiki-Version: 12.10
Content-Type: text/plain;charset=UTF-8
Content-Language: en
Content-Length: 45
Connection: close

Malformed URL: the spaces section is invalid.

I tried to SELECT all spaces from the xwikispace table with the names as shown in the URL above and all of them returned one row each and so did the name of the page itself. I confirmed that the XWS_NAME and XWS_PARENT columns are appropriately set and that’s the case. Since editing other pages under the same parent page works fine, this leads me again to issue with the page’s name.

Now I tried to create a new page with a slash in its name and test with it - for some reason, it is experiencing the exact same issue as described above. However, as mentioned, there is an existing page with a / character in its name and there aren’t any problems editing it.

So maybe it is indeed caused by the collation and the two pages were created under two different set collations which might lead me here?

As mentioned, I’ll try fixing the database issue and I’ll keep you updated. Thank you for your help.