Well, I still don’t know why the urls were being written incorrectly after 11.3 to 11.6.1 upgrade.
I reinstalled the xwiki software, but that made no difference - so it wasn’t a problem with the installation files.
I’ve uninstalled as many unneeded extensions as I could (some weird ones got in there, don’t know how)
I found that setting the xwiki.home and xwiki.url.protocol properties fixed some of the urls, but not all.
Finally, I used the nginx reverse proxy to rewrite any of the offending urls to their proper format on the fly. Annoying, but that’s the only solution I have until I make a brand new wiki installation and migrate everything to it.