I found an old discussion here Can't make rights work in XWiki 8.4.4 - #10 by douglasl and it turns out that the programming rights are the problem. These checkbox was activated for the XWikiAllGroup (but only this one!), causing all admin rights propagated to all sites and wikis. Once unchecked all went back to normal and expected behaviour.
This is somewhat confusing and my suggestion would be to insert an explicit warning, that giving just programming rights to the AllGroup results in global admin rights.
But anyway, this forum was again very helpful!
Norbert