Proxy-Parameters beeing ignored

Help / Discuss
#1

(PRE: I read all the other threads about xwiki and proxy… )

I am using the latest Xwiki Docker Container and docker-compose to set up xwiki.

Since the (final) host on which xwiki is build/run has no direct internet connection, I need xwiki to use a proxy for initial setup and downloading extensions.

As I read in the other threads, I use JAVA_OPTS in the compose-file to pass the right parameters, and when the container starts, I see those parameters are recognized:

xwiki  | 13-Aug-2019 09:42:54.745 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version:        Apache Tomcat/8.5.43
xwiki  | 13-Aug-2019 09:42:54.746 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built:          Jul 4 2019 20:53:15 UTC
xwiki  | 13-Aug-2019 09:42:54.746 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server number:         8.5.43.0
xwiki  | 13-Aug-2019 09:42:54.746 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name:               Linux
xwiki  | 13-Aug-2019 09:42:54.748 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version:            3.10.0-514.el7.x86_64
xwiki  | 13-Aug-2019 09:42:54.748 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture:          amd64
xwiki  | 13-Aug-2019 09:42:54.748 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home:             /opt/java/openjdk/jre
xwiki  | 13-Aug-2019 09:42:54.748 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version:           1.8.0_222-b10
xwiki  | 13-Aug-2019 09:42:54.748 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor:            AdoptOpenJDK
xwiki  | 13-Aug-2019 09:42:54.748 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:         /usr/local/tomcat
xwiki  | 13-Aug-2019 09:42:54.748 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:         /usr/local/tomcat
xwiki  | 13-Aug-2019 09:42:54.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/usr/local/tomcat/conf/logging.properties
xwiki  | 13-Aug-2019 09:42:54.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
xwiki  | 13-Aug-2019 09:42:54.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dhttp.proxyHost=10.xx.xx.xx -Dhttp.proxyPort=8080 -Dhttp.proxyUser=xxx -Dhttp.proxyPassword=xxx -Dhttps.proxyUser=xxx -Dhttps.proxyPassword=xxx -Dhttps.proxyHost=10.xx.xx.xx -Dhttps.proxyPort=8080
xwiki  | 13-Aug-2019 09:42:54.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Xmx1024m
xwiki  | 13-Aug-2019 09:42:54.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true
xwiki  | 13-Aug-2019 09:42:54.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.connector.CoyoteAdapter.ALLOW_BACKSLASH=true
xwiki  | 13-Aug-2019 09:42:54.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.security.egd=file:/dev/./urandom
xwiki  | 13-Aug-2019 09:42:54.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048
xwiki  | 13-Aug-2019 09:42:54.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
xwiki  | 13-Aug-2019 09:42:54.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
xwiki  | 13-Aug-2019 09:42:54.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs=
xwiki  | 13-Aug-2019 09:42:54.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/usr/local/tomcat
xwiki  | 13-Aug-2019 09:42:54.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/usr/local/tomcat
xwiki  | 13-Aug-2019 09:42:54.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/usr/local/tomcat/temp
xwiki  | 13-Aug-2019 09:42:54.750 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent Loaded APR based Apache Tomcat Native library [1.2.23] using APR version [1.6.3].
xwiki  | 13-Aug-2019 09:42:54.750 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true].
xwiki  | 13-Aug-2019 09:42:54.750 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR/OpenSSL configuration: useAprConnector [false], useOpenSSL [true]

As you see, with “13-Aug-2019 09:42:54.749 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dhttp.proxyHost=10.xx.xx.xx -Dhttp.proxyPort=8080 -Dhttp.proxyUser=xxx -Dhttp.proxyPassword=xxx -Dhttps.proxyUser=xxx -Dhttps.proxyPassword=xxx -Dhttps.proxyHost=10.xx.xx.xx -Dhttps.proxyPort=8080” Xwiki should use the proxy.

While initializing a new installation, Xwiki connects to store.xwiki.com in order to download flavors/extenstions/etc.

That does not work with the proxy.

I get the following errors with defined proxy-parameters:

xwiki  | 2019-08-13 09:43:08,963 [Core extension repository updater] ERROR aultExtensionRepositoryManager - Unexpected error when trying to find extension [commons-daemon:commons-daemon/1.1.0] in repository [store.xwiki.com:xwiki:https://store.xwiki.com/xwiki/rest]
xwiki  | org.xwiki.extension.ResolveException: Failed to create extension object for extension [commons-daemon:commons-daemon/1.1.0]
xwiki  |        at org.xwiki.extension.repository.xwiki.internal.XWikiExtensionRepository.resolve(XWikiExtensionRepository.java:344)
xwiki  |        at org.xwiki.extension.repository.internal.DefaultExtensionRepositoryManager.resolve(DefaultExtensionRepositoryManager.java:286)
xwiki  |        at org.xwiki.extension.repository.internal.core.DefaultCoreExtensionScanner.updateExtensions(DefaultCoreExtensionScanner.java:116)
xwiki  |        at org.xwiki.extension.repository.internal.core.DefaultCoreExtensionRepository$1.run(DefaultCoreExtensionRepository.java:138)
xwiki  |        at java.lang.Thread.run(Thread.java:748)
xwiki  | Caused by: java.io.IOException: Failed to request [https://store.xwiki.com/xwiki/rest/repository/extensions/commons-daemon%3Acommons-daemon/versions/1.1.0]
xwiki  |        at org.xwiki.extension.repository.xwiki.internal.XWikiExtensionRepository.getRESTResource(XWikiExtensionRepository.java:247)
xwiki  |        at org.xwiki.extension.repository.xwiki.internal.XWikiExtensionRepository.getRESTObject(XWikiExtensionRepository.java:305)
xwiki  |        at org.xwiki.extension.repository.xwiki.internal.XWikiExtensionRepository.resolve(XWikiExtensionRepository.java:350)
xwiki  |        at org.xwiki.extension.repository.xwiki.internal.XWikiExtensionRepository.resolve(XWikiExtensionRepository.java:340)
xwiki  |        ... 4 common frames omitted
xwiki  | Caused by: java.net.UnknownHostException: store.xwiki.com
xwiki  |        at java.net.InetAddress.getAllByName0(InetAddress.java:1281)
xwiki  |        at java.net.InetAddress.getAllByName(InetAddress.java:1193)
xwiki  |        at java.net.InetAddress.getAllByName(InetAddress.java:1127)
xwiki  |        at org.apache.http.impl.conn.SystemDefaultDnsResolver.resolve(SystemDefaultDnsResolver.java:45)
xwiki  |        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:112)
xwiki  |        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:373)
xwiki  |        at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:394)
xwiki  |        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:237)
xwiki  |        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
xwiki  |        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
xwiki  |        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
xwiki  |        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
xwiki  |        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
xwiki  |        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
xwiki  |        at org.xwiki.extension.repository.xwiki.internal.XWikiExtensionRepository.getRESTResource(XWikiExtensionRepository.java:244)
xwiki  |        ... 7 common frames omitted

And the Step 2 remains EMPTY:

xwiki-proxy-problem

If I remove the proxy-parameters, everything works - but the host needs to have a direct internet-connection - and that is not possible in the final environment.

The proxy parameters are correct, because Docker itself can pull the images using that proxy-server.

I tested on a separate system with proxy AND real internet-access.

With the proxy-parameters, the know problem and errors, without everything works and I can select the default flavor.

I also traced with tcpdump, there is no use of the configured proxy at all - it just fails.

What may be the problem?

A xwiki-docker-container-version-specific problem?

Any help is REALLY welcome!

With best regards
Frank

#2

The proxy-server is available from the xwiki-docker-container.

I can connect to the proxy from inside (using docker exec -it xxx /bin/bash and curl -v …) the Xwiki-Container to the proxy-server…

It just seems that the parameters have just not the wanted effect…?

#3

Reading https://memorynotfound.com/configure-http-proxy-settings-java/, it says:

Authenticating Proxy

Setting http.proxyUser and http.proxyPassword will not automatically authenticate via a proxy.

Taking the above into account. Here is how you can Authenticate via a proxy. This initialisation code is typically executed at application startup. So make sure you register this authenticator before you make any HTTP Requests that require Proxy Authentication.

// settings proxy credentials
System.setProperty("http.proxyUser", "proxyUser");
System.setProperty("http.proxyPassword", "secret");

// Java ignores http.proxyUser. Here come's the workaround.
Authenticator.setDefault(new Authenticator() {
    @Override
    protected PasswordAuthentication getPasswordAuthentication() {
        if (getRequestorType() == RequestorType.PROXY) {
            String prot = getRequestingProtocol().toLowerCase();
            String host = System.getProperty(prot + ".proxyHost", "");
            String port = System.getProperty(prot + ".proxyPort", "80");
            String user = System.getProperty(prot + ".proxyUser", "");
            String password = System.getProperty(prot + ".proxyPassword", "");
            if (getRequestingHost().equalsIgnoreCase(host)) {
                if (Integer.parseInt(port) == getRequestingPort()) {
                    return new PasswordAuthentication(user, password.toCharArray());
                }
            }
        }
        return null;
    }
});

Is it possible that Xwiki has yet to “learn” to use a proxy with auth the right way??