Xwiki hibernate cannot open the transaction to postgres

Hello,

we are using xwiki-tomcat-postgres docker installation. It is part of a stack running on two nodes(db, xwiki). We also are using OIDCAuthenticator with Keycloak.

Versions:

  • xwiki:12.3
  • postgres:9.5
  • OIDCAuthenticator:1.21.2
  • Keycloak:9.0.0

We do not have any custom settings(except OIDC)

Sometimes it looks like it cannot get connection to db, it cannot open the transaction.
It happens only sometimes, mostly after longer inactivity(30-40minutes) if I come back to the page, I get 504 error.

Here are logs:

2020-09-07T06:55:39.666787255Z 2020-09-07 06:55:39,666 [http://our-xwiki/oidc/authenticator/callback?state=some_state8&session_state=some_session_state] DEBUG o.x.c.o.a.i.OIDCUserManager    - updateUser formattedSubject: subject-uuid 
2020-09-07T06:55:39.666923215Z 2020-09-07 06:55:39,666 [http://our-xwiki/oidc/authenticator/callback?state=some_state8&session_state=some_session_state] DEBUG o.x.c.o.a.i.s.OIDCUserStore    - searchDocument issuer: https://our-auth-server, subject: subject-uuid 
2020-09-07T06:55:39.667082176Z 2020-09-07 06:55:39,666 [http://our-xwiki/oidc/authenticator/callback?state=some_state8&session_state=some_session_state] DEBUG o.x.c.o.a.i.s.OIDCUserStore    - searchDocument query from doc.object(XWiki.OIDC.UserClass) as oidc where oidc.issuer = :issuer and oidc.subject = :subject 
2020-09-07T06:55:39.689826806Z 2020-09-07 06:55:39,683 [http://our-xwiki/oidc/authenticator/callback?state=some_state8&session_state=some_session_state] DEBUG c.x.x.i.s.h.HibernateStore     - Trying to get session from pool 
2020-09-07T06:55:39.691331133Z 2020-09-07 06:55:39,689 [http://our-xwiki/oidc/authenticator/callback?state=some_state8&session_state=some_session_state] DEBUG c.x.x.i.s.h.HibernateStore     - Taken session from pool [SessionImpl(107955503<open>)] 
2020-09-07T06:55:39.691377287Z 2020-09-07 06:55:39,690 [http://our-xwiki/oidc/authenticator/callback?state=some_state8&session_state=some_session_state] DEBUG c.x.x.i.s.h.HibernateStore     - Trying to open transaction

And nothing more. It is trying to authenticate the user, but ends up here… trying to open transaction. After few refreshes of xwiki page, it suddenly can open new transaction and communicate with db.
After another 15 minutes even the hung transaction is closed.

We are running other postgres services in the same swarm and on the same nodes and we do not have any connection issues with them.

Do you have any suggestions what can be possibly wrong?
Thanks in advance.

EDIT: i updated postgres image to 12-alpine, but the problem stays…

I found the real problem is in docker and closing of connection, the solution is to set endpoint_mode: dnsrr for postgres service
explanation is here:

Cool, thanks for posting back the solution.