César Ferreira (Aug 06 2025 at 10:14): César Ferreira (Aug 06 2025 at 10:14):Hi there, we have a Dataverse installation using Docker containers and whenever the Dataverse container loses the connection with the Database container, for example if there is a DB container failure, the server responds with this page, as expected:
HTTP Status 500 - Internal Server Error
type Exception report
messageInternal Server Error
descriptionThe server encountered an internal error that prevented it from fulfilling this request.
exception
jakarta.ejb.EJBException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 4.0.1.payara-p1.v202304041433): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: org.postgresql.util.PSQLException: Connection has been closed.
Error Code: 0
Call: SELECT ID, CONTENT, LANG, NAME FROM SETTING WHERE ((NAME = ?) AND (LANG IS NULL))
bind => [1 parameter bound]
Query: ReadAllQuery(name="Setting.findByName" referenceClass=Setting sql="SELECT ID, CONTENT, LANG, NAME FROM SETTING WHERE ((NAME = ?) AND (LANG IS NULL))")
note The full stack traces of the exception and its root causes are available in the Payara Server 6.2023.8 #badassfish logs.
Payara Server 6.2023.8 #badassfish
After the DB container is recovered this page persists and the only solution we found to recover the connection is to restart the Dataverse container. Is there any more "elegant" solution we could use to force the connection to be reestablished from the Dataverse side and solve this problem instead of the good old "restart the service" solution?
Philip Durbin 🚀 (Aug 06 2025 at 13:12):Well, it's easy enough for me to replicate this! In Docker Desktop I stopped the postgres container and get the same error.
Philip Durbin 🚀 (Aug 06 2025 at 13:13):Then I clicked "start" on the postgres container and refreshed my browser.
Yeah, I see what you mean. Dataverse is still dead in the water. 
Philip Durbin 🚀 (Aug 06 2025 at 13:13):@César Ferreira do you feel like creating an issue about this? https://github.com/IQSS/dataverse/issues
Philip Durbin 🚀 (Aug 06 2025 at 13:31):This seems related:
Is there a way to reopen the connection pools to database without needing to restart all server? - https://github.com/payara/Payara/issues/887
Philip Durbin 🚀 (Aug 06 2025 at 13:31):"Use connection checking in the datasource configuration" they say.
César Ferreira (Aug 06 2025 at 13:33):I was creating an issue, but I can take a look on that issue first.
Philip Durbin 🚀 (Aug 06 2025 at 13:33):"You can also enable connection validation in the administration console as this will check a connection before handing it out of the pool and if it is broken it will reconnect to the database.
See http://blog.payara.fish/an-introduction-to-connection-pools-in-payara-server "
Philip Durbin 🚀 (Aug 06 2025 at 13:34):Sure, whatever you want.
Philip Durbin 🚀 (Aug 06 2025 at 13:34):Also, please note that there is a whole world of database settings we can use: https://guides.dataverse.org/en/6.7.1/installation/config.html#advanced-database-settings
César Ferreira (Aug 06 2025 at 13:40):I am using v6.2, but I am currently on the update process of Dataverse to the latest version. Since I am using a custom .war file it will probably take some time until I catch up with v6.7. This could probably be fixed with this setting dataverse.db.is-connection-validation-required then.
Philip Durbin 🚀 (Aug 06 2025 at 13:50):All of those options are available in 6.2 as well: https://guides.dataverse.org/en/6.2/installation/config.html#advanced-database-settings
César Ferreira (Aug 06 2025 at 13:51):OK, good news. I will check that setting then.
César Ferreira (Aug 06 2025 at 15:11):After some tests, I got it to work as expected with the following settings and the database connection is now automatically recovered:
asadmin create-jvm-options "-Ddataverse.db.is-connection-validation-required=true"
asadmin create-jvm-options "-Ddataverse.db.connection-validation-method=table"
asadmin create-jvm-options "-Ddataverse.db.validation-table-name=setting"
asadmin create-jvm-options "-Ddataverse.db.validate-atmost-once-period-in-seconds=60"
I wasn't sure which table I should choose for validation, I guess any table can be chosen for that purpose.
@Philip Durbin 🚀 Thank you for the support :grinning_face_with_smiling_eyes:
Philip Durbin 🚀 (Aug 06 2025 at 15:23):The setting table seems fine, I guess. It's pretty fundamental. :smile:
I'm glad you got it working! Do you want to add this to the Dataverse documentation somewhere? :thinking:
César Ferreira (Aug 06 2025 at 15:30):I think it would be interesting to document this situation as an example. I can look into that tomorrow. Can you send me the instructions on how can I do it? It has been a while since I have contributed to the documentation.
Philip Durbin 🚀 (Aug 06 2025 at 15:32):Sure, here's the latest advice: https://guides.dataverse.org/en/6.7.1/contributor/documentation.html
Philip Durbin 🚀 (Aug 06 2025 at 15:35):I guess I would suggest a new section under https://guides.dataverse.org/en/6.7.1/installation/config.html#database-persistence after "Advanced Database Settings".
Maybe "Database Configuration Tips"? And then a nested bullet for "Database Connection Recovery" (and then all those details you figured out).
César Ferreira (Aug 07 2025 at 08:11):Yes, I could do that.
Philip Durbin 🚀 (Aug 07 2025 at 12:30):#11730 looks great! (Preview at https://dataverse-guide--11730.org.readthedocs.build/en/11730/installation/config.html#database-configuration-tips ). Thanks!
I left a little feedback, especially about init.d vs configuring it once. Maybe I'm misunderstanding. :sweat_smile:
César Ferreira (Aug 07 2025 at 13:12):I only suggested an init.d script because we are using docker containers and as the project is still under development the container may need to be recreated and in that case the payara configs are lost.
Philip Durbin 🚀 (Aug 07 2025 at 13:20):Ok. Maybe it should we worded more generically so the advice can be used by "classic" (non-Docker) installations as well. What do you think?
César Ferreira (Aug 07 2025 at 13:24):Yes, I agree. I will change that to be more generic.
Philip Durbin 🚀 (Aug 07 2025 at 13:26):Awesome, thanks!
Philip Durbin 🚀 (Aug 11 2025 at 14:15):@César Ferreira we merged #11730! Thanks!
Notification Bot (Aug 12 2025 at 08:56):César Ferreira has marked this topic as resolved.
Last updated: Oct 30 2025 at 06:21 UTC