Philip Durbin 🚀 (Jul 17 2024 at 14:51): Philip Durbin 🚀 (Jul 17 2024 at 14:51):A new thread by @Asbjørn Skødt - https://groups.google.com/g/dataverse-community/c/HgZRJ3WoKUs/m/YP9JFm1sAgAJ
Oliver Bertuch (Jul 17 2024 at 15:13):I just replied, pointing him here to continue the discussion :smiley: https://groups.google.com/g/dataverse-community/c/HgZRJ3WoKUs/m/1cKsKudvAgAJ
Philip Durbin 🚀 (Jul 17 2024 at 15:52):Yeah, I thought here might be better. Thanks.
Oliver Bertuch (Jul 17 2024 at 16:25):Maybe @Asbjørn Skødt is interested in trying out the experimental TOML support :money_face:
Philip Durbin 🚀 (Jul 17 2024 at 17:24):Exactly! I think he'd love it.
Philip Durbin 🚀 (Jul 17 2024 at 17:25):@Asbjørn Skødt see Feature Request: Enabling configuration of Dataverse using simple file(s) #10684
Asbjørn Skødt (Jul 18 2024 at 08:01):Hey! Yes, let's continue here. What terrible lies the ChatGPT gives you! What is this TOML, tell me more :)
Oliver Bertuch (Jul 18 2024 at 08:06):Hi @Asbjørn Skødt !
I've been announcing this in #containers > toml configuration... I created a fake JDBC driver that we can use to deploy JVM options that are MPCONFIG enabled (note that not all are, especially the storage subsystem isn't yet) as TOML files.
There's also an upstream issue that describes it with some examples: #10684
Oliver Bertuch (Jul 18 2024 at 13:24):@Asbjørn Skødt feel free to join the container meeting in 5 or watch the recording later. Will give a demo of the TOML config thing. (Also with a demo app, not Dataverse...)
Philip Durbin 🚀 (Jul 18 2024 at 13:25):See #containers > weekly meeting or https://ct.gdcc.io
Asbjørn Skødt (Jul 22 2024 at 07:44):So I think the conclusion for us was to wait for Oliver's TOML update.
Oliver Bertuch (Jul 22 2024 at 07:46):Don't forget to vote :+1: on https://github.com/payara/Payara/issues/6822 y'all!!!
Philip Durbin 🚀 (Jul 22 2024 at 13:50):So, yes, please vote, and yes, I'm looking forward to the TOML file, but...
... should we also write better documentation about how to use MicroProfile Config today?
Oliver Bertuch (Jul 22 2024 at 15:48):No idea. It's complicated. And the configuration guide is a mess. I already added a section on safe password handling. Not sure if we put more gasoline into the fire.
Oliver Bertuch (Jul 22 2024 at 15:49):Should we send a notice to the community mailing list to add more votes for the issue? I'm already alerting some other channels to get more votes in.
Philip Durbin 🚀 (Jul 22 2024 at 15:53):Heh, if you want to email the google group, for it.
That said, at 15 votes, the TOML issue is already number one! Number two has 9 votes: https://github.com/payara/Payara/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc
Jay Sundu (Sep 10 2025 at 16:39):Hi there.
Did TOML support get implemented in Dataverse? We have also been looking into using a microprofile-config.properties file but need some more guidance on where to put the file and how to get Payara to recognize it. What are folks doing to avoid editing the domain.xml file and handle configuration like PID settings?
Thanks, Jay
Oliver Bertuch (Sep 10 2025 at 17:49):Never ever edit the properties file directly. It is absolutely not meant to be edited and you may Ende up getting in trouble doing so.
Oliver Bertuch (Sep 10 2025 at 17:50):Yes, Payara enabled TOML support.
Oliver Bertuch (Sep 10 2025 at 17:50):Here's how you activate it: https://docs.payara.fish/community/docs/6.2025.9/Technical%20Documentation/MicroProfile/Config/TOML.html
Oliver Bertuch (Sep 10 2025 at 17:52):So far I've mostly used env vars in containers, but looking forward to using TOML now, mounted from a K8s ConfigMap.
Jay Sundu (Sep 10 2025 at 19:37):Thanks @Oliver Bertuch I'll try to get TOML working
Daniel C Schmidt (Sep 11 2025 at 16:22):@Oliver Bertuch Following up on Jay's Q, right now we're deploying Dataverse to VMs per the documentation (which among things, involves using asadmin to deploy the app, which results in a mutable <application> section of that file). The root motivation behind Jay's Q is that we want to full extract our config customization (a la what you'd do with a containerized deployment). (We run containers for everything but Dataverse, but opted not to for it because it didn't seem well-supported at the time.) Do you have any pointers for us? Maybe a GitHub link? From your comment it sounds like you're mostly using the environment, but we'd be curious about details (e.g. what remains, if anything, in domain.xml).
Philip Durbin 🚀 (Sep 11 2025 at 16:43):Here's an example using MPConfig with environment variables in Dataverse:
[dataverse@dev1 ~]$ /usr/local/payara6/glassfish/bin/asadmin stop-domain
Picked up JAVA_TOOL_OPTIONS: -Djdk.util.zip.disableZip64ExtraFieldValidation=true --add-opens=java.base/java.io=ALL-UNNAMED
Waiting for the domain to stop ....
Command stop-domain executed successfully.
[dataverse@dev1 ~]$
[dataverse@dev1 ~]$ export DATAVERSE_VERSION=foo
[dataverse@dev1 ~]$
[dataverse@dev1 ~]$ /usr/local/payara6/glassfish/bin/asadmin start-domain
Picked up JAVA_TOOL_OPTIONS: -Djdk.util.zip.disableZip64ExtraFieldValidation=true --add-opens=java.base/java.io=ALL-UNNAMED
Waiting for domain1 to start ...................
Successfully started the domain : domain1
domain Location: /usr/local/payara6/glassfish/domains/domain1
Log File: /usr/local/payara6/glassfish/domains/domain1/logs/server.log
Admin Port: 4848
Command start-domain executed successfully.
[dataverse@dev1 ~]$
[dataverse@dev1 ~]$ curl http://localhost:8080/api/info/version
{"status":"OK","data":{"version":"foo","build":"11747-review-dataset-type-48ccfc8f54"}}
DATAVERSE_VERSION comes from here in the code: https://github.com/IQSS/dataverse/blob/v6.7.1/src/main/java/edu/harvard/iq/dataverse/settings/JvmSettings.java#L42
(Don't worry about my custom build number. I set that with a script before I built the war file.)
Daniel C Schmidt (Sep 11 2025 at 16:51):Thanks @Philip Durbin 🚀 , that looks promising. I'm thinking we could package everything into a .env file and (in our current setup) pass that to the service via the systemd service configuration (i.e. EnvironmentFile). Do you know if everything can be provided that way, or would we still have to resort to templating domain.xml or running asadmin scripts pre-startup for some settings? I'm coming into this from another project so am a bit less familiar with Dataverse than @Jay Sundu , but my understanding from his testing is that not all configurations can be set that way.
Philip Durbin 🚀 (Sep 11 2025 at 18:09):There are a few stragglers but most of the old JVM settings we used to set in domain.xml can now be set via MPConfig, such as with environment variables, as shown above.
Philip Durbin 🚀 (Sep 11 2025 at 18:10):Storage can't be configured via MPConfig yet. See #9998.
Philip Durbin 🚀 (Sep 11 2025 at 18:12):I think that's the big one. Please let us know what else you need.
Philip Durbin 🚀 (Sep 11 2025 at 18:12):But yeah, you should be able to source an .env file before you start Payara.
Jay Sundu (Sep 11 2025 at 21:17):Thanks @Philip Durbin 🚀 . Can I use environment variables for the Per Provider Settings? For instance, I have the variable dataverse.pid.ez1.type and I tried setting DATAVERSE_PID_EZ1_TYPE but that doesn't seem to work.
Daniel C Schmidt (Sep 11 2025 at 21:32):@Jay Sundu Did you try using DATAVERSE_PID_EZID_ as the prefix? That would seem to be it based on the code linked above. Granted I'm just grokking the constants there, not looking into how things are actually fetched from ENV, so might be off here
Jay Sundu (Sep 11 2025 at 23:22):Thanks @Daniel C Schmidt** I ended up rebuilding the dev environment and now DATAVERSE_PID_EZ1_TYPE works there.
Philip Durbin 🚀 (Sep 11 2025 at 23:27):Working? Great! :tada:
Philip Durbin 🚀 (Sep 12 2025 at 00:35):Also, if you have suggestions of how we should improve the docs, let's talk! I can definitely help with a PR if you want to start one.
Oliver Bertuch (Sep 12 2025 at 05:21):Personally, I'd always favor using a TOML file over any .env file. But if systemd + env file works for you, go go go! :smile:
Philip Durbin 🚀 (Sep 12 2025 at 10:48):Makes sense, but you gotta walk before you can run. :smile:
@Oliver Bertuch what do you think? Should document the DATAVERSE_VERSION example above? I think of it as the "hello world" of using MPConfig in a classic installation.
Oliver Bertuch (Sep 12 2025 at 12:08):Maybe instead of _VERSION use the build number thingy?
Oliver Bertuch (Sep 12 2025 at 12:08):But sure, we can make some example for MPCONFIG and include it in the docs
Oliver Bertuch (Sep 12 2025 at 12:13):It probably would help to have the full picture in there how to go about the env file and systemd, to be extended with TOML etc
Philip Durbin 🚀 (Sep 12 2025 at 13:06):DATAVERSE_BUILD has no effect if you build the war with src/main/java/BuildNumber.properties in place: https://github.com/IQSS/dataverse/blob/v6.7.1/src/main/java/edu/harvard/iq/dataverse/util/SystemConfig.java#L106-L120
(I tried it yesterday. Didn't work. Looked at the code above.)
Philip Durbin 🚀 (Sep 12 2025 at 13:07):The problem I'm trying to solve with an example is this: people may be unclear how to even use MPConfig and just give up and hack on domain.xml.
Daniel C Schmidt (Sep 12 2025 at 14:29):Philip Durbin 🚀 said:
The problem I'm trying to solve with an example is this: people may be unclear how to even use MPConfig and just give up and hack on domain.xml.
That was our experience coming into the deployment process as a team with a lot of container/orchestration experience but only a very, very distant Java background. (I suspect the clean path is more obvious to an experienced Java developer.) The use of asadmin, for example, which is referenced quite a bit in release notes—we didn't realize that not only does it modify domain.xml, but that some of the modifications are (without hacky workarounds) impossible to back into configuration management. Application deployments come to mind due to changes to the <applications> config section.
Philip Durbin 🚀 (Sep 12 2025 at 14:34):Makes total sense. Many years of hacking on domain.xml (for this project) has made me lose perspective on how unnatural it is. Hopefully you like the MPConfig way better. And TOML (which builds on MPConfig) should be better still. More like configuring any service on Linux. :sweat_smile:
Last updated: Oct 30 2025 at 05:14 UTC