Frank Smutniak (May 18 2026 at 16:58): Frank Smutniak (May 18 2026 at 16:58):Our production dataverse runs 6.10.1 on an 8 virtual CPU server with 32GB memory.
We want to make sure our settings make full use of the resources.
We build the server using the Harvard ansible scripts but expect there are
some server specific adjustments we need to make beyond initial setup.
Our current settings look like the following:
domain1/config/domain.xml
<jvm-options>-Xmx16384m</jvm-options>
For domain1/config/domain.xml what are suggested min-thread-pool-size and max-thread-pool-size for the <thread-pool>
associated with http-listener-1 (http-thread-pool)
We currently do not have these set but are considering:
<jvm-options>-XX:MaxMetaspaceSize=4096m</jvm-options>
<jvm-options>-XX:MetaspaceSize=2048m</jvm-options>
<jvm-options>-Xms12288m</jvm-options>
In solr/bin/solr.in.sh we currently have no MEM settings. Should we consider?
SOLR_JAVA_MEM="-Xms4g -Xmx4g"
For apache, is a connection pool set of 300 optimal?
For postgres, are there settings you've found to be optimal for working with dataverse?
Thank you for any guidance.
Don Sizemore (May 22 2026 at 15:42):Hello,
I wouldn't try to run Dataverse in a JVM heap smaller than 16g as you've specified. Dataverse usage can be bursty; the greatest memory spike will happen during tabular ingest.
I've never played with the min/max thread pool sizes for the http listeners, but I remember reading that MaxMetaspaceSize and MetaspaceSize should be set to the same value to avoid performance problems (this may now be historic).
UNC Dataverse currently runs in a Solr JVM heap setting of 1g; 4g should be plenty.
The connection pool size for Apache depends on your usage; I'd imagine 300 should be plenty as long as the LLM scrapers stay away.
I've never had to tune Postgres; our production RDS instance has 2 CPU and 8GB of RAM; the biggest single effect I've seen on performance depends on administrators calling ANALYZE; (or not) following a database import.
I'd love to hear what other community members have to say here.
Don
Thomas van Erven (May 23 2026 at 07:29):Most "tuning" I needed to do on Postgres was cleanup a rather large ACTIONLOGRECORD table (years of buildup) and a lot of notifications. Runs fine under default settings, Postgres being a saint and all.
I run demo/test dataverses on smaller VM's, but despite best effort, I find Payara has serious difficulty getting out of bed for less than 2 GB heap minimal and the sweet spot is somewhere around 3-4 GB.
Workloads are on the spiky side of ops as Don indicates, tabular ingest can be an evil sod occasionally as users will happily add large tabular data files. Treat any post-processing (workflows) you do as those Mario enemies with spikes on all sides - expect to see trouble from them at least once even if they look innocent.
Thomas van Erven (May 23 2026 at 07:32):One thing I'd like to do at some point is mess with a pgvector database, for that matter. Having a db that can natively store embeddings in a relatively efficient way to act as a search index (rather than Payara) might be trading disk space for memory but that's a decent deal in this case. I had some good luck with FFF-driven vector space being a relatively cheap win with searching.
Frank Smutniak (May 26 2026 at 15:22):Thanks Don and Thomas,
The bots are a big drain on resources in bursts but with our recent changes it is having less downtime or slow service. Thank you for your suggestions and confirmation.
Frank
Frank Smutniak (May 26 2026 at 15:23):Removing ACTIONLOGRECORD was done as we recently ported from the old amazon linux to AL 23. It saved a lot of space!
Last updated: May 30 2026 at 09:11 UTC