Ash Manda (Oct 05 2026 at 13:47): Ash Manda (Oct 05 2026 at 13:47):I wanted to collect some thoughts on this PR: https://github.com/uncch-rdmc/dataverse/pull/99
Ash Manda (Oct 05 2026 at 13:47):It builds on top of the work in: https://github.com/IQSS/dataverse/pull/12783
Ash Manda (Oct 05 2026 at 13:48):What does this PR do?
It splits the current integration tests into 3 groups that take about the same time to run.
This PR causes 6 GitHub Action runners to run concurrently at the same time, but it pushes down the entire process from PR to test results in about 20-25 minutes flat.
Build Containers ──┬── JSF Tests (chromium) ~ 20 mins
├── JSF Tests (firefox) ~ 23 mins
├── JSF Tests (webkit) ~ 21 mins
└── Integration Tests (Group 1) ~ 18 mins
└── Integration Tests (Group 2) ~ 18 mins
└── Integration Tests (Group 3) ~ 18 mins
So overall this is a very major improvement for development speed.
So, what's the catch? Well, GitHub Free Organizations (like IQSS) have a concurrency limit of 20 runners for the WHOLE org. So this means at max 3 PRs can be processed every 20 minutes for all repos within the org. I linked some numbers onto the PR.
Philip Durbin 🚀 (Oct 05 2026 at 13:54):Does "each on its own runner with a fresh Dataverse" mean you're spinning up three instances of Dataverse?
Ash Manda (Oct 05 2026 at 13:58):that's right
Ash Manda (Oct 05 2026 at 13:58):actually in this PR, six instances
Philip Durbin 🚀 (Oct 05 2026 at 14:00):Sounds like a good way to get more work done more quickly! :tada:
Ash Manda (Oct 05 2026 at 14:25):If we want to make this experimental PR happen without causing traffic bottlenecks for other developers, we might have to create a separate testing org with a GitHub Teams subscription. That's why I wanted to just start the conversation on how this might look. I don't want to pay GitHub any money.
Philip Durbin 🚀 (Oct 05 2026 at 14:30):Do you have a sense of how close we are to maxing out our limits? I'm not even very aware of them. :sweat_smile:
Ash Manda (Oct 05 2026 at 14:33):
So P99 is essentially worst case, P90 is the average experience. I modeled based on past week's usage, but release times might be more hectic
Right now for P90 we're nowhere near the limits. After 12783 is merged, P90 goes to 53 seconds, and P99 goes to 6 minutes, but this is completely acceptable, since the tests finish 50% faster and the runners get released fast, so overall it is a win no matter what.
With the experimental PR, P90 goes to 3 min and P99 goes to 10 mins. Realistically, this means we would erase any savings that we got with the optimization, and create more overhead. Especially during releases.
Philip Durbin 🚀 (Oct 05 2026 at 14:37):Ash Manda said:
Right now for P90 we're nowhere near the limits.
I'm glad to hear this! :sweat_smile:
Ash Manda (Oct 05 2026 at 14:40):If I'm able to find a safe way to run the Integration Tests and JSF tests on the same instance in parallel, we would kill two birds with one shot
Don Sizemore (Oct 05 2026 at 17:28):@Philip Durbin 🚀 and @Ash Manda : I'm clearing Jenkins from the docs (and code comments) as we convert to containerized GitHub Actions. What say y'all about
./src/test/java/edu/harvard/iq/dataverse/api/UsersIT.java: // This test is disabled because it is only compatible with the containerized development environment and would cause the Jenkins job to fail.
./src/test/java/edu/harvard/iq/dataverse/api/FilesIT.java: // This test is disabled because it is only compatible with the containerized development environment and would cause the Jenkins job to fail.
?
Philip Durbin 🚀 (Oct 05 2026 at 18:44):@Don Sizemore please don't mess with those lines. I'm messing with them (enabling the disabled tests!) in this PR: support SPA dev - #12746
Please see also #containers > support SPA dev #12746
Last updated: Nov 02 2026 at 07:01 UTC