Ash Manda (Apr 09 2026 at 18:55): Ash Manda (Apr 09 2026 at 18:55):Hi @Philip Durbin 🚀 .Just wanted to get your thoughts on something.
@Don Sizemore has tasked me now with figuring out a way to permanently move away from Jenkins and use GA.
So this is how the workflow might look
Currently afaik, there is no good way in which we can display BOTH develop branch commit reports and PR commit reports directly on GitHub. So, we will have to feed the JaCoCo reports somewhere, maybe that could be a different website with a backend that is hosted on GitHub Pages. If that is the route we decide to go in, I will build the website myself too. I have a prototype workflow too, and it works mostly as intended with some additional tweaking I'll be doing in the coming weeks.
What do you think? Any thoughts/improvements/suggestions about this workflow?
Philip Durbin 🚀 (Apr 09 2026 at 18:58):@Ash Manda I love it! But, yes, you're right that the reporting might be difficult. Any idea what other projects do? Jacoco is fairly popular, I believe. :thinking:
Ash Manda (Apr 09 2026 at 19:04):Most corporate and open source projects that I am aware of use a separate reporting dashboard. In our case that is Jenkins, others use Codecov or Coveralls or SonarCloud.
Codecov for example is a paid option BUT it is free forever for public repositories with unlimited users. We would have to set it up org-side. But just like with GitHub Actions we dunno if Microsoft or Codecov will ever change their minds and remove the unlimited runs part.
With our own dashboard that we could build for Dataverse, the initial time cost of building it and maintaining it will be high, but in return IQSS retains permanent control over the reporting.
Philip Durbin 🚀 (Apr 09 2026 at 19:05):We do use Coveralls already, if that helps: https://coveralls.io/github/IQSS/dataverse
Ash Manda (Apr 09 2026 at 19:06):Coveralls fully supports JaCoCo reports! We could place them there.
Philip Durbin 🚀 (Apr 09 2026 at 19:07):Great! And I believe we already do. :smile:
Philip Durbin 🚀 (Apr 09 2026 at 19:08):Please check this out: https://github.com/IQSS/dataverse/blob/v6.10.1/.github/workflows/maven_unit_test.yml#L153-L163
Ash Manda (Apr 09 2026 at 19:08):Oh. We don't need Jenkins at all then?
Philip Durbin 🚀 (Apr 09 2026 at 19:09):Well, the goal is to retire Jenkins, it sounds like. We still use it today, obviously, for our CI and for building the guides.
Philip Durbin 🚀 (Apr 09 2026 at 19:10):When I first setup Coveralls it was in Travis. But @Oliver Bertuch moved it to GitHub Actions in #7944.
Ash Manda (Apr 09 2026 at 19:10):Seems like the existing GitHub Actions cover everything Jenkins does, that's why I was confused.
Ash Manda (Apr 09 2026 at 19:11):If there's anything that Jenkins specifically does that GA isn't doing already, then I could work on filling that gap. But seems like y'all have already thought of this!
Philip Durbin 🚀 (Apr 09 2026 at 19:12):The small thing that Jenkins is doing for us now is building the guides: https://guides.dataverse.org/en/6.10.1/developers/making-releases.html#build-the-guides-for-the-release
The big thing that Jenkins does for us is running the API test suite.
Philip Durbin 🚀 (Apr 09 2026 at 19:12):... and it sounds like you'd like to run the API test suite in containers, right? Sounds great!
Ash Manda (Apr 09 2026 at 19:12):Oh so we need 1. Integration Tests running in GA and 2. Guides built in GA for Jenkins to be redundant. Is that accurate?
Philip Durbin 🚀 (Apr 09 2026 at 19:13):Yes. I tend to call them API tests. @Oliver Bertuch gets grumpy about me calling tests the wrong things. Maybe we can name them his way this time! :sweat_smile:
Ash Manda (Apr 09 2026 at 19:15):Thank you so much! That clarifies a lot.
Ash Manda (Apr 09 2026 at 19:17):One last stupid question, is there already a workflow to build the guides or would you need one for that too?
Philip Durbin 🚀 (Apr 09 2026 at 19:17):Sure! I don't know if this helps or complicates things but https://github.com/gdcc/api-test-runner used to work. By that I mean that I could run API tests against the develop branch (for example) against Dataverse running in containers.
Ash Manda (Apr 09 2026 at 19:18):I will take a look and see if we can reuse that
Philip Durbin 🚀 (Apr 09 2026 at 19:21):Currently, we build the production version of the guides with Jenkins. Here's how: https://guides.dataverse.org/en/6.10.1/developers/making-releases.html#build-the-guides-for-the-release
So yes, if we're going to retire Jenkins, we'll need something else.
We do already have two checks in GitHub for guides....
... but they don't deploy to https://guides.dataverse.org . Only Jenkins does that.
Ash Manda (Apr 09 2026 at 19:23):Afaik GAs should be able to cover this, I'll let you know how it goes Phil. Thank you once again, you've been a great help. Otherwise I would've been chasing my own tail.
Philip Durbin 🚀 (Apr 09 2026 at 19:24):Oh, sure, I set up all this stuff (with @Don Sizemore) that you're keen to retire. :sweat_smile:
I know where the bodies are buried! :scream:
Philip Durbin 🚀 (Apr 09 2026 at 19:35):Want me to explain the two checks for the guides? Of course you do! :sweat_smile:
Philip Durbin 🚀 (Apr 09 2026 at 19:38):@Don Sizemore created the "Guides Build Status" one in #7838. If there's a syntax problem in a PR, it will catch it. Here's how it looks as of this writing: https://github.com/IQSS/dataverse/blob/v6.10.1/.github/workflows/guides_build_sphinx.yml
Philip Durbin 🚀 (Apr 09 2026 at 19:40):@Oliver Bertuch created the "docs/readthedocs.org" one. As explained in #8650, it's for previewing doc changes in HTML format for pull requests. We talk about it here: https://guides.dataverse.org/en/6.10.1/contributor/documentation.html#quick-fix
Instead of a GitHub Action, it uses a webhook, which looks like this:
Ash Manda (Apr 21 2026 at 16:52):2026-04-21T14:30:41.0336131Z [ERROR] Tests run: 20, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 288.2 s <<< FAILURE! -- in edu.harvard.iq.dataverse.api.SearchIT
2026-04-21T14:30:41.0344827Z [ERROR] edu.harvard.iq.dataverse.api.SearchIT.testSearchFilesAndUrlImages -- Time elapsed: 8.977 s <<< FAILURE!
2026-04-21T14:30:41.0346353Z java.lang.AssertionError:
2026-04-21T14:30:41.0346993Z 1 expectation failed.
2026-04-21T14:30:41.0347594Z JSON path data.items[0].image_url doesn't match.
2026-04-21T14:30:41.0348219Z Expected: a string containing "/logo"
2026-04-21T14:30:41.0348763Z Actual: null
2026-04-21T14:30:41.0349029Z
2026-04-21T14:30:41.0349856Z at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
2026-04-21T14:30:41.0351353Z at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
2026-04-21T14:30:41.0355704Z at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
2026-04-21T14:30:41.0357117Z at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:73)
2026-04-21T14:30:41.0358655Z Apr 21, 2026 2:30:41 PM edu.harvard.iq.dataverse.api.UtilIT getRestAssuredBaseUri
2026-04-21T14:30:41.0359995Z at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:108)
2026-04-21T14:30:41.0361688Z at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:277)
2026-04-21T14:30:41.0363367Z at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure.validate(ResponseSpecificationImpl.groovy:512)
2026-04-21T14:30:41.0365065Z at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure$validate$1.call(Unknown Source)
2026-04-21T14:30:41.0367098Z at io.restassured.internal.ResponseSpecificationImpl.validateResponseIfRequired(ResponseSpecificationImpl.groovy:696)
2026-04-21T14:30:41.0368918Z at io.restassured.internal.ResponseSpecificationImpl.this$2$validateResponseIfRequired(ResponseSpecificationImpl.groovy)
2026-04-21T14:30:41.0370555Z at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
2026-04-21T14:30:41.0371725Z at java.base/java.lang.reflect.Method.invoke(Method.java:580)
2026-04-21T14:30:41.0372954Z at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43)
2026-04-21T14:30:41.0374747Z at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:198)
2026-04-21T14:30:41.0376766Z at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:62)
2026-04-21T14:30:41.0378198Z at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:185)
2026-04-21T14:30:41.0379579Z at io.restassured.internal.ResponseSpecificationImpl.body(ResponseSpecificationImpl.groovy:270)
2026-04-21T14:30:41.0380891Z at io.restassured.specification.ResponseSpecification$body$1.callCurrent(Unknown Source)
2026-04-21T14:30:41.0382200Z at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49)
2026-04-21T14:30:41.0383491Z at io.restassured.specification.ResponseSpecification$body$1.callCurrent(Unknown Source)
2026-04-21T14:30:41.0384814Z at io.restassured.internal.ResponseSpecificationImpl.body(ResponseSpecificationImpl.groovy:117)
2026-04-21T14:30:41.0386456Z at io.restassured.internal.ValidatableResponseOptionsImpl.body(ValidatableResponseOptionsImpl.java:244)
2026-04-21T14:30:41.0387859Z at edu.harvard.iq.dataverse.api.SearchIT.testSearchFilesAndUrlImages(SearchIT.java:2032)
2026-04-21T14:30:41.0389033Z at java.base/java.lang.reflect.Method.invoke(Method.java:580)
2026-04-21T14:30:41.0389846Z at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
2026-04-21T14:30:41.0390654Z at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
2026-04-21T14:17:53.1497050Z [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 1, Time elapsed: 0.549 s <<< FAILURE! -- in edu.harvard.iq.dataverse.api.MakeDataCountApiIT
2026-04-21T14:17:53.1498527Z [ERROR] edu.harvard.iq.dataverse.api.MakeDataCountApiIT.testMakeDataCountGetMetric -- Time elapsed: 0.416 s <<< FAILURE!
2026-04-21T14:17:53.1499477Z java.lang.AssertionError:
2026-04-21T14:17:53.1499851Z 1 expectation failed.
2026-04-21T14:17:53.1500200Z Expected status code <200> but was <400>.
2026-04-21T14:17:53.1500478Z
2026-04-21T14:17:53.1501183Z at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
2026-04-21T14:17:53.1502406Z at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
2026-04-21T14:17:53.1503292Z at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
2026-04-21T14:17:53.1503930Z at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:73)
2026-04-21T14:17:53.1504712Z at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:108)
2026-04-21T14:17:53.1505998Z at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:277)
2026-04-21T14:17:53.1506869Z at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure.validate(ResponseSpecificationImpl.groovy:512)
2026-04-21T14:17:53.1507702Z at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure$validate$1.call(Unknown Source)
2026-04-21T14:17:53.1508531Z at io.restassured.internal.ResponseSpecificationImpl.validateResponseIfRequired(ResponseSpecificationImpl.groovy:696)
2026-04-21T14:17:53.1509411Z at io.restassured.internal.ResponseSpecificationImpl.this$2$validateResponseIfRequired(ResponseSpecificationImpl.groovy)
2026-04-21T14:17:53.1510408Z at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
2026-04-21T14:17:53.1511244Z at java.base/java.lang.reflect.Method.invoke(Method.java:580)
2026-04-21T14:17:53.1511851Z at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43)
2026-04-21T14:17:53.1512742Z at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:198)
2026-04-21T14:17:53.1513612Z at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:62)
2026-04-21T14:17:53.1514323Z at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:185)
2026-04-21T14:17:53.1515045Z at io.restassured.internal.ResponseSpecificationImpl.statusCode(ResponseSpecificationImpl.groovy:135)
2026-04-21T14:17:53.1516000Z at io.restassured.specification.ResponseSpecification$statusCode$0.callCurrent(Unknown Source)
2026-04-21T14:17:53.1516876Z at io.restassured.internal.ResponseSpecificationImpl.statusCode(ResponseSpecificationImpl.groovy:143)
2026-04-21T14:17:53.1517730Z at io.restassured.internal.ValidatableResponseOptionsImpl.statusCode(ValidatableResponseOptionsImpl.java:89)
2026-04-21T14:17:53.1518528Z at edu.harvard.iq.dataverse.api.MakeDataCountApiIT.testMakeDataCountGetMetric(MakeDataCountApiIT.java:68)
2026-04-21T14:17:53.1519124Z at java.base/java.lang.reflect.Method.invoke(Method.java:580)
2026-04-21T14:17:53.1519506Z at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
2026-04-21T14:11:46.7179540Z [ERROR] Tests run: 62, Failures: 1, Errors: 0, Skipped: 4, Time elapsed: 377.5 s <<< FAILURE! -- in edu.harvard.iq.dataverse.api.DatasetsIT
2026-04-21T14:11:46.7206380Z [ERROR] edu.harvard.iq.dataverse.api.DatasetsIT.testUpdateMultipleFileMetadata -- Time elapsed: 4.884 s <<< FAILURE!
2026-04-21T14:11:46.7207686Z java.lang.AssertionError:
2026-04-21T14:11:46.7208133Z 1 expectation failed.
2026-04-21T14:11:46.7208502Z Expected status code <200> but was <403>.
2026-04-21T14:11:46.7208807Z
2026-04-21T14:11:46.7209497Z at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
2026-04-21T14:11:46.7210765Z at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
2026-04-21T14:11:46.7211950Z at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
2026-04-21T14:11:46.7212874Z at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:73)
2026-04-21T14:11:46.7214250Z at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:108)
2026-04-21T14:11:46.7216042Z at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:57)
2026-04-21T14:11:46.7217338Z at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:263)
2026-04-21T14:11:46.7218635Z at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:277)
2026-04-21T14:11:46.7220130Z at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure.validate(ResponseSpecificationImpl.groovy:512)
2026-04-21T14:11:46.7221653Z at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure$validate$1.call(Unknown Source)
2026-04-21T14:11:46.7223517Z at io.restassured.internal.ResponseSpecificationImpl.validateResponseIfRequired(ResponseSpecificationImpl.groovy:696)
2026-04-21T14:11:46.7225112Z at io.restassured.internal.ResponseSpecificationImpl.this$2$validateResponseIfRequired(ResponseSpecificationImpl.groovy)
2026-04-21T14:11:46.7226842Z at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
2026-04-21T14:11:46.7227881Z at java.base/java.lang.reflect.Method.invoke(Method.java:580)
2026-04-21T14:11:46.7228974Z at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43)
2026-04-21T14:11:46.7230638Z at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:198)
2026-04-21T14:11:46.7232200Z at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:62)
2026-04-21T14:11:46.7233474Z at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:185)
2026-04-21T14:11:46.7234784Z at io.restassured.internal.ResponseSpecificationImpl.statusCode(ResponseSpecificationImpl.groovy:135)
2026-04-21T14:11:46.7236305Z at io.restassured.specification.ResponseSpecification$statusCode$0.callCurrent(Unknown Source)
2026-04-21T14:11:46.7237581Z at io.restassured.internal.ResponseSpecificationImpl.statusCode(ResponseSpecificationImpl.groovy:143)
2026-04-21T14:11:46.7239003Z at io.restassured.internal.ValidatableResponseOptionsImpl.statusCode(ValidatableResponseOptionsImpl.java:89)
2026-04-21T14:11:46.7240544Z at edu.harvard.iq.dataverse.api.DatasetsIT.testUpdateMultipleFileMetadata(DatasetsIT.java:7105)
2026-04-21T14:11:46.7241519Z at java.base/java.lang.reflect.Method.invoke(Method.java:580)
2026-04-21T14:11:46.7242200Z at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
2026-04-21T14:11:46.7242856Z at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
2026-04-21T14:14:40.2467362Z [ERROR] Tests run: 5, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 65.16 s <<< FAILURE! -- in edu.harvard.iq.dataverse.api.DataRetrieverApiIT
2026-04-21T14:14:40.2469046Z Apr 21, 2026 2:14:40 PM edu.harvard.iq.dataverse.api.UtilIT getRestAssuredBaseUri
2026-04-21T14:14:40.2469845Z INFO: Base URL for tests: http://localhost:8080
2026-04-21T14:14:40.2471146Z [ERROR] edu.harvard.iq.dataverse.api.DataRetrieverApiIT.testRetrieveMyDataCollections -- Time elapsed: 11.05 s <<< FAILURE!
2026-04-21T14:14:40.2472441Z org.opentest4j.AssertionFailedError: expected: <16> but was: <17>
2026-04-21T14:14:40.2473482Z at org.junit.jupiter.api.AssertionFailureBuilder.build(AssertionFailureBuilder.java:151)
2026-04-21T14:14:40.2475008Z at org.junit.jupiter.api.AssertionFailureBuilder.buildAndThrow(AssertionFailureBuilder.java:132)
2026-04-21T14:14:40.2476363Z at org.junit.jupiter.api.AssertEquals.failNotEqual(AssertEquals.java:197)
2026-04-21T14:14:40.2477309Z at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:150)
2026-04-21T14:14:40.2478284Z at org.junit.jupiter.api.AssertEquals.assertEquals(AssertEquals.java:145)
2026-04-21T14:14:40.2479393Z at org.junit.jupiter.api.Assertions.assertEquals(Assertions.java:531)
2026-04-21T14:14:40.2480455Z at edu.harvard.iq.dataverse.api.DataRetrieverApiIT.testRetrieveMyDataCollections(DataRetrieverApiIT.java:164)
2026-04-21T14:14:40.2481466Z at java.base/java.lang.reflect.Method.invoke(Method.java:580)
2026-04-21T14:14:40.2482062Z at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
2026-04-21T14:14:40.2482645Z at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
These are the four integration tests that fail on every attempt
Don Sizemore (Apr 21 2026 at 17:34):we think we need to cobble in some more DB settings and jvm-options from Dataverse-Ansible and we should be good!
Ash Manda (Apr 27 2026 at 20:01):https://github.com/IQSS-Dataverse-Development/dataverse/actions/runs/25013696261
Ash Manda (Apr 27 2026 at 20:05):@Philip Durbin 🚀 Hi Phil! Hope you're doing good!
These three tests seem to fail no matter what, and I am not entirely sure if its just configuration or requires test code changes, think anyone can lend me a hand for a few minutes to quickly see what's up with these?
Ash Manda (Apr 27 2026 at 20:06):Just 3 tests flaking on us out of 2511, so we're almost there
Philip Durbin 🚀 (Apr 27 2026 at 20:18):I'm back, I'm back! Only 3?!? We're almost there! Good job! 
:tada:
Philip Durbin 🚀 (Apr 27 2026 at 20:19):@Ash Manda how do I see the details for the SearchIT failure, for example? :thinking:
Ash Manda (Apr 27 2026 at 20:20):Oops my bad. https://github.com/IQSS-Dataverse-Development/dataverse/runs/73261506538
Philip Durbin 🚀 (Apr 27 2026 at 20:20):Expected: a string containing "/logo"
Actual: null
Thanks!
Philip Durbin 🚀 (Apr 27 2026 at 20:44):@Ash Manda can we see if this helps? https://github.com/IQSS-Dataverse-Development/dataverse/pull/4
Ash Manda (Apr 27 2026 at 20:54):will start the workflow soon
Ash Manda (Apr 27 2026 at 21:04):Hi Phil, could you please resubmit PR directly to the develop branch?
Ash Manda (Apr 27 2026 at 21:04):then the workflow can run
Philip Durbin 🚀 (Apr 27 2026 at 21:06):Sure, here you go: https://github.com/IQSS-Dataverse-Development/dataverse/pull/5
Ash Manda (Apr 27 2026 at 21:13):Running: https://github.com/IQSS-Dataverse-Development/dataverse/actions/runs/25019823115
Ash Manda (Apr 27 2026 at 21:45):Hi Phil, it still seems to fail. But please no pressure, and no rush on this, Jenkins still works, we can figure this out slowly :smile:
Philip Durbin 🚀 (Apr 27 2026 at 22:04):I think my PR fixed SearchIT. We're now down to two failures: https://github.com/IQSS-Dataverse-Development/dataverse/runs/73282463436
Ash Manda (Apr 27 2026 at 22:17):Philip Durbin 🚀 said:
I think my PR fixed SearchIT. We're now down to two failures: https://github.com/IQSS-Dataverse-Development/dataverse/runs/73282463436
oops looked at the wrong run, you're absolutely right, tysm!
Philip Durbin 🚀 (Apr 27 2026 at 23:18):@Oliver Bertuch any idea why https://github.com/IQSS-Dataverse-Development/dataverse/blob/ddb6da543df74b9d6583de81ed450772784b9995/src/test/java/edu/harvard/iq/dataverse/util/MailSessionProducerIT.java#L258 is failing?
Philip Durbin 🚀 (Apr 27 2026 at 23:19):I'm not sure why the actual was 17 instead of 16 at https://github.com/IQSS-Dataverse-Development/dataverse/blob/ddb6da543df74b9d6583de81ed450772784b9995/src/test/java/edu/harvard/iq/dataverse/api/DataRetrieverApiIT.java#L164
Don Sizemore (Apr 28 2026 at 12:09):@Philip Durbin 🚀 Ash noted that there was a root element, but I'm curious why these tests pass in a VM but fail in containers. One difference is the underlying OS (Rocky->Ubuntu) which has posed charset problems here and there over the years.
Oliver Bertuch (Apr 28 2026 at 12:24):Hey folks, just catching up on this.
Oliver Bertuch (Apr 28 2026 at 12:24):Should I still look into this or are y'all good?
Oliver Bertuch (Apr 28 2026 at 12:25):I am pretty sure that I (who wrote the MailSessionProducerIT) did test with a container at the time.
Oliver Bertuch (Apr 28 2026 at 12:27):Also happy do some pair programming over VC if that helps.
Philip Durbin 🚀 (Apr 28 2026 at 12:38):@Oliver Bertuch yeah, if you could look at MailSessionProducerIT, I'd appreciate it
Oliver Bertuch (Apr 28 2026 at 12:43):Hmm the MaSePrIT is no API test, but a real integration test. So it should already run as part of the regular jobs on IQSS. Is it failing over there, too?
Oliver Bertuch (Apr 28 2026 at 12:45):Nope, all good over there. (See https://github.com/IQSS/dataverse/actions/runs/25045635432/job/73360393320 for an example)
Oliver Bertuch (Apr 28 2026 at 12:49):I'm wondering why the test is even executed. While the original Jenkins job used mvn test (surefire), you've opted to use failsafe now (mvn verify). It looks like failsafe picks up the actual integration tests and not just the API tests as intended. (This doesn't answer the why it fails, but still)
Oliver Bertuch (Apr 28 2026 at 12:50):To me it looks like the "-Dtest=..." is a noop for failsafe, looking at their documentation. https://maven.apache.org/surefire/maven-failsafe-plugin/verify-mojo.html
Oliver Bertuch (Apr 28 2026 at 12:54):Yeah, for failsafe it has to be "-Dit.test", not "-Dtest".
Oliver Bertuch (Apr 28 2026 at 12:54):We really should use <group> instead and put a @Tag on the test classes.
Ash Manda (Apr 28 2026 at 13:08):Omg amazing catch @Oliver Bertuch I did not realize surefire, and failsafe accepted different flags
Ash Manda (Apr 28 2026 at 13:08):pushed the fix
Ash Manda (Apr 28 2026 at 13:11):you might've just saved me a whole day of banging my head on the wall
Oliver Bertuch (Apr 28 2026 at 13:12):I did? Why? :blushing:
Oliver Bertuch (Apr 28 2026 at 13:14):OK moving on... I can't find a reasonable explanation why this wouldn't work so far. Educated guess: the setup somehow has some more system properties being set.
Oliver Bertuch (Apr 28 2026 at 13:14):These real integration tests do not even touch a single Dataverse container, they just run within the local JVM
Oliver Bertuch (Apr 28 2026 at 13:15):Should we pursue this further or do we stop investigating to concentrate on the API tests?
Oliver Bertuch (Apr 28 2026 at 13:18):@Ash Manda do you mind switching of the unit tests? -DskipUnitTests
Ash Manda (Apr 28 2026 at 13:22):on it
Ash Manda (Apr 28 2026 at 13:23):done
Oliver Bertuch (Apr 28 2026 at 13:24):BTW this makes no sense. - name: Build Dataverse containers via Maven
run: |
set -euo pipefail
mvn clean package -Pct -P all-unit-tests \
-DskipUnitTests=true \
-Djacoco.skip=false \
-Dmaven.test.failure.ignore=true
Oliver Bertuch (Apr 28 2026 at 13:25):On the one hand you are including via profile all the unit tests
Oliver Bertuch (Apr 28 2026 at 13:25):But then you skip them all in package
Oliver Bertuch (Apr 28 2026 at 13:25):What will happen anyway, because that's default for profile ct
Oliver Bertuch (Apr 28 2026 at 13:25):But the comment says "force unit tests to run"
Oliver Bertuch (Apr 28 2026 at 13:25)::thinking:
Ash Manda (Apr 28 2026 at 13:26):I switched them off rn, the tests skip by default
Ash Manda (Apr 28 2026 at 13:26):cause you asked me to switch them off?
Oliver Bertuch (Apr 28 2026 at 13:26):hehehe in mvn verify :wink:
Oliver Bertuch (Apr 28 2026 at 13:26):No point in running the unit tests there
Oliver Bertuch (Apr 28 2026 at 13:27):At least for now they are only creating loads of clutter
Ash Manda (Apr 28 2026 at 13:27):mvn verify doesn't run the unit tests
Ash Manda (Apr 28 2026 at 13:27):only the integration tests
Oliver Bertuch (Apr 28 2026 at 13:27):Nope
Oliver Bertuch (Apr 28 2026 at 13:27):mvn verify means mvn test phase runs, too
Oliver Bertuch (Apr 28 2026 at 13:28):https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html
Ash Manda (Apr 28 2026 at 13:28):give the workflow a few minutes
Ash Manda (Apr 28 2026 at 13:28):you'll see the two thousand tests or so disappear
Oliver Bertuch (Apr 28 2026 at 13:29):Are we looking at the same thing? I'm looking at https://github.com/IQSS-Dataverse-Development/dataverse/actions/runs/25055406452/workflow
Oliver Bertuch (Apr 28 2026 at 13:29):Job is currently running
Oliver Bertuch (Apr 28 2026 at 13:30):THere is no skipUnitTests on https://github.com/IQSS-Dataverse-Development/dataverse/actions/runs/25055406452/workflow#L209-L217, so Maven will happily go through the test phase of the life cycle and execute the unit tests.
Ash Manda (Apr 28 2026 at 13:30):my bad, I was looking at a different workflow
Oliver Bertuch (Apr 28 2026 at 13:31):test phase -> surefire, picks up *Test.java
verify phase -> failsafe, picks up *IT.java
Oliver Bertuch (Apr 28 2026 at 13:41):Oh oh... No more tests detected...
Ash Manda (Apr 28 2026 at 13:43):I genuinely have no idea what just happened
Oliver Bertuch (Apr 28 2026 at 13:43):I may have an idea.
Oliver Bertuch (Apr 28 2026 at 13:44):Yeah I suspect the default group create a problem here
Oliver Bertuch (Apr 28 2026 at 13:45):The it.test is an additional filter, not overriding the group
Oliver Bertuch (Apr 28 2026 at 13:45):<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<configuration>
<groups>${it.groups}</groups>
<!--
Note: The @{} syntax here is used for late variable binding.
Otherwise we would not be able to override the defaults in <properties> with prepare-agent above.
See also https://www.jacoco.org/jacoco/trunk/doc/prepare-agent-mojo.html
It's ignored by IntelliJ argLine parsing.
-->
<argLine>@{failsafe.jacoco.args} ${argLine}</argLine>
<skip>${skipIntegrationTests}</skip>
<systemPropertyVariables>
<postgresql.server.version>${postgresql.server.version}</postgresql.server.version>
</systemPropertyVariables>
</configuration>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>
So far we only had the "real integration tests" in scope for this, as so far, the API tests were always executed by surefire as part of mvn test -Dtest=...
Ash Manda (Apr 28 2026 at 13:47):we need all the ~2500 Jenkins tests running in this workflow right?
Oliver Bertuch (Apr 28 2026 at 13:48):That depends.
Ash Manda (Apr 28 2026 at 13:54):let me poke around a little with the workflow
Philip Durbin 🚀 (Apr 28 2026 at 13:55):Should we carve out some time to talk about testing during tech hours this afternoon? Thanks for all the hard work!
Oliver Bertuch (Apr 28 2026 at 13:55):I did a lot of things already to split up the Jacoco bits
Oliver Bertuch (Apr 28 2026 at 13:55):So we can just run the API tests
Oliver Bertuch (Apr 28 2026 at 13:55):And then merge the Jacoco reports for unit, integration and API tests
Oliver Bertuch (Apr 28 2026 at 13:56):This might make things easier and faster, as we don't have to keep repeating all of them
Oliver Bertuch (Apr 28 2026 at 13:56):We already have the unit and integration testing job in maven_unit_test.yml
Oliver Bertuch (Apr 28 2026 at 13:56):Which BTW then hands over the container workflow to build the containers
Oliver Bertuch (Apr 28 2026 at 13:57):We should probably integrate these pipelines more
Oliver Bertuch (Apr 28 2026 at 13:57):But that is a good next step
Oliver Bertuch (Apr 28 2026 at 13:57):For now, we'd only need to decide how we wanna go about executing which tests
Ash Manda (Apr 28 2026 at 14:32):If I can figure out a way to get those 2 tests to pass in this workflow, then honestly we wouldn't need to do much else in the way of combined reports or touching other workflows, it would make life easier for everyone. As I understand, there's no hurry to replace Jenkins immediately so I'm sure eventually we can get all of them to pass in a single workflow :smile:
I'll look at Jenkins, and the already existing workflows to see if there's a config mismatch or something.
Ash Manda (Apr 28 2026 at 17:23):down to one failing test
Ash Manda (Apr 28 2026 at 17:23):https://github.com/IQSS-Dataverse-Development/dataverse/actions/runs/25065778729
Philip Durbin 🚀 (Apr 28 2026 at 17:33):That pesky DataRetrieverApiIT. Hmm. :thinking:
Philip Durbin 🚀 (Apr 28 2026 at 17:34):// User1 gets the list of Dataverses/Collections it has access to
retrieveMyCollectionListResponse = UtilIT.retrieveMyCollectionList(User1ApiToken, null);
retrieveMyCollectionListResponse.prettyPrint();
// The count should show the list size to be User1's + Root Dataverse count
items = retrieveMyCollectionListResponse.getBody().jsonPath().getList("data.items");
assertEquals(rootCount + user1DataverseCount, items.size());//BOOM!💣
Can we just put a greater than or equal to there
Ash Manda (Apr 28 2026 at 17:37):Like y'all do for the superuser
Ash Manda (Apr 28 2026 at 17:38):I mean if there's more data collections than expected, that still means the test is valid right?
Philip Durbin 🚀 (Apr 28 2026 at 17:54):mvn test -Dtest=DataRetrieverApiIT#testRetrieveMyDataCollections works on my machine :trademark:
Philip Durbin 🚀 (Apr 28 2026 at 17:58):@Steven Winship you wrote these tests. Any ideas?
Philip Durbin 🚀 (Apr 28 2026 at 18:03):The test is failing because user1 has access to an extra collection. It created 15 and has access to the root collection. So it should have access to 16 collections but it has access to 17 collections for some reason. 
Philip Durbin 🚀 (Apr 28 2026 at 18:05):This seems to be the extra collection: "id": 449
Ash Manda (Apr 28 2026 at 18:05):I'm trying to comb through the logs for: dv89758e93 which is id 449
Philip Durbin 🚀 (Apr 28 2026 at 18:05):I say this because the 15 that were created for the test have higher numbers and were all in sequence, from 727 to 741.
Philip Durbin 🚀 (Apr 28 2026 at 18:06):@Ash Manda this container is no longer up, right? I assume it gets destroyed at the end.
Ash Manda (Apr 28 2026 at 18:06):2026-04-28T17:07:12.390204076Z [#|2026-04-28T17:07:12.390+0000[1;93m|WARNING|[0mPayara 7.2026.2|[1;94medu.harvard.iq.dataverse.DataverseServiceBean|[0m_ThreadID=267;_ThreadName=http-thread-pool::http-listener-1(37);_TimeMillis=1777396032390;_LevelValue=900;|
2026-04-28T17:07:12.390228021Z Unable to find a single dataverse using alias "dv89758e93": jakarta.persistence.NoResultException: getSingleResult() did not retrieve any entities.|#]
Philip Durbin 🚀 said:
Ash Manda this container is no longer up, right? I assume it gets destroyed at the end.
yep, it gets destroyed every single time, but I make sure every single execution log is saved
Philip Durbin 🚀 (Apr 28 2026 at 18:07):Can I see that log, please?
Ash Manda (Apr 28 2026 at 18:07): Philip Durbin 🚀 (Apr 28 2026 at 18:09):Thanks. This is available online as well?
Ash Manda (Apr 28 2026 at 18:10):yep
Ash Manda (Apr 28 2026 at 18:10):https://github.com/IQSS-Dataverse-Development/dataverse/actions/runs/25065778729
Philip Durbin 🚀 (Apr 28 2026 at 18:14):Well, it's reproducible. For the earlier run it was also id 449, like the current run. :smile: But why? :thinking: What's different about running this in GitHub Actions vs Jenkins and local? :thinking:
Ash Manda (Apr 28 2026 at 18:15):@Don Sizemore pointed out that it might be that Solr is not getting enough time to index because thread.sleep is flaky on GitHub Actions, so I increased the timeout from 4000 to 40000 just to test if that's the issue
Philip Durbin 🚀 (Apr 28 2026 at 18:16):@Ash Manda when I click "raw output" I see the output for just that one failing test, which is great. Is it possible to also get all the output in one file?
Philip Durbin 🚀 (Apr 28 2026 at 18:17):Ash Manda said:
Solr
Yeah, it's calling into /api/mydata/retrieve/collectionList which is powered by Solr, I'm pretty sure.
Philip Durbin 🚀 (Apr 28 2026 at 18:19):@Ash Manda by the way, I think you're solving this issue for us, which is huge:
Let contributors know why tests are failing #9916
I just tried an incognito window and I can see all the juicy details. :lemon:
Ash Manda (Apr 28 2026 at 18:20):Philip Durbin 🚀 said:
Ash Manda when I click "raw output" I see the output for just that one failing test, which is great. Is it possible to also get all the output in one file?
It probably is, let me look into it.
Ash Manda (Apr 28 2026 at 18:24):https://github.com/IQSS-Dataverse-Development/dataverse/actions/runs/25065778729/job/73432388515
If you go to the failed job, click the gear icon next to the workflow name, you will see an option named download log archive
Ash Manda (Apr 28 2026 at 18:25):That contains the full execution log
Philip Durbin 🚀 (Apr 28 2026 at 18:28):12.6 MB. Thanks!
Philip Durbin 🚀 (Apr 28 2026 at 18:32):This is a little sus to me...
2026-04-28T17:07:12.7763138Z {
2026-04-28T17:07:12.7763303Z "id": 334,
2026-04-28T17:07:12.7763536Z "assignee": ":authenticated-users",
2026-04-28T17:07:12.7763863Z "assigneeName": "Anyone with a Dataverse account",
2026-04-28T17:07:12.7764158Z "roleId": 5,
2026-04-28T17:07:12.7764383Z "roleName": "Dataset Creator",
2026-04-28T17:07:12.7764751Z "roleDescription": "A person who can add datasets within a dataverse.",
2026-04-28T17:07:12.7765134Z "_roleAlias": "dsContributor",
2026-04-28T17:07:12.7765388Z "definitionPointId": 449,
2026-04-28T17:07:12.7765658Z "definitionPointName": "dv89758e93",
2026-04-28T17:07:12.7765941Z "definitionPointType": "Dataverse"
2026-04-28T17:07:12.7766189Z }
... I wonder why all authenticated users have access to id 449. :thinking:
Ash Manda (Apr 28 2026 at 18:35):I had the exact same thought, but if that were not the intended behavior then all tests should've thrown one extra. Only user1's does.
Philip Durbin 🚀 (Apr 28 2026 at 18:36):Yeah, it's still a puzzle... what's different about this env? :thinking:
Philip Durbin 🚀 (Apr 28 2026 at 18:37):@Ash Manda we've seen DataRetrieverApiIT fail twice for sure. Has it failed other times? Does it always fail? Do you want to run it again just for fun? Can we run just that test class in isolation? I'm just grasping at straws here. :smile:
Ash Manda (Apr 28 2026 at 18:37):it has been failing since the first time I wrote the workflow
Philip Durbin 🚀 (Apr 28 2026 at 18:37):boo
Ash Manda (Apr 28 2026 at 18:38):I even thought it might be a multi-threaded race condition, and forced the entire suite to run single-core single-thread, but that did not fix it either
Philip Durbin 🚀 (Apr 28 2026 at 18:38):bleh
Philip Durbin 🚀 (Apr 28 2026 at 18:39):@Steven Winship can we just delete that test? :crazy:
Steven Winship (Apr 28 2026 at 18:46):You can comment it out and see if the other tests pass
Ash Manda (Apr 28 2026 at 18:47):yep, everything except that does pass
Philip Durbin 🚀 (Apr 28 2026 at 18:47):or add @Disabled
Steven Winship (Apr 28 2026 at 18:47):@Disabled
Philip Durbin 🚀 (Apr 28 2026 at 18:47):that way we can punt on this until another day :sweat_smile:
Philip Durbin 🚀 (Apr 28 2026 at 18:48):@Ash Manda do you want a PR to add @Disabled or have you got it?
Steven Winship (Apr 28 2026 at 18:48):Is it always the same alias/id? or is it a newly created dataverse each time?
Philip Durbin 🚀 (Apr 28 2026 at 18:49):Philip Durbin 🚀 said:
Well, it's reproducible. For the earlier run it was also id 449, like the current run. :smile: But why? :thinking: What's different about running this in GitHub Actions vs Jenkins and local? :thinking:
@Steven Winship for two runs, at least, the id was 449. ^^
Steven Winship (Apr 28 2026 at 18:50):Then that tells me we don't blow away the DB for each run. As we should
Steven Winship (Apr 28 2026 at 18:50):Most tests do not clean up
Ash Manda (Apr 28 2026 at 18:50):for every failed run, it is the same id 449 although the alias changes
Steven Winship (Apr 28 2026 at 18:51):Makes sense since the dataverses are created in the same order
Steven Winship (Apr 28 2026 at 18:51):They would have the same id but not alias
Ash Manda (Apr 28 2026 at 18:51):changing assertsequal to greater than or equal also fixes it. I think the same is done for superuser
Ash Manda (Apr 28 2026 at 18:53):Philip Durbin 🚀 said:
Ash Manda do you want a PR to add
@Disabledor have you got it?
I got it, but the workflow ain't in prod rn, so no hurry yet?
Steven Winship (Apr 28 2026 at 18:53):Since we probably can't get it to fail locally changing it to >= is fine with me
Steven Winship (Apr 28 2026 at 18:54):It's better than skipping it
Philip Durbin 🚀 (Apr 28 2026 at 18:59):@Ash Manda right, no hurry but please go ahead with any of the suggestions above! :smile: I want to see it go green! :green_heart:
Ash Manda (Apr 28 2026 at 20:59):Tests: Green
https://github.com/IQSS-Dataverse-Development/dataverse/actions/runs/25075466484
Ash Manda (Apr 28 2026 at 21:00):Had to disable the whole test, unfortunately
Ash Manda (Apr 28 2026 at 21:00):but the rest 2497 of them run like butter
Philip Durbin 🚀 (Apr 28 2026 at 21:00):Green? Yay! :tada:
Don't worry, @Steven Winship will fix it later. :crazy:
Ash Manda (Apr 28 2026 at 21:01):now its upto IQSS team to decide if this workflow is a good substitute for what Jenkins does, or if something is missing
Philip Durbin 🚀 (Apr 28 2026 at 21:02):I mean, we might as well merge it, right? Belt and suspenders, for now! Are you planning to make a pull request?
Ash Manda (Apr 28 2026 at 21:02):yep opening one right now
Ash Manda (Apr 28 2026 at 21:02):after I clean up the commit history
Ash Manda (Apr 28 2026 at 21:02):or is it okay even like this?
Ash Manda (Apr 28 2026 at 21:05):actually the PR has to come from https://github.com/uncch-rdmc/dataverse and I don't have perms there to create a new branch and make a PR, I'll do it tomorrow, and clean up the commit history at the same time
Ash Manda (Apr 28 2026 at 21:06):Massive thank you to all of you for helping me work through this even though you're busy and have other responsibilities
Philip Durbin 🚀 (Apr 28 2026 at 21:06):Happy to help! Great work! So exciting! :tada:
Philip Durbin 🚀 (May 26 2026 at 13:26):@Ash Manda I'm looking at https://github.com/IQSS/dataverse/pull/12368 and jobs are failing. Can you please take a look?
I'm re-running https://github.com/IQSS/dataverse/actions/runs/25817871801?pr=12368 manually to see if I can get it to pass.
Ash Manda (May 26 2026 at 13:39):Hi Phil. Good morning and welcome back! There's no way they'll pass, I couldn't rebase with develop fast enough, let me rebase and force push, that should fix it.
Philip Durbin 🚀 (May 26 2026 at 13:40):Sounds good. For the job above, it was a transient GitHub problem.
Philip Durbin 🚀 (May 26 2026 at 14:41):@Ash Manda it looks like there's a failure at https://github.com/IQSS/dataverse/actions/runs/26452806612/job/77879134373?pr=12368
Ash Manda (May 26 2026 at 14:54):That's... surprising to me, I just verified it passed at: https://github.com/uncch-rdmc/dataverse/actions/runs/26452804941
It tests against the same develop branch. I'm looking into it.
Ash Manda (May 26 2026 at 15:02):They ran against the exact same commit too. Might have to just re-trigger it?
Philip Durbin 🚀 (May 26 2026 at 15:35):Sure, I just kicked it off again:
Philip Durbin 🚀 (May 26 2026 at 15:35):Here it goes: https://github.com/IQSS/dataverse/actions/runs/26452806612?pr=12368
Ash Manda (May 26 2026 at 15:41):thank you so much! Fingers crossed that it doesn't flake again, GitHub in general and especially Actions has been giving me hell for the past few weeks
Philip Durbin 🚀 (May 26 2026 at 15:44):Actually, we just had a little chat at the end of standup about Jenkins. Obviously, we don't want UNC to keep paying for it but it does have some nice attributes:
Philip Durbin 🚀 (May 26 2026 at 15:44):Maybe we can talk about it at tech hours some day. Mostly it was me, @Leo Andreev @Juan Pablo Tosca Villanueva and @Steven Winship talking about it. Steven mentioned some experience with GitLab CI.
Oliver Bertuch (May 26 2026 at 15:52):Maybe have another look at Dagger? You can now use Java to write pipelines, too
Philip Durbin 🚀 (May 26 2026 at 16:01):Right. See also #dev > From Jenkins to GHA to Dagger?
Ash Manda (May 26 2026 at 16:13):Depending on how and where the IQSS team wants it, I can write it in Jenkins or Gitlab CI or Woodpecker or even Codeberg Forgejo. At the end of the day, it's just some YAML actions running on a container. The actual hard work of getting the containers up was done by y'all, so I'm just building convenience on top of it.
My boss @Don Sizemore wants me to retire Jenkins, that's my ultimate goal, and I'm happy to write the testing in any tool that helps IQSS achieve that without incurring costs!
Ash Manda (May 26 2026 at 16:15):Also, the tests passed so yay!
Oliver Bertuch (May 26 2026 at 16:19):That's what I'm saying: using Dagger we may be able to define pipelines in a much more reusable way. We don't rely on the YAML, but write code. Define pipelines once, run anywhere. The YAML will still be necessary but only for triggering dagger.
Ash Manda (May 26 2026 at 16:23):If I understand Dagger's docs correctly, it still has to run on some infrastructure (Dagger Cloud or local or own cloud) at the end of the day, it's not very different from Jenkins when it comes to costs? Or am I understanding this incorrectly?
Oliver Bertuch (May 26 2026 at 17:11):Yes, any CI needs to run somewhere.
Oliver Bertuch (May 26 2026 at 17:12):But - defining your pipelines should be opaque to the runner involved if you aim for sovereignty
Oliver Bertuch (May 26 2026 at 17:13):You may write a bunch of scripts and run those on some infrastructure as CI.
Oliver Bertuch (May 26 2026 at 17:13):The point is: at some point, you will end up with a complex system, taking care of everything.
Oliver Bertuch (May 26 2026 at 17:14):So instead if writing all of this ourselves, all pipeline engineering was done using a services YAML config file
Oliver Bertuch (May 26 2026 at 17:14):And these things become very brittle once things get more complex
Oliver Bertuch (May 26 2026 at 17:15):A good example is the container maintenance. I had to write my own scripts to capture the complexity involved
Oliver Bertuch (May 26 2026 at 17:17):So using a framework gives you more control over the definition. Plus, you get to run your pipelines anywhere the same way. As the pipelines are defined within the framework, you can execute the framework within some runner or on your machine.
Oliver Bertuch (May 26 2026 at 17:17):So moving from Jenkins to Github Actions to whatever becomes much less tedious, as you only need to wire dagger to the runner execution, not the whole workflow.
Philip Durbin 🚀 (May 26 2026 at 17:34):Ash Manda said:
Also, the tests passed so yay!
I just merged #12368! Thanks again, @Ash Manda!
Oliver Bertuch (May 26 2026 at 17:43):However, before we start investing into Dagger, it might be a better idea to invest into Testcontainers. There is a huge overlap between both projects, but Testcontainers probably will feel more native to JUnit.
Ash Manda (May 26 2026 at 17:47):I agree with all the technical points. And honestly, y'all have been working on this project for years, know all the pain points, and have a good amount of context. I really wouldn't know enough to make any technical recommendations to you or Phil, so I'm very happy to write stuff and experiment in Dagger if you want me to.
BUT (there's always a but) that Jenkins hardware in our data center, it might have its power cable pulled soon enough, and I want to make sure we aren't left hanging when that happens. And the entire reason UNC wants to retire Jenkins isn't because UNC doesn't like Jenkins, but solely because of costs. No matter where we move to, someone has to foot the costs for the CI/CD whether IQSS or GDCC or UNC. With GitHub Actions, the benefit is the Microsoft foots the cost for public CI/CD unlimited and that doesn't appear to change anytime soon
Oliver Bertuch (May 26 2026 at 18:20):Yeah, I'm not saying we should quit on GH actions. Just make things more portable if we need to jump ship. It's great we are now moving away from Jenkins, which is a great first step!!!
Oliver Bertuch (May 26 2026 at 18:20):Thanks for putting all the effort into this and making it happen! :folded_hands:
Ash Manda (May 26 2026 at 18:26):I should be thanking you both for helping me so much through all this, being immensely co-operative, and understanding. I'm always happy to help
Ash Manda (May 26 2026 at 20:29):Also Oliver, I remember you telling me in a meeting that the ideal goal would be to run the full unit tests, api tests, and integration tests using a single mvn verify command in a single-go and discard the integration tests txt file. If I remember correctly during my weeks of testing, we are exactly one single failing test away from accomplishing that in this workflow, and that test is MailSessionProducerIT.
Philip Durbin 🚀 (May 26 2026 at 20:33):/me runs git blame MailSessionProducerIT.java
Oliver Bertuch (May 26 2026 at 20:34):Yeah. That's what I've been referring to when saying it might be better to invest in Testcontainers rather than Dagger.
Oliver Bertuch (May 26 2026 at 20:34):If executing the tests is as easy as running a single Maven command, the required amount of pipeline engineering is relatively low.
Ash Manda (May 26 2026 at 20:37):After porting the build guides over, I will put my focus towards making everything run in a single step. Long term, it should make things easier hopefully. Then I'll come back with more specific details about what's failing and why.
Oliver Bertuch (May 26 2026 at 20:38):If you need any input or pointers about moving the guides, let me know (separate thread, maybe in #docs ?)
Philip Durbin 🚀 (May 26 2026 at 20:38):@Ash Manda are you still working on #containers > Docker mode when spinning up EC2 instances as well? (We can talk more in that topic!) :sweat_smile:
Philip Durbin 🚀 (May 26 2026 at 20:43):Also, to return to the Jenkins topic, in #12368 we disabled a test called DataRetrieverApiIT#testRetrieveMyDataCollections but in #12372 we are re-enabling it. I like how we can see the "+1 test" at https://github.com/IQSS/dataverse/pull/12372#issuecomment-4547662917 like this:
Philip Durbin 🚀 (May 26 2026 at 20:46):It's similar to what we can see on the Jenkins side (-1 skipped and +1 diff):
Philip Durbin 🚀 (May 26 2026 at 20:46):On Jenkins I can drill down and see the test has been fixed:
Philip Durbin 🚀 (May 26 2026 at 20:49):On the GitHub Actions side I'm not sure how to drill down to a particular test. If I go to https://github.com/IQSS/dataverse/actions/runs/26466698096/job/77928873945?pr=12372 for example I see the tests taking 28+ minutes and there's a ton of output under there, but I don't think it's organized at all:
Philip Durbin 🚀 (May 26 2026 at 20:50):I mean, in practice, passing tests aren't so interesting. When there's a failing test, I'm pretty sure I saw how GitHub Actions will point us directly to it, which is good.
Philip Durbin 🚀 (May 26 2026 at 21:01):Sweet, @Victoria Lubitch merged the new GitHub Action into her pull request at https://github.com/IQSS/dataverse/pull/11507 . Hopefully, she'll be able to see for herself if there are any API test failures!
Oliver Bertuch (May 26 2026 at 21:06):It might be interesting to use a Junit extension to inject the Github Action Logs Grouping.
Ash Manda (May 26 2026 at 21:07):There should be a full dashboard that drills down into every single test passed or failed, that's the reason why I liked this GitHub Action so much. Suddenly I'm not able to pinpoint the failing test. I am looking into it
Ash Manda (May 26 2026 at 21:09):This is an example: https://github.com/uncch-rdmc/dataverse/runs/77951732414
Ash Manda (May 26 2026 at 21:10):I am not working on the frontend container docker mode because there have been more immediate priority tasks assigned to me, but its in the back of my mind
Ash Manda (May 26 2026 at 21:17):Well, this is some GitHub bs, we need to write another workflow because its coming from a fork apparently.
Technically you can download the maven reports and see the exact failure, but yeah annoying
Oliver Bertuch (May 26 2026 at 21:18):No access to secrets or privileged access to anything this way.
Oliver Bertuch (May 26 2026 at 21:18):(From a fork with trigger PR)
Oliver Bertuch (May 26 2026 at 21:19):Only aroubd that: pull_request_target trigger plus merging or using a comment command as trigger.
Oliver Bertuch (May 26 2026 at 21:19):Command being the more secure variant
Ash Manda (May 27 2026 at 01:57):I just remembered, there's a pretty clean workaround. They can open a PR from the feature branch in their fork to their develop branch in their fork. This is how I do it. And I think it's a clean way to do it without any security struggles
Philip Durbin 🚀 (May 29 2026 at 13:35):Philip Durbin 🚀 said:
Actually, we just had a little chat at the end of standup about Jenkins. Obviously, we don't want UNC to keep paying for it but it does have some nice attributes:
- open source
- we are in control of the service and data
- not locked into GitHub
One idea we floated is that IQSS could start hosting Jenkins.
Philip Durbin 🚀 (May 29 2026 at 13:36):Or rather, return to hosting Jenkins, I suppose.
Philip Durbin 🚀 (May 29 2026 at 13:36):Would it be possible to commit the latest configs to https://github.com/IQSS/dataverse-jenkins ? For safe keeping.
Oliver Bertuch (May 29 2026 at 13:41):When we want to go back to self hosting, have you considered looking at alternatives to Jenkins like Woodpecker, Tekton, etc?
Philip Durbin 🚀 (May 29 2026 at 13:41):I haven't heard of those.
Oliver Bertuch (May 29 2026 at 13:44):Especially Woodpecker (or it's origin, Drone) seems to be worth a look. Jenkins is one big fish to ship around.
Philip Durbin 🚀 (May 29 2026 at 13:46):Well, it's good to have options if we need to get away from GitHub Actions in the future for whatever reason.
Oliver Bertuch (May 29 2026 at 13:46):So why start running a Jenkins server now???
Philip Durbin 🚀 (May 29 2026 at 13:47):Start? :smile:
Oliver Bertuch (May 29 2026 at 13:47):"One idea we floated is that IQSS could start hosting Jenkins."
Philip Durbin 🚀 (May 29 2026 at 13:48):ok, "keep Jenkins running in parallel for a while"
Philip Durbin 🚀 (May 29 2026 at 13:53):belt and suspenders
Ash Manda (May 29 2026 at 13:53):If IQSS wants to run Jenkins in parallel, all for the better! Like Phil once told me, belt and suspenders :smile:
Ash Manda (May 29 2026 at 13:53):Jinx!
Philip Durbin 🚀 (May 29 2026 at 13:53):jinx!
Oliver Bertuch (May 29 2026 at 13:54):I still don't get it. For what use cases would you like to keep Jenkins around?
Philip Durbin 🚀 (May 29 2026 at 13:54):It wasn't my suggestion.
Oliver Bertuch (May 29 2026 at 13:55):You as in "IQSS"
Philip Durbin 🚀 (May 29 2026 at 13:55):I can't speak for everyone.
Oliver Bertuch (May 29 2026 at 13:56):I wasn't suggesting that. Merely wondering what motivated the idea to have IQSS run a Jenkins server. "In parallel" to what? Is there some build mechanics aside from the guides still missing to be moved?
Philip Durbin 🚀 (May 29 2026 at 13:57):This thread is about CI, not guides.
Ash Manda (May 29 2026 at 13:57):Well Jenkins does two main things. The API tests, and docs push. The API tests, there's a workflow sitting in the repo rn. The guides, I can assist migration depending on where the guides server goes. They're both interconnected. That informs our strategy.
Philip Durbin 🚀 (May 29 2026 at 14:16):yes, exactly
Last updated: May 30 2026 at 09:11 UTC