Ash Manda (Aug 31 2026 at 19:21): Ash Manda (Aug 31 2026 at 19:21):Hello IQSS team. Good afternoon!
I've been looking at the recent issues that have been popping up on the GitHub issues page. Very specifically one, regarding default custom licenses preventing dataset creation. This is a good example of a bug/case that can be turning into a test preventing this from ever happening again. I discussed this very briefly with Phil. I want to help in whatever way I can to prevent the team from firefighting bugs.
I strongly believe that having a pipeline for bug -> fix -> test will help make Dataverse resilient over time and make it anti-fragile. I wasn't entirely sure on where to go from having the idea, and was hoping that the IQSS team could provide some recommendations on whether this is a good idea and if it is, where can we go from here? Maybe a board/doc that lists these bugs that are suitable for being turned into tests? In an ideal world, I would want to establish a repeatable process for resolving these issues, writing some replication bullets, and put them in a backlog for test implementations.
Philip Durbin 🚀 (Sep 01 2026 at 17:40):@Ash Manda sounds like a great topic for tech hours! Would you like to lead a discussion on this some day?
Also, did you mean to have JSF in the Zulip topic? The approach sound reusable across a variety of contexts.
Ash Manda (Sep 01 2026 at 19:15):I would love to! I initially only envisioned this being used for the JSF tests, but if y'all can find more use for the process, then I'm more than glad to pitch in. That's why I had JSF in the topic.
Philip Durbin 🚀 (Sep 01 2026 at 20:18):Cool. I added it to the agenda for next week but it can be whenever.
Ash Manda (Sep 08 2026 at 15:46):Just to re-iterate on the idea that I had: Whenever a bug gets fixed right, especially critical bugs that people encounter in production, if a test isn't included with the PR, there's no way to know if that bug will just regress and reoccur in the future. So I wanted to come up with some kind of frictionless process, where critical bugs can be put into a backlog so that someone (for JSF, me) can write tests and make sure it doesn't occur again. Or request tests for critical paths that don't exist yet.
The Google Doc that I'm maintaining on UNC's behalf: https://docs.google.com/document/d/1AFNDZvNWDa-TG5G6Mu9AiwBcjmfEiyMsyFgabostj4w/edit?usp=sharing
UNC's fork where I'm writing these new tests: https://github.com/uncch-rdmc/dataverse-jsf-tests
Philip Durbin 🚀 (Sep 08 2026 at 18:54):Yes, makes complete sense. If you fix a bug, include a test so it doesn't come back.
Again, this why I think the tests should be in the same codebase. That way, the fix and the test can be in the same PR.
Philip Durbin 🚀 (Sep 08 2026 at 18:58):But until we get the JSF tests into the main repo, sure tests can also be added to the separate repo for critical bugs, or any bugs!
Philip Durbin 🚀 (Sep 09 2026 at 20:07):#12679 is an example of a regression we should add a test for
Philip Durbin 🚀 (Sep 09 2026 at 20:48):Fixed by #12681 (thanks, Jim)
Philip Durbin 🚀 (Sep 09 2026 at 21:02):Another regression in 6.12 rc: #12682 
Ash Manda (Sep 09 2026 at 21:14):I've added targeted fixes for both, we'll catch them going forward
Philip Durbin 🚀 (Sep 10 2026 at 15:31):Yet another regression in 6.12 rc: #12692 (summary fields not showing)
Ash Manda (Sep 10 2026 at 15:39):This is really tricky for just a JSF test to cover. It'll need some kind of regression test where it queries the API for the fields, and make sure those fields are present on the page. I'll create an issue.
Philip Durbin 🚀 (Sep 10 2026 at 15:42):Well, out of the box, it's a static list of fields.
Philip Durbin 🚀 (Sep 10 2026 at 15:43):So I think it's fine to hard code a test against those fields, for now.
Philip Durbin 🚀 (Sep 10 2026 at 15:44):Step two would be to check Get Summary Field Names and assert on those fields.
But again, for the first iteration, I'd suggest checking for the hard-coded fields.
Ash Manda (Sep 10 2026 at 15:45):Makes sense, thank you!
Ash Manda (Sep 10 2026 at 15:48):Phil could you tag these issues with "regression-test-needed" or a similar tag on GitHub with your thoughts about tests in the comments? That way I could filter these issues directly there and keep everything in one place
Ash Manda (Sep 10 2026 at 15:48):Right now it just says "Bug".
Ash Manda (Sep 10 2026 at 15:49):We probably won't be able to turn every bug into a test
Philip Durbin 🚀 (Sep 10 2026 at 16:04):My answer depends on if we're allowed to move the tests to the main repo or not. :smile:
Philip Durbin 🚀 (Sep 16 2026 at 19:35):This is where we put regressions, right?
✘ 11 [suite-chromium] › tests/suite/11-create-edit-metadata-template.spec.ts:11:5
From https://github.com/IQSS/dataverse/actions/runs/35137507075/job/104933445055?pr=12717
Ash Manda (Sep 16 2026 at 19:52):ooh that got caught real fast
Ash Manda (Sep 16 2026 at 19:53):bb2fc891b93a73c85a7be376b3e347d5b2e9b8ee.webm
For future reference
Philip Durbin 🚀 (Sep 16 2026 at 19:55):Thanks! I should try to get that test running locally. :thinking:
Philip Durbin 🚀 (Sep 17 2026 at 13:33):Jim posted a related comment about that 11-create-edit-metadata-template failure: https://github.com/IQSS/dataverse/pull/12719/#issuecomment-5714899544
Philip Durbin 🚀 (Oct 01 2026 at 19:41):I recommend a JSF test to check for #12769 - there shouldn't be a 500 error if you go to a non-existent collection.
Philip Durbin 🚀 (Oct 28 2026 at 09:03):Another one to add: JSF Deaccession Dialog Fix #12855
Last updated: Nov 02 2026 at 07:01 UTC