Stream: dev

Topic: Improve OpenAPI descriptions with AI


view this post on Zulip BalΓ‘zs Pataki (Jun 04 2026 at 16:35):

Hi,

I just created this issue

https://github.com/IQSS/dataverse/issues/12437

and this PR with the proposed changes:

https://github.com/IQSS/dataverse/pull/12439

@Philip Durbin πŸš€ suggested to make @Juan Pablo Tosca Villanueva aware of it, so I just did that :slight_smile:

The idea is to Make OpenAPI Great (Again) by adding currently missing annotations that provide descriptions and more context for the REST endpoints, making them more machine-actionable. My ultimate goal is to implement a Dataverse MCP server based on this improved OpenAPI documentation.

view this post on Zulip Philip Durbin πŸš€ (Jun 05 2026 at 00:03):

Only 2,655 lines to review in that PR. :smile:

view this post on Zulip Philip Durbin πŸš€ (Jun 05 2026 at 00:03):

I should back up and say I support the direction and goals! MCP! Yes!

view this post on Zulip Philip Durbin πŸš€ (Jun 05 2026 at 00:04):

@BalΓ‘zs Pataki giving the high line count in the PR, is it fair to assume all the descriptions are AI-generated?

view this post on Zulip Philip Durbin πŸš€ (Jun 05 2026 at 00:04):

Also, I'm sure you realize that this PR will be prone to merge conflicts.

view this post on Zulip Ash Manda (Jun 05 2026 at 00:25):

Almost certainly this specific PR cannot be merged. It just can't. HOWEVER. Since the changes are mostly annotations, we can keep track of what needs to be changed where, and how and carve this into say 12-15 PRs with 3-4 commits each. Within 3 months or so. This will reduce review burden and merge conflicts a lot. This is an important target for UNC, so I'm willing to stick with this and see it through. After a few PRs are merged, maybe BalΓ‘zs can start prototyping the MCP server based on the improved OpenAPI spec.

Does this sound reasonable to y'all?

view this post on Zulip BalΓ‘zs Pataki (Jun 05 2026 at 06:18):

Yes, these are all AI generated and it is a lot, that's why I raised the issue to discuss how to cope with this. I know from experience that sometimes even 4 lines of changes can take months to get tested and approved.

I assume that the annotations are nearly all correct, though the phrasing of the descriptions may not always be. But still, if we only need to review the description texts (for methods and arguments) it is still a more manageable job then adding these annotations from scratch.

Having said that, since these changes only affect the method headings (adding annotations and some reformatting, plus some import at the beginning of the files) I don't actually expect too much merge conflicts.

I was also thinking about creating a specific review tool for going through these changes in case the github review tools are not sufficient here.

Regarding the MCP server: I already have implemented one based on FastMCP. As you will see I also added tags to the methods, which allow configuration of various "profiles" for the MCP. 580+ tools are too much and users usually don't need all of them, so DV installations could be able to filter by tags what kind of tools they want to expose via the MCP. Such profiles could be "data steward", or "read-only discovery" and then only tools supporting these use cases would be offered by the MCP. This implementation is now a separate python process that need to be run and managed, but we can also think about building a Java implementation right into DV.

view this post on Zulip Philip Durbin πŸš€ (Jun 05 2026 at 09:47):

Makes sense. We have a weekly developer meeting ("tech hours") that I believe you both have attended. Starting this week I made an agenda/notes doc and just put this topic on it for the next meeting. How does that sound?

view this post on Zulip BalΓ‘zs Pataki (Jun 05 2026 at 13:04):

06.16 is OK with me!

view this post on Zulip Philip Durbin πŸš€ (Jun 22 2026 at 17:02):

@BalΓ‘zs Pataki thanks for attending tech hours and explaining your PR to us! Your slides were great as well.

view this post on Zulip Philip Durbin πŸš€ (Jul 10 2026 at 19:36):

@BalΓ‘zs Pataki I make a few tweaks to #12439 and approved it. Great stuff! From 10 to 98 out of 100 according to vacuum! :tada:

view this post on Zulip BalΓ‘zs Pataki (Jul 10 2026 at 19:44):

Wow, great, thank you very much! I’m on vacation I can help next week if anything is needed.

view this post on Zulip Philip Durbin πŸš€ (Jul 10 2026 at 19:45):

Meh, I don't think so. I added some docs here: https://dataverse-guide--12439.org.readthedocs.build/en/12439/developers/api-design.html#openapi

view this post on Zulip Philip Durbin πŸš€ (Jul 10 2026 at 19:47):

Sorry about DELETE /pids/{id}/delete. That was my bad: https://github.com/IQSS/dataverse/commit/8aacc9251c8d1288a3c0cc725a39ee9909a353c8

view this post on Zulip Philip Durbin πŸš€ (Jul 13 2026 at 18:04):

I merged #12439! :tada:

view this post on Zulip Philip Durbin πŸš€ (Jul 13 2026 at 18:13):

Now that the PR has been merged, I posted some follow up messages in related topics:

view this post on Zulip BalΓ‘zs Pataki (Sep 11 2026 at 08:12):

Just did a little test with the latest version. It seems there’s a bit of a regression with opaque Response return types:

https://github.com/IQSS/dataverse/pull/12439#issuecomment-5631440124

view this post on Zulip Philip Durbin πŸš€ (Sep 11 2026 at 13:41):

@BalΓ‘zs Pataki hmm, could you please open a new issue? And do you think we should fix this before we ship 6.12? :thinking:

view this post on Zulip BalΓ‘zs Pataki (Sep 11 2026 at 14:39):

I can investigate this further, especially why this error didn’t show up when we tried it before.

view this post on Zulip BalΓ‘zs Pataki (Sep 14 2026 at 15:23):

I created this: https://github.com/IQSS/dataverse/issues/12701

Downgrading the SmallRye OpenAPI plugin would be the easiest solution, but I don't know what else it might break. I think it's only used for OpenAPI JSON generation, so hopefully not much.

view this post on Zulip Philip Durbin πŸš€ (Sep 14 2026 at 15:28):

Hmm, I'm not sure I like the idea of staying on an old version of SmallRye. Is there a SmallRye issue we can track for this? That is, is there hope that they will fix things in a future release?

view this post on Zulip BalΓ‘zs Pataki (Sep 14 2026 at 16:00):

It seems that up to 4.1.1 we can achieve 97/100, then starting with 4.2.0 it goes back to 89/100.

This is my 4.1.1 run:

> vacuum lint -r ~/tmp/dvvacuum.yaml 'http://localhost:8080/openapi?format=json'

 β–ˆβ–ˆβ•—   β–ˆβ–ˆβ•— β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—  β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β–ˆβ–ˆβ•—   β–ˆβ–ˆβ•—β–ˆβ–ˆβ•—   β–ˆβ–ˆβ•—β–ˆβ–ˆβ–ˆβ•—   β–ˆβ–ˆβ–ˆβ•— γ€Šγ€Šγ€Šβ”€β•β”€β•β”€β”€ Β·* Β· Λ™*
 β–ˆβ–ˆβ•‘   β–ˆβ–ˆβ•‘β–ˆβ–ˆβ•”β•β•β–ˆβ–ˆβ•—β–ˆβ–ˆβ•”β•β•β•β•β•β–ˆβ–ˆβ•‘   β–ˆβ–ˆβ•‘β–ˆβ–ˆβ•‘   β–ˆβ–ˆβ•‘β–ˆβ–ˆβ–ˆβ–ˆβ•— β–ˆβ–ˆβ–ˆβ–ˆβ•‘γ€Šγ€Šγ€Šβ”€β”€β•β”€β•β”€β”€Β· Β·Β·* Λ™Λ™
 β–ˆβ–ˆβ•‘   β–ˆβ–ˆβ•‘β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•‘β–ˆβ–ˆβ•‘     β–ˆβ–ˆβ•‘   β–ˆβ–ˆβ•‘β–ˆβ–ˆβ•‘   β–ˆβ–ˆβ•‘β–ˆβ–ˆβ•”β–ˆβ–ˆβ–ˆβ–ˆβ•”β–ˆβ–ˆβ•‘γ€Šγ€Šγ€Šβ”€β”€β”€β•β”€β•β”€Β·Β·Β· Λ™Λ™ Λ™
 β•šβ–ˆβ–ˆβ•— β–ˆβ–ˆβ•”β•β–ˆβ–ˆβ•”β•β•β–ˆβ–ˆβ•‘β–ˆβ–ˆβ•‘     β–ˆβ–ˆβ•‘   β–ˆβ–ˆβ•‘β–ˆβ–ˆβ•‘   β–ˆβ–ˆβ•‘β–ˆβ–ˆβ•‘β•šβ–ˆβ–ˆβ•”β•β–ˆβ–ˆβ•‘γ€Šγ€Šβ”€β”€β•β”€β•β”€β”€Β·* Β·Β· Λ™Λ™
  β•šβ–ˆβ–ˆβ–ˆβ–ˆβ•”β• β–ˆβ–ˆβ•‘  β–ˆβ–ˆβ•‘β•šβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•—β•šβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•”β•β•šβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ•”β•β–ˆβ–ˆβ•‘ β•šβ•β• β–ˆβ–ˆβ•‘ γ€Šγ€Šβ”€β•β”€β•β”€β”€* Β· Β· Λ™
   β•šβ•β•β•β•  β•šβ•β•  β•šβ•β• β•šβ•β•β•β•β•β• β•šβ•β•β•β•β•β•  β•šβ•β•β•β•β•β• β•šβ•β•     β•šβ•β• γ€Šβ”€β”€β”€β•β”€β•β”€Β· Β·Β·* Λ™Λ™ Λ™

 version: 0.29.2 | compiled: Sat, 06 Jun 2026 09:45:50 UTC
 https://quobix.com/vacuum/ | https://github.com/daveshanley/vacuum

 using ruleset '/Users/balazs/tmp/dvvacuum.yaml' (containing 51 rules)
 vacuuming file 'http://localhost:8080/openapi?format=json' against 51 rules: https://quobix.com/vacuum/rulesets/recommended

 category              βœ— errors      β–² warnings    ● info
 ────────────────────  ────────────  ────────────  ────────────────────────────────────────────────────────
 Operations            0             4             0
 Tags                  0             1             0
 Validation            0             1             0
 Descriptions          0             2             0
 ────────────────────  ────────────  ────────────  ────────────────────────────────────────────────────────
 total                 0             8             0

 rule                                      violations    quality impact
 ────────────────────────────────────────  ────────────  ──────────────────────────────────────────────────
 no-request-body                           4             β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
 oas3-parameter-description                2             β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
 oas3-api-servers                          1             β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
 operation-tags                            1             β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
 ────────────────────────────────────────  ────────────  ──────────────────────────────────────────────────
 total                                     8

 --> use the <dashboard> command to be able to navigate results interactively <--

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                                                                                                         β”‚
β”‚  Quality Score: 97/100 [A+]                                                                             β”‚
β”‚                                                                                                         β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

 β”‚
 β”‚ β–² Passed, but with 8 warnings and 0 informs

view this post on Zulip Philip Durbin πŸš€ (Sep 14 2026 at 16:45):

Sounds like you're narrowing down where the regression in SmallRye was introduced. Are you thinking about opening an issue with them?

view this post on Zulip BalΓ‘zs Pataki (Sep 14 2026 at 16:47):

Maybe it is not a regression but a fix from their side. :-)

view this post on Zulip Philip Durbin πŸš€ (Sep 14 2026 at 16:47):

heh

view this post on Zulip BalΓ‘zs Pataki (Sep 14 2026 at 18:19):

I may have found a straightforward solution, by adding a specific annotation, but this would touch again 17 files. Added comment about it in the issue.

view this post on Zulip Philip Durbin πŸš€ (Sep 14 2026 at 19:52):

@BalΓ‘zs Pataki you're welcome to make that PR but we're in code freeze for 6.12. :cold_face: Please see #community > Release 6.12 Timeline

view this post on Zulip Philip Durbin πŸš€ (Sep 14 2026 at 19:52):

Good job figuring out a solution! :dataverse_man:

view this post on Zulip BalΓ‘zs Pataki (Sep 14 2026 at 19:53):

Yeah, I know there was already code freeze, but hope we will have a 6.13 someday :grinning:

view this post on Zulip Philip Durbin πŸš€ (Sep 14 2026 at 19:54):

Ha, definitely. No timeline thread yet, though. :smile:

view this post on Zulip Oliver Bertuch (Sep 15 2026 at 15:11):

Hey @BalΓ‘zs Pataki what is the motivation behind getting the score back up? Looking at the PR I'm a little worried. While the score may get back up (quantitativly), it's not a qualitative improvement of our OpenAPI spec.

As the way our Java REST API endpoints responds has grown organically (one might even say out of control) towards not using DTOs, there's not a lot of schema that we can add now to the spec. On the other hand, that's one of the most essential things a client needs when using the spec to implement some action against the API.

So while the score is back up, the client will not have a lot of more information. The schema will literally say "expect a JSON object" and so on. I'm not sure this will help anyone. It's like code coverage: a high number doesn't tell you of what quality the test are made, how well they cover edge cases etc.

My vote would be to be honest and stick with a 89. It's not too bad, but looking at a 98 and then getting a lot of nothing seems contraproductive as an expectation to set and communicate.

That said: I very much fear it will be a lot of work digging ourselves out of the hole we dug... (But that's true for a lot of things gone wrong with the API architecture.)

view this post on Zulip Oliver Bertuch (Sep 15 2026 at 15:18):

I've invested a few tokens a while ago by asking Qwen3.8-Flash-Next to review the API Datasets class (was testing the LLM deployment on a server). A god class example par excellence with 7000 lines. Takes ~60kT / ~500MB of KV context cache to dig through. The TL;DR for a refactoring was:

  1. Split into focused resource classes (or sub-resources): DatasetResource, DatasetVersionResource, FileEmbargoResource, FileRetentionResource, GlobusTransferResource, DatasetTypeResource, CurationStatusResource, ThumbnailsResource, etc. A base AbstractDatasetResource can hold the shared findDatasetOrDie / response(…) helpers.
  2. Extract the per-file attribute logic (embargo, retention) into a shared generic helper or a small service.
  3. Standardize on one dataset-lookup pattern, one permission-check pattern, and one error-response pattern.
  4. Move handleVersion + DsVersionHandler into a package-level utility.
  5. Standardize parameter names ({id} everywhere for the dataset identifier).
  6. Reduce injected dependencies per class to only what each sub-resource needs.
  7. Break long methods into validation / orchestration / response steps.

The class is clearly functional (it's a production API for a real system), but from a maintainability and testability standpoint it is far from well-designed.

And it's not even mentioning response schemas, as the other tasks are far more important and complex... :see_no_evil: I might do this again and throw GitNexus into the mix, so the model has a better chance at understanding the Dataverse codebase context (haha).

view this post on Zulip BalΓ‘zs Pataki (Sep 15 2026 at 15:41):

Yes, the current fix is to explicitly declare that we return "an object", although we don't specify its shape. The regression was that it didn't even say that it was "an object" and now it does . The vacuum score is not about how nice your API is functionally (I mean, whether you return a nicely shaped object or just an opaque object type), but whether it is properly defined so that it is machine interpretable. That's why I think 98 is reasonable with our current implementation and fix.

Of course the real solution - as I also suggested in the issue - is to have proper DTO-s returned, which then SmallRye will define proper JSON schemas for and so users would know the exact shape of the returned values.

Until then I think it is a valid and honest declaration that we return "an object" or "a string" and it is still more useful and correct than saying nothing.

view this post on Zulip Philip Durbin πŸš€ (Sep 15 2026 at 15:46):

fight, fight, fight! :swords:

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 08:41):

Alright, I'll bite. Both arguments sound convincing, depending on what perspective and world view you have.

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 08:42):

So let's not argue over bike shedding, let's make it more constructive.

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 08:42):

I went ahead and used your PR to generate an updated OpenAPI spec.

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 08:43):

As one of the intentions for OpenAPI specs these days isn't just implementing an autogenerated client, but agentic AI usage, I went ahead and threw an agent at this.

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 08:44):

Using Pi Agent backed with a Qwen3.8:27B model, I asked this question: "Can you please read the file target/classes/META-INF/openapi.yaml, analyze it and give me a thorough and honest overview of problems with this OpenAPI spec for a consumer (looking to implement a client or using it in an agentic AI context)? "

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 08:56):

It took a while, installed Python openapi-schema-validator and openapi-spec-validator plus NPM-based Redocly CLI, looked at the source code and came up with a long result: https://iffmd.fz-juelich.de/81LtXoJlQniOz2FVBz1uxg

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 08:56):

I haven't proof-read all of them yet, but some things are already standing out to me as valid things.

view this post on Zulip BalΓ‘zs Pataki (Sep 16 2026 at 08:59):

In the meantime I ran the same prompt in chatgpt and it gave similar result: the biggest problem being the missing response types. ("~80% of responses have no schema" in your report)

https://chatgpt.com/share/6aaa5a33-111c-83eb-8bee-c64b9050f857

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:02):

  1. Looking at https://demo.dataverse.org/openapi, there really is no "servers" block mixed into the "template". The missing API base URL is probably bad, too. (Verified with YAML from Demo)
  2. The error schema is a problem. @Johannes D and @Vera Clemens have already proposed adapting RFC9457 going forward, which is a good idea.
  3. The "no response schemas" part is obvious and we know that, that will be a larger endavour.
  4. The request contracts might be worth a proof read. We thought we did quite good on those, but normal validation tools can only get so far...
  5. Some of the security things look legit - we probably don't do a good job on those.
  6. We should double check the path collisions Redocly flags. That would be an easy fix.

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:05):

I know that a lot of API endpoints have strange content types attached, like "/". In another run it did mention those as a potential source of problems for clients/agents as well. (Misleading what parameters should be sent, etc)

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:07):

I'm not sure who's to blame here: did we do a poor job at annotating with API specs, are some of our Jersey content type annotations bad or is the generator doing a poor job. Needs further evaluation.

view this post on Zulip BalΓ‘zs Pataki (Sep 16 2026 at 09:07):

So, how should we proceed with knowing all this? These are mostly deep architectural/source code issues we already knew about. The current OpenAPI work just made it more visible.

Of course, where we can just fix something with OpenAPI declarations, which is relatively easy, we should do that.

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:07):

Yeah, I'd favor we address anything we can address with OpenAPI specs.

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:07):

At least for this PR

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:07):

Here's a thought.

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:08):

How about writing a DEP for all of this?

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:08):

As you said, this is a larger thing than slamming a few more annotations on endpoints here and there

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:09):

IMHO it would be a good idea to write up a thorough problem description and come up with a plan of attack

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:09):

Alas, a "Dataverse Enhancement Proposal"

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:09):

WDYT?

view this post on Zulip BalΓ‘zs Pataki (Sep 16 2026 at 09:09):

Yeah, so Dataverse 7.0 :-)

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:12):

Not necessarily

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:12):

Honestly, many of these points can be addressed without a breaking change

view this post on Zulip BalΓ‘zs Pataki (Sep 16 2026 at 09:13):

We should have a call about it maybe to discuss?

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:15):

Sure! We could try to raise attention via the mailing list. There is not a working group for such a thing (yet?), but maybe it can be a discussion during a community call. IMO what would justify investing work here would be feedback from users telling us about how they use Dataverse with agents.

view this post on Zulip BalΓ‘zs Pataki (Sep 16 2026 at 09:15):

Good idea!

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:20):

I don't know about the actual status of https://github.com/gdcc/mcp-dataverse (there is some commit activity, but the actual MCP service seems dead and I dunno if they implemented the new stateless MCP 2.0, maybe @Philip Durbin πŸš€ or @Ken Lui knows more) and recently @Jan Range added https://gdcc.github.io/pyDataverse/mcp/server to pyDataverse. From where I'm standing, I'd say a good OpenAPI spec would let agents go a long way.

(As a side note: at least within Helmholtz due to the availability of Blablador, a Helmholtz wide (free) inference service, using agents for scientific works is becoming increasingly more common)

view this post on Zulip BalΓ‘zs Pataki (Sep 16 2026 at 09:21):

Well, I started to work on this OpenAPI enhancement thing to have a complete/configurable MCP covering all/configured part of a DV server. We plan to add it as part of our ARP project.

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:22):

What would be the most pressing endpoints for your use cases? (No list necessary, better tell me the Java API classes involved)

view this post on Zulip BalΓ‘zs Pataki (Sep 16 2026 at 09:23):

That's why I added tags to semantically group functionalities, so that you can have a "datasteward" like MCP which is about search, metadatada editing, publication. Anothor one for DV sysadmins, etc.

view this post on Zulip BalΓ‘zs Pataki (Sep 16 2026 at 09:24):

So, in my FastAPI based MCP server I can configure "profiles" like this, based on tags in the openapi:

# Profile: Data steward
# For creating, editing, searching, publishing, and organizing datasets/files.
INCLUDE_TAGS=Datasets,Dataverses,Files,Search,Licenses,Dataset Fields,Metadata
EXCLUDE_TAGS=Admin,Users,Roles,Notifications,Workflows,External Tools,Dataverse Featured Items

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:25):

That's pretty cool!

view this post on Zulip BalΓ‘zs Pataki (Sep 16 2026 at 09:25):

(you don't need to specify both include/exclude, this is just for an example)

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:27):

Looking at your example, would "Datasets" and "Dataverses" be the classes which would deliver the most-bang-for-buck if they'd have all the shiny OpenAPI things?

view this post on Zulip BalΓ‘zs Pataki (Sep 16 2026 at 09:28):

Yes, definitely. And Search, and Files. And Metadata :slight_smile:

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:28):

Haha slow your roll :wink: The Datasets class alone is a god class with 7k lines.

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:29):

But maybe we can actually identify groups of endpoints in these classes that would be most helpful and start moving them to separate classes. Come up with a better package structure and start adding DTOs.

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:29):

So we can start shifting things around without any big-boom moves

view this post on Zulip BalΓ‘zs Pataki (Sep 16 2026 at 09:29):

Good strategy, I like it!

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:30):

Hence the idea to write a DEP :wink:

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:34):

It might be interesting to hear perspectives from people like @Tilo Mathes (RSpace) who integrate their products with Dataverse. They may have a take on how valuable a good OpenAPI spec is (both from their own product as well as from someone using the Dataverse API)

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 09:50):

@Philip Durbin πŸš€ can you setup a Google Doc for the DEP plz? :pray:

view this post on Zulip Philip Durbin πŸš€ (Sep 16 2026 at 12:00):

Oliver Bertuch said:

  1. Looking at https://demo.dataverse.org/openapi

I would suggest looking at https://beta.dataverse.org/openapi instead because it has the recent OpenAPI improvements that will be in 6.12. The demo server is still running 6.11.

(The demo server always runs the most recent release. The beta server runs unreleased code.)

view this post on Zulip Philip Durbin πŸš€ (Sep 16 2026 at 12:28):

Oliver Bertuch said:

the actual MCP service seems dead

@Slava Tykhonov can you please restart https://mcp.dataverse.org ? We're getting "bad gateway".

view this post on Zulip Philip Durbin πŸš€ (Sep 16 2026 at 12:29):

Oliver Bertuch said:

Jan Range added https://gdcc.github.io/pyDataverse/mcp/server to pyDataverse

I opened this: Add pyDataverse MCP server to AI GuideΒ #12537

view this post on Zulip Philip Durbin πŸš€ (Sep 16 2026 at 12:32):

Oliver Bertuch said:

can you setup a Google Doc for the DEP plz?

Sure, here you go: https://docs.google.com/document/d/1hC7SXOrJWKRjniviIpWVH1aAZL1vJhc451LBb9mKDvk/edit?usp=sharing

view this post on Zulip Slava Tykhonov (Sep 16 2026 at 12:51):

Screenshot 2026-09-16 at 14.49.44.png
MCP restarted and seems to be working

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 12:54):

Philip Durbin πŸš€ said:

Oliver Bertuch said:

  1. Looking at https://demo.dataverse.org/openapi

I would suggest looking at https://beta.dataverse.org/openapi instead because it has the recent OpenAPI improvements that will be in 6.12. The demo server is still running 6.11.

(The demo server always runs the most recent release. The beta server runs unreleased code.)

I will reverify!!!

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 14:08):

No servers block, /api, or v1 inside the OpenAPI docs on beta either!

view this post on Zulip Philip Durbin πŸš€ (Sep 16 2026 at 14:08):

:doh:

view this post on Zulip Oliver Bertuch (Sep 16 2026 at 14:09):

<sarcasm>Also a valid way to protect yourself from rogue AI agents, just send them into the void...</sarcasm>

view this post on Zulip BalΓ‘zs Pataki (Sep 16 2026 at 14:56):

That's too bad and vacuum doesn't report it because servers is optional in OpenAPI.

view this post on Zulip BalΓ‘zs Pataki (Sep 17 2026 at 10:39):

So, my openapi enhancement attempts just tried to document what we have with more (human) descriptions without changing any API-s. In this respect with all these nicer description, we are of course nearly as bad with most of the input-output semantics as before.

For example, we have this endpoint:

    @POST
    @AuthRequired
    @Path("{identifier}/metadatablockfacets/isRoot")
    @Consumes(MediaType.APPLICATION_JSON)
    @Produces(MediaType.APPLICATION_JSON)
    @Operation(summary = "Assign metadata block facet root status",
            description = "Changes whether a dataverse is a metadata-block-facet root.")
    @RequestBody(description = "Boolean text indicating whether the dataverse is a metadata-block-facet root.")
    public Response updateMetadataBlockFacetsRoot(@Context ContainerRequestContext crc,
            @Parameter(description = "Dataverse alias, id, or persistent identifier.", required = true)
            @PathParam("identifier") String dvIdtf,
            @RequestBody(description = "Boolean text indicating whether the dataverse is a metadata-block-facet root.")
            String body) {
        try {
            final boolean blockFacetsRoot = parseBooleanOrDie(body);

These were already defined:

    @POST
    @AuthRequired
    @Path("{identifier}/metadatablockfacets/isRoot")
    @Consumes(MediaType.APPLICATION_JSON)
    @Produces(MediaType.APPLICATION_JSON)

But then in code we have

final boolean blockFacetsRoot = parseBooleanOrDie(body);

which clearly doesn't expect a JSON as input but a true or false or similar string value.

What I want to say is that a good vacuum value do not mean a semantically correct API, but conformance to OpenAPI spec.

view this post on Zulip Notification Bot (Sep 17 2026 at 10:53):

2 messages were moved from this topic to #dev > expired cert on guides, demo, etc. by Philip Durbin πŸš€.

view this post on Zulip Tilo Mathes (Sep 17 2026 at 13:50):

Oliver Bertuch said:

It might be interesting to hear perspectives from people like Tilo Mathes (RSpace) who integrate their products with Dataverse. They may have a take on how valuable a good OpenAPI spec is (both from their own product as well as from someone using the Dataverse API)

I can relay that question to our team, we've recently moved to OpenAPI 3.0 from our previous Swagger 2.0

view this post on Zulip Oliver Bertuch (Sep 24 2026 at 14:49):

@BalΓ‘zs Pataki here's a treat for you. For the new bulk export endpoint I went ahead and did a proper DTO for the requests including OpenAPI annotations. See https://github.com/IQSS/dataverse/pull/12737. If you feel like it, have a go and see if this would be what we eventually want for all of the API.

view this post on Zulip BalΓ‘zs Pataki (Sep 24 2026 at 15:13):

I just glanced through it, and the request record looks nice indeed!

I see that the response is still just Response. I didn't check what exactly is expected to be returned here, but could we type the response as well?

Also, I only see a 400 response. Shouldn't we have a 200 as well, with the expected response type?

view this post on Zulip Oliver Bertuch (Sep 24 2026 at 22:03):

Maybe? Need to look into that! It's a streaming response, as we just push out metadata in different formats. So there is never an option to have a structured response object.


Last updated: Nov 02 2026 at 07:01 UTC