Stream: troubleshooting

Topic: metadatablocks api

view this post on Zulip Björn Selent (Aug 08 2024 at 07:52):

Hi all,

we encounter a inconsistent behaviour when using the /api/metadatablocks endpoint. When for example querying the citation metadatablock the first parent is "otherId" with child "otherIdAgency" etc. The entry for "otherIdAgency" is then shown again after the otherId block, which we think it shouldn't as it is a child entry. The same holds for all other child entries, too, i.e. they are all shown twice. The entries are not filtered during casting from list to set via treeSet in JsonPrinter.java, ln. 642.
Additionally we noticed that for the codeMeta20 block the children displayOrder are numbered always starting from 0 (which we think is the proper way to do it) whereas in all other blocks the entries are numbered consecutively. The counting form 0 can mitigate the filtering issue but only if a certain order within the DB of parents and children are followed.
It seems like using a non-unique identifier, i.e. displayOrder, for sorting the json export leads to undesired side-effects.
Is anybody else seeing this behaviour?

We are on dataverse version 6.3.
Thank you and regards
Björn (FoKUS, University of Stuttgart)

view this post on Zulip Philip Durbin 🚀 (Aug 08 2024 at 13:09):

Hmm, looking at https://demo.dataverse.org/api/metadatablocks/citation I see what you mean:

{
  "status": "OK",
  "data": {
...
    "fields": {
...
      "otherId": {
        "name": "otherId",
...
        "childFields": {
          "otherIdAgency": {
            "name": "otherIdAgency",
...
      "otherIdAgency": {
        "name": "otherIdAgency",

view this post on Zulip Florian Fritze (Aug 09 2024 at 08:49):

Philip Durbin schrieb:

Hmm, looking at https://demo.dataverse.org/api/metadatablocks/citation I see what you mean:

{
  "status": "OK",
  "data": {
...
    "fields": {
...
      "otherId": {
        "name": "otherId",
...
        "childFields": {
          "otherIdAgency": {
            "name": "otherIdAgency",
...
      "otherIdAgency": {
        "name": "otherIdAgency",

Hello Philip,
I have just created a Pull Request that should fix the issue: https://github.com/IQSS/dataverse/pull/10764
Kind regards
Florian (FoKUS, University of Stuttgart)
P.S. I hope I fulfilled all the requirements for a proper Pull Request.

view this post on Zulip Philip Durbin 🚀 (Aug 12 2024 at 14:40):

@Florian Fritze Thanks.

edu.harvard.iq.dataverse.api.DataversesIT.testListMetadataBlocks is failing with this:

1 expectation failed.
JSON path data[1].fields.size() doesn't match.
Expected: <10>
  Actual: <2>

Here: edu.harvard.iq.dataverse.api.DataversesIT.testListMetadataBlocks(DataversesIT.java:779)

view this post on Zulip Philip Durbin 🚀 (Aug 12 2024 at 14:43):

It looks like the code was last edited by @Guillermo Portas in PR #10642 to fix #10637.

view this post on Zulip Guillermo Portas (Aug 12 2024 at 19:36):

@Philip Durbin The issue that Florian describes is something that was present in the original code before I modified it. It's a behavior that I remember discussing in the past, and we decided to keep it this way for backwards compatibility. That is, showing the child fields also repeated as root fields in the response. We need to decide whether we want to finally change this behavior or keep it as is.

view this post on Zulip Guillermo Portas (Aug 12 2024 at 19:37):

@Philip Durbin I'm in favor of changing it, although it might be good to double-check first.

view this post on Zulip Philip Durbin 🚀 (Aug 12 2024 at 19:46):

Ah, thanks @Guillermo Portas. I didn't remember discussing it!

view this post on Zulip Philip Durbin 🚀 (Aug 12 2024 at 19:47):

@Florian Fritze at minimum, if we do decide to make a backward-incompatible change, we should document it at a future version of https://guides.dataverse.org/en/6.3/api/changelog.html

view this post on Zulip Philip Durbin 🚀 (Aug 12 2024 at 19:55):

Can you please edit doc/sphinx-guides/source/api/changelog.rst in your PR?

view this post on Zulip Philip Durbin 🚀 (Aug 12 2024 at 19:57):

Oh and please create a release note snippet too: https://guides.dataverse.org/en/6.3/developers/version-control.html#writing-a-release-note-snippet

view this post on Zulip Florian Fritze (Aug 13 2024 at 11:12):

Philip Durbin schrieb:

Can you please edit doc/sphinx-guides/source/api/changelog.rst in your PR?

I am not quite familiar with this. What should I enter in the changelog.rst file? What would you expect from me. Currently on the top there is:

API Changelog (Breaking Changes)
================================

This API changelog is experimental and we would love feedback on its usefulness. Its primary purpose is to inform API developers of any breaking changes. (We try not ship any backward incompatible changes, but it happens.) To see a list of new APIs and backward-compatible changes to existing API, please see each version's release notes at https://github.com/IQSS/dataverse/releases

.. contents:: |toctitle|
:local:
:depth: 1

v6.3

view this post on Zulip Florian Fritze (Aug 13 2024 at 11:15):

And an additional question: Should I make the changes in my branch which I used for the Pull Request? Thanks in advance.

view this post on Zulip Philip Durbin 🚀 (Aug 13 2024 at 14:08):

@Florian Fritze Hi! v6.3 is the latest released version. You'd start a new section called v6.4 and add a bullet describing the change.

view this post on Zulip Philip Durbin 🚀 (Aug 13 2024 at 14:38):

And yes, all in the same branch, please. Thanks!

view this post on Zulip Florian Fritze (Aug 14 2024 at 06:15):

Philip Durbin schrieb:

And yes, all in the same branch, please. Thanks!

Good morning, it's done.

view this post on Zulip Philip Durbin 🚀 (Aug 20 2024 at 14:09):

@Florian Fritze thanks! I just left another review: https://github.com/IQSS/dataverse/pull/10764#pullrequestreview-2248119241

view this post on Zulip Florian Fritze (Dec 16 2024 at 12:03):

Philip Durbin 🚀 schrieb:

Florian Fritze thanks! I just left another review: https://github.com/IQSS/dataverse/pull/10764#pullrequestreview-2248119241

Hello Philip, I am still waiting for an approval of my Pull Request. Could it be that it is fallen under the table?

view this post on Zulip Florian Fritze (Dec 16 2024 at 12:05):

Florian Fritze schrieb:

Philip Durbin 🚀 schrieb:

Florian Fritze thanks! I just left another review: https://github.com/IQSS/dataverse/pull/10764#pullrequestreview-2248119241

Hello Philip, I am still waiting for an approval of my Pull Request. Could it be that it is fallen under the table?

Thanks in advance Florian

view this post on Zulip Philip Durbin 🚀 (Dec 16 2024 at 13:16):

@Steven Winship can you please take a look at this?

view this post on Zulip Philip Durbin 🚀 (Dec 16 2024 at 14:21):

@Florian Förster I can't push to your fork so I made a PR for you. Please take a look: https://github.com/IQSS/dataverse/pull/10764#issuecomment-2545755220

Last updated: Oct 30 2025 at 05:14 UTC