Stream: troubleshooting

Topic: Metadata format for Dataset Migration API in 6.5


view this post on Zulip mjlassila (Jan 20 2025 at 10:53):

I'm experimenting with Migration API to import data from our legacy repository system. My goal is to create JSON-LD metadata which is compatible with Dataset Migration API in 6.5.

To test the API, I exported some metadata records out of my 6.5 test installation in OAI-ORE format, as instructed in Migration API documents, made some trivial modifications to metadata and tried to re-import metadata using Migration API.

However, I get a 400 bad request error when reimporting the data, Dataverse complaining about:
The API request cannot be completed with the parameters supplied. Details: Compound field values must be JSON objects, field: author

The example file I loaded from Migration API documentation imports fine.

view this post on Zulip Philip Durbin ๐Ÿš€ (Jan 21 2025 at 12:06):

How is your JSON-LD file different from the example file? Can you share it with us?

view this post on Zulip mjlassila (Jan 21 2025 at 13:30):

At first I tried a file which I exported from Dataverse 6.5 using OAI-ORE export format because I supposed that exported JSONLD works directly with Dataset Migration API.

It didn't work out but by trial and error I managed to modify the example file to include additional information our use case requires (eg. geospatial metadata). My exported file from 6.5 and modified, functional import file based on example file are here: https://gist.github.com/mjlassila/ecdbd11447ccdf87995db20bfc5e686c

view this post on Zulip Philip Durbin ๐Ÿš€ (Jan 21 2025 at 14:10):

Ah, great, so you got it working.

view this post on Zulip mjlassila (Jan 22 2025 at 09:21):

Yep, it works now as intended.

Perhaps it would be useful add to Dataset Migration API documentation that exported OAI-ORE data is not directly compatible with the API?

In case anyone other than us uses BaseX and XQuery for data wrangling, here is a snippet of XQuery for generating JSON-LD compatible with the Dataset Migration API. https://gist.github.com/mjlassila/ecdbd11447ccdf87995db20bfc5e686c

view this post on Zulip Philip Durbin ๐Ÿš€ (Jan 22 2025 at 12:25):

Yes! Are you interested in making a pull request?

view this post on Zulip mjlassila (Jan 24 2025 at 07:16):

I can try :smile: Is the documentation source in the Dataverse Github repo current or is there a separate repository for documentation?

view this post on Zulip Philip Durbin ๐Ÿš€ (Jan 24 2025 at 12:23):

Same repo. Here's the file: https://github.com/IQSS/dataverse/blob/develop/doc/sphinx-guides/source/developers/dataset-migration-api.rst

view this post on Zulip Philip Durbin ๐Ÿš€ (Jan 24 2025 at 12:23):

This might help as well: https://guides.dataverse.org/en/6.5/contributor/documentation.html

view this post on Zulip Philip Durbin ๐Ÿš€ (Jan 24 2025 at 12:24):

Even if you just write Markdown I'm happy to convert it to rst for you.

view this post on Zulip mjlassila (Jan 28 2025 at 09:06):

Hopefully I got it right: https://github.com/IQSS/dataverse/pull/11192

view this post on Zulip Philip Durbin ๐Ÿš€ (Jan 28 2025 at 12:41):

Thanks for the PR!

make html works fine. You can preview the changes at https://dataverse-guide--11192.org.readthedocs.build/en/11192/developers/dataset-migration-api.html

I added that link to your PR description.

view this post on Zulip Philip Durbin ๐Ÿš€ (Jan 28 2025 at 12:41):

As to the actual content, it looks like Jim (qqmyers) has some feedback for you.

view this post on Zulip Philip Durbin ๐Ÿš€ (Jan 28 2025 at 12:42):

I'm surprised that siteUrl is needed but Jim would know better than I would.

view this post on Zulip Philip Durbin ๐Ÿš€ (Feb 03 2025 at 19:35):

@mjlassila merged! Thanks!


Last updated: Oct 30 2025 at 06:21 UTC