Stream: troubleshooting

Topic: Validation Errors Creating a Dataset w/ pyDataverse

view this post on Zulip Sherry Lake (Nov 01 2023 at 20:25):

The following json file creates a dataset with no errors using "curl" API command:
curl -H "X-Dataverse-key:$API_TOKEN" -X POST "$SERVER_URL/api/dataverses/$PARENT/datasets" --upload-file dataset-finch1.json -H 'Content-type:application/json

but that same json file dataset-finch1.json gives a validation error in pyDataverse:

jsonschema.exceptions.ValidationError: {'name': 'CC0 1.0', 'uri': 'http://creativecommons.org/publicdomain/zero/1.0'} is not of type 'string'

Failed validating 'type' in schema['properties']['datasetVersion']['properties']['license']:
{'$id': '#/properties/datasetVersion/properties/license',
'type': 'string'}

My file is attached:
dataset-finch1.json

What is missing to make the json validate in pyDataverse?

view this post on Zulip Philip Durbin 🚀 (Nov 01 2023 at 21:07):

Hmm. What version of pyDataverse are you using?

view this post on Zulip Sherry Lake (Nov 01 2023 at 21:26):

I'm using V0.3.1 - https://pydataverse.readthedocs.io/en/latest/index.html

I can create a dataset with pyDataverse if I leave out the license part, but then the license on my newly created dataset is "Custom", which I don't want.

view this post on Zulip Philip Durbin 🚀 (Nov 01 2023 at 22:32):

I guess I'd suggest opening an issue at https://github.com/gdcc/pyDataverse/issues

view this post on Zulip Oliver Bertuch (Nov 02 2023 at 12:41):

@Sherry Lake there is a huge chance that the schemas in https://github.com/gdcc/pyDataverse/tree/master/src/pyDataverse/schemas/json are outdated. Please create an issue, that would help a lot tracking this!

view this post on Zulip Oliver Bertuch (Nov 02 2023 at 12:42):

On a related note: @Don Sizemore @Jan Range @Philip Durbin (and to some degree me) are trying to get the release train rolling again for pyDataverse. Stay tuned!

view this post on Zulip Sherry Lake (Nov 02 2023 at 12:47):

Thanks, for the information. I will create a issue.

Issue created: https://github.com/gdcc/pyDataverse/issues/161

view this post on Zulip Philip Durbin 🚀 (Nov 02 2023 at 13:25):

Thanks, Sherry. @Jan Range check this out ^^

view this post on Zulip Jan Range (Nov 02 2023 at 13:28):

Hi @Sherry Lake! This is related to a missing header in the POST request. There is a fix on a fork of mine:

https://github.com/JR-1991/pyDataverse

Can you try it with this version? Maybe you need to uninstall pyDataverse first for the changes to come into effect.

view this post on Zulip Sherry Lake (Nov 02 2023 at 13:42):

Even with the validation errors, I still ran the create dataset command -resp = api.create_dataset("uvacs", ds.json())

Dataset was created, but with "Custom terms" - it ignored the license part (assuming because of the failed validation).

view this post on Zulip Sherry Lake (Nov 02 2023 at 13:43):

@Jan Range I am not sure how to uninstall pyDataverse and test your version. I'm on a Mac. Any hints on how to do that? I can try later this afternoon (East Coast US).

view this post on Zulip Jan Range (Nov 02 2023 at 13:45):

Of course, here is the command to run:

pip uninstall pyDataverse
pip install git+https://github.com/JR-1991/pyDataverse

If it doesn't work we may need to figure out where the issue is.

view this post on Zulip Sherry Lake (Nov 02 2023 at 20:33):

Unfortunately, I got the same error with your version @Jan Range (see the install lines below) . Problem seems to be in this json schema:
https://github.com/gdcc/pyDataverse/blob/master/src/pyDataverse/schemas/json/dataset_upload_default_schema.json

I uninstalled pyDataverse - After uninstalling, I typed this
pip install git+https://github.com/JR-1991/pyDataverse

which gave me results:

Defaulting to user installation because normal site-packages is not writeable
Collecting git+https://github.com/JR-1991/pyDataverse
  Cloning https://github.com/JR-1991/pyDataverse to /private/var/folders/r6/84fn85zs085b3c5n5hcg59xm0000gq/T/pip-req-build-gpqbzqwo
  Running command git clone --filter=blob:none --quiet https://github.com/JR-1991/pyDataverse /private/var/folders/r6/84fn85zs085b3c5n5hcg59xm0000gq/T/pip-req-build-gpqbzqwo
  Resolved https://github.com/JR-1991/pyDataverse to commit 0fcfcd3fbc6bf1aec869899f715a51dca25e91be
  Preparing metadata (setup.py) ... done
Requirement already satisfied: requests>=2.12.0 in /Users/sah/Library/Python/3.9/lib/python/site-packages (from pyDataverse==0.3.1) (2.31.0)
Requirement already satisfied: jsonschema>=3.2.0 in /Users/sah/Library/Python/3.9/lib/python/site-packages (from pyDataverse==0.3.1) (4.18.4)
Requirement already satisfied: attrs>=22.2.0 in /Users/sah/Library/Python/3.9/lib/python/site-packages (from jsonschema>=3.2.0->pyDataverse==0.3.1) (23.1.0)
Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /Users/sah/Library/Python/3.9/lib/python/site-packages (from jsonschema>=3.2.0->pyDataverse==0.3.1) (2023.7.1)
Requirement already satisfied: referencing>=0.28.4 in /Users/sah/Library/Python/3.9/lib/python/site-packages (from jsonschema>=3.2.0->pyDataverse==0.3.1) (0.30.0)
Requirement already satisfied: rpds-py>=0.7.1 in /Users/sah/Library/Python/3.9/lib/python/site-packages (from jsonschema>=3.2.0->pyDataverse==0.3.1) (0.9.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/sah/Library/Python/3.9/lib/python/site-packages (from requests>=2.12.0->pyDataverse==0.3.1) (3.2.0)
Requirement already satisfied: idna<4,>=2.5 in /Users/sah/Library/Python/3.9/lib/python/site-packages (from requests>=2.12.0->pyDataverse==0.3.1) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/sah/Library/Python/3.9/lib/python/site-packages (from requests>=2.12.0->pyDataverse==0.3.1) (2.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /Users/sah/Library/Python/3.9/lib/python/site-packages (from requests>=2.12.0->pyDataverse==0.3.1) (2023.7.22)
Building wheels for collected packages: pyDataverse
  Building wheel for pyDataverse (setup.py) ... done
  Created wheel for pyDataverse: filename=pyDataverse-0.3.1-py3-none-any.whl size=32602 sha256=c58d4c133d30bbe3c64b726d087de264ddf6da74e8dd2ec7a6318afc6a1d0f2d
  Stored in directory: /private/var/folders/r6/84fn85zs085b3c5n5hcg59xm0000gq/T/pip-ephem-wheel-cache-yzpofxys/wheels/61/2a/0a/07fb58afbbadb2ed13b0efe12205b86aaff3841ac8792fb06d
Successfully built pyDataverse
Installing collected packages: pyDataverse
Successfully installed pyDataverse-0.3.1

view this post on Zulip Philip Durbin 🚀 (Nov 02 2023 at 20:38):

On a related note, there's a new JSON Schema for Dataverse datasets at #9463 if you'd like to play with it.

Last updated: Oct 30 2025 at 06:21 UTC