Dataverse.jl

This package is about interfaces to the Dataverse data collections and APIs.

👉 demo notebook and notebook code

Warning

This package is in early development stage.

For example, list and download files from the default Harvard Dataverse instance:

using Dataverse

DOI = "doi:10.7910/DVN/EE3C40"
files = Dataverse.file_list(DOI)
Dataverse.file_download(DOI, files.filename[1])

Use base_url to connect to another Dataverse installation:

base_url = "https://data.example.edu"
files = Dataverse.file_list("doi:10.1234/EXAMPLE"; base_url=base_url)
header, dataverses, datasets = Dataverse.dataverse_scan(:root; base_url=base_url)

The documentation build does not execute these network examples. The CI integration tests use an isolated Dataverse instance created for each test run.

Julia Dataverse API

Dataverse.restDataverse.file_list — Function
file_list(DOI::String="doi:10.7910/DVN/ODM2IQ"; base_url=DEFAULT_BASE_URL)

Use HTTP, JSON, and DataFrames to list files in a dataset.

Return a DataFrame with filename, filesize, id, and download URL. Use base_url to connect to a Dataverse installation other than Harvard Dataverse.

file_list("doi:10.7910/DVN/ODM2IQ")
file_list("doi:10.1234/EXAMPLE"; base_url="https://data.example.edu")
source
file_list(nam::Symbol=:OCCA_clim; base_url=DEFAULT_BASE_URL)

Lookup DOI from list of demo data sets (:OCCAclim or :ECCOclim).

source
Dataverse.downloads.file_download — Function
file_download(DOI::String, name::String, path=tempdir(); base_url=DEFAULT_BASE_URL)

Download a named file from a dataset. Use base_url to connect to a Dataverse installation other than Harvard Dataverse.

DOI="doi:10.7910/DVN/OYBLGK"
filename="polygons_MBON_seascapes.geojson"
Dataverse.file_download(DOI, filename)
source
file_download(list::DataFrame,nam::String,pth::String)
lst=Dataverse.file_list("doi:10.7910/DVN/RNXA2A")
Dataverse.file_download(lst,lst.filename[2],tempdir())
source
Dataverse.restDataverse.dataverse_scan — Function
dataverse_scan(nam::Symbol=:ECCOv4r2; base_url=DEFAULT_BASE_URL)

Use HTTP, JSON, and DataFrames to list contents in a dataverse.

Returns header (Dict), dataverses (DataFrame), and datasets (DataFrame).

(header,dataverses,datasets)=Dataverse.dataverse_scan()
Dataverse.file_list(datasets.persistentUrl[1])
source
Dataverse.downloads.unzip — Function

function unzip(file,exdir="")

Source : @sylvaticus, https://discourse.julialang.org/t/ how-to-extract-a-file-in-a-zip-archive-without-using-os-specific-tools/34585/5

source