ο‚›Edit on GitHub

Dataverse.jl

This package is about interfaces to the Dataverse data collections and APIs.

πŸ‘‰ demo notebook and notebook code

Warning

This package is in early development stage.

For example:

using Dataverse

DOI="doi:10.7910/DVN/EE3C40"
files=Dataverse.file_list(DOI)
Dataverse.file_download(DOI,files.filename[1])
Downloading file : /tmp/MITprof_mar2016_argo0708.nc

or 

(header,dataverses,datasets)=Dataverse.dataverse_scan(:ECCOv4r2)
(Dict{String, Any}("ownerId" => 2843116, "name" => "ECCO Version 4 Release 2 Dataverse", "id" => 2843117, "description" => "<br> 'ECCO Version 4 Release 2' is a global ocean state estimate that covers the period from 1992 to 2011 (Forget et al. 2015, 2016). It was produced on behalf of the ECCO consortium (http://ecco-group.org/) with major support provided NASA’s Physical Oceanography Program. General documentation of the `ECCO Version 4 Release 2' dataverse and all included datasets can be found at http://dx.doi.org/10.7910/DVN/ODM2IQ (see README.pdf in that dataset)\r\n\r\n<br><br> References: \r\n<br> - Forget, G., J.-M. Campin, P. Heimbach, C. N. Hill, R. M. Ponte, and C. Wunsch, 2015: ECCO version 4: an integrated framework for non-linear inverse modeling and global ocean state estimation. Geoscientific Model Development, 8, 3071-3104, http://dx.doi.org/10.5194/gmd-8-3071-2015\r\n<br> - Forget, G., J.-M. Campin, P. Heimbach, C. N. Hill, R. M. Ponte, and C. Wunsch, 2016: ECCO Version 4: Second Release, http://hdl.handle.net/1721.1/102062\r\n<br><br>", "dataverseContacts" => Any[Dict{String, Any}("contactEmail" => "gforget@mit.edu", "displayOrder" => 0)], "theme" => Dict{String, Any}("tagline" => "http://www.ecco-group.org/", "linkUrl" => "http://www.ecco-group.org/", "textColor" => "888888", "id" => 606, "logo" => "seal_ecco.jpg", "linkColor" => "428BCA", "backgroundColor" => "F5F5F5"), "permissionRoot" => true, "affiliation" => "Massachusetts Institute of Technology", "alias" => "ECCOv4r2", "dataverseType" => "RESEARCH_PROJECTS"…), 1Γ—3 DataFrame
 Row β”‚ id       type       title
     β”‚ Int64    String     String
─────┼───────────────────────────────────────────────────────
   1 β”‚ 2845520  dataverse  ECCO version 4 release 2 inputs …, 11Γ—3 DataFrame
 Row β”‚ id       type     persistentUrl
     β”‚ Int64    String   String
─────┼─────────────────────────────────────────────────────
   1 β”‚ 2844389  dataset  https://doi.org/10.7910/DVN/ODM2…
   2 β”‚ 2844461  dataset  https://doi.org/10.7910/DVN/3HPR…
   3 β”‚ 2845372  dataset  https://doi.org/10.7910/DVN/H8W5…
   4 β”‚ 2845392  dataset  https://doi.org/10.7910/DVN/OOB8…
   5 β”‚ 2845408  dataset  https://doi.org/10.7910/DVN/AMIY…
   6 β”‚ 2845452  dataset  https://doi.org/10.7910/DVN/VQR2…
   7 β”‚ 2845519  dataset  https://doi.org/10.7910/DVN/WJNX…
   8 β”‚ 2845523  dataset  https://doi.org/10.7910/DVN/NXYK…
   9 β”‚ 2845782  dataset  https://doi.org/10.7910/DVN/EE3C…
  10 β”‚ 2847266  dataset  https://doi.org/10.7910/DVN/T093…
  11 β”‚ 3377869  dataset  https://doi.org/10.7910/DVN/AVVG…)

then

files=Dataverse.file_list(datasets.persistentUrl[1])
12Γ—4 DataFrame
Rowfilenamefilesizeidurl
StringInt64Int64String
1available_diagnostics.log328802862636https://dataverse.harvard.edu/api/access/datafile/2862636
2costfunction001230472862637https://dataverse.harvard.edu/api/access/datafile/2862637
3eccov4_lonlat.m46512862635https://dataverse.harvard.edu/api/access/datafile/2862635
4ECCOV4R2_depiction.pdf213251362863359https://dataverse.harvard.edu/api/access/datafile/2863359
5ECCOV4R2_gcmfaces.pdf14384292968694https://dataverse.harvard.edu/api/access/datafile/2968694
6ECCOV4R2_MITslides.pdf146067012863358https://dataverse.harvard.edu/api/access/datafile/2863358
7ECCOV4R2_userGuide.pdf1200642968695https://dataverse.harvard.edu/api/access/datafile/2968695
8estimate_history.log13092862638https://dataverse.harvard.edu/api/access/datafile/2862638
9nctiles45x45.jpg4553122862639https://dataverse.harvard.edu/api/access/datafile/2862639
10nctiles90x90.jpg3846752862640https://dataverse.harvard.edu/api/access/datafile/2862640
11README.pdf729872968696https://dataverse.harvard.edu/api/access/datafile/2968696
12STDOUT.0000150120462862642https://dataverse.harvard.edu/api/access/datafile/2862642

and 

Dataverse.file_download(files,files.filename[1])
Downloading file : /tmp/available_diagnostics.log

Julia Dataverse API

Dataverse.restDataverse.file_list β€” Function
file_list(DOI::String="doi:10.7910/DVN/ODM2IQ")

Use HTTP, JSON, and DataFrames to list files in a dataset.

Return a DataFrame with filename, filesize, and id.

file_list("doi:10.7910/DVN/ODM2IQ")
source
file_list(nam::Symbol=:OCCA_clim)

Lookup DOI from list of demo data sets (:OCCAclim or :ECCOclim).

source
Dataverse.downloads.file_download β€” Function
file_download(DOI::String,nam::String,pth=tempdir())
DOI="doi:10.7910/DVN/OYBLGK"
filename="polygons_MBON_seascapes.geojson"
Dataverse.file_download(DOI,filename)
source
file_download(list::DataFrame,nam::String,pth::String)
lst=Dataverse.file_list("doi:10.7910/DVN/RNXA2A")
Dataverse.file_download(lst,lst.filename[2],tempdir())
source
Dataverse.restDataverse.dataverse_scan β€” Function
dataverse_scan(nam::Symbol=:ECCOv4r2)

Use HTTP, JSON, and DataFrames to list contents in a dataverse.

Returns header (Dict), dataverses (DataFrame), and datasets (DataFrame).

(header,dataverses,datasets)=Dataverse.dataverse_scan()
Dataverse.file_list(datasets.persistentUrl[1])
source