Stream: geospatial

Topic: โœ” plot arrays from HDF5


view this post on Zulip Jan Range (Mar 22 2023 at 08:59):

Do you think its feasible to embed a plotting routine for the given arrays within an HDF5 file? I think that even with a simple visualization this would be very handy to inspect an HDF5 file. Similar to what H5Web does.

view this post on Zulip Philip Durbin ๐Ÿš€ (Mar 22 2023 at 11:21):

I like to say that anything is possible with code. :happy:

However, I'm a bit concerned about the effort involved.

Generally speaking, we allow external tools to do dataviz. That was one of our the architectural changes when we did the big rewrite of DVN 3 into Dataverse 4.

So here I am fretting that perhaps you want this in the core? In Java? Or were you thinking about an external tool making plots from arrays? If it's the latter, sure, especially if it's contributed!

That said, perhaps it would be faster to get H5Web working? :thinking:

view this post on Zulip Jan Range (Mar 22 2023 at 12:26):

Alright, that makes sense. At first I was thinking about core, but you are correct, an external tool is way better and easier to achieve. Since H5Web is a React Library we could build a simple page for that.

view this post on Zulip Jan Range (Mar 28 2023 at 14:49):

Good news! H5Web is working right out of the box. We can use their example to simply host an instance and supply the files via an URL. They are currently hosting a demo instance to which you can send data, but I am not sure if they are happy when the complete Dataverse community is causing traffic on their server :-D

view this post on Zulip Philip Durbin ๐Ÿš€ (Mar 28 2023 at 14:50):

Hmm! @Jan Range can you please DM me a link? You're right, we should be careful not to overload it. :sweat_smile:

view this post on Zulip Jan Range (Mar 28 2023 at 14:51):

Here is an example taken from Harvard Dataverse, where we just need to supply the direct URL to the file. That aligns pretty nice with the external tools workflow. Next step would be to write a JSON manifest.

https://h5web.panosc.eu/h5wasm?url=https://dataverse.harvard.edu/api/access/datafile/6938813

view this post on Zulip Philip Durbin ๐Ÿš€ (Mar 28 2023 at 14:54):

Awesome! Thanks! I must say, though, the first impression is a little confusing... "No visualization available for this entity."

Screen-Shot-2023-03-28-at-10.52.36-AM.png

But, once you click under "DATA" you start seeing good stuff!

Screen-Shot-2023-03-28-at-10.54.00-AM.png

@Ana Trisovic check this out!! ^^

view this post on Zulip Jan Range (Mar 28 2023 at 14:59):

Yea, I was also confused at first :-D

view this post on Zulip Jan Range (Mar 28 2023 at 15:00):

I would suggest we take their code and containerize it so that other Dataverse installations can host it themselves to use it for file previews. What do you think?

view this post on Zulip Philip Durbin ๐Ÿš€ (Mar 28 2023 at 15:21):

Oh, I thought we would add it to https://github.com/gdcc/dataverse-previewers instead. Is that a possibility?

Generally speaking each previewer wraps a javascript library that does something, like show a preview of a PDF.

view this post on Zulip Jan Range (Mar 28 2023 at 16:08):

I dont know if this is possible since this is a React application. Or is it possible to host it there too?

view this post on Zulip Philip Durbin ๐Ÿš€ (Mar 28 2023 at 16:15):

Not sure. I see two paths, possibly not mutually exclusive:

If this doesn't make sense, lemme know. I'll keep typing or we can do a quick call. :happy:

view this post on Zulip Jan Range (Mar 28 2023 at 17:28):

I can build it to HTML, CSS and JS - That should work at the file previewers repository

view this post on Zulip Philip Durbin ๐Ÿš€ (Mar 28 2023 at 18:18):

Great!

view this post on Zulip Jan Range (Mar 28 2023 at 21:24):

Switched to Vite by the way. Definitely superior to create-react-app!

view this post on Zulip Philip Durbin ๐Ÿš€ (Mar 28 2023 at 21:44):

Heh. Nice.

view this post on Zulip Jan Range (Apr 13 2023 at 21:20):

Had to fight through some bugs, but I finally managed to provide H5Web as static site! Here you can find an example hosted on my server.:

http://82.165.61.139:3000//?url=https://dataverse.harvard.edu/api/access/datafile/6938813&name=test.hdf5

You may replace url and name to try try other files.

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 13 2023 at 21:32):

Amazing! Great work, @Jan Range !

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 13 2023 at 21:34):

I don't know what "D" is but it works!

From http://82.165.61.139:3000//?url=https://dev1.dataverse.org/api/access/datafile/1038&name=ICOADS_R3.0.0_1662-10.nc

Screenshot-2023-04-13-at-5.34.08-PM.png

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 13 2023 at 21:42):

Ah, D is wind direction.

view this post on Zulip Jan Range (Apr 14 2023 at 06:44):

Thanks a lot! Happy to see it is also working for you :-)

view this post on Zulip Jan Range (Apr 14 2023 at 11:15):

I can also change the design of the web-app. The current color scheme looks kinda meh :laughing: Maybe adapt it to the Dataverse scheme or something more "friendly"?

image.png

view this post on Zulip Jan Range (Apr 14 2023 at 13:12):

Another quick styling :-)

image.png

view this post on Zulip Jan Range (Apr 14 2023 at 13:16):

Also add a Dataverse logo?

view this post on Zulip Jan Range (Apr 14 2023 at 13:25):

Sorry for spamming here, but is it possible to embed the external tool within the "Preview" tab?

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 14 2023 at 14:25):

Ha! Yes, we can definitely embed the tool within the Preview tab. We just make sure "types" includes "preview": https://guides.dataverse.org/en/5.13/api/external-tools.html#terminology

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 14 2023 at 14:31):

As for the design, logos, and etc., I was thinking that it might be nice to add HD5Web to the standard "previewers" repo at https://github.com/gdcc/dataverse-previewers

If it helps, here where I added the NcML previewer: https://github.com/gdcc/dataverse-previewers/pull/18

It's actually a good example of "types":["preview"].

If it makes more sense to have a separate, standalone repo for HD5Web, that's fine too. I hope we aren't forking! :sweat_smile: Typically, we wrap a nice, existing tool (PDF preview, for example) written in Javascript in our "previewers" framework. That said, we don't have to follow this pattern if it's limiting.

view this post on Zulip Jan Range (Apr 14 2023 at 14:41):

Perfect! I am really excited to see it working on an installation :dataverse_woman:

view this post on Zulip Jan Range (Apr 14 2023 at 14:44):

I think we can push it to Dataverse Previewers although the built files are everything but readable. Along this I would add the React source code from which it was built. This way we can can modify it easily.

view this post on Zulip Jan Range (Apr 14 2023 at 14:44):

Afaik the previewers are hosted on a GitHub page?

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 14 2023 at 15:40):

Yes, exactly. GItHub Pages. For example: https://gdcc.github.io/dataverse-previewers/previewers/betatest/NcmlPreview.html?fileid=6987711&siteUrl=https://dataverse.harvard.edu&datasetid=6987552&datasetversion=1.0&locale=en

view this post on Zulip Jan Range (Apr 14 2023 at 17:33):

Alright, will submit a PR once back home.

view this post on Zulip Jan Range (Apr 14 2023 at 17:34):

Bit off topic, but which app do you use to obtain these nice window screenshots? They look great!

view this post on Zulip Jan Range (Apr 14 2023 at 21:40):

Tested it on Github.io, works like a charm. Going to open a PR at GDCC previewers.

https://jr-1991.github.io/?url=https://dataverse.harvard.edu/api/access/datafile/6938813&name=test.hdf5

view this post on Zulip Jan Range (Apr 14 2023 at 23:23):

Opened a PR:

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 15 2023 at 14:39):

@Jan Range incredible! Thank you!

We absolutely should consider maintainability which means the source should go somewhere. Let touch base with Jim Myers, who created and has contributed the most to the previewers repo, if he thinks it's a good fit for HD5Web.

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 15 2023 at 14:40):

I'm just using the normal "take a screenshot" built into Mac.

view this post on Zulip Ana Trisovic (Apr 15 2023 at 16:35):

This is really amazing @Jan Range :smiley:

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 15 2023 at 17:39):

@Jan Range I just added it as an external tool on my dev1 server but here's how the query parameters are passed:

https://dev1.dataverse.org/dataexplore/dataverse-previewers/previewers/betatest/HDF5Preview.html?fileid=1038&siteUrl=https://dev1.dataverse.org&datasetid=1036&datasetversion=2.0&locale=en

So the question is, can we change hdf5.js a bit to handle fileid=1038 and siteUrl=https://dev1.dataverse.org as separate parameters?

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 15 2023 at 17:40):

By the way, since we have two preview tools for NetCDF files now, you can see how the UI lets you toggle between them: https://dev1.dataverse.org/file.xhtml?fileId=1038&version=2.0

view this post on Zulip Jan Range (Apr 15 2023 at 19:38):

@Philip Durbin thanks for testing it! Of course, I can change the query parameters to match the default ones. The name parameter is also not that important as it is just a display of the file name within the viewer. The ID will also work. I am currently not at home, will implement it once back and push the update to the PR.

view this post on Zulip Jan Range (Apr 16 2023 at 07:41):

Regarding the source, what do you think about an extra directory which hosts React code and an action that deploys it automatically to the betatest directory? This would open up to other previewers for files. Was thinking about Plotly for the visualisation of tabular data.

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 16 2023 at 12:11):

Hmm, for the source maybe you could add it to a separate branch and link to it from the PR, just to show how much there is and your idea of where it might go.

view this post on Zulip Jan Range (Apr 17 2023 at 06:18):

Alright, sounds like a plan :ok:

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 17 2023 at 12:16):

@Jan Range I went ahead and put HD5Web on the agenda for today's meeting: https://docs.google.com/document/d/1cZJr0JsASzOTxl70RynWtMjIa5dssCCpKPhyV-Bvx0c/edit?usp=sharing

view this post on Zulip Jan Range (Apr 20 2023 at 07:02):

I've managed to reproduce the error and it seems like, when the Previewers HTML filename is not index it fails to load it. I try to adjust it on the React side, my guess its something within the JS files that expect the index.html and isnt working if any other name is given.

view this post on Zulip Jan Range (Apr 20 2023 at 07:17):

Tried it with a different name, coming from React and still not possible using serve -s .. It Only works, when its called index.html - I may be missing something here. Do you know how to fix this?

view this post on Zulip Jan Range (Apr 25 2023 at 17:26):

I happen to make it work with a different name locally!

view this post on Zulip Jan Range (Apr 25 2023 at 17:27):

Seems like there was some expectations from react-router

view this post on Zulip Jan Range (Apr 25 2023 at 17:27):

Going to push the new build. @Philip Durbin can you try it out once its up?

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 25 2023 at 17:27):

Absolutely!

view this post on Zulip Jan Range (Apr 25 2023 at 17:43):

Its up now :dataverse_man:

view this post on Zulip Jan Range (Apr 25 2023 at 17:44):

Used this query for testing:

http://localhost:3000/HDF5Preview?siteUrl=https://dataverse.harvard.edu/&fileid=6938813

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 25 2023 at 17:50):

Deployed but no preview at https://dev1.dataverse.org/file.xhtml?fileId=1038&version=2.0

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 25 2023 at 17:50):

Do these query parameters look right? https://dev1.dataverse.org/dataexplore/dataverse-previewers/previewers/betatest/HDF5Preview.html?fileid=1038&siteUrl=https://dev1.dataverse.org&datasetid=1036&datasetversion=2.0&locale=en

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 25 2023 at 17:51):

I'm happy to change the manifest to line up with what you're expecting.

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 25 2023 at 17:52):

https://dev1.dataverse.org/js/hdf5.js is a 404

view this post on Zulip Jan Range (Apr 25 2023 at 17:53):

Oh thats a pity :sad:

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 25 2023 at 17:53):

I changed it from /js to ./js

view this post on Zulip Jan Range (Apr 25 2023 at 17:54):

Console says that apparently it is not able to find the corresponding css and js

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 25 2023 at 17:55):

That's better!

Screenshot-2023-04-25-at-13-54-25-ICOADS_R3.0.0_1662-10.nc-Root.png

view this post on Zulip Jan Range (Apr 25 2023 at 17:55):

Niiice! How did you make it work?

view this post on Zulip Jan Range (Apr 25 2023 at 17:56):

Ah alright, you changed the path to the js within the HTML?

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 25 2023 at 17:56):

    <script type="module" crossorigin [-src="/js/hdf5.js"></script>-]{+src="./js/hdf5.js"></script>+}
    <link type="text/css" rel="stylesheet" [-href="/css/preview.css"-]{+href="./css/preview.css"+} />
    <link rel="stylesheet" [-href="/css/hdf5.css"-]{+href="./css/hdf5.css"+} />

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 25 2023 at 17:56):

yeah

view this post on Zulip Jan Range (Apr 25 2023 at 17:56):

Sometimes its the simple stuff ... :laughter_tears:

view this post on Zulip Jan Range (Apr 25 2023 at 17:57):

image.png

view this post on Zulip Jan Range (Apr 25 2023 at 17:58):

This change should then work?

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 25 2023 at 18:01):

Yes, exactly.

view this post on Zulip Jan Range (Apr 25 2023 at 18:01):

Cool! Will push it now

view this post on Zulip Jan Range (Apr 25 2023 at 18:02):

That saved my day :laughing:

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 25 2023 at 18:02):

Heh. The last thing is to update the markdown file with curl commands.

view this post on Zulip Jan Range (Apr 25 2023 at 18:04):

Alright, currently editing it

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 25 2023 at 18:05):

Awesome.

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 25 2023 at 18:05):

Oh, and we still need to decide where to put the source code, right?

view this post on Zulip Jan Range (Apr 25 2023 at 18:07):

Yes, I will create another PR that proposes a new dir and action to support React apps built with vite

view this post on Zulip Jan Range (Apr 25 2023 at 18:08):

One question, we were talking about it to be a previewer and an explorer - Is explorer then the right addition to types

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 25 2023 at 18:08):

Ok. I'm also asking Jim about it at https://iqss.slack.com/archives/CVB2SMDFX/p1682446081208379?thread_ts=1682445613.157419&cid=CVB2SMDFX and @mentioned you. Might be faster. :happy:

view this post on Zulip Jan Range (Apr 25 2023 at 18:09):

Saw it on Slack :smile:

view this post on Zulip Jan Range (Apr 25 2023 at 18:11):

So great its working now! Been biting my teeth out on this last mile :laughing:

view this post on Zulip Philip Durbin ๐Ÿš€ (Apr 25 2023 at 18:13):

This is how I added the tool, twice because there are two content types. You may need fewer query parameters. And you should probably use the regular toolUrl pattern instead of my local URL: tool.txt

view this post on Zulip Jan Range (May 09 2023 at 18:46):

@Philip Durbin glad I have seen the comment to the NcML previewer. The PR at the previewers needs to be updated with the most recent source.

view this post on Zulip Philip Durbin ๐Ÿš€ (May 09 2023 at 18:47):

@Jan Range sorry, probably doing too many things at once but what do you mean? :sweat_smile:

view this post on Zulip Jan Range (May 09 2023 at 18:48):

The issue with ncml.js not using the API Token upon fetching. Just saw the comment at your dataverse PR that something is not working.

view this post on Zulip Jan Range (May 09 2023 at 18:49):

Haven't synced the H5Web build with the source code yet. There the API-Token header is included.

view this post on Zulip Philip Durbin ๐Ÿš€ (May 09 2023 at 18:51):

If you're saying your previewer works with drafts, great! Someday I'll go fix mine. :happy:

view this post on Zulip Philip Durbin ๐Ÿš€ (May 11 2023 at 15:00):

Good news! #9480 is in the sprint!

At minimum this will involve adding the new H5Web previewer to the list in the Dataverse Guides.

I'm also happy to test it more, help figure out where to put the source code, etc.

view this post on Zulip Philip Durbin ๐Ÿš€ (May 16 2023 at 14:40):

Did you know H5Web works with Nexus files? https://github.com/IQSS/dataverse/issues/7947#issuecomment-1549809295

view this post on Zulip Jan Range (May 16 2023 at 15:28):

Just saw the comment on GH - Thats awesome!!

view this post on Zulip Philip Durbin ๐Ÿš€ (May 16 2023 at 15:31):

I also just gave a demo of H5Web at the community call. The map of the US was right side up this time!

view this post on Zulip Jan Range (May 16 2023 at 15:36):

Nice! Can you send me the link to the file that has the map? Would love to see it :-)

view this post on Zulip Philip Durbin ๐Ÿš€ (May 16 2023 at 15:39):

Sure it's bi_1985.nc (I forget where we got this). Currently at https://dev1.dataverse.org/file.xhtml?fileId=1037&version=2.0

view this post on Zulip Philip Durbin ๐Ÿš€ (May 16 2023 at 15:40):

I'm pretty sure it came from https://www.northwestknowledge.net/metdata/data/bi_1985.nc

view this post on Zulip Philip Durbin ๐Ÿš€ (May 16 2023 at 15:42):

That northwest site is linked from the bottom of our design doc: https://docs.google.com/document/d/1Ax_sMdgx5ROkIBA7-IC4_hySvgXkk6O8qTZLIvWWnqE/edit?usp=sharing

view this post on Zulip Jan Range (May 16 2023 at 15:44):

Wow this is beautiful! You can even go through the time dimension :heart_eyes:

view this post on Zulip Jan Range (May 16 2023 at 15:45):

This would be a perfect demo file for our slides

view this post on Zulip Philip Durbin ๐Ÿš€ (May 16 2023 at 15:54):

orly

view this post on Zulip Philip Durbin ๐Ÿš€ (May 16 2023 at 15:55):

How do you go through the time dimension?

view this post on Zulip Jan Range (May 16 2023 at 17:28):

image.png

You can use the slider on the left. Very nice!

view this post on Zulip Philip Durbin ๐Ÿš€ (May 16 2023 at 17:46):

I see. I guess I'm on day 161 of 1985. Now I'll go back to the future.

view this post on Zulip Philip Durbin ๐Ÿš€ (May 18 2023 at 21:14):

I just made a PR into your PR (and left some comments): https://github.com/gdcc/dataverse-previewers/pull/30#issuecomment-1553653523

Overall, this is looking good! :rocket:

view this post on Zulip Philip Durbin ๐Ÿš€ (May 19 2023 at 13:53):

I just made this PR: add H5Web to guides #9600

view this post on Zulip Philip Durbin ๐Ÿš€ (May 19 2023 at 13:54):

Including a screenshot of the map, right side up :happy:

Screen-Shot-2023-05-19-at-9.50.41-AM.png

view this post on Zulip Philip Durbin ๐Ÿš€ (May 19 2023 at 13:54):

@Jan Range for the screenshot I manually removed the tool title and the blue box.

view this post on Zulip Jan Range (Jun 12 2023 at 18:46):

The PR is up :raised_hands:

https://github.com/gdcc/dataverse-previewers/pull/35

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 12 2023 at 18:58):

Looks great! I'm a little confused though. Can I go ahead and close the older PR now and say that the newer one is replacing it?

view this post on Zulip Jan Range (Jun 12 2023 at 19:03):

Yes, I left it open though for the case if Jim might not accept the addition of the PR. But we can re-open it anyway if thats the case.

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 12 2023 at 19:11):

Ok, I went ahead and closed it. Yes, we can re-open it if necessary.

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 12 2023 at 19:11):

I did click "request review" from Jim.

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 20 2023 at 14:20):

@Jan Range did you get a chance to read Jim's feedback? https://github.com/gdcc/dataverse-previewers/pull/35#issuecomment-1593671142

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 20 2023 at 14:23):

One part of the feedback is pretty straightforward. I think he wants you to move your stuff into the "betatest" directory.

view this post on Zulip Jan Range (Jun 20 2023 at 15:10):

Hi @Philip Durbin :wave: Yes, I saw it and will implement the changes today. Indeed, they look straightforward :-)

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 20 2023 at 15:20):

No rush, but thanks!

view this post on Zulip Jan Range (Jun 21 2023 at 11:09):

Updated the PR - Now puts everything in betatest as requested by Jim

view this post on Zulip Jan Range (Jun 21 2023 at 11:09):

@Philip Durbin is it possible to test it at your dev instance?

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 21 2023 at 13:03):

Yes! I got hit with some paperwork but soon! Thanks!

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 11:33):

@Jan Range I need to look closer but https://dev1.dataverse.org/dataexplore/dataverse-previewers/previewers/betatest/HDF5Preview.html?fileid=1066&siteUrl=https://dev1.dataverse.org&datasetid=1065&datasetversion=1.0&locale=en isn't working.

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 11:34):

It might be a relative vs absolute path thing.

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 11:34):

It's trying to pull down https://dev1.dataverse.org/js/hdf5.js which probably isn't right.

view this post on Zulip Jan Range (Jun 22 2023 at 13:12):

@Philip Durbin thanks for testing! Yes, the js and css paths need to be relative in the resulting html. Going to check if this can be taken care of in the build phase.

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 13:12):

Thanks so much!

view this post on Zulip Jan Range (Jun 22 2023 at 13:31):

Fixed it :smile: Vite is so nice! Was able to include this in the build file and now it should work

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 13:42):

Closer! I pulled but now I get a different error:

Screenshot-2023-06-22-at-9.41.57-AM.png

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 13:43):

Same URL as above if you'd like to try it.

view this post on Zulip Jan Range (Jun 22 2023 at 13:54):

Thanks! Do you use a private dataset or a public one?

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:00):

Public. Here's the file: https://dev1.dataverse.org/file.xhtml?fileId=1066&version=1.0

view this post on Zulip Jan Range (Jun 22 2023 at 14:09):

image.png

Seems like there is some authorization error. I tested your link, but exchanged siteURL and fileID which point to Harvard's Dataverse. Here everything works fine, will check where the diffs are and hopefully find a solution.

https://dev1.dataverse.org/dataexplore/dataverse-previewers/previewers/betatest/HDF5Preview.html?siteUrl=https://dataverse.harvard.edu/&fileid=6938813

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:11):

Ah, maybe we saw this before? Or maybe it's just deja vu. :sweat_smile:

view this post on Zulip Jan Range (Jun 22 2023 at 14:11):

Haha yea I have the feeling we've been there already

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:13):

I can download with wget https://dev1.dataverse.org/api/access/datafile/1066

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:13):

No auth problems.

view this post on Zulip Jan Range (Jun 22 2023 at 14:13):

Maybe the issue is that I pass an empty "X-Dataverse-key" header since the URL does not include apiToken. Could that be the source?

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:14):

Huh. Maybe?!?

view this post on Zulip Jan Range (Jun 22 2023 at 14:14):

Because that is the novelty compared to the previous builds we've tested

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:15):

Want me to copy https://github.com/JR-1991.keys into my dev1 server so you can log in?

view this post on Zulip Jan Range (Jun 22 2023 at 14:18):

Okay, if you wait fro long enough H5Web returns an error message, that the given file is not a valid HDF5 - Thats quite odd!

image.png

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:20):

Weird. This one is super small but it's still spinning: https://dev1.dataverse.org/dataexplore/dataverse-previewers/previewers/betatest/HDF5Preview.html?fileid=1038&siteUrl=https://dev1.dataverse.org&datasetid=1036&datasetversion=2.0&locale=en

view this post on Zulip Jan Range (Jun 22 2023 at 14:21):

I get the same error with H5Web's instance

https://h5web.panosc.eu/h5wasm

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:21):

So maybe it doesn't matter how big the file is... maybe it'll eventually just give that error about not being a valid file.

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:22):

Hey! It DOES work for a draft file!

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:23):

Here, I'll paste the URL and then reset my API token: https://dev1.dataverse.org/dataexplore/dataverse-previewers/previewers/betatest/HDF5Preview.html?fileid=1073&siteUrl=https://dev1.dataverse.org&apiToken=6c626b0a-7f7f-4f47-816c-34fc44a3c6a9&datasetid=1072&datasetversion=:draft&locale=en

view this post on Zulip Jan Range (Jun 22 2023 at 14:23):

Philip Durbin schrieb:

Hey! It DOES work for a draft file!

Awesome!!!

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:24):

So yeah, maybe try not sending the token if there is no token??

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:24):

Your idea above :big_smile:

view this post on Zulip Jan Range (Jun 22 2023 at 14:25):

Alright, I'll fix that and push an update

view this post on Zulip Jan Range (Jun 22 2023 at 14:33):

The changes are now online, but it is still not working. This might be a problem related to H5Web itself. If we are not able to resolve it, I will open an issue there

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:34):

Online? I haven't pulled yet. Should I?

view this post on Zulip Jan Range (Jun 22 2023 at 14:35):

Ah okay, thought it would be updated automatically. The built files are updated in the PR though

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:35):

Cool, let me go pull. I'm hosting it locally:
"toolUrl":"/dataexplore/dataverse-previewers/previewers/betatest/HDF5Preview.html",

view this post on Zulip Jan Range (Jun 22 2023 at 14:35):

image.png

Thats so odd since they claim nc files are working

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:36):

https://dev1.dataverse.org/dataexplore/dataverse-previewers/previewers/betatest/HDF5Preview.html?fileid=1038&siteUrl=https://dev1.dataverse.org&datasetid=1036&datasetversion=2.0&locale=en looks good! That's the small file, only 120 KB or so

view this post on Zulip Jan Range (Jun 22 2023 at 14:37):

Does it work for you? I am stuck in the loading page

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:37):

Screenshot-2023-06-22-at-10.37.30-AM.png

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:37):

Works!

view this post on Zulip Jan Range (Jun 22 2023 at 14:39):

Niiice!

view this post on Zulip Jan Range (Jun 22 2023 at 14:39):

Ah now it works on my side too :raised_hands:

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:41):

Screenshot-2023-06-22-at-10.40.59-AM.png

view this post on Zulip Philip Durbin ๐Ÿš€ (Jun 22 2023 at 14:48):

I just moved https://github.com/IQSS/dataverse/pull/9600 to "in review"... just a doc change to advertise the new previewer.

view this post on Zulip Notification Bot (Aug 11 2023 at 18:27):

Philip Durbin has marked this topic as resolved.


Last updated: Nov 01 2025 at 14:11 UTC