Should we move dataverse.org from OpenScholar to GitHub? We recently did this for the GDCC website, moving it to https://github.com/gdcc/www.gdcc.io
Benefits include:
I have a specific technical question, however. @Ceilyn Boyd and I both agree that some some of static site would be best. The GDCC website is static and uses Sphinx (same as guides.dataverse.org) but we're interested in hearing about other static site generators. Do you know of any good ones? Here are our requirements so far:
The static site generators I've used the most are:
Please feel free to weigh in on the idea of moving itself as well as which static site generator to use. Thanks! :heart:
Don't forget the Asciidoc universe. One of the more well known tools for it are https://docs.asciidoctor.org and https://antora.org/. You can even use Asciidoc in Javadocs. https://github.com/asciidoctor/asciidoclet It might be worth thinking about Asciidoc also in relation to skill building for #docs At least it's something you can also use from a Maven build, not just standalone. (Also note that Jekyll and Hugo have plugins to support Asciidoc)
Right. Asciidoc. Hmm. I think @Ceilyn Boyd and I were thinking Markdown but I didn't put it above under "requirements so far". :thinking: Thanks for the suggestion.
I started playing with a Sphinx based site at https://sphinx.dataverse.org
The code is at https://github.com/IQSS/sphinx.dataverse.org and I used https://github.com/gdcc/www.gdcc.io as a starting point.
I dunno, I'm already having trouble controlling header links: https://github.com/IQSS/sphinx.dataverse.org/issues/1 ![]()
We definitely don't have to use Sphinx if there's something better.
@Vaida Plankytฤ @Dieuwertje Bloemen @Philipp Conzett @Amรฉlie Fiocca @Dimitri Szabo rather than cross-posting in #docs I'll just mention you here. :grinning:
I'm curious what you think of https://sphinx.dataverse.org so far. And about the whole idea of moving dataverse.org to GitHub.
Wow, this works pretty well: https://urltomarkdown.com
@Philip Durbin ๐ The new site looks very cool and tidy! A thing that bothers me with GitHub-based websites is that it seems it's not possible to have URLs to be open in a new tab or window.
@Philipp Conzett hmm, it should be possible. Which page?
Also, to all, I just pushed a lot more content to https://sphinx.dataverse.org and reorganized a bit. Feedback is welcome!
Oh? Last I checked, I couldn't find a way to define an open_blank URL with Markdown.
@Philipp Conzett I think I found a solution. I enabled myst_links_external_new_tab from https://github.com/executablebooks/MyST-Parser/pull/857.
Please go to https://sphinx.dataverse.org/about and click "An Introduction to the Dataverse Network as an Infrastructure for Data Sharing". It should open in a new tab. There's no visual indication that a tab will be opened though. I hope that's ok.
@Ceilyn Boyd and I just talked through a new doc I'm calling dataverse.org on GitHub information architecture. Comments are welcome!
Ah, great! Thanks, @Philip Durbin ๐ Could you enable this feature on gdcc.io?
Yep! Can you please create an issue? I have to update the version of myst.
Ok, I just pushed another update to https://sphinx.dataverse.org
Following that doc, we're trying out top level sections called "Project" and "Learn". Feedback welcome.
oh! this is a great idea. having the website be directly editable in a communal way would make it so much easier to iteratively improve it. though since sphinx (and any documentation-focused framework) are so text-heavy, it'd be helpful to think on how we could break up the text so it's easier to navigate.
Yes! Absolutely. I'd love to break up the text a bit.
Ceilyn and I have some ideas for the homepage. There's a tab called "homepage" in that doc above.
Lately, I've been trying to focus on the hierarchy. Where pages live. This is kind of a mess in the old site. :sweat_smile:
would love to make some time during the docs meet to help review some of this! ah yes, 1-2 big call-to-action buttons would be really good!
being able to quickly iterate on what's added to the header is so good! thanks for pushing ahead with all this :sunglasses:
Yeah, that big "download" button is for you. :grinning:
The header is a bit tricky. I complained about this at https://github.com/IQSS/sphinx.dataverse.org/issues/1
I'd like to figure out how to exclude a top-level page or directory from the header.
aw! what would be a static page generator without quirks...
Looks great, thanks Phil!
At some point we should ask @Julian Gautier what he thinks about moving https://dataverse.org/cts-guide to https://guides.dataverse.org :sweat_smile:
Come to think of it, "Quickstarts" should probably go under "Learn". :thinking:
I actually like Quick Starts where it is. But maybe consider adding a Download button to the top of that page, too?
Philip Durbin ๐ said:
At some point we should ask Julian Gautier what he thinks about moving https://dataverse.org/cts-guide to https://guides.dataverse.org :sweat_smile:
Yeah let me know when you wanna talk about it
Well, the main thing I'm thinking is that if we're ok with moving it, it might be nice to move it before 6.5 is released so that we can link to it. (Redirect to it.)
Ah actually I misunderstood your question lol. You're asking about moving the CTS guide from Dataverse.org to the Dataverse Guides. Why do you want to move it to the guides?
Meh, I dunno. It's a guide. Seems like a better fit.
Actually, if you have a couple minutes I could walk you through that doc above.
I'm also wondering about the Best Practices docs. Some of this content never gets touched. Maybe it's perfect already. :halo:
Yeah do you want to talk at 3:30 today?
Sure, sounds good.
At a high level, I'm trying to simplify things a bit. Spring cleaning. :broom:
https://harvard.zoom.us/my/pdurbin?pwd=em1WNUZGbnY2YjhxNEdSbjJJMXNSUT09
@Julian Gautier whenever you're ready. And anyone else who's interested, I guess!
Ok, we decided to try putting the CoreTrustSeal guide under "Best Practices". Thanks!
And talked about all kinds of other stuff, of course. :grinning:
I mean, https://sphinx.dataverse.org is kinda getting there, right? :sweat_smile:
@Jan Range should we consider using MkDocs for dataverse.org? I know you have an open pull request to switch pyDataverse to it.
@kuhlaid should we consider Docusaurus for dataverse.org? You set up that nice demo site for us a while back and it's still up.
Thatโs a tough one. I really like MkDocs for its simplicity and how quickly it allows you to create something useful. Its seamless integration with GitHub Pages is another big plus. However, it always gives off a very "documentation-like" vibe, which might not be ideal for a website overview, as it can appear somewhat cluttered. I use it for pyDataverse since itโs widely adopted in the Python community.
I just came across Docusaurus, and at first glance, it looks fantastic! The MDX integration is an excellent feature that provides a lot of flexibility. While you can embed HTML/JS into Markdown files with MkDocs, it sometimes feels a bit clunky. For Dataverse.org, Iโd personally prefer Docusaurus over MkDocs because of its flexibility and more polished appearance.
@Jan Range thanks! And do you have an opinion on Sphinx or Hugo? I've added notes for these in the dataverse.org on GitHub information architecture and static site generator bakeoff doc. Or Jekyll? Jekyll isn't in that doc but which we use it for https://py.gdcc.io.
Iย donโt have any experience with Sphinx, but Iโve used Hugo a bit and think itโs a great tool for building simple websites. However, things start to feel cumbersome when you move beyond the default templates and try to customize them. Not because of Golang itself, but due to the complexity of the template structure, at least with the ones Iโve worked with.
For example, I built https://enzymeml.org with a colleague using Hugo. It was straightforward at first, but when we wanted to make structural changes, it became a real challenge. Thatโs why weโre transitioning to MkDocs now. Itโs easier to maintain and more practical to hand off to a student for ongoing updates, but imo it cannot compete in terms of looks unfortunately.
I still like Docusaurus. I build another site that uses it for documenting a Dataverse collection: https://jocoknow.vercel.app/docs/datasets/10.7910_DVN_SKP9IB/overview
Docusaurus has built in broken-link/build checking so it will complain if you missed something. Anyway, it works for me. I have not figured out how to get it to push to GitHub pages but it builds for free on Vercel easily.
It does look nice, thanks.
on first impression, the Docusaurus demo site looks so good! is it possible to create additional "static-style" pages like the landing page? Maybe we could review the various items in the dataverse.org header and see where they point, and see if the content can be consolidated into just a few pages. Then the rest would really just be the main documentation right?
re:Hugo/Jekyll, I'd echo that while they're nice for custom websites and some templating, they don't come with out-of-the-box niceties that are specifically designed for heavy documentation use, and could easily become a major maintenance weight :weight_lift:
@kuhlaid if you're up for it, I'm happy to create a repo at https://github.com/IQSS/docusaurus.dataverse.org if you want to stub out a prototype for us. Like I've done at https://github.com/IQSS/sphinx.dataverse.org (and you're welcome to borrow some of that Markdown if you like).
@Vaida Plankytฤ well, to be clear, we're talking about the brochure site at dataverse.org, not the documentation site at guides.dataverse.org . So I wouldn't say the use case is "heavy documentation use". It's more, I dunno, "sell Dataverse to the world", I guess. :grinning: Make it look awesome. :grinning:
aha, i misunderstood! that makes sense. how complex is the existing site? it seems there's a few template-needing content, like blog posts, publications, etc. but otherwise it's mostly a collection of static pages?
Medium complex? Please see the "hierarchy" tab in the dataverse.org on GitHub information architecture and static site generator bakeoff doc.
As a reminder, we have this topic of moving the Dataverse website to GitHub pretty high on the agenda for the docs working group meeting in a couple hours.
A new mockup from the docs working group meeting ^^
And we took a lot of notes.
The "image with people collaborating" is inspired by https://dspace.org
Screenshot 2024-11-22 at 11-08-03 DSpace Home - DSpace.png
I kinda wish we had recorded it. It was a good discussion!
I set up a prototype using Hugo at https://hugo.dataverse.org
Source at https://github.com/IQSS/hugo.dataverse.org
Screenshot 2024-12-07 at 06-59-33 The Dataverse Project.png
Especially after our conversation in the recent docs meeting, I'm feeling less constrained to keep things the same as the existing dataverse.org website.
Especially the home page.
This attitude and the flexibility of Hugo itself feels a bit freeing.
With Sphinx, I haven't figured out how to decouple the header links from the top level content. That's what https://github.com/IQSS/sphinx.dataverse.org/issues/1 is about.
With Hugo, I simply specify the links I want: https://github.com/IQSS/hugo.dataverse.org/blob/d5b2f7580106e87683748d1394392e9ae44ea145/hugo.yml#L5-L18
menu:
main:
- identifier: about
name: About
url: /about/
weight: 10
- identifier: community
name: Community
url: /community/
weight: 20
- identifier: download
name: Download/GitHub
url: https://github.com/IQSS/dataverse/releases
weight: 30
This feels freeing because it's easier to decide if we want to put events at /events or /community/events.
With Sphinx, having events at /events means that "Events" shows up in the header as a link. So I end up feeling like Sphinx is forcing us to put content into fewer top level buckets.
I shouldn't say Sphinx. I really mean the PyData theme we're using.
Anyway, Hugo is fun so far. But I might need some help from someone who's better at website design and coding. :sweat_smile:
I must be having fun because I keep tweaking this new Hugo site. Here's the latest:
Screenshot 2024-12-07 at 12-27-22 The Dataverse Project.png
What do you think? :sweat_smile:
Ok, I'm trying to be patient but let's face it, I'm not. :sweat_smile: @Vaida Plankytฤ @Dieuwertje Bloemen @Amรฉlie Fiocca @Dimitri Szabo @Philipp Conzett @Oliver Bertuch @Ceilyn Boyd @Julian Gautier @Dwayne Liburd @Sonia Barbosa et. al, what do you think of https://hugo.dataverse.org ? :grinning:
Looks all good to me! Is Hugo also what will be used for the guides?
Heya Phil. Thanks for pinging me. Sorry I don't have time right now to check it out. Trying to make progress on other projects
Hate to say it, please don't take this the wrong way: the design seems rather old school, not very modern. More like amateurs hacking on an open source software than a professional product you can put your trust in.
@Philipp Conzett no, the plan for now anyway is to stick with Sphinx for the guides.
@Oliver Bertuch can you please link to some examples that look more professional, modern, and trustworthy?
Although I hate our menu, I like the overall structure of my home org https://www.fz-juelich.de/en
Also pretty good structure and nice layout at a competitor: https://dspace.org/
I also like the look and feel of https://www.harvard.edu/
Thanks, this helps.
Do you think we should use a different Hugo theme?
Or use something other than Hugo?
I'm asking this because I'd like to settle on a platform that's good enough and git-based. Then people can make pull requests to improve the website.
ooh, this is all exciting! snowed under a bit (metaphorically) but hoping to review this very soon. :geek:
Philip Durbin ๐ said:
I'm asking this because I'd like to settle on a platform that's good enough and git-based. Then people can make pull requests to improve the website.
Hugo/Jekyll + md is a fine tech choice. I was just complaining about design and potentially content. I approve and appreciate GitOps.
Ok, thanks. Is Sphinx also an acceptable tech choice for you? I have a prototype going at https://sphinx.dataverse.org as well but I think I like Hugo a bit better.
Part of the problem here is that I'm not a web designer. I would love for someone to help make the site more modern, professional, and trustworthy.
Sphinx is fine as well. Nice combo with MD possible too. If the header is too much if a burden, ditch it
Yeah I'm not at design as well :grinning_face_with_smiling_eyes:
My most popular tweet is "if you think experts are expensive you should see how much amateurs cost". :sweat_smile:
Would it be worth it to reconsider tackling the migration and redesign separately? That's what's been done for the SPA for the most part, to make things more manageable
That's a good thought. Using Sphinx (well, the PyData theme, especially the header) was sort of forcing me to make the site less flat but otherwise I was trying to keep https://sphinx.dataverse.org mostly true to the original site.
With https://hugo.dataverse.org I went in a different direction, trying to incorporate feedback on how we could make the homepage better.
But maybe I could create a https://hugo2.dataverse.org that tries to stay true to the original. I think the site can be as flat as we want with Hugo.
Hi, Philip ! I don't really know the technical options of these tools but my first view is visual and I'm not fan of the big empty space on the sides. I suppose it's possible to customize but yes we need a designer :sweat_smile:
It's true. There is quite a bit of empty space on the sides.
heh, we've been having the same conversation at RSpace about our design. I made it "vertical" so that it would be easier to make mobile-friendly, but it results in a lot of scrolling and doesn't feel like the best use of space. the more marketingy websites tend to be good at filling up the whole space, while I know a lot of technical-focused projects tend to have more of a "here's an intro to our thing in markdown" look that feels more like scrolling an article.
True. There's less empty space on mobile. :sweat_smile:
Another thing I was thinking while hacking on https://hugo.dataverse.org is where the blog should go. With the theme I'm using (PaperMod) it gets added to the bottom.
Generally speaking, I'm wondering if where content that changes (such as a blog) should be. The old/current site at https://dataverse.org has the blog and tweets at the bottom (though the tweets seem to be broken).
i'm off for a long winter break, but i'd love to discuss this more in the new year! this is all great stuff and it's good we're talking about how exactly to best present the content as well as the technical stack.
Oh, and the map. The map also changes. It's toward the bottom, just above the blog.
New Years Resolution: new website for Dataverse! :tada:
hm, having a preview to the latest few blog posts on the landing page could be quite nice, or have it be a little "notice" at the very top:
image.png
the map is quite powerful, i'd say it can go pretty high up, alongside the key "metrics" we discussed :sunglasses:
something like https://zeon.studio/preview?project=hugoplate is a good example of using up horizontal space, as well as showcasing features with a list of bullets and case studies/testimonials! and a nice hero section at the top that gives a TL;DR and action item :sunglasses:
alright, time for me to go have some celebratory bubble tea! :tea: take care y'all, was great meeting and working with you this year! happy holidays :holiday_tree:
Ah, this one: https://themes.gohugo.io/themes/hugoplate/
Interesting. Thanks!
I agree with @Julian Gautier that maybe the design and the migration should be separated. Would it be an idea to offer a way for people who have time to make up some options/mock-ups (can also be done in photoshop or whatever) to get some concrete ideas going that can be compared or chosen between. If there's a design, it's always easier to turn it into code than to have some html and make it look pretty without any idea of where you're going design-wise in the end, I think.
While primarily utilized for simultaneous code and design generation, you might consider trying v0. Its chat-based interface allows you to explore design options and brainstorm ideas. I also use it to create designs since I'm not particularly skilled at it. For instance, you can chat and modify the designs further.
Here's an example where I prompted to convert the existing page into a more modern design:
https://v0.dev/chat/XBemEnIRQ9O?b=b_HkA5I57ZFhx
I made a quick and simple mock-up losely based on some of the examples discussed (and the IQSS website, which is great for color inspiration because of the orange: https://www.iq.harvard.edu/). My mock-up example. If we want a full redesign, I would just start collecting ideas and then picking the best options/elements and finalizing a design. But maybe that's something to take our time for (possibly something for the community meeting, or is that too far out?).
Wow! Thanks, @Dieuwertje Bloemen! I'm just going to grab a screenshot to put inline:
Screenshot 2024-12-11 at 7.02.59โฏAM.png
You too, @Jan Range! Another screenshot:
Screenshot 2024-12-11 at 07-05-10 v0.png
The only thing that makes me nervous, personally, is having some design and not being able deliver on making it real. Given my limited skills in this area, it's much easier for me to add content (Markdown) to an existing theme (let's say say a Hugo theme) than to create the theme myself.
So maybe that's a question... does anyone here have the skills (or know someone who does) to create a Hugo theme? That's assuming we go with Hugo. I can tell you that any Hugo theme I come up with is going to be worse than PaperMod or Hugoplate.
I have a slight preference for just picking a theme that's "good enough" and shipping a new site.
If we have someone who can create a theme for us, that changes everything. It gives us more options and we can throw whatever designs we want at this talented individual.
ohh we are cooking :cooking:
Yes! I'm loving all the designs and feedback! :heart:
I think that heavily depends on what you want to achieve using the website. While it is nice to have a beautiful design, the question is if it is necessary for users. In its current state, it is informative, and the goal is to transport this information effectively while being maintainable/developable. Hence, I lean to @Philip Durbin ๐ suggestion of taking a theme and leaving it as such - I would value the ease of use and feasibility over a design, given it is a community project.
My experience customizing Hugo has not been great, but that may depend on your template. In a nutshell, it is HTML sprinkled with GoLang directives/variables. In itself, it is not complex, but those template file structures and namings can be a pain :grinning:
there's an additional difficulty to having a customised theme -- adding new elements and making tweaks would have more of a bottleneck because we'd be relying on a specific persom who is able to make the design consistent. i feel that'd add friction to keeping the website up to date :big_smile: echoing what Jan has said basically
I agree, from an experience with my association's website... we had someone create a beautiful template (https://informasciences.ch/) at the start of the project but he left and now we're in a problem because no one really has the skills to, update or modify it easily, so we're thinking of going back to something more generic and easier to maintain.
i think the following approach could be good, which we have started on already:
step 3 would essentially be the migration to GitHub step as well.
idk if we should shortlist 2 or so setups and then have us try and edit the content, e.g. add a blog post, change some text, and see which one feels less painful? :thinking:
the content would also need to consider what angle we're taking; is this page a marketing site promoting dataverse to potential institutions/users? what do they care about knowing? what do existing users want as quick links? the initial hugo page that was shared had a lot of that figured out imo!
@Amรฉlie Fiocca interesting that you're considering a switch to a simpler, more maintainable site! Makes sense. I hope you get to keep the robots and general sci-fi vibes. :big_smile:
@Vaida Plankytฤ yes to all of that. I do think the choice of Hugo or Sphinx or whatever is important. I'd encourage anyone who thinks they might want to edit the site locally (on their laptop or whatever) to clone these repos and see if you can get them working:
If the README doesn't explain the setup well, please let me know!
I would suggest making some local edits. Play around. Which do you like better, Hugo or Sphinx?
If you like neither one, please say so! :big_smile:
By the way I looked at this mockup a lot when hacking together https://hugo.dataverse.org
It's the one the #docs working group came up with the other day. And I referred to the notes.
I do think we're circling ever closer to agreeing on the content we want. :big_smile:
Oh, and by the way, I started trying to add a "trusted by" section to the Hugo site but I couldn't figure out how to place logos side by side.
This is the kind of stuff I could use help with. :heart:
One more quick point: we recently moved the GDCC website from OpenScholar (Drupal) to GitHub Pages and from my perspective the benefit of opening up the editing of the site via pull requests is huge.
So I'm in favor of moving quickly and reaping that benefit. Let the community make pull requests soon, I say!
sounds like a plan! i'll have a go myself once i'm back. i've played with Hugo before and I liked how easy it was to add content once the various templates were set up. looking forward to seeing any new stuff you all cook up in the meantime! :sunglasses:
@Dwayne Liburd
@Sonia Barbosa yesterday @Dwayne Liburd and I had a nice chat about all this.
We discussed this today at our monthly meeting. There's a new comment at https://github.com/IQSS/dataverse-pm/issues/349#issuecomment-2898908802
BOOM! :bomb: A new contender by @Germรกn Saracca! Astro!
Code at https://github.com/IQSS/astro.dataverse.org
Didn't know about Astro - another example of static site generators. As I know Jekyll and Hugo better, I looked for a comparison - maybeย this oneย helps others as well.
I just gave the Astro site (and this topic in general) a shout out in the community news: https://groups.google.com/g/dataverse-community/c/WGAZpOfRvHk/m/uCLTf-VgAgAJ
For anyone new to this topic, https://github.com/IQSS/dataverse-pm/issues/349 is the issue to track this project and dataverse.org on GitHub information architecture and static site generator bakeoff is the main doc to look at.
@topic I want to thank everyone who contributed to the conversation about the design of the dataverse.org home page. In particular, I'd like to thank @Dieuwertje Bloemen and @Vaida Plankytฤ ๐จ who I believe made the following mockups:
@Dwayne Liburd @Sonia Barbosa @Ceilyn Boyd @Danny Ebanks @Cheng Shi Rebecca and I just reviewed these mockups and they were very useful!
@Philip Durbin ๐ yes, i made that second one in Paint! :smile: really glad they were useful. let us know how the thinking on the website evolves, I'd love to help out.
dataverse.org was migrated from OpenScholar to vanilla Drupal today. I just announced it here: https://groups.google.com/g/dataverse-community/c/IMheyu5XkOA/m/p0EGFtMiAQAJ
On a related note, I have a database backup of the vanilla Drupal content but I don't know how to get it running locally (so that I can hopefully add a REST API and get content out to migrate to a static site). Can anyone help with this? Here's the docker compose I tried: docker-compose.yml
Last updated: Nov 01 2025 at 14:11 UTC