Deirdre Kirmis (Jun 11 2026 at 23:53): Deirdre Kirmis (Jun 11 2026 at 23:53):Hi all,
During the Large Data Working Group discussion at DCM2026, it became clear that many of the challenges we face around large data are really part of broader conversations about supporting and operating Dataverse. While our time was limited and we weren't able to dive deeply into many of them, the discussion highlighted that some participants are focused on researcher support and data management workflows, while others are focused on the technical infrastructure and operational side of running Dataverse.
That got me thinking that, rather than splitting the existing Large Data Working Group into two groups (as we discussed), there may be value in creating a separate Infrastructure & Operations Working Group. The goal would be to provide a forum for repository administrators, system administrators, developers, devops engineers, and others responsible for operating Dataverse and its supporting services. I'll admit that I have a personal interest in this as well .. I would love the opportunity to learn from others who are solving these kinds of infrastructure and operational challenges. :slight_smile:
Potential discussion topics could include storage and store configuration, cloud and on-prem architectures, backup and disaster recovery planning, operating system and application upgrades, security, monitoring, automation, performance tuning, scalability, Globus integrations, and other operational best practices. While large data would certainly remain an important topic, the scope would be broader and focused on the technologies and processes that support Dataverse and its users.
Would there be interest in a group like this? If so, I'd love to hear what topics you'd find most valuable and whether you'd be interested in participating.
Oliver Bertuch (Jun 12 2026 at 09:54):Let me just point out that this additional working group may have a huge overlap with the existing Containerization Working Group. We discuss (mostly containerized) ops on a regular basis in our meetings and in the channel.
I'm happy to discuss merging things, if that helps to avoid scattering into too many working groups.
Philip Durbin 🚀 (Jun 12 2026 at 09:57):Yep, secretly a devops group as I said in a recent talk:
Deirdre Kirmis (Jun 12 2026 at 14:26):Ah .. I guess I've been missing out then. Good to know .. okay nm this idea.
Oliver Bertuch (Jun 12 2026 at 14:53):We do try to keep things container related. But we could open up if people would like to attend. We have only very few people joining us for container related stuff. So this might be an opportunity.
What I'm trying to say: happy to expand the meeting scope. But we do need to talk about containers, too. Might be scary for people.
Philip Durbin 🚀 (Jun 12 2026 at 15:24):Yeah, one thing about the container meeting is that we talk a fair amount about dev stuff since we use containers for dev. I wouldn't want the infra and ops folks to get bored! :sweat_smile:
Deirdre Kirmis (Jun 12 2026 at 16:51):I'm guessing most infra/ops folks are very familiar with and support a variety of containerized apps these days. Many of the apps that I maintain are running in containers and I have deployed a containerized installation of Dataverse (using your WG procedures). However, I'm not a developer so I'm generally more interested in the operational side of running and maintaining the platform than in designing the containerization approach.
The current group does seem very dev-heavy. IMO if the preference is a combined working group, the focus should be "Operations & Infrastructure" and containerization should be one subtopic among all of the other categories of things that infra/ops folks do. The terrain is much larger than just the running application, as it includes all of the other services that support it (ie: integrations, monitoring, security, networking, storage, backups).
I guess I missed the WG meeting yesterday so I will try to attend next month and see if I can keep up! :wink:
Philip Durbin 🚀 (Jun 12 2026 at 16:54):@Deirdre Kirmis here's an idea. What if we make infra/ops the main topic for a community call? And if people are really into it, you could ask if people want to form a working group.
Deirdre Kirmis (Jun 12 2026 at 18:35):I don't think it needs to be the main topic, but I can add it as a community question and see if anyone else is interested. Thanks!
Philip Durbin 🚀 (Jun 12 2026 at 18:53):@Deirdre Kirmis sounds good. Please feel free to go ahead and add it to the next agenda.
Oliver Bertuch (Jun 12 2026 at 23:04):@Deirdre Kirmis I hear you! The two most present persons on the call are @Philip Durbin 🚀 and me, and we're both devs. But at least I am a dev-op, as I run Jülich DATA and all the necessary infrastructure for it. (AFAIK Phil is not involved in any of that for Harvard DV, but I might be wrong). So I am more than happy to talk about ops during these meetings.
This is especially important when I put the dev hat back on: I always try to align the dev side to what the op side needs! But I need feedback about what works, what doesn't, etc. Maybe we should rephrase the website for the CT WG somewhat to make it clear we very much welcome ops attending!
Also, @Philip Durbin 🚀 does a great job at moderating these meetings 
. (Greatest CCO of all time!) We already prioritze any questions from people coming in before hitting the agenda. Maybe we need to be even mor explicit about this and make it more obvious that any ops questions can be addressed first, before we hit any dev topics.
Philip Durbin 🚀 (Jun 13 2026 at 21:42):Right, I'm not involved in infra or ops for Harvard Dataverse, though I'm happy to give opinions! :sweat_smile:
Philip Durbin 🚀 (Jun 13 2026 at 21:43):I did work in ops in previous jobs at Harvard and Berklee and a couple other places.
Philip Durbin 🚀 (Jun 13 2026 at 21:47):I kicked off a topic at #zulip > #ops channel? about if we should start a new channel here in Zulip.
Philip Durbin 🚀 (Jun 13 2026 at 21:52):And yeah, I'm happy to talk about ops stuff first in the Containerization working group. And then let the ops folks go before we dive into dev stuff.
Dieuwertje Bloemen (Jul 07 2026 at 15:01):I'm gonna drop a comment here, just to stay updated via email on any of this. I'm gonna have to explore who of our team might be able to join in on this.
Deirdre Kirmis (Jul 07 2026 at 15:36):Just a note .. this WG idea was not meant to replace the Large Data Support working group, it is a spin-off that is meant to delve more deeply into infra and ops overall, not just related to large data support issues. Some questions that I would be interested in discussing:
What storage options does your org utilize in Dataverse?
How are you configuring your local and remote stores?
What OS are you using and how many servers do you run?
Are you using autoscaling? Target optimizer? Other cloud services?
What are your backup, DR and preservation workflows?
Are you using RDS and other AWS services such as ECR/ECS for microservices (ie: solr)?
How are you configuring Globus and have you tried to access files in glacier?
How do you migrate to a new OS/server/bucket?
Best practices for running microservices and external tools (ie: separate server? same server?)
How do you have bot control configured? What tools do you use? What rules?
How do you get community-dev tools configured properly?
How have you configured various integrations?
How is your shibboleth configured? Have you configured KeyCloak and/or new auth methods?
How are you dealing with security vulnerabilities and accessibility requirements in your org?
How are you developing/using AI for your installation?
Many more .. :smile: .. anything you can think of that sysadmins/devops deal with in the ever-changing landscape.
Philip Durbin 🚀 (Jul 07 2026 at 18:19):@Deirdre Kirmis thanks for talking up the idea of a working group today at the community call!
Philip Durbin 🚀 (Jul 07 2026 at 18:19):@topic we've created a new channel to geek out in: #infra-ops
Have fun!
Last updated: Aug 18 2026 at 08:27 UTC