Jan Range (Feb 13 2026 at 13:06): Jan Range (Feb 13 2026 at 13:06):@Oliver Bertuch unfortunately pyFileSystem2 has turned out to be incompatible with newer versions, because it is using pkg_resources and its support has been removed. Huge bummer!
Luckily the current pyDataverse 0.4.0 implementation simply uses their base class to implement necessary methods, but not the entire eco-system. Hence, I can restore compatibility by simply not inheriting and removing the dependency (see completed vs failed run).
I am now looking into alternatives, since other users have experienced the same issues. One of which is fsspec which looks very promising. What are your thoughts? To me it looks like it is a viable alternative and possibly offers even more.
Oliver Bertuch (Feb 13 2026 at 13:21):That looks like a well maintained library!
Oliver Bertuch (Feb 13 2026 at 13:21):They even provide chaining and local caching out of the box!
Oliver Bertuch (Feb 13 2026 at 13:22):Also their argument for serializable filesystem requests seem very reasonable!
Oliver Bertuch (Feb 13 2026 at 13:22):I'm not a Python expert, but it may be worth a shot!
Oliver Bertuch (Feb 13 2026 at 13:25):I like their overview of alternatives (e.g. https://github.com/piskvorky/smart_open), but as we seek abstraction, this seems to be a good fit
Jan Range (Feb 13 2026 at 17:44):I agree, looking through the docs this is an awesome library! I will test it out in the upcoming week :slight_smile:
Jan Range (Jun 16 2026 at 10:33):@Oliver Bertuch I managed adding the fsspec to pyDataverse. What is extremely nice, is that any fsspec-compatible library can use the Dataverse filesystem via dataverse://. Pandas and Polars for instance can make use of that:
import pandas as pd
import pyDataverse
url = "dataverse://darus.uni-stuttgart.de/results/summary.tab?persistentId=doi:10.18419/DARUS-5539"
df = pd.read_csv(url, sep="\t")
print("Columns:", list(df.columns))
print(df.head())
Huh. Interesting. Where does support for dataverse:// come from? pyDataverse?
Jan Range (Jun 16 2026 at 11:21):Yes, it stems from pyDataverse and is "registered" once pyDataverse is imported
Jan Range (Jun 16 2026 at 11:21):
Philip Durbin 🚀 (Jun 16 2026 at 11:42):Very cool. I can imagine a short "Pandas and Polars with pyDataverse" talk. :smile:
Oliver Bertuch (Jun 16 2026 at 15:46):Pretty cool! Nice job! :tada:
Last updated: Aug 18 2026 at 08:27 UTC