Johannes D (Feb 21 2024 at 12:10): Johannes D (Feb 21 2024 at 12:10):We noticed that we have quite a few export files cached in our S3 storage location. Is the an option to configure dataverse to store the cached exports on a different storage location (i.e local disk) but keep the uploaded files in a different location (S3)?
Oliver Bertuch (Feb 21 2024 at 12:50):Nope, such a concept doesn't exist in Dataverse. It actually would mean larger changes in the storage subsystem, as currently all files of a dataset are understood as living on the same storage.
Johannes D (Feb 21 2024 at 13:03):Thanks Oliver! Is there an option to disable the caching mechanism? If not, would it be feasible to implement one? Sure, disabling caching has some drawbacks and is rather an obscure feature not requested by many .
Philip Durbin 🚀 (Feb 22 2024 at 14:26):Hmm, I wouldn't call exports are an obscure feature. They are used for harvesting (OAI-PMH), for example, which is a popular feature.
Philip Durbin 🚀 (Feb 22 2024 at 14:27):@Johannes D you are welcome to create an issue about the problem you're having!
Johannes D (Feb 22 2024 at 14:36):Philip Durbin said:
Hmm, I wouldn't call exports are an obscure feature. They are used for harvesting (OAI-PMH), for example, which is a popular feature.
I'm mean disabling a cache is not a typical thing to do...
Philip Durbin 🚀 (Feb 22 2024 at 15:21):Oh, right. :grinning:
Philip Durbin 🚀 (Feb 22 2024 at 15:21):Yes, the cache is there for a reason. :grinning:
Johannes D (Feb 22 2024 at 15:22):I know, is there a chance to get a feature flag that disables this cache merged?
Philip Durbin 🚀 (Feb 22 2024 at 15:56):I'm asking in Slack for you.
Philip Durbin 🚀 (Feb 22 2024 at 15:59):@Johannes D "why" "what's the rationale behind it?"
Philip Durbin 🚀 (Feb 22 2024 at 17:19):Could having a trusted remote store help?
"In addition to having the type “remote” and requiring a label, Trusted Remote Stores are defined in terms of a baseURL - all files managed by this store must be at a path starting with this URL, and a baseStore - a file, s3, or swift store that can be used to store additional ancillary dataset files (e.g. metadata exports, thumbnails, auxiliary files, etc.)."
From https://guides.dataverse.org/en/6.1/installation/config.html#trusted-remote-storage
Philip Durbin 🚀 (Feb 22 2024 at 17:20):@Johannes D also, @Leo Andreev said you should feel free to create an issue about disabling the caching.
Last updated: Nov 01 2025 at 14:11 UTC