shan (Jul 10 2024 at 01:59): shan (Jul 10 2024 at 01:59):I try to change the default upload dataset directory to "/sdcfs/data_collect_prod/publication" by running the following command "./asadmin create-jvm-options "\-Ddataverse.files..directory=/sdcfs/data_collect_prod/publication",
image.png
but when I upload the dataset, the files still located under the default directory "/usr/local/payara6/glassfish/domains/domain1/files/"
Anybody has ideas?
Thanks
Oliver Bertuch (Jul 10 2024 at 06:25):That's not how this works. Your command is adding an arbitrary system property that will never be picked up.
Oliver Bertuch (Jul 10 2024 at 06:27):You need to replace the <id> part with the actual ID of your file storage. Please look into your domain.xml to extract that information if you don't remember the ID.
Oliver Bertuch (Jul 10 2024 at 06:29):If you're using the classic way of installing Dataverse, the ID will be file by default.
Oliver Bertuch (Jul 10 2024 at 06:31):As you're talking about uploaded files, please also keep in mind that any upload will have temporary storage locations before being transferred to the actual storage. See the dataverse.files.* JVM options.
Oliver Bertuch (Jul 10 2024 at 06:33):@Philip Durbin once upon a time I made a diagram for that. Should we include this in the guides somewhere?
Philip Durbin π (Jul 10 2024 at 13:42):@Oliver Bertuch sure, I've referenced that diagrams a couple times in the past few months. It's quite useful.
Philip Durbin π (Jul 10 2024 at 13:43):@shan are you able to put the right values in by editing domain.xml (classic installation) or by editing your compose.yml file (Docker installation)?
shan (Jul 11 2024 at 02:17):I used the classic installation, and rerun the command "./asadmin create-jvm-options "\-Ddataverse.files..directory=/sdcfs/data_collect_prod""
but it still didn't work, the upload files still located in the default directory "/usr/local/payara6/glassfish/domains/domain1/files"
here is the screenshot of the domain.xmlοΌ
image.png
Philip Durbin π (Jul 12 2024 at 15:10):@shan ok, classic installation. In that case you can just edit the domain.xml file with vi or emacs or whatever, if that's easier.
I'm a little confused my your screenshot. The word "dataverse" doesn't appear anywhere. Did you screenshot the correct part of the file?
Bikram (Jul 12 2024 at 21:55):We created a symlink for default location βfilesβ to a directory in separate disk.
shan (Jul 15 2024 at 07:22):@Bikram that sounds a good idea ~
Oliver Bertuch (Jul 15 2024 at 07:55):Symlinks are a bad idea. Mount a file system instead https://linuxize.com/post/how-to-mount-and-unmount-file-systems-in-linux/ There's also the option to use bind mounts if really necessary.
Using configuration, you can also change the destination folders. Just in case you can't change your mount points for some reason. See https://guides.dataverse.org/en/latest/installation/config.html#dataverse-files-directory and following, as well as https://guides.dataverse.org/en/latest/installation/config.html#temporary-upload-file-storage
shan (Jul 15 2024 at 08:36):@Oliver Bertuch Ok, I Checked the configuration in domain.xml, there are two dataverse related directory configurations,
image.png
Do I need to change the configuration of both parts? what's the difference between the two configurations (Ddataverse.files.directory and Ddataverse.files.file.directory)in the image οΌ
Oliver Bertuch (Jul 15 2024 at 08:38):Please read about the details in the extensive configuration guide:
https://guides.dataverse.org/en/latest/installation/config.html#storage-files-dir
https://guides.dataverse.org/en/latest/installation/config.html#dataverse-files-directory
Oliver Bertuch (Jul 15 2024 at 08:43):And don't ask ChatGPT about it - it has no actual clue what it's about... Tries to compensate by halucinating... :see_no_evil: https://chatgpt.com/share/5559fe4f-25ef-4e5d-ac4b-3afef4f938f4
Bikram (Jul 15 2024 at 17:53):okay thank you, just wondering why symlinks are bad idea! We have had our Dataverse instances setup this way from last 8-9years, Never ran into any issues and it makes easier to use separate data disks with different sizes and repurpose those.
But yes, one should use the available configuration options. Lot of these options were not available with the initial release of Dataverse4.
shan (Jul 16 2024 at 08:27):@Philip Durbin I edit the domain.xml file with vi, and change the <jvm-options>-Ddataverse.files.directory=/sdcfs/data_collect_prod/dataverse</jvm-options>, then restart the domain by running "./asadmin restart-domains /home/dataverse/dataverse-6.2/target/dataverse-6.2.war"
but after that, I return to the home page of dataverse, http://mydataverse.domain:8080, it shows that payara is running , instead of the dataverse page I have created before
image.png
Philip Durbin π (Jul 16 2024 at 13:36):@shan hmm, that's no good. Can you attach your server.log file?
shan (Jul 17 2024 at 00:55):@Philip Durbin sadly no log files appeared in the /usr/local/payara6/glassfish/domains/domain1/logs/server.log, the latest log is
image.png
shan (Jul 17 2024 at 01:37):@Philip Durbin it seems to because of the wrong GID of the file "server.log", I've corrected it to the right one, and get the following messages from the server.log:
image.png
I have tested from the command line that both "root" and "dataverse" have the permissions to write and read the directory
Philip Durbin π (Jul 17 2024 at 02:06):Whatever user Payara is running as (usually "dataverse") needs to be able to write there.
shan (Jul 17 2024 at 08:26):@Philip Durbin Finally, the problem fixed, it is because of the difference between the uid in Linux system and uid in the filesystem. Anyway, thanks !
Philip Durbin π (Jul 17 2024 at 12:58):Interesting. I'm glad you figured it out! Good job. :trophy:
Notification Bot (Jul 17 2024 at 12:58):Philip Durbin has marked this topic as resolved.
Last updated: Oct 30 2025 at 06:21 UTC