Thomas van Erven (Oct 11 2023 at 12:47): Thomas van Erven (Oct 11 2023 at 12:47):Hitting some troubles with moving existing envs to s3fs. The gameplan:
Step 6 is interrupted unfortunately, because I'm seeing IOErrors when opening and uploading files:
[#|2023-10-11T14:35:32.161+0200|WARNING|Payara 5.2022.3|edu.harvard.iq.dataverse.dataaccess.ImageThumbConverter|_ThreadID=95;_ThreadName=http-thread-pool::jk-connector(1);_TimeMillis=1697027732161;_LevelValue=900;|
caught IOException trying to open an input stream for file://18b13f045f2-ef595f3a5aa6java.io.IOException: Failed to open local file file:///data/dataverse/files/10.5072/TAR/Y05ZDI/18b13f045f2-ef595f3a5aa6|#]
[#|2023-10-11T14:35:45.004+0200|SEVERE|Payara 5.2022.3|edu.harvard.iq.dataverse.util.FileUtil|_ThreadID=98;_ThreadName=http-thread-pool::jk-connector(4);_TimeMillis=1697027745004;_LevelValue=1000;|
Failed to create filesTempDirectory: /data/dataverse/files/temp|#]
[#|2023-10-11T14:35:45.004+0200|WARNING|Payara 5.2022.3|edu.harvard.iq.dataverse.EditDatafilesPage|_ThreadID=98;_ThreadName=http-thread-pool::jk-connector(4);_TimeMillis=1697027745004;_LevelValue=900;|
Failed to process and/or save the file blank_template_melting_bard.jpg; Temp directory is not configured.|#]
Thoughts? : )
Oliver Bertuch (Oct 11 2023 at 12:54):It kind of looks like that the S3 file sytem cannot create a folder for the temp file dir
Oliver Bertuch (Oct 11 2023 at 12:55):Is it writable?
Oliver Bertuch (Oct 11 2023 at 12:55):Can you actually create a directory in s3fs without any content? (In S3 AFAIK folders are just a concept like we do it in Dataverse, it's metadata of a file object)
Oliver Bertuch (Oct 11 2023 at 12:56):I'm not sure about the first one
Oliver Bertuch (Oct 11 2023 at 12:56):That could come from different things...
Thomas van Erven (Oct 11 2023 at 12:56):Yeap, can confirm if I just use mkdir on the mount directly.
Oliver Bertuch (Oct 11 2023 at 12:56):Not sure how your config and your mounts look like...
Oliver Bertuch (Oct 11 2023 at 12:57):Also as user dataverse?
Oliver Bertuch (Oct 11 2023 at 12:57):(Or payara, as we might talk containers)
Thomas van Erven (Oct 11 2023 at 12:58):I'm debugging with umask at 000 at present, shouldn't be a permission issue (though can be). It's a thread to pull at.
Oliver Bertuch (Oct 11 2023 at 12:58):I am tempted to move this to #troubleshooting ... It's not a dev topic (yet?)
Thomas van Erven (Oct 11 2023 at 12:58):Yes, please move.
Notification Bot (Oct 11 2023 at 12:58):This topic was moved here from #dev > S3fs woes by Oliver Bertuch.
Thomas van Erven (Oct 11 2023 at 12:58):Apologies for creating it here in the first place : )
Thomas van Erven (Oct 11 2023 at 12:58):Apologies for starting in dev in the first place.
Thomas van Erven (Oct 11 2023 at 12:58):I briefly am about to get yelled at, brb after this.
Notification Bot (Oct 11 2023 at 12:58):A message was moved here from #dev > S3fs woes by Oliver Bertuch.
Oliver Bertuch (Oct 11 2023 at 13:00):Thomas van Erven said:
I'm debugging with umask at 000 at present, shouldn't be a permission issue (though can be). It's a thread to pull at.
Well what happens in the background is that if you upload a file and the temp dir does not yet exist, it will create it for you before storing the upload for further processing
Oliver Bertuch (Oct 11 2023 at 13:01):So if Dataverse cannot create this directory for you, but you can directories on the mounted FS, this certainly indicates some sort of permission problem
Thomas van Erven (Oct 11 2023 at 13:55):Temp directory exists; it booms at creating the file.
Thomas van Erven (Oct 11 2023 at 13:56):Side note; when in a discussion with management, do not say "of course management idea quality is poor, I wouldn't expect anything else from management". :smiling_face_with_tear:
Oliver Bertuch (Oct 11 2023 at 14:04):Thomas van Erven said:
Temp directory exists; it booms at creating the file.
That's not what the error message is telling us...? :thinking:
Thomas van Erven (Oct 11 2023 at 14:15):The error sits on a throne of lies:
[#|2023-10-11T16:15:03.480+0200|SEVERE|Payara 5.2022.3|edu.harvard.iq.dataverse.util.FileUtil|_ThreadID=96;_ThreadName=http-thread-pool::jk-connector(2);_TimeMillis=1697033703480;_LevelValue=1000;|
Failed to create filesTempDirectory: /data/dataverse/files/temp|#]
[#|2023-10-11T16:15:03.480+0200|WARNING|Payara 5.2022.3|edu.harvard.iq.dataverse.EditDatafilesPage|_ThreadID=96;_ThreadName=http-thread-pool::jk-connector(2);_TimeMillis=1697033703480;_LevelValue=900;|
Failed to process and/or save the file DeanSpencer-spotcol-dwarfforge.jpg; Temp directory is not configured.|#]
[root@tar31 temp]# pwd
/data/dataverse/files/temp
However, you gave me an excellent idea. Ownership.
Thomas van Erven (Oct 11 2023 at 14:17):As concept, not as a file property, you understand )
Oliver Bertuch (Oct 11 2023 at 14:18):See that's why I am introducing the startup checks in https://github.com/IQSS/dataverse/pull/9819
Philip Durbin 🚀 (Oct 11 2023 at 14:27):I'm glad the yelling was brief! :grinning:
Thomas van Erven (Oct 11 2023 at 14:55):It wasn't permission issue either.
Thomas van Erven (Oct 11 2023 at 14:55):But; I've got a slightly different error, which suggests it might be rights orientated.
Thomas van Erven (Oct 16 2023 at 09:26):You were correct; it works now. Topic may be resolved.
Thomas van Erven (Oct 16 2023 at 09:28):(As a side note; the dataverse file behaviours for temp/draft/final take quite long in terms of the interface that keeps loading, for large files and larger sets of files. I suspect I'm going to consult your migration scripts and use the Keystone implementation directly, rather than a fuse mount).
Notification Bot (Oct 16 2023 at 14:07):Thomas van Erven has marked this topic as resolved.
Last updated: Oct 30 2025 at 06:21 UTC