Király Péter (Oct 29 2024 at 07:46): Király Péter (Oct 29 2024 at 07:46):Hi, I have a problem. Just upgraded Dataverse from 5.14 to 6.0. I followed the release notes, and I ran into a trouble. Those dataverses that are configured to work with S3 (we use CEPH S3 not AWS), are inaccessible, they produce the following on the web side:
XML Parsing Error: mismatched tag. Expected: </br>.
Location: https://data.goettingen-research-online.de/dataverse/pkiraly
Line Number 355, Column 9:
</div>
--------^
Payara server.log says: "java.io.IOException: ERROR: S3AccessIO - Failed to look up bucket gro-data (is AWS properly configured?): null (Service: Amazon S3; Status Code: 400; Error Code: InvalidArgument; Request ID: tx00000feb01b793cfaf500-0067208fa0-5263351e-default; S3 Extended Request ID: 5263351e-default-default; Proxy: null).
I can access the bucket with command line:
aws --endpoint=https://s3.gwdg.de s3 ls --human-readable --summarize s3://gro-data/
and I have the following JVM options:
-Ddataverse.files.s3.bucket-name=gro-data
-Ddataverse.files.s3.custom-endpoint-url=https://s3.gwdg.de
-Ddataverse.files.s3.download-redirect=true
-Ddataverse.files.s3.ingestsizelimit=1073741824
-Ddataverse.files.s3.label=S3 storage
-Ddataverse.files.s3.type=s3
-Ddataverse.files.s3.upload-redirect=true
I rechecked Dataverse config guide, but I do not found what is the problem. Do you have any suggestion what to check/restart? Many thanks for any suggestions!
Notification Bot (Oct 29 2024 at 10:28):A message was moved here from #community > AWS S3 and class transitions by Philip Durbin 🐉.
Philip Durbin 🚀 (Oct 29 2024 at 10:29):@Király Péter hmm, it looks like the error is coming from here: https://github.com/IQSS/dataverse/blob/v6.0/src/main/java/edu/harvard/iq/dataverse/dataaccess/S3AccessIO.java#L117
Philip Durbin 🚀 (Oct 29 2024 at 10:29):throw new IOException("ERROR: S3AccessIO - Failed to look up bucket "+bucketName+" (is AWS properly configured?): " + sce.getMessage());
Philip Durbin 🚀 (Oct 29 2024 at 10:29):bucketName is null for you.
Philip Durbin 🚀 (Oct 29 2024 at 10:32):Perhaps you could add some additional print line debugging? From a quick look your JVM settings look ok to me. :thinking:
Király Péter (Oct 29 2024 at 10:47):@Philip Durbin 🐉 Many thanks for the tip, but I disagree this time. The error message starts with "Failed to look up bucket gro-data (is AWS properly configured?)", so the bucketName is gro-data. The null comes from the sce.getMessage() call, but it is not clear from there what is null. In the details one value is null: the Proxy. In dataverse documentation there is a optional configuration for proxy:
JVM Option: dataverse.files.<id>.proxy-url
Value: <?>
Description: URL of a proxy protecting the S3 store. Optional.
Default value: (none)
Király Péter (Oct 29 2024 at 10:54):Is there probably a service I should restart?
Philip Durbin 🚀 (Oct 29 2024 at 11:12):Oh, sorry, sorry, you're right. The bucketName is not null, obviously.
Philip Durbin 🚀 (Oct 29 2024 at 11:19):So a SdkClientException is being caught but its getMessage() method is null? But the message shows "null" and keeps going (...properly configured?): null (Service: Amazon S3; Status...). Strange. Are you able to add any extra debugging? I'm not sure what to say about the proxy. I don't believe I've ever used it myself.
Philip Durbin 🚀 (Oct 29 2024 at 11:20):#8285 by @Don Sizemore show a non-null message from SdkClientException
Leo Andreev (Oct 29 2024 at 12:37):Let's eliminate all the straightforward/simple possibilities. Could you please confirm that the new Payara 6 is running under the same user as Payara 5/Dataverse 5.14 used to (presumably the same user for whom the aws command line is still working).
Last updated: Oct 30 2025 at 06:21 UTC