Guido Schmutz (Oct 09 2023 at 11:37): Guido Schmutz (Oct 09 2023 at 11:37):Hi, I have just started working with Dataverse and today I'm working on adding the necessary configuration to use S3 for storage. I know from the documentation that AWS credentials need to be in the credentials file. I was just wondering if environment variables wouldn't work as well (ASW_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY) as it is normally the standard behavior: https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html. Looking at the source code I see why it is not working, as you build your own AWSCredentialsProviderChain. Now I was wondering if you would accept a PR which adds the EnvironmentVariableCredentialsProvider into the chain as well, so that environment variables are supported as well, similar to the default behavior. Especially in a development environment with docker-compose this would be easier that mapping a credential file into the docker container.
Guido Schmutz (Oct 09 2023 at 11:54):Just realized that I have missed the two properties dataverse.files.<id>.access-key and dataverse.files.<id>.secret-key which solves the problem, but I think supporting the environment variables would still be a nice, of course with a WARNING in the doc to not use it for production deployments :big_smile:
Oliver Bertuch (Oct 09 2023 at 12:41):Hi @Guido Schmutz and welcome :wave:
Oliver Bertuch (Oct 09 2023 at 12:42):Please note that both dataverse.files.<id>.access-key and dataverse.files.<id>.secret-key are MicroProfile Config enabled already, thus you can already use environment variables to get them in
Oliver Bertuch (Oct 09 2023 at 12:44):You can also use any other MPC source, as linked in the guides.
Oliver Bertuch (Oct 09 2023 at 12:44):If you are using containers, you might be interested in http://preview.guides.gdcc.io/en/develop/container/index.html
Oliver Bertuch (Oct 09 2023 at 12:45):And if you're using containers in a cloud / K8s scenario, you might want to use the cloud provider's KMS solution - there are some sources available builtin with Payara
Oliver Bertuch (Oct 09 2023 at 12:46):That said, please be aware that the other dataverse.files.<...> have not yet been enabled to be used with MicroProfile Config. It's on the roadmap in Milestone D.
Oliver Bertuch (Oct 09 2023 at 12:47):And last but not least: the container nerds hang in #containers - feel free to ask questions there
Oliver Bertuch (Oct 09 2023 at 12:47):We also have a weekly working group call, see https://ct.gdcc.io/ for notes and recordings
Oliver Bertuch (Oct 09 2023 at 12:48):Oliver Bertuch said:
Please note that both
dataverse.files.<id>.access-keyanddataverse.files.<id>.secret-keyare MicroProfile Config enabled already, thus you can already use environment variables to get them in
To make that easier: the env vars are DATAVERSE_FILES_<ID>_ACCESS_KEY and DATAVERSE_FILES_<ID>_SECRET_KEY
Guido Schmutz (Oct 09 2023 at 12:51):@Oliver Bertuch thanks for your help and reply. Yes I have already used the two env variables for the access key and secret and it works perfectly! Have only seen it in the doc after I have sent my first message. I know RTFM ;-) which I did just didn't scroll down until the end of the many dataverse.files.xxxx settings :smiley:
Oliver Bertuch (Oct 09 2023 at 12:52):Yeah, saw your second message but thought I'd provide some more pointers and context... :wink:
Guido Schmutz (Oct 09 2023 at 12:52):Oliver Bertuch said:
And last but not least: the container nerds hang in #containers - feel free to ask questions there
yes that's where I was but then thought it is more a general questions and that's why I posted it here ....
Oliver Bertuch (Oct 09 2023 at 12:53):I agree - the whole dataverse.files.<id> complex is still TODO and will be beneficial for all kinds of use cases / installation types.
Notification Bot (Oct 09 2023 at 13:00):A message was moved from this topic to #dev > Platys App Stack by Oliver Bertuch.
Philip Durbin 🚀 (Oct 09 2023 at 13:10):This week I plan to start working on this issue: Add S3 tests to the regular integration test suite #6783
Philip Durbin 🚀 (Oct 09 2023 at 13:10):So hopefully I'll start exercising some of this code. It's good timing.
Last updated: Nov 01 2025 at 14:11 UTC