Philip Durbin 🚀 (May 22 2024 at 13:38): Philip Durbin 🚀 (May 22 2024 at 13:38):I'd like to publish a jar to Maven Central but I'm hitting various roadblocks.
Locally, from my laptop, using my "pdurbin" account, I'm able to publish a snapshot to org.dataverse but not io.gdcc. (I've emailed Central Support about getting permission to publish to io.gdcc.) For example, a snapshot jar, is available, built from a hello app.
I'm using credentials from my pdurbin account (User Access Token) and placing them in ~/.m2/settings.xml like this:
<settings>
<servers>
<server>
<id>ossrh</id>
<username>REDACTED</username>
<password>REDACTED</password>
</server>
</servers>
</settings>
Then I type mvn deploy to publish the snapshot. Pretty easy from local, but...
Philip Durbin 🚀 (May 22 2024 at 13:43):I have a snapshot GitHub Action set up for the hello app, but it's showing (as @Juan Pablo Tosca Villanueva noticed) this debug output:
[DEBUG] Failed to decrypt password for server github: org.sonatype.plexus.components.sec.dispatcher.SecDispatcherException: java.io.FileNotFoundException: /home/runner/.m2/settings-security.xml (No such file or directory)
Then it fails with this error:
Error: Failed to execute goal org.sonatype.plugins:nexus-staging-maven-plugin:1.6.13:deploy (injected-nexus-deploy) on project hello: Failed to deploy artifacts: Could not transfer artifact org.dataverse.test:hello:jar:0.0.1-20240522.132907-4 from/to ossrh (https://s01.oss.sonatype.org/content/repositories/snapshots): authentication failed for https://s01.oss.sonatype.org/content/repositories/snapshots/org/dataverse/test/hello/0.0.1-SNAPSHOT/hello-0.0.1-20240522.132907-4.jar, status: 401 Unauthorized -> [Help 1]
I'm using the same credentials, in repository secrets under https://github.com/gdcc/hello/settings/secrets/actions
Why does it work locally but not in GitHub Actions? :thinking:
Juan Pablo Tosca Villanueva (May 22 2024 at 13:45):Is there anything in runners on the other repo that is missing from this?
Juan Pablo Tosca Villanueva (May 22 2024 at 13:45):
Philip Durbin 🚀 (May 22 2024 at 13:57):Do you mean https://github.com/IQSS/UNF/settings/actions/runners , for example? Nothing there:
Screenshot-2024-05-22-at-9.57.26-AM.png
Juan Pablo Tosca Villanueva (May 22 2024 at 14:09):Yes but from https://github.com/gdcc/dataverse-exporters
Philip Durbin 🚀 (May 22 2024 at 14:58):Nothing at https://github.com/gdcc/dataverse-exporters/settings/actions/runners either.
Juan Pablo Tosca Villanueva (May 22 2024 at 15:16):I wonder where this settings-security.xml is setup
Juan Pablo Tosca Villanueva (May 22 2024 at 15:18):Does this seems similar? @Philip Durbin https://github.com/orgs/community/discussions/27165
Philip Durbin 🚀 (May 22 2024 at 15:20):Yes, it does seem related.
Philip Durbin 🚀 (May 22 2024 at 15:22):I just tried to push a snapshot to another org.dataverse project and it also says "401 Unauthorized": https://github.com/IQSS/UNF/actions/runs/9194059373
Maybe I should add the extra debugging.
Juan Pablo Tosca Villanueva (May 22 2024 at 15:24):The ‘settings-security.xml’ file contains the master password used to encrypt remote repository credentials. The default location of this file is ‘~<user_home>/.m2/settings-security.xml’.
Normally, if this file does not exist, or if it does not contain valid content, the ‘mvn --encrypt-password’ command line will fail with the error like:
So prob this is on the cache of the github action that was previously set up by someone? (not an expert just trying to figure this out :sweat_smile:)
Philip Durbin 🚀 (May 22 2024 at 15:25):I added extra debugging: https://github.com/IQSS/UNF/actions/runs/9194126200
No mention of settings-security.xml. Seems like it's just straight up 401 unauthorized.
Juan Pablo Tosca Villanueva (May 22 2024 at 15:27)::open_mouth: https://www.bitshifted.co/blog/deploy-maven-central-github-actions-step-by-step-guide/
Philip Durbin 🚀 (May 22 2024 at 18:30):I just heard back from Maven Central Support: "It appears that you are an authorized publisher on org.dataverse, but not yet io.gdcc, which explains the issues you're experiencing."
Philip Durbin 🚀 (May 22 2024 at 18:30):"In order to grant publishing access on io.gdcc, we'll need an email on this thread approving your request from one of the accounts already authorized to publish under that domain."
Juan Pablo Tosca Villanueva (May 22 2024 at 18:32)::upside_down:
Juan Pablo Tosca Villanueva (May 22 2024 at 18:35):But how we can publish from one repo
Juan Pablo Tosca Villanueva (May 22 2024 at 18:35):Unless whoever set this up used credentials valid for gdcc.io
Juan Pablo Tosca Villanueva (May 22 2024 at 18:36):But we have credentials for dataverse.org
Philip Durbin 🚀 (May 22 2024 at 18:50):I feel like this is only the first step, authorizing my pdurbin account to publish to io.gdcc.
Philip Durbin 🚀 (May 22 2024 at 18:50):There's still a lot of other weird stuff going on.
Juan Pablo Tosca Villanueva (May 22 2024 at 18:59):At least I hopes that is a fix, I was running out of ideas :laughing:
Philip Durbin 🚀 (May 22 2024 at 18:59):I don't think we're out of the woods yet.
Juan Pablo Tosca Villanueva (May 22 2024 at 19:00):have you tried turning it off and on again
Philip Durbin 🚀 (May 22 2024 at 20:18):It looks like they key we use to sign is not yet expired (but will soon, at the end of July):
$ gpg --show-keys --keyid-format 0xshort privatekey.txt
sec# rsa4096/0xC031F36B 2021-07-30 [SC] [expires: 2024-07-29]
2DFF887456235B4550B857DFBAFC446FC031F36B
uid Dataverse Bot (Signing Bot for Dataverse and GDCC packages) <dataversebot@gdcc.io>
That prob explains why the other one stopped working (?)
Philip Durbin 🚀 (May 22 2024 at 20:19):well, it isn't expired
Juan Pablo Tosca Villanueva (May 22 2024 at 20:20):The one from UNF?
Philip Durbin 🚀 (May 22 2024 at 20:20):They all use the same key, the one above.
Juan Pablo Tosca Villanueva (May 22 2024 at 20:20)::thinking:
Philip Durbin 🚀 (May 22 2024 at 20:20):DATAVERSEBOT_GPG_KEY
Philip Durbin 🚀 (May 22 2024 at 20:32):I'm hopeful that this is working, locally: mvn -Prelease -Dgpg.keyname=$DATAVERSEBOT_GPG_KEYNAME -Dgpg.passphrase=$DATAVERSEBOT_GPG_PASSWORD deploy
Philip Durbin 🚀 (May 22 2024 at 20:33):I had to import the private key first: gpg --import privatekey.txt
Philip Durbin 🚀 (May 22 2024 at 20:37):[INFO] * Upload of locally staged artifacts finished.
[INFO] * Closing staging repository with ID "orgdataverse-1003".
Waiting for operation to complete...
..........
[INFO] Remote staged 1 repositories, finished with success.
[INFO] Remote staging repositories are being released...
Waiting for operation to complete...
......
[INFO] Remote staging repositories released.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 57.213 s
[INFO] Finished at: 2024-05-22T16:37:17-04:00
[INFO] ------------------------------------------------------------------------
Nothing at https://repo1.maven.org/maven2/org/dataverse/ though.
Juan Pablo Tosca Villanueva (May 22 2024 at 20:56):So they give you access already? I got lost
Juan Pablo Tosca Villanueva (May 22 2024 at 20:56):Or that will take some time
Philip Durbin 🚀 (May 22 2024 at 20:57):No, I still don't have access to io.gdcc. Waiting. I replied from the dataversebot email account and said that Phil guy is trustworthy. :grinning:
Philip Durbin 🚀 (May 22 2024 at 20:57):Above I'm trying to publish "hello" to org.dataverse. To real Maven Central, not staging.
Philip Durbin 🚀 (May 22 2024 at 21:04):There it is: https://repo1.maven.org/maven2/org/dataverse/test/hello/0.0.1/
Philip Durbin 🚀 (May 22 2024 at 21:04):In real Maven Central.
Juan Pablo Tosca Villanueva (May 22 2024 at 21:04)::partying_face:
Philip Durbin 🚀 (May 22 2024 at 21:05):https://central.sonatype.com/artifact/org.dataverse.test/hello
Philip Durbin 🚀 (May 22 2024 at 21:06):I created that "test" area so we can practice.
Juan Pablo Tosca Villanueva (May 22 2024 at 21:06):This was deployed locally? No github action right
Philip Durbin 🚀 (May 22 2024 at 21:06):Right, from my laptop.
Philip Durbin 🚀 (May 22 2024 at 21:07):At least I know it's possible :sweat_smile:
Philip Durbin 🚀 (May 22 2024 at 21:07):And I'm ok pushing the croissant jar this way, if I have to.
Philip Durbin 🚀 (May 22 2024 at 21:07):But I would like to get GitHub Actions working.
Juan Pablo Tosca Villanueva (May 22 2024 at 21:07):I hope once we have the right keys will be easy :rolling_on_the_floor_laughing:
Philip Durbin 🚀 (May 22 2024 at 21:18):Well, the gpg key is fine. I just proved that.
Philip Durbin 🚀 (May 22 2024 at 21:22):And the dataversebot sonotype username and password work from dataverse-exporters, as we discussed in the spreadsheet we made:
(I just updated it to say that I can push hello to org.dataverse final with my pdurbin credentials. To real Maven Central.)
Philip Durbin 🚀 (May 22 2024 at 21:23):Here are the docs I just wrote: https://github.com/gdcc/hello/commit/d2a2fa230af6889aec112717be6fb7e5de31e3d5
Philip Durbin 🚀 (May 23 2024 at 02:05):I added a step to let me ssh into the runner. This is what I see:
runner@fv-az973-728:~/work/hello/hello$ cd ~/.m2/
runner@fv-az973-728:~/.m2$ ls
repository settings.xml toolchains.xml
runner@fv-az973-728:~/.m2$ cat settings.xml
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd">
<servers>
<server>
<id>github</id>
<username>${env.GITHUB_ACTOR}</username>
<password>${env.GITHUB_TOKEN}</password>
</server>
</servers>
</settings>runner@fv-az973-728:~/.m2$
That id reminds me that the error is "Failed to decrypt password for server github".
Instead of "github" it should be "ossrh".
And the username and password should be updated, of course.
Philip Durbin 🚀 (May 23 2024 at 02:11):https://github.com/actions/setup-java/blob/main/docs/advanced-usage.md#Publishing-using-Apache-Maven
Philip Durbin 🚀 (May 23 2024 at 02:21):Interesting. The settings.xml above was from the "hello" app. I just added tmate to dataverse-exporters and the values are updated correctly:
runner@fv-az1108-124:~/work/dataverse-exporters/dataverse-exporters$ cd ~/.m2
runner@fv-az1108-124:~/.m2$ ls
repository settings.xml toolchains.xml
runner@fv-az1108-124:~/.m2$ cat settings.xml
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd">
<servers>
<server>
<id>ossrh</id>
<username>${env.MAVEN_USERNAME}</username>
<password>${env.MAVEN_PASSWORD}</password>
</server>
</servers>
</settings>runner@fv-az1108-124:~/.m2$
(Kudos to Oliver for suggesting tmate: https://mxschmitt.github.io/action-tmate/ )
Juan Pablo Tosca Villanueva (May 23 2024 at 02:54)::open_mouth:
Juan Pablo Tosca Villanueva (May 23 2024 at 02:54):Oh this looks cool
Philip Durbin 🚀 (May 23 2024 at 10:28):Interesting, when I move the tmate debugging to just after setup-java, the values are there on the hello project:
runner@fv-az568-806:~/work/hello/hello$ cd ~/.m2
runner@fv-az568-806:~/.m2$ ls
settings.xml toolchains.xml
runner@fv-az568-806:~/.m2$ cat settings.xml
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd">
<servers>
<server>
<id>ossrh</id>
<username>${env.MAVEN_USERNAME}</username>
<password>${env.MAVEN_PASSWORD}</password>
</server>
</servers>
</settings>runner@fv-az568-806:~/.m2$
Maybe it's the caching? I moved debugging to just after the caching and the values are not there:
runner@fv-az1433-508:~/work/hello/hello$ cd ~/.m2
runner@fv-az1433-508:~/.m2$ ls
repository settings.xml toolchains.xml
runner@fv-az1433-508:~/.m2$ cat settings.xml
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd">
<servers>
<server>
<id>github</id>
<username>${env.GITHUB_ACTOR}</username>
<password>${env.GITHUB_TOKEN}</password>
</server>
</servers>
</settings>runner@fv-az1433-508:~/.m2$
It works! :tada: :tada: :tada: I removed the "Cache Maven Packages" step:
https://github.com/gdcc/hello/actions/runs/9206483018
https://s01.oss.sonatype.org/content/groups/staging/org/dataverse/test/hello/0.0.2-SNAPSHOT/
Philip Durbin 🚀 (May 23 2024 at 11:12):Hmm, what we do is slightly different than https://github.com/actions/cache/blob/v4.0.2/examples.md#java---maven
Philip Durbin 🚀 (May 23 2024 at 11:13):Maybe we are suffering from this: Maven documentation example leads to cached settings.xml - https://github.com/actions/cache/issues/1195
Philip Durbin 🚀 (May 23 2024 at 11:30):Ok, I added the cache step back and publishing still works.
Philip Durbin 🚀 (May 23 2024 at 12:07):But why does the old caching config work on dataverse-exporters?
As is the failure on the UNF repo due to caching or something else?
Juan Pablo Tosca Villanueva (May 23 2024 at 12:21):So the action for the snapshot of Hello works now right?
Juan Pablo Tosca Villanueva (May 23 2024 at 12:24):the time of validity for the key defaults to *2 years*
Juan Pablo Tosca Villanueva (May 23 2024 at 12:24):Last successful run was 8 months ago
Juan Pablo Tosca Villanueva (May 23 2024 at 12:24)::thinking:
Juan Pablo Tosca Villanueva (May 23 2024 at 12:25):With the debugger can you re-export the key and test that locally?
Philip Durbin 🚀 (May 23 2024 at 12:46):Yes, the snapshot action for "hello" works now. It was the cache.
Now I'll move on to making sure .github/workflows/maven-release.yml works.
Juan Pablo Tosca Villanueva (May 23 2024 at 13:14):So in theory this should also work for deploying the croissant exporter to croissant?
Philip Durbin 🚀 (May 23 2024 at 13:46):Yes, I'm hoping the same tweak regarding caching will work on https://github.com/gdcc/exporter-croissant
Philip Durbin 🚀 (May 23 2024 at 13:46):Also, I just heard back from Maven Central Support: "We have provisioned your pdurbin OSSRH account on https://s01.oss.sonatype.org as a publisher for the io.gdcc namespace."
Philip Durbin 🚀 (May 23 2024 at 14:11):I just tested, and yes, I can push from pdurbin to io.gdcc, the exporter-croissant repo. I updated the spreadsheet:
Juan Pablo Tosca Villanueva (May 23 2024 at 14:14):Wait, i thought you were able to push hello with the github action?
Juan Pablo Tosca Villanueva (May 23 2024 at 14:15):Oh the image is not updated
Philip Durbin 🚀 (May 23 2024 at 14:15):I am, I am. I just updated the image again:
green.png
Philip Durbin 🚀 (May 23 2024 at 14:15):more green
Philip Durbin 🚀 (May 23 2024 at 14:16):shoot I missed one
Philip Durbin 🚀 (May 23 2024 at 14:16):
Juan Pablo Tosca Villanueva (May 23 2024 at 14:20):It is looking good!
Juan Pablo Tosca Villanueva (May 23 2024 at 14:20):So what did you have to do with the cache?
Philip Durbin 🚀 (May 23 2024 at 14:21):The issue is basically this: https://github.com/actions/cache/issues/1195
Philip Durbin 🚀 (May 23 2024 at 14:21):And the fix is to change the path.
Philip Durbin 🚀 (May 23 2024 at 14:45):I removed some green. Still working on "pdurbin creds GitHub Action (hello)" for org.dataverse. To push a final version using actions only.
Juan Pablo Tosca Villanueva (May 23 2024 at 15:31):Ok it seems to me you are out of the woods :tree: :tree: :tree: :tree: :tree:
Juan Pablo Tosca Villanueva (May 23 2024 at 15:31)::rolling_on_the_floor_laughing:
Juan Pablo Tosca Villanueva (May 23 2024 at 15:31):Going to do some PPR stuff but ping me if I can help with anything
Philip Durbin 🚀 (May 23 2024 at 15:39):I think so! I hope so! I feel less blocked than I have in days. :relieved:
Philip Durbin 🚀 (May 24 2024 at 14:57):There's still a mystery here. The dataversebot sonatype creds I have on file don't work. I tried in GitHub Actions and locally (settings.xml) with the "hello" app.
Philip Durbin 🚀 (May 24 2024 at 14:58):I've been assuming the dataversebot creds are being used for dataverse-exporters but maybe it's using Oliver's creds? They are set at the org (gdcc) level.
Philip Durbin 🚀 (May 24 2024 at 14:59):In short, now that I've reset the password for the databasebot sonatype account, I'm thinking about regenerating the creds (a username and token).
Juan Pablo Tosca Villanueva (May 24 2024 at 15:01):We could verify the value of the keys with the debugger right?
Juan Pablo Tosca Villanueva (May 24 2024 at 15:02):I don’t think github lets you see them after they are saved?
Philip Durbin 🚀 (May 24 2024 at 15:02):From the dataverse-exporters, repo, yes, I should be able to look at settings.xml.
Juan Pablo Tosca Villanueva (May 24 2024 at 15:05):But that is because you had those locally, but I was wondering if we can verify the credentials for dataverse bot and check if these are Oliver’s or something else
Philip Durbin 🚀 (May 24 2024 at 15:05):No, for dataverse-exporters the secrets were already defined at the org (gdcc) level.
Philip Durbin 🚀 (May 24 2024 at 15:18):Oh, right, it's just this:
<username>${env.MAVEN_USERNAME}</username>
<password>${env.MAVEN_PASSWORD}</password>
Not sure how to get the actual values. :thinking:
Juan Pablo Tosca Villanueva (May 24 2024 at 15:24):That is what I was thinking
Juan Pablo Tosca Villanueva (May 24 2024 at 15:24):Can you echo
Juan Pablo Tosca Villanueva (May 24 2024 at 15:24):Or write them with the debuger?
Juan Pablo Tosca Villanueva (May 24 2024 at 15:24):Debugger
Philip Durbin 🚀 (May 24 2024 at 15:27):Got them! With this: https://github.com/gdcc/dataverse-exporters/commit/01e7d2aaa664829eae60a3fa62ed156eb9d9d9e6
https://stackoverflow.com/questions/67503656/echo-github-action-environment-variables
And they are different than what I have on file!
Juan Pablo Tosca Villanueva (May 24 2024 at 15:28):Aha!
Juan Pablo Tosca Villanueva (May 24 2024 at 15:30):That is kind of wild those
Juan Pablo Tosca Villanueva (May 24 2024 at 15:30):Tho
Juan Pablo Tosca Villanueva (May 24 2024 at 15:30):Lol
Juan Pablo Tosca Villanueva (May 24 2024 at 15:30):Reading this
Philip Durbin 🚀 (May 24 2024 at 15:30):Let me try them on "hello"
Juan Pablo Tosca Villanueva (May 24 2024 at 15:30):‘You can't show secrets through echo otherwise there would be a huge security problem (even using env variables as an intermediary).’
Juan Pablo Tosca Villanueva (May 24 2024 at 15:31):That would make me think that you wouldn’t be able but :stuck_out_tongue:
Philip Durbin 🚀 (May 24 2024 at 15:31):wait, wait, didn't I already try them, by getting them at the org (gdcc) level? :thinking:
Philip Durbin 🚀 (May 24 2024 at 15:31):I'll try them locally first, from settings.xml
Philip Durbin 🚀 (May 24 2024 at 15:33):
Philip Durbin 🚀 (May 24 2024 at 15:35):The gddc org creds work from my laptop.
Philip Durbin 🚀 (May 24 2024 at 15:40):Nice, for the "hello" app, I switched to the gddc org creds (gpg too, while I was in there) and they work, published: https://github.com/gdcc/hello/actions/runs/9226506854/job/25386421143
Philip Durbin 🚀 (May 24 2024 at 15:41):So between the wrong dataversebot creds and the cache thing I really didn't stand a chance.
Philip Durbin 🚀 (May 24 2024 at 19:47):For the UNF repo, I simply updated DATAVERSEBOT_SONATYPE_USERNAME and DATAVERSEBOT_SONATYPE_TOKEN and now it works. Phew. I was about to mess with the cache setting. Not sure why I didn't have to. :thinking:
Philip Durbin 🚀 (May 24 2024 at 20:00):Changing path: ~/.m2 to path: ~/.m2/repository was enough to get exporters-croissant to upload a snapshot from GitHub Actions: https://github.com/gdcc/exporter-croissant/commit/de84d164beb51fd85aeb2be4a39633065f29d9f7
Philip Durbin 🚀 (May 24 2024 at 20:08):I just left a comment here: https://github.com/actions/cache/issues/1195#issuecomment-2130284022
Philip Durbin 🚀 (May 24 2024 at 20:09):I don't have any more red. All green for pushes I've tried at least:
green.png
Juan Pablo Tosca Villanueva (May 24 2024 at 20:10)::walking::evergreen_tree::evergreen_tree::evergreen_tree::evergreen_tree:
Last updated: Nov 01 2025 at 14:11 UTC