Philip Durbin 🚀 (Jul 24 2024 at 18:41): Philip Durbin 🚀 (Jul 24 2024 at 18:41):Huh, I'm getting a crazy error at https://github.com/IQSS/dataverse/actions/runs/10078623902/job/27869639991?pr=10721
Philip Durbin 🚀 (Jul 24 2024 at 18:41):[INFO] DOCKER> ERROR: failed to solve: process "/bin/bash -euo pipefail -c # Create pathes\n mkdir -p \"${HOME_DIR}\" \"${PAYARA_DIR}\" \"${DEPLOY_DIR}\" \"${CONFIG_DIR}\" \"${SCRIPT_DIR}\"\n mkdir -p \"${STORAGE_DIR}\" \"${SECRETS_DIR}\" \"${DUMPS_DIR}\"\n # Create user\n addgroup --gid ${GID} payara\n adduser --system --uid ${UID} --no-create-home --shell /bin/bash --home \"${HOME_DIR}\" --gecos \"\" --ingroup payara payara\n echo payara:payara | chpasswd\n # Set permissions\n # Note: Following OpenShift best practices for arbitrary user id support:\n # https://docs.openshift.com/container-platform/4.14/openshift_images/create-images.html#use-uid_create-images\n chown -R payara:0 \"${HOME_DIR}\" \"${STORAGE_DIR}\" \"${SECRETS_DIR}\" \"${DUMPS_DIR}\"\n chmod -R g=u \"${HOME_DIR}\" \"${STORAGE_DIR}\" \"${SECRETS_DIR}\" \"${DUMPS_DIR}\"\n\n" did not complete successfully: exit code: 127
Philip Durbin 🚀 (Jul 24 2024 at 18:41):This is a new one on me. Any ideas? I'm not doing anything Docker-related in PR #10721.
Philip Durbin 🚀 (Jul 24 2024 at 19:20):I get the same error when I run mvn -f modules/container-base -Pct package on a Rocky 8 box.
Philip Durbin 🚀 (Jul 24 2024 at 19:28):If I remove all the "RUN <<EOF" stanzas I can build the base image but I have a feeling we need them. :grinning:
Philip Durbin 🚀 (Jul 24 2024 at 19:42):From this file, I mean:
modules/container-base/src/main/docker/Dockerfile
Philip Durbin 🚀 (Jul 24 2024 at 19:43):For the first stanza, if I delete several lines at the end, it works:
RUN <<EOF
# Create pathes
mkdir -p "${HOME_DIR}" "${PAYARA_DIR}" "${DEPLOY_DIR}" "${CONFIG_DIR}" "${SCRIPT_DIR}"
mkdir -p "${STORAGE_DIR}" "${SECRETS_DIR}" "${DUMPS_DIR}"
# Create user
EOF
The next line is this:
addgroup --gid ${GID} payara
And come to think of it, https://github.com/IQSS/dataverse/actions/runs/10078623902/job/27869639991?pr=10721 does say [INFO] DOCKER> 0.711 /bin/bash: line 5: addgroup: command not found :thinking:
Philip Durbin 🚀 (Jul 24 2024 at 19:47):Hmm... "[Bug]: Error building with temurin-17: adduser not found" - https://github.com/adoptium/containers/issues/606
Philip Durbin 🚀 (Jul 24 2024 at 19:47):"From what I understand, this commit from 2 week ago changed the default base image from Ubuntu focal (20.04) to Ubuntu noble (24.04).
The official ubuntu:focal image used to have useradd installed by default, but ubuntu:noble does not:"
Philip Durbin 🚀 (Jul 24 2024 at 19:59):Hmm, you'd think this would fix it but I'm still getting Ubuntu 24:
$ git diff
diff --git a/modules/container-base/src/main/docker/Dockerfile b/modules/container-base/src/main/docker/Dockerfile
index 93f9fa4f0c..4e03b3ee94 100644
--- a/modules/container-base/src/main/docker/Dockerfile
+++ b/modules/container-base/src/main/docker/Dockerfile
@@ -22,7 +22,7 @@
# Make the Java base image and version configurable (useful for trying newer Java versions and flavors)
-ARG JAVA_IMAGE="eclipse-temurin:17-jre"
+ARG JAVA_IMAGE="eclipse-temurin:17-jre-focal"
FROM $JAVA_IMAGE
```
This works:
-ARG JAVA_IMAGE="eclipse-temurin:17-jre"
-FROM $JAVA_IMAGE
+FROM eclipse-temurin:17-jre-focal
But I'm not sure why I can't just change the env variable.
Philip Durbin 🚀 (Jul 24 2024 at 20:26):I created an issue: Upgrade from Ubuntu 22 to 24 breaks container images #10722
Philip Durbin 🚀 (Jul 24 2024 at 20:42):I went ahead and pushed some fixes into this PR, which I renamed:
add local_lib mvn repo and primefaces themes, pin base image to Ubuntu 22 #10721
Philip Durbin 🚀 (Jul 25 2024 at 13:21):I closed that PR #10721 and am focusing on this one instead:
Security optimizations for the container base image #10672
Philip Durbin 🚀 (Jul 25 2024 at 13:22):However, I'm getting a weird error: [ERROR] DOCKER> Unable to inspect image [solr:] : {"message":"invalid reference format"} (Bad Request: 400) [{"message":"invalid reference format"} (Bad Request: 400)]
More lines of output at https://github.com/IQSS/dataverse/pull/10672#issuecomment-2250307844
Oliver Bertuch (Jul 25 2024 at 13:22):There is a tag missing
Philip Durbin 🚀 (Jul 25 2024 at 13:23):Oh. Did I do something wrong?
Oliver Bertuch (Jul 25 2024 at 13:23):Not sure... :smile:
Oliver Bertuch (Jul 25 2024 at 13:26):Nope, fails here too.
Oliver Bertuch (Jul 25 2024 at 13:26):Let me check
Oliver Bertuch (Jul 25 2024 at 13:27):That's odd - it doesn't push the tag into the configbaker build...
Kris Dekeyser (KU Leuven) (Jul 25 2024 at 13:28):If changing the ARG value does not work, you may be calling docker build with a --buildarg argument overriding your default setting in the Dockerfile.
Oliver Bertuch (Jul 25 2024 at 13:28):I think there was an upstream issue about this... Let me go look
Oliver Bertuch (Jul 25 2024 at 13:28):https://github.com/fabric8io/docker-maven-plugin/issues/1800
Oliver Bertuch (Jul 25 2024 at 13:29):Merged 5 days ago...
Oliver Bertuch (Jul 25 2024 at 13:29):https://github.com/fabric8io/docker-maven-plugin/issues/1776
Oliver Bertuch (Jul 25 2024 at 13:30):Closes a while ago as fixed, but missing a release...
Oliver Bertuch (Jul 25 2024 at 13:30):And still no newer release... :grimacing:
Oliver Bertuch (Jul 25 2024 at 13:31):So maybe we downgrade to 0.43.4 again and tell people to use the 0.45-SNAPSHOT if they run into the driver flag problem?
Oliver Bertuch (Jul 25 2024 at 13:54):We talked about this during the containerization meeting: https://docs.google.com/document/d/13Zxg_iFiTOYfIJkUSmb7ytL-OeIc2vPGCuoOvtPAd6w/
Oliver Bertuch (Jul 25 2024 at 13:58):@Philip Durbin it is possible to auto-active a profile just for Macs. Put the links in the meeting notes doc
Philip Durbin 🚀 (Jul 25 2024 at 14:38):Thanks!
Philip Durbin 🚀 (Jul 25 2024 at 16:31):Ok, I merged #10672.
Philip Durbin 🚀 (Jul 25 2024 at 16:32):Before doing so I put docker maven plugin back to version it was before I messed with it.
Philip Durbin 🚀 (Jul 25 2024 at 16:33):I tested on Rocky 8. Let's worry about Macs in some other effort.
Oliver Bertuch (Jul 27 2024 at 19:39):DMP 0.45 is out. https://github.com/fabric8io/docker-maven-plugin/releases/tag/v0.45.0
Oliver Bertuch (Jul 27 2024 at 19:39):@Philip Durbin do you want to give it a shot on your Mac?
Oliver Bertuch (Jul 27 2024 at 19:39):Then we can maybe FastTrack an update of the plugin
Philip Durbin 🚀 (Jul 29 2024 at 12:15):@Oliver Bertuch sure! Do you want to go ahead and make a PR? If I can build it on my Mac I'll merge it.
Philip Durbin 🚀 (Jul 29 2024 at 13:42):@Oliver Bertuch I went ahead and made a PR: upgrade docker maven plugin to 0.45 #10730
Philip Durbin 🚀 (Jul 29 2024 at 13:43):If you can approve and merge it, I'll appreciate it.
Oliver Bertuch (Jul 29 2024 at 13:48):Builds on my machine!
Oliver Bertuch (Jul 29 2024 at 13:49):Looks like CI is done too
Oliver Bertuch (Jul 29 2024 at 13:49):Looks like good to go for me!
Philip Durbin 🚀 (Jul 29 2024 at 13:51):Great. I stuck it in "ready for review". Please feel free to approve (at minimum) and merge (I hope!).
Oliver Bertuch (Jul 29 2024 at 13:51):Done and done and done
Philip Durbin 🚀 (Jul 29 2024 at 13:52):Perfect! Thanks!
Balázs Pataki (Aug 26 2024 at 09:57):Hi!
I just wanted to do a clean install of our 6.1 based installation and got bitten by the addgroup: command not found thing.
Is there any way to somehow backport these changes to older Dataverse versions? And these changes are still just in develop and not in main, right?
What I tried is to copy the modules/* from develop to our own install. This way mvn -Pct -f modules/container-base install works, but then mvn -Pct docker:run fails with
dev_dataverse> /opt/payara/scripts/init_2_configure.sh: line 59: POSTBOOT_COMMANDS: unbound variable
Any idea how to make our old installation work? Is there maybe a way to get the older version of the eclipse-temurin:11-jre image?
Balázs Pataki (Aug 26 2024 at 13:03):Ok, I solved it by just copying the necessary groupadd, useradd commands, like these:
# Remove the default user if present (do not fail build if not, introduced by Ubuntu 24.04)
userdel --force --remove ubuntu || true
groupdel -f ubuntu || true # for some reason, groupdel on Ubuntu 22.04 does not like --force
# Create user
groupadd --gid "${GID}" "${LINUX_GROUP}"
useradd --system --uid "${UID}" --no-create-home --shell /bin/false --home "${HOME_DIR}" --gid "${LINUX_GROUP}" "${LINUX_USER}"
echo "${LINUX_USER}:$LINUX_PASSWORD" | chpasswd
# Set permissions
# Note: Following OpenShift best practices for arbitrary user id support:
# https://docs.openshift.com/container-platform/4.14/openshift_images/create-images.html#use-uid_create-images
chown -R "${LINUX_USER}:0" "${HOME_DIR}" "${STORAGE_DIR}" "${SECRETS_DIR}" "${DUMPS_DIR}"
chmod -R g=u "${HOME_DIR}" "${STORAGE_DIR}" "${SECRETS_DIR}" "${DUMPS_DIR}"
With this fix can now use the original 6.1 docker scripts as is.
Philip Durbin 🚀 (Aug 26 2024 at 13:46):@Balázs Pataki I'm glad you figured out a work around. One of our priorities is to start tagging both the base image and dataverse image with a version number. Would this help? If you could pin the base image you use to version 6.1 or whatever? (I'm just using 6.1 as an example. We would only tag new images.)
Balázs Pataki (Aug 26 2024 at 14:24):In most cases, that would be great. However in this specific case a dependency (eclipse-temurin:11-jre) changed in an unpredictable, backward way. So, even if the dataverse base images are tagged, but you need to rebuild it, the changed dependency would still cause problems.
I am kind of disappointed that dockerhub tags are not permanent and tagging doesn't guarantee that the tagged image will be the same once it is tagged, forever. Instead the tag is just a name, pointer that may point to anything and changed in the future.
Of course for dataverse base images there could be a methodology (an agreement of the docker image publisher), which guarantees tag permanence when an image is stored in dockerhub. Do you plan to do something like this and make the base images available in dockerhub or some other repository?
Philip Durbin 🚀 (Aug 26 2024 at 14:36):Well, I think it's up to us what our tagging policy is. We could push a tag once and never change it. Or we could push the same tag over and over, for security updates, for example.
Oliver Bertuch (Aug 26 2024 at 15:15):As I wrote in https://dataverse.zulipchat.com/#narrow/stream/375812-containers/topic/change.20version.20scheme.20base.20image.3F/near/452365150, let's follow VMWare's Bitnami example.
Balázs Pataki (Aug 26 2024 at 15:17):Oliver Bertuch said:
As I wrote in https://dataverse.zulipchat.com/#narrow/stream/375812-containers/topic/change.20version.20scheme.20base.20image.3F/near/452365150, let's follow VMWare's Bitnami example.
That's great, thanks!
Last updated: Oct 30 2025 at 05:14 UTC