Deirdre Kirmis (Sep 05 2023 at 19:47): Deirdre Kirmis (Sep 05 2023 at 19:47):@Philip Durbin i have tried starting up the dataverse container using compose .. i see it going through creating all of the containers for solr, dataverse, postgres, maildev, etc .. and building the db, etc .. but i see a lot of errors that look like:
"dev_bootstrap | 2023-09-05T19:38:21Z ERR Expectation failed error="failed to establish an http connection, caused by: Get \"http://dataverse:8080/api/info/version\": dial tcp 172.18.0.7:8080: connect: connection refused" address=http://dataverse:8080/api/info/version
dev_bootstrap | 2023-09-05T19:39:23Z INF [HTTP] Checking the http://dataverse:8080/api/info/version ...
dev_bootstrap | 2023-09-05T19:39:23Z ERR Expectation failed error="the status code doesn't expect" actual=404 expect=200"
Philip Durbin 🚀 (Sep 05 2023 at 19:47):Hmm, a few of those errors are ok at the beginning.
Philip Durbin 🚀 (Sep 05 2023 at 19:48):Is it still going? Still doing stuff?
Deirdre Kirmis (Sep 05 2023 at 19:48):do i need to edit the docker-compose-dev.yml file and change the hostname? and/or do i still need to build the image?
Deirdre Kirmis (Sep 05 2023 at 19:49):it is running (looks like it) but i can't reach it from <mydomain>:8080 (I added a dns entry pointing to a load balancer, pointing to the instance that I am running docker in)
Philip Durbin 🚀 (Sep 05 2023 at 19:49):You shouldn't need to build the image.
Philip Durbin 🚀 (Sep 05 2023 at 19:51):Did you see this at the end? "Done, your instance has been configured for development. Have a nice day!"
Deirdre Kirmis (Sep 05 2023 at 19:51):okay it might be okay and i just need to adjust something with security groups or something else .. it looks like it is running, i am seeing payara messages that look like normal ones
Deirdre Kirmis (Sep 05 2023 at 19:51):oh i didn't see that message you mentioned ..
Philip Durbin 🚀 (Sep 05 2023 at 19:53):That message should come at the end. It's how you know Dataverse has been configured.
Philip Durbin 🚀 (Sep 05 2023 at 19:53):You shouldn't need to edit any hostnames.
Deirdre Kirmis (Sep 05 2023 at 19:53):the last thing that i see is the payara log and the message "HarvesterService: checking timer 2"
Philip Durbin 🚀 (Sep 05 2023 at 19:57):Hmm, would you be able to attach as much of the output as possible here as a text file?
Deirdre Kirmis (Sep 05 2023 at 19:58):yes, let me stop and start it again .. here are all the containers running:
docker-stuff.JPG
Deirdre Kirmis (Sep 05 2023 at 19:58):just a sec .. will get the output
Deirdre Kirmis (Sep 05 2023 at 20:29):here is all that i was able to get .. not sure it is getting all of the errors
docker-output4.txt
Philip Durbin 🚀 (Sep 05 2023 at 20:31):Thanks. Something is fishy :fish:
Philip Durbin 🚀 (Sep 05 2023 at 20:33):This is a bad sign: dev_bootstrap | Error: context deadline exceeded
Philip Durbin 🚀 (Sep 05 2023 at 20:36):Anything we should know about the machine you're running Docker on? Is it Linux? Some server somewhere?
Deirdre Kirmis (Sep 05 2023 at 20:36):i'm running it on ubuntu 22 in aws
Deirdre Kirmis (Sep 05 2023 at 20:36):maybe i should try centos or rocky linux?
Philip Durbin 🚀 (Sep 05 2023 at 20:37):Ok, my first thought is to simply try again. That is kill all the running containers (Ctrl-c might be enough). rm -rf docker-dev volumes. Then try again. Fresh terminal for fresh output. :big_smile:
Deirdre Kirmis (Sep 05 2023 at 20:38):okay will do .. i deleted all the images and containers before running this time, but didn't kill the dev volumes .. thanks! will let you know how that goes :-)
Philip Durbin 🚀 (Sep 05 2023 at 20:39):Awesome. Thanks. And Ubuntu should be fine. I'm using a Mac.
Deirdre Kirmis (Sep 05 2023 at 20:39)::+1: thanks for your help!
Philip Durbin 🚀 (Sep 05 2023 at 20:41):Sure. Sorry it didn't just work.
Deirdre Kirmis (Sep 05 2023 at 20:41):oh that's okay .. it's probably just me! i have a way of making things not work until i put hours of time into figuring out what I messed up! :-D
Philip Durbin 🚀 (Sep 05 2023 at 20:43):I mean, something else I could have you try is the "alpha" images. That's what @Thomas van Erven is using. They are equivalent to 5.14 or the "master" branch. You're getting the "unstable" images, which is the tip of the "develop" branch.
Deirdre Kirmis (Sep 05 2023 at 20:46):i think i did clone the master branch :grinning_face_with_smiling_eyes: .. should i clone a different one?
Philip Durbin 🚀 (Sep 05 2023 at 20:47):Well, we switched our default branch from master/main to develop. You could run a "git status" to check.
Deirdre Kirmis (Sep 05 2023 at 20:47):ah yea, it is "develop"
Deirdre Kirmis (Sep 05 2023 at 21:19):i killed everything and created it all again .. here is the output .. don't see any obvious errors, but I also don't see the "Done" message
docker-output4.txt
Deirdre Kirmis (Sep 05 2023 at 21:19):i can try the master branch images .. haven't done that yet
Deirdre Kirmis (Sep 05 2023 at 21:34):or try installing maven, java, etc and build/run it that way
Philip Durbin 🚀 (Sep 05 2023 at 22:13):This is good! dev_dataverse | Loading application [dataverse] at [/]|#]
Philip Durbin 🚀 (Sep 05 2023 at 22:15):But again we have this: dev_bootstrap | Error: context deadline exceeded
Philip Durbin 🚀 (Sep 05 2023 at 22:17):Ah, you should be able to override the timeout: https://github.com/IQSS/dataverse/blob/2246d666017fbc169552e4dfc74f4c635d84ce28/modules/container-configbaker/scripts/bootstrap.sh#L27
Philip Durbin 🚀 (Sep 05 2023 at 22:19):Adding something like this might help:
diff --git a/docker-compose-dev.yml b/docker-compose-dev.yml
index ab44dbc180..07ccf67e44 100644
--- a/docker-compose-dev.yml
+++ b/docker-compose-dev.yml
@@ -38,6 +38,8 @@ services:
container_name: "dev_bootstrap"
image: gdcc/configbaker:unstable
restart: "no"
+ environment:
+ - TIMEOUT:3m
command:
- bootstrap.sh
- dev
I think. :sweat_smile:
Deirdre Kirmis (Sep 05 2023 at 22:21):will try it .. thanks!
Thomas van Erven (Sep 06 2023 at 05:33):Maybe a bit late to the party (and possibly you've hit the point already), but what size machine are you trying to run it on? JVM instances are typically a bit on the "heavy" side in terms of underlying resource requirements.
Oliver Bertuch (Sep 06 2023 at 05:52):The Dataverse container is typically limited to 2GB of RAM. Stats show that's often enough for development and running tests.
Philip Durbin 🚀 (Sep 06 2023 at 14:34):I get the same failure if I lower the timeout from 2m to 1m like this:
diff --git a/docker-compose-dev.yml b/docker-compose-dev.yml
index ab44dbc180..41a450616c 100644
--- a/docker-compose-dev.yml
+++ b/docker-compose-dev.yml
@@ -38,6 +38,8 @@ services:
container_name: "dev_bootstrap"
image: gdcc/configbaker:unstable
restart: "no"
+ environment:
+ - TIMEOUT=1m
command:
- bootstrap.sh
- dev
Actually, I'm getting dev_bootstrap | Error: context deadline exceeded when I revert back to 2m. :grimacing:
Philip Durbin 🚀 (Sep 06 2023 at 15:00):This is on my Mac, I mean.
Philip Durbin 🚀 (Sep 06 2023 at 15:00):I'm getting this:
dev_dataverse | Exception while loading the app|#]
dev_dataverse | Caused by: org.flywaydb.core.internal.command.DbMigrate$FlywayMigrateException: Migration V4.11__5513-database-variablemetadata.sql failed
And yet, it's working fine at https://github.com/gdcc/api-test-runner
Philip Durbin 🚀 (Sep 06 2023 at 15:01):Why?
Oliver Bertuch (Sep 06 2023 at 15:07):I suspect IO problems.
Philip Durbin 🚀 (Sep 06 2023 at 15:20):Who knows. And my problem seems different than Deirdre's.
Philip Durbin 🚀 (Sep 06 2023 at 18:52):My Mac had been up for 196 days. I just rebooted (and updated to macOS 12.6.8 while I was at it).
docker compose -f docker-compose-dev.yml up works fine now. Phew.
Philip Durbin 🚀 (Sep 06 2023 at 18:53):@Deirdre Kirmis how are you making out? Sorry for the noise above. My troubles seem unrelated to yours. :sweat_smile:
Deirdre Kirmis (Sep 06 2023 at 18:53):sorry just got back to this .. i am using a t3a.medium instance .. i can try increasing that .. i tried setting the timeout to 3m but still failed with the context deadline exceeded error .. i looked that up and it seems like it could indicate network errors so I could have some security group issues (although I have port 8080, 80, and 443 open) .. so may try to just create a public instance
Deirdre Kirmis (Sep 06 2023 at 18:54):or i could try a higher timeout?
Philip Durbin 🚀 (Sep 06 2023 at 18:59):Hmm, t3a.medium seems to have 4 GB of memory according to https://aws.amazon.com/ec2/instance-types/t3/
The runners in GitHub Actions have 7 GB of memory according to https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources
So, yeah, could you try a t3a.large? That should give you 8 GB of memory.
Deirdre Kirmis (Sep 06 2023 at 19:08):okay will do that now :fingers_crossed:
Deirdre Kirmis (Sep 06 2023 at 19:20):sorry i know open ports don't matter if i'm just trying to get the app running :smile:
Deirdre Kirmis (Sep 06 2023 at 19:21):have increased the instance size and starting it up now (with 3m timeout)
Philip Durbin 🚀 (Sep 06 2023 at 19:21):Ha, no worries. Occupied ports do matter. When I rebooted I had a local postgres running that I had to kill. :big_smile:
Deirdre Kirmis (Sep 06 2023 at 19:23):argh got the same error
Deirdre Kirmis (Sep 06 2023 at 19:23):wonder if i should try a different instance type altogether .. we just use the t3a instances by default in our dev account to start with
Deirdre Kirmis (Sep 06 2023 at 19:24):or, i'm okay with installing java and maven and trying to build/run it that way .. just haven't tried yet
Philip Durbin 🚀 (Sep 06 2023 at 19:24):You see dev_bootstrap | Error: context deadline exceeded?
Deirdre Kirmis (Sep 06 2023 at 19:26):yes .. was watching closely and saw it right in the midst of a bunch of other payara messages .. it ends with "JMXStartupService has started JMXConnector on JMXService URL service:jmx:rmi" and then just stops
Philip Durbin 🚀 (Sep 06 2023 at 19:26):Any errors about the app failing to deploy?
Philip Durbin 🚀 (Sep 06 2023 at 19:27):Does curl http://localhost:8080/api/info/version work?
Philip Durbin 🚀 (Sep 06 2023 at 19:27):If there's a deployment failure sometimes you have to scroll up quite a bit.
Deirdre Kirmis (Sep 06 2023 at 19:29):yikes, let me do it all again and save the output .. i forgot to redirect output to a file!
Philip Durbin 🚀 (Sep 06 2023 at 19:29):If the containers are still running you can do docker logs dataverse-1
Philip Durbin 🚀 (Sep 06 2023 at 19:30):Or maybe docker logs dev_dataverse
Deirdre Kirmis (Sep 06 2023 at 19:30):oh right! still learning docker, too :-)
Deirdre Kirmis (Sep 06 2023 at 19:36):here is the output .. i see errors, but mostly the ones you said may be normal .. and the dev_bootstrap error
docker-output5.txt
Deirdre Kirmis (Sep 06 2023 at 19:37):docker logs dev-dataverse worked :smile:
Deirdre Kirmis (Sep 06 2023 at 19:37):curl http://localhost:8080/api/info/version works
Philip Durbin 🚀 (Sep 06 2023 at 19:48):I'm a little confused by what I'm seeing in the logs. The difference between the first messages from dev_bootstrap and the last seems to be under two minutes:
$ grep ^dev_bootstrap docker-output5.txt | head -2
dev_bootstrap | Waiting for http://dataverse:8080 to become ready in max 2m.
dev_bootstrap | 2023-09-06T19:31:21Z INF [HTTP] Checking the http://dataverse:8080/api/info/version ...
$
$ grep ^dev_bootstrap docker-output5.txt | tail -3
dev_bootstrap | 2023-09-06T19:33:19Z ERR Expectation failed error="the status code doesn't expect" actual=404 expect=200
dev_bootstrap | Error: context deadline exceeded
dev_bootstrap exited with code 124
So I'm wondering if our attempts to bump it to 3m is working or not. :thinking:
Philip Durbin 🚀 (Sep 06 2023 at 19:49):Also, there's a message in the middle with timeout=3s that I don't understand. 3s instead of 3m? Weird.
dev_bootstrap | 2023-09-06T19:32:15Z ERR Expectation failed error="timed out while making an http call, caused by: Get \"http://dataverse:8080/api/info/version\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" timeout=3s
Deirdre Kirmis (Sep 06 2023 at 19:50):errr maybe not? here is what i changed:
config-changes.jpg
Philip Durbin 🚀 (Sep 06 2023 at 19:52):Yeah, looks good. But maybe this env TIMEOUT thing doesn't work. I'm sure @Oliver Bertuch could tell us if we're barking up the wrong tree. :tree: :dog:
Philip Durbin 🚀 (Sep 06 2023 at 19:54):It looks like the 3m isn't getting into the script:
$ grep 'in max' docker-output5.txt
dev_bootstrap | Waiting for http://dataverse:8080 to become ready in max 2m.
but it did seem to in your issue above .. there was an impact when you changed it to 1m?
Philip Durbin 🚀 (Sep 06 2023 at 20:01):Well, I was having all sorts of trouble. I think rebooting it fixed it.
Philip Durbin 🚀 (Sep 06 2023 at 20:02):I can teach you how to bump up the timeout in the script itself if you want. You'll need mvn installed though.
Deirdre Kirmis (Sep 06 2023 at 20:03):okay i'll go ahead and install maven and java 11
Philip Durbin 🚀 (Sep 06 2023 at 20:04):Java 17, you mean :big_smile:
Deirdre Kirmis (Sep 06 2023 at 20:04):oops yea
Deirdre Kirmis (Sep 06 2023 at 20:13):okay got mvn v3.6.3 and jdk 17.0.8.1 installed .. should i go through the quick start/build/run or is there a way to still use compose to run after editing the script?
Deirdre Kirmis (Sep 06 2023 at 20:14):have deleted all the containers, images, and the docker-dev-volumes directory again so all clean!
Oliver Bertuch (Sep 06 2023 at 20:15):I'm on my cell, but we could do a quick Zoom if y'all want.
Oliver Bertuch (Sep 06 2023 at 20:16):Maybe it helps looking at things as they happen
Deirdre Kirmis (Sep 06 2023 at 20:18):sure if you are all available and willing! i don't want to be a pain :sweat_smile:
now that i have mvn and java installed i could probably go through the quick start and see if that works
Philip Durbin 🚀 (Sep 06 2023 at 20:19):@Deirdre Kirmis well, we started this thread with docker compose so I guess we should stick with that. :big_smile:
Deirdre Kirmis (Sep 06 2023 at 20:19):okay sounds good .. however i can get it to work! i'm learning a lot about docker
Philip Durbin 🚀 (Sep 06 2023 at 20:19):Please edit vim modules/container-configbaker/scripts/bootstrap.sh and increase the timeout.
Oliver Bertuch (Sep 06 2023 at 20:19):@Philip Durbin can you light a fireplace to gather round?
Philip Durbin 🚀 (Sep 06 2023 at 20:20):Then try this
mvn -Pct docker:build -Ddocker.filter=dev_bootstrap
i increased it to 3m
Philip Durbin 🚀 (Sep 06 2023 at 20:20):(As I just re-documented! See #containers > iterating on configbaker ) :big_smile:
Deirdre Kirmis (Sep 06 2023 at 20:21):running the mvn command ...
Philip Durbin 🚀 (Sep 06 2023 at 20:21):You should see something like this:
[INFO] DOCKER> [gdcc/configbaker:unstable] "dev_bootstrap": Built image sha256:c5e0d
still going ...
Philip Durbin 🚀 (Sep 06 2023 at 20:23):It should only take a few seconds...
[INFO] Total time: 4.181 s
Might be taking a while to download the Solr image
Philip Durbin 🚀 (Sep 06 2023 at 20:24):Ah, right
Deirdre Kirmis (Sep 06 2023 at 20:24):it is downloading things from the maven repo
Oliver Bertuch (Sep 06 2023 at 20:24):First timers, cold cache
Philip Durbin 🚀 (Sep 06 2023 at 20:24):brr! :cold_face:
Oliver Bertuch (Sep 06 2023 at 20:24):Oh dear, Maven downloading the internet
Oliver Bertuch (Sep 06 2023 at 20:24):That's a very cold cache
Philip Durbin 🚀 (Sep 06 2023 at 20:25):@Deirdre Kirmis welcome to our world :sweat_smile:
Deirdre Kirmis (Sep 06 2023 at 20:25):it is doing this:
Downloading from central-repo: https://repo1.maven.org/maven2/com/google/cloud/google-cloud-security-private-ca-bom/2.5.1/google-cloud-security-private-ca-bom-2.5.1.pom
Downloaded from central-repo: https://repo1.maven.org/maven2/com/google/cloud/google-cloud-security-private-ca-bom/2.5.1/google-cloud-security-private-ca-bom-2.5.1.pom (3.5 kB at 140 kB/s)
Downloading from payara-nexus-artifacts: https://nexus.payara.fish/repository/payara-artifacts/com/google/cloud/google-cloud-servicedirectory-bom/2.4.1/google-cloud-servicedirectory-bom-2.4.1.pom
Downloading from payara-patched-externals: https://raw.github.com/payara/Payara_PatchedProjects/master/com/google/cloud/google-cloud-servicedirectory-bom/2.4.1/google-cloud-servicedirectory-bom-2.4.1.pom
Oliver Bertuch (Sep 06 2023 at 20:25):Maybe we should put a note about that in the docs :see_no_evil:
Deirdre Kirmis (Sep 06 2023 at 20:25):but many more things ..
Oliver Bertuch (Sep 06 2023 at 20:26):Yeah, basically each and every dependency necessary for the project.
Philip Durbin 🚀 (Sep 06 2023 at 20:26):@Oliver Bertuch I'm having flashbacks to getting Steve to try Docker. :sweat_smile:
Deirdre Kirmis (Sep 06 2023 at 20:26):hope i made the volume large enough! :sweat_smile:
Oliver Bertuch (Sep 06 2023 at 20:27):Can't try now but still wondering about timeout env not being picked up...
Oliver Bertuch (Sep 06 2023 at 20:27):Have you tried changing it with a command argument?
Oliver Bertuch (Sep 06 2023 at 20:27):Instead of repackaging...
Oliver Bertuch (Sep 06 2023 at 20:28):Now the only arg is "dev". You can give more arguments.
Oliver Bertuch (Sep 06 2023 at 20:28):See bootstrap.sh -h for deets
Deirdre Kirmis (Sep 06 2023 at 20:29):no but i can if you think that's a better idea? i could do it in another console while this is running, if that would work
Oliver Bertuch (Sep 06 2023 at 20:29):Sure, that's no problemo
Oliver Bertuch (Sep 06 2023 at 20:29):You could try with docker run first to get a feeling for the command
Oliver Bertuch (Sep 06 2023 at 20:30):So like docker run -it --rm gdcc/configbaker:unstable bootstrap.sh -h
Deirdre Kirmis (Sep 06 2023 at 20:31):what is that command doing?
Oliver Bertuch (Sep 06 2023 at 20:32):It will start an interactive container for you, run the specified shell script inside the container and when that ends will stop and remove the container for you
Deirdre Kirmis (Sep 06 2023 at 20:32):oh right okay i see .. it did all that
Oliver Bertuch (Sep 06 2023 at 20:33):You should see some help messages in the output :smile:
Deirdre Kirmis (Sep 06 2023 at 20:33):yes, timeout is one of them
Oliver Bertuch (Sep 06 2023 at 20:34):IIRC, it should be sth like bootstrap.sh -t 3m dev
Oliver Bertuch (Sep 06 2023 at 20:34):Could you try that with docker run for me, see if it gets picked up?
Oliver Bertuch (Sep 06 2023 at 20:35):You can always exit the script with ctrl-c as with any other shell script you're running on your machine
Deirdre Kirmis (Sep 06 2023 at 20:35):so .. docker run -it --rm gdcc/configbaker:unstable bootstrap.sh -h -t 3m dev ??
Oliver Bertuch (Sep 06 2023 at 20:36):Please skip the -h, we don't want the help output again :wink:
Oliver Bertuch (Sep 06 2023 at 20:37):Other than that: fire at will
Deirdre Kirmis (Sep 06 2023 at 20:37):it does say " to become ready in max 3m"
Oliver Bertuch (Sep 06 2023 at 20:37):Ha!
Oliver Bertuch (Sep 06 2023 at 20:37):So correct syntax
Oliver Bertuch (Sep 06 2023 at 20:37):So let's put this in the docker compose
Deirdre Kirmis (Sep 06 2023 at 20:37):that's a lot easier .. so i would add to the end of the compose command?
Philip Durbin 🚀 (Sep 06 2023 at 20:38):Something like this:
diff --git a/docker-compose-dev.yml b/docker-compose-dev.yml
index ab44dbc180..2126aa2c0e 100644
--- a/docker-compose-dev.yml
+++ b/docker-compose-dev.yml
@@ -39,7 +39,7 @@ services:
image: gdcc/configbaker:unstable
restart: "no"
command:
- - bootstrap.sh
+ - bootstrap.sh -t 3m
- dev
networks:
- dataverse
(adding "dev" isn't strictly necessary)
Oliver Bertuch (Sep 06 2023 at 20:38):Careful... args are a it tricky in commands when not from CLI
Philip Durbin 🚀 (Sep 06 2023 at 20:38):oh?
Oliver Bertuch (Sep 06 2023 at 20:38):It's an array for a reason
Oliver Bertuch (Sep 06 2023 at 20:39):Let me go get deets from the compose refernce
Deirdre Kirmis (Sep 06 2023 at 20:40):i thought you meant .. "docker compose -f docker-compose-dev.yml bootstrap.sh -t 3m up" or something like that?
Deirdre Kirmis (Sep 06 2023 at 20:41):the maven build is done btw :big_smile:
Oliver Bertuch (Sep 06 2023 at 20:41):https://docs.docker.com/compose/compose-file/05-services/#command
Philip Durbin 🚀 (Sep 06 2023 at 20:42):Whoops, you're right:
dev_bootstrap | [dumb-init] bootstrap.sh -t 1m: No such file or directory
If given as array, we must use
- bootstrap.sh
- -t
- 3m
- dev
So just split at every space char
Deirdre Kirmis (Sep 06 2023 at 20:43):ahhhh .. glad i didn't try my idea .. okay will edit the yml again
Deirdre Kirmis (Sep 06 2023 at 20:45):okay like this?
yml-changes.JPG
Oliver Bertuch (Sep 06 2023 at 20:45):Maybe while at it go for 5m
Oliver Bertuch (Sep 06 2023 at 20:45):Reducing later is easy... :wink:
Deirdre Kirmis (Sep 06 2023 at 20:45):done! .. will try the original compose command again
Oliver Bertuch (Sep 06 2023 at 20:46):BTW if you don't want all the clutter, just start compose in background with -d
Oliver Bertuch (Sep 06 2023 at 20:46):You can get the logs from just the bootstrap container
Oliver Bertuch (Sep 06 2023 at 20:47):Probably easier to monitor for changes
Oliver Bertuch (Sep 06 2023 at 20:47):There is an entire subsection in the guide about running in foreground/background
Deirdre Kirmis (Sep 06 2023 at 20:49):thanks for the guide and all the info and help
Oliver Bertuch (Sep 06 2023 at 20:49):http://preview.guides.gdcc.io/en/develop/container/dev-usage.html#running
Oliver Bertuch (Sep 06 2023 at 20:49):Oh don't thank me. Most of it probably was @Philip Durbin
Deirdre Kirmis (Sep 06 2023 at 20:50):thank you phil! :tada:
Philip Durbin 🚀 (Sep 06 2023 at 20:50):Docs, docs, docs. Docker docs!
Deirdre Kirmis (Sep 06 2023 at 20:50):it worked!
Deirdre Kirmis (Sep 06 2023 at 20:50):i got the "have a nice day" message!
Deirdre Kirmis (Sep 06 2023 at 20:52):i still can't connect in a browser, but that is probably my instance and SG settings .. will work on that
Philip Durbin 🚀 (Sep 06 2023 at 20:52):that's awesome
Philip Durbin 🚀 (Sep 06 2023 at 20:52):welcome to Club Docker
Deirdre Kirmis (Sep 06 2023 at 20:52):thank you both so much! at least i know the docker container is running dataverse!
Philip Durbin 🚀 (Sep 06 2023 at 20:53):You could curl http://localhost:8080/api/metadatablocks/citation if you want
Deirdre Kirmis (Sep 06 2023 at 20:53):it should be running in the browser on port 8080 right?
so my SG should just allow that port?
Philip Durbin 🚀 (Sep 06 2023 at 20:54):Hmm, good question. I've only run this on my laptop.
Oliver Bertuch (Sep 06 2023 at 20:54):To be frank: I have next to never ever used AWS, so I'm afraid I'm not of much help here
Deirdre Kirmis (Sep 06 2023 at 20:54):that curl command worked .. i can figure out the SG thing .. i think i know what the issue is .. thank you, thank you, thank you!!!!
Oliver Bertuch (Sep 06 2023 at 20:55):But... there probably are good tutorials about how to expose a container on AWS
Philip Durbin 🚀 (Sep 06 2023 at 20:55):@Deirdre Kirmis you could come to the container meeting tomorrow if you like. A bit early for you though. :grimacing: https://ct.gdcc.io
Deirdre Kirmis (Sep 06 2023 at 20:55):yes, i will look it up .. you all are awesome!!!!!
Oliver Bertuch (Sep 06 2023 at 20:55):Yes, please come and hang with us! We need people to tell us what to improve to make the experience smoother
Deirdre Kirmis (Sep 06 2023 at 20:56):yes, i will look it up .. you all are awesome!!!!!
Deirdre Kirmis (Sep 06 2023 at 20:56):i may attend the container meeting as a fly on the wall .. i can't contribute, but i can learn
Philip Durbin 🚀 (Sep 06 2023 at 20:56):For the frontend repo they use nginx: https://github.com/IQSS/dataverse-frontend/blob/b44205da066698fe2b1015be7d264f45b274e27b/dev-env/docker-compose-dev.yml#L4-L16
Deirdre Kirmis (Sep 06 2023 at 20:56):ah yes, i need to configure nginx! will do that
Philip Durbin 🚀 (Sep 06 2023 at 20:56):@Deirdre Kirmis when you're ready, a new Zulip topic about ports, please! :big_smile:
Oliver Bertuch (Sep 06 2023 at 20:56):If you want let's encrypt, I suggest Traefik
Philip Durbin 🚀 (Sep 06 2023 at 20:57):@Oliver Bertuch should we bump up the default timeout to 3m?
Oliver Bertuch (Sep 06 2023 at 20:57):Fine with me!
Oliver Bertuch (Sep 06 2023 at 20:58):Thinking along the lines of options to detect where we are running to auto adapt...
Oliver Bertuch (Sep 06 2023 at 20:58):But maybe just not worth the effort
Oliver Bertuch (Sep 06 2023 at 20:59):@Deirdre Kirmis @Philip Durbin would it make sense to document steps taken on AWS for this?
Oliver Bertuch (Sep 06 2023 at 20:59):It might help others to know which EC2 flavor has been successfully used before
Oliver Bertuch (Sep 06 2023 at 21:00):Maybe we can ask @Loïc and @Thomas van Erven to add their XP with Google and Azure
Philip Durbin 🚀 (Sep 06 2023 at 21:05):Here's a PR to bump from 2m to 3m: https://github.com/IQSS/dataverse/pull/9886
Deirdre Kirmis (Sep 06 2023 at 21:08):yes, for sure, as soon as i get this figured out i'll document how i got it working .. trying to figure out the config for nginx to run it ... i just set it up recently to run an angular app so must be somewhat similar .. ??
Deirdre Kirmis (Sep 06 2023 at 21:10):and will document the instance size that i'm using .. i could try downgrading it back to t3a.medium and see if it still runs
Deirdre Kirmis (Sep 06 2023 at 21:15):gotta go to a meeting but will work on all of this after .. thanks again!
Philip Durbin 🚀 (Sep 07 2023 at 10:42):Yeah, good idea. Maybe a t3a.medium is enough. Worth trying.
Deirdre Kirmis (Sep 07 2023 at 16:46):i downgraded to t3a.medium and re-ran the deployment and it worked! .. i am able to reach it from http://dataverse-docker.lib.asu.edu:8080/; still working on getting it running behind a load balancer with https, port 443, etc .. had it working last night but messed something up and now it isn't :neutral: .. think i will have to figure out how to run nginx in a container to do proxy pass
Deirdre Kirmis (Sep 07 2023 at 16:48):with containers, in order to customize branding, set jvm options, etc .. do you have to just build that into the build?
Deirdre Kirmis (Sep 07 2023 at 16:49):errr image i mean?
Deirdre Kirmis (Sep 07 2023 at 17:22):(deleted)
Philip Durbin 🚀 (Sep 07 2023 at 17:53):I think we roll up all of that under "customization" in our proposal, linked from https://ct.gdcc.io
Philip Durbin 🚀 (Sep 07 2023 at 17:53):It's mostly a TODO, I think.
Philip Durbin 🚀 (Sep 07 2023 at 17:53):Direct link to the proposal: https://docs.google.com/document/d/14DHDB24Cp_kzpYqhHCKCtnzOw8_WuLOOONyqJHSsaYM/edit
Philip Durbin 🚀 (Sep 07 2023 at 17:54):@Deirdre Kirmis please feel free to create new topics under #containers for this or that issue.
Philip Durbin 🚀 (Sep 07 2023 at 17:54):Also! During our meeting today I realized I told you TIMEOUT:3m instead of TIMEOUT=3m above!
Deirdre Kirmis (Sep 07 2023 at 17:54)::+1:
Philip Durbin 🚀 (Sep 07 2023 at 17:55):TIMEOUT=7m works for Guillermo
Deirdre Kirmis (Sep 07 2023 at 17:55):oh maybe i'll try that and see if it works too
Philip Durbin 🚀 (Sep 07 2023 at 17:55):Yeah, and I'd like to verify it on my machine as well.
Philip Durbin 🚀 (Sep 07 2023 at 17:55):But good to know we don't have to rebuild the configbaker image. Two ways to configure the timeout, I guess. :big_smile:
Deirdre Kirmis (Sep 07 2023 at 17:56):so many options! :big_smile:
Philip Durbin 🚀 (Sep 07 2023 at 17:56):As for JVM options, you can add a lot of them to your compose file.
Philip Durbin 🚀 (Sep 07 2023 at 17:57):See https://guides.dataverse.org/en/5.14/container/app-image.html#tunables
Philip Durbin 🚀 (Sep 07 2023 at 17:59):Examples:
environment:
- DATAVERSE_DB_HOST=postgres
- dataverse_files_s3_upload__redirect=true
i will try to add some
Philip Durbin 🚀 (Sep 07 2023 at 17:59):You're also wondering about branding?
Deirdre Kirmis (Sep 07 2023 at 18:00):yea we have a lot of custom branding on our live site to match our other web communications so eventually we would want to be able to do that
Philip Durbin 🚀 (Sep 07 2023 at 18:01):Generally speaking, you'll probably want to create your own "persona" instead of the "dev" personal to configure stuff like that.
Philip Durbin 🚀 (Sep 07 2023 at 18:59):I just tried this and it works fine...
diff --git a/docker-compose-dev.yml b/docker-compose-dev.yml
index ab44dbc180..41a450616c 100644
--- a/docker-compose-dev.yml
+++ b/docker-compose-dev.yml
@@ -38,6 +38,8 @@ services:
container_name: "dev_bootstrap"
image: gdcc/configbaker:unstable
restart: "no"
+ environment:
+ - TIMEOUT=1m
command:
- bootstrap.sh
- dev
... docker logs dev_bootstrap shows "Waiting for http://dataverse:8080 to become ready in max 1m."
Sorry for the confusion yesterday!
Deirdre Kirmis (Sep 07 2023 at 19:04):oh no worries .. this is all a learning experience for me! :big_smile: .. every little bit helps!
Deirdre Kirmis (Sep 07 2023 at 19:05):i tried it on my 2nd dev installation and it worked, too :tada:
Philip Durbin 🚀 (Sep 07 2023 at 19:05):nice!
Last updated: Oct 30 2025 at 05:14 UTC