Stream: containers

Topic: update-fields.sh


view this post on Zulip Philip Durbin πŸš€ (Nov 07 2024 at 13:34):

Do we already run update-fields.sh in the dev persona? As far as I can tell, we don't. We rely on the fact that schema.xml already has all the fields for the 6 metadata blocks we ship.

I'm asking because I'm thinking about loading the CodeMeta block in the "dev" persona. But it looks like I'll need to call update-fields.sh afterwards.

view this post on Zulip Philip Durbin πŸš€ (Nov 07 2024 at 13:41):

We already had a draft issue for this so I just converted it into a real one:

document updating metadata block (tsv) and Solr in Dockerized dev env #11004

view this post on Zulip Philip Durbin πŸš€ (Nov 07 2024 at 13:50):

It looks like @luddaniel talked about this back in April: https://docs.google.com/document/d/1toiFxJKGvrYMy9KrW9GZ4U6FLGhMaCipJ9ggtvZFwgc/edit?usp=sharing

Screenshot 2024-11-07 at 8.47.33β€―AM.png

view this post on Zulip Philip Durbin πŸš€ (Nov 07 2024 at 13:51):

My use case it to enable the CodeMeta metadata block for testing in #11001.

view this post on Zulip Philip Durbin πŸš€ (Nov 07 2024 at 13:56):

Ah, the script is over at #dev > Run dataverse-external-vocab-support demo under docker (loading cvocdemo.tsv)

view this post on Zulip luddaniel (Nov 07 2024 at 14:27):

Hi @Philip Durbin πŸ‰, as we have our own customized .tsv (like citation, I know bad practice) we regularly have the need to run the update-fields.sh script while testing/developing. For CVOC as an example.

view this post on Zulip luddaniel (Nov 07 2024 at 14:28):

Philip Durbin πŸ‰ said:

Ah, the script is over at #dev > Run dataverse-external-vocab-support demo under docker (loading cvocdemo.tsv)

I still run parts of that procedure.

view this post on Zulip Philip Durbin πŸš€ (Nov 07 2024 at 14:39):

Right. I remember. And I even went back and watched part of the recording from April. :grinning:

view this post on Zulip Philip Durbin πŸš€ (Nov 07 2024 at 14:46):

I have configbaker saving schema.xml into /shared

But now I need to run update-fields.sh somehow.

view this post on Zulip Philip Durbin πŸš€ (Nov 07 2024 at 14:47):

Maybe I'm thinking about this wrong but I was hoping I could run a container after configbaker/bootstrap runs like this:

view this post on Zulip Philip Durbin πŸš€ (Nov 07 2024 at 14:47):

  dev_solr_finisher:
    container_name: "dev_solr_finisher"
    image: gdcc/configbaker:unstable
    depends_on:
      - dev_bootstrap
    restart: "no"
    command:
      - sh
      - -c
      - "cat /shared/schema.xml"
    volumes:
      - ./docker-dev-volumes/solr/data:/var/solr
      - ./docker-dev-volumes/solr/conf:/solr-template
      - ./docker-dev-volumes/shared:/shared

view this post on Zulip Philip Durbin πŸš€ (Nov 07 2024 at 14:48):

But this new "dev_solr_finisher" container doesn't wait for "dev_bootstrap". It runs right away and can't find /shared/schema.xml :thinking:

view this post on Zulip luddaniel (Nov 07 2024 at 15:00):

I think @Oliver Bertuch will have the correct idea.
In my mind I'm thinking about a "toolbox ready to be use", you give one .tsv and it updates everything. As you did for https://dataverse-guide--10940.org.readthedocs.build/en/10940/container/running/demo.html#multiple-languages
@JΓ©rΓ΄me Roucou I don't reminber, but you might have a .sh for this.

view this post on Zulip Philip Durbin πŸš€ (Nov 07 2024 at 15:02):

I was hoping this would work but it doesn't:

    depends_on:
      dev_bootstrap:
        condition: service_completed_successfully

view this post on Zulip Philip Durbin πŸš€ (Nov 07 2024 at 15:03):

I end up with [**ERROR**] DOCKER> [gdcc/configbaker:unstable] "dev_bootstrap": Timeout after 10267 ms while waiting on exit code 0

view this post on Zulip luddaniel (Nov 07 2024 at 15:03):

Philip Durbin πŸ‰ said:

I was hoping this would work but it doesn't:

    depends_on:
      dev_bootstrap:
        condition: service_completed_successfully

There is a something that wait for http://localhost:8080 to be started maybe you use the same idea.

view this post on Zulip Philip Durbin πŸš€ (Nov 07 2024 at 15:12):

condition: service_completed_successfully works fine here: https://github.com/IQSS/dataverse-frontend/pull/180/files

view this post on Zulip Philip Durbin πŸš€ (Nov 07 2024 at 15:28):

Waiting 3 minutes like this is dumb but it works and unblocks me :sweat_smile:

sleep 180 && cat /shared/schema.xml

view this post on Zulip Philip Durbin πŸš€ (Nov 07 2024 at 21:12):

Nice, I got something hacky working:

dev_solr_finisher> Β  Β  <copyField source="codeVersion" dest="_text_" maxChars="3000"/>

view this post on Zulip Philip Durbin πŸš€ (Nov 08 2024 at 20:24):

Ok, I got a proper solution working. I'm happy for feedback though: https://github.com/IQSS/dataverse/pull/11001/commits/5afad8e6ac390461d42603f56efa583e70d87421


Last updated: Oct 30 2025 at 05:14 UTC