Renee L. (Aug 05 2026 at 15:12): Renee L. (Aug 05 2026 at 15:12):I'm seeing a curious issue where dataverses appear in the search results but datasets do not. Datasets can be accessed through the links in Notifications, but they don't show up in the search at all. This first appeared when I was making changes to citation.properties and geospatial.properties.
I've tried all of the commands on the Solr Search Index page with no success. The commands appear to be successful, but still no datasets appear in the UI.
For example, if I try to re-index a single dataset (curl http://localhost:8080/api/admin/index/dataset?persistentId=doi:10.5072/FK2/AAA000), I see the result:
{"status":"OK","data":{"message":"Reindexed dataset perma:DV/ZWIP4O","id":5,"persistentId":"perma:DV/ZWIP4O","versions":[{"semanticVersion":"1.0","id":1}]}}
If I clear the search index (curl http://localhost:8080/api/admin/index) the dataverses disappear except for the root, but when I index again (curl http://localhost:8080/api/admin/index), only the dataverses reappear. The datasets do not. And yes, they are published!
{"status":"OK","data":{"availablePartitionIds":[0],"args":{"numPartitions":1,"partitionIdToProcess":0},"message":"indexAllOrSubset has begun of 4 dataverses and 1 datasets."}}
The result for the Async Reindex shows the correct number of dataverses and dataset(s).
I am using podman with containers. Any advice is appreciated.
Philip Durbin 🚀 (Aug 05 2026 at 15:32):Do you see any errors in the logs from the main Dataverse container?
Philip Durbin 🚀 (Aug 05 2026 at 15:33):(For a non-Docker installation, I'm talking about server.log: https://guides.dataverse.org/en/6.11/admin/troubleshooting.html#finding-the-payara-log-file )
Renee L. (Aug 05 2026 at 18:56):I don't see any errors, actually. Here is a copy of the file, attached.
dataverse_datacore_server.log
Philip Durbin 🚀 (Aug 05 2026 at 18:58):Wow, almost nothing in that log. No noise about timers and whatnot? Has the instance been up for a while?
The noise I'm talking about looks like this:
[#|2026-08-05T18:50:01.004+0000|INFO|Payara 7.2026.2|edu.harvard.iq.dataverse.timer.DataverseTimerServiceBean|_ThreadID=288;_ThreadName=__ejb-thread-pool12;_TimeMillis=1785955801004;_LevelValue=800;|
HarvesterService: checking timer 1|#]
[#|2026-08-05T18:50:01.004+0000|INFO|Payara 7.2026.2|edu.harvard.iq.dataverse.timer.DataverseTimerServiceBean|_ThreadID=288;_ThreadName=__ejb-thread-pool12;_TimeMillis=1785955801004;_LevelValue=800;|
HarvesterService: checking timer 2|#]
No, because I keep stopping the containers and spinning them up again, trying different things. So the instance hasn't been up long.
Philip Durbin 🚀 (Aug 05 2026 at 19:05):I think we should verify that logging is working. :sweat_smile:
Renee L. (Aug 05 2026 at 19:05):That would explain it! :smile:
Philip Durbin 🚀 (Aug 05 2026 at 19:05):You're doing something like this?
docker logs -f dataverse
(For me, in dev, it's dev_dataverse.)
Philip Durbin 🚀 (Aug 05 2026 at 19:08):What if you go to http://localhost:8080/api/dataverses/doesNotExist
In the logs, I see this:
[#|2026-08-05T19:07:22.961+0000|WARNING|Payara 7.2026.2|edu.harvard.iq.dataverse.DataverseServiceBean|_ThreadID=148;_ThreadName=http-thread-pool::http-listener-1(4);_TimeMillis=1785956842961;_LevelValue=900;|
Unable to find a single dataverse using alias "doesNotExist": jakarta.persistence.NoResultException: getSingleResult() did not retrieve any entities.|#]
Can you get something like that to show up in the logs?
Renee L. (Aug 05 2026 at 19:11):When I run podman logs -f dataverse I see something like you describe. The previous log was from /appserver/glassfish/domains/domain1/logs/server.log
Philip Durbin 🚀 (Aug 05 2026 at 19:13):Ah, makes sense. We definitely want the live log.
Now! Can you please try reindexing a dataset? The command should be something like
curl http://localhost:8080/api/admin/index/dataset?persistentId=doi:10.5072/FK2/AAA000
From https://guides.dataverse.org/en/6.11/admin/solr-search-index.html#reindexing-datasets
Renee L. (Aug 05 2026 at 19:17):Here are the results from the live log, showing the "page not found" warning on down.
"methodology" is one of the fields from the custom metadata .tsv file that I added.
Philip Durbin 🚀 (Aug 05 2026 at 19:19):Yeah, it's the one that's preventing indexing.
As you say: unknown field 'methodology'
Philip Durbin 🚀 (Aug 05 2026 at 19:20):You made a brand new tsv file, right?
Philip Durbin 🚀 (Aug 05 2026 at 19:22):foobar.tsv instead of citation.tsv I mean
Renee L. (Aug 05 2026 at 19:28):Yes, according to these instructions: Metadata Block Setup
Looks like the "Updating the Solr Schema" step was the one that was failing. Possibly I was running it in the wrong container.
Philip Durbin 🚀 (Aug 05 2026 at 19:32):Gotcha. If it helps, I wrote up some container-specific instructions here: https://guides.dataverse.org/en/6.11/container/running/demo.html#additional-metadata-blocks
Philip Durbin 🚀 (Aug 05 2026 at 19:32):Sorry this is so hard and weird, by the way! It's just the way it's always been! :grimacing:
Renee L. (Aug 05 2026 at 19:33)::joy:
Renee L. (Aug 05 2026 at 20:11):Shall I create a pull request on GitHub for a podman-specific command to update the Solr schema? The equivalent of
curl http://localhost:8080/api/admin/index/solr/schema | docker run -i --rm -v ./docker-dev-volumes/solr/data:/var/solr gdcc/configbaker:unstable update-fields.sh /var/solr/data/collection1/conf/schema.xml
Also, I am unable to find the schema.xml file.
Philip Durbin 🚀 (Aug 05 2026 at 20:16):Hmm, yes, let's figure out how to document podman commands. I'd like to suggest we talk more in #containers > podman instead of docker and tomorrow's container meeting! See #containers > meetings
Philip Durbin 🚀 (Aug 05 2026 at 20:16):As for the schema.xml file, it's got to be in there somewhere! :sweat_smile:
Philip Durbin 🚀 (Aug 05 2026 at 20:19):Oh, right! Instead of docker-dev-volumes/solr you probably have data/solr?
Philip Durbin 🚀 (Aug 05 2026 at 20:20):docker-compose-dev.yml
81: - ./docker-dev-volumes/app/data:/dv
112: - ./docker-dev-volumes/app/data:/dv
docker/compose/demo/compose.yml
42: - ./data/app/data:/dv
75: - ./data/app/data:/dv
Ah yes, it's in data/solr/conf/conf . Thank you!
Philip Durbin 🚀 (Aug 05 2026 at 20:50):Great! Sure!
Since this is under https://guides.dataverse.org/en/6.11/container/running/demo.html (demo stuff rather than dev stuff ), I should have used "data/solr", obviously.
If you feel like making a PR, please go ahead! But for now, please keep it as "docker" instead of "podman" if you do. :smile:
Philip Durbin 🚀 (Aug 06 2026 at 17:42):@Renee L. as discussed we created this issue today:
In container guide, solr updates, change docker-dev-volumes to data #12588
Are you interested in making a PR? If not, I can or add "good first issue" label on it.
Philip Durbin 🚀 (Aug 06 2026 at 17:52):Nevermind! It's much quicker for me to fix.
I just made PR #12589 and I'd appreciate a review!
Last updated: Aug 18 2026 at 08:27 UTC