Stream: troubleshooting

Topic: Question about the search (API)


view this post on Zulip Laura Huis in 't Veld (Apr 02 2025 at 14:09):

Hello all,

Does anyone know if there is a way to exclude some results when doing a search query?
I got this example from a user:

https://archaeology.datastations.nl/dataverse/root?q=&fq0=dansSpatialCoverageText_ss%3A%22West+Betuwe%22&types=dataverses%3Adatasets%3Afiles&sort=dateSort&order=desc

Where we would like to exclude all datasets that have 'Portable Antiquities of the Netherlands' as author name. Is this somehow possible? I don't think the boolean 'NOT' will work here?

Laura

view this post on Zulip Philip Durbin πŸš€ (Apr 02 2025 at 14:12):

Please try it with double quotes rather than single quotes, like this:

NOT "Portable Antiquities of the Netherlands"

view this post on Zulip Philip Durbin πŸš€ (Apr 02 2025 at 14:15):

This is known as a phrase: https://solr.apache.org/guide/solr/9_7/query-guide/standard-query-parser.html#specifying-terms-for-the-standard-query-parser

view this post on Zulip Laura Huis in 't Veld (Apr 02 2025 at 14:39):

Thanks for the hint Phil.
I am not an expert at all with search queries.
I tried this in the Advanced Search UI:
Screenshot 2025-04-02 at 16.34.48.png
Screenshot 2025-04-02 at 16.35.08.png

This results in https://archaeology.datastations.nl/dataverse/root/?q=%28authorName%3A%22Portable+Antiquities+of+the+Netherlands%22+AND+dansSpatialCoverageText%3AWest%2BBetuwe%29

It ignores the 'NOT'. Probably a dumb question, but where do I place the 'NOT' in my query?

view this post on Zulip Philip Durbin πŸš€ (Apr 02 2025 at 14:41):

Hmm, I see you have spatial coverage in there as well. Let's focus on one field first. Walk before we run. :big_smile:

view this post on Zulip Philip Durbin πŸš€ (Apr 02 2025 at 14:42):

You're right that the Advanced Search page eats the NOT. :doh:

view this post on Zulip Philip Durbin πŸš€ (Apr 02 2025 at 14:42):

What if you type this directly in the search box?

NOT authorName:"Portable Antiquities of the Netherlands"

view this post on Zulip Laura Huis in 't Veld (Apr 02 2025 at 14:45):

Yes, that is working.

view this post on Zulip Laura Huis in 't Veld (Apr 02 2025 at 14:46):

But yeah, the user would like to filter on another metadata field as well.
We have a custom field called 'dansSpatialCoverageText'.

view this post on Zulip Philip Durbin πŸš€ (Apr 02 2025 at 14:48):

When you say it isn't working, can you please be more specific?

The default sort is "relevance" and if I flip it to "newest" I see datasets that don't seem to have that search phrase. So it feels like it's working. :thinking:

view this post on Zulip Laura Huis in 't Veld (Apr 02 2025 at 14:49):

Ah, I tried this in the search box, and I think this gives the result I needed:
NOT authorName:"Portable Antiquities of the Netherlands" AND dansSpatialCoverageText:"West Betuwe"

view this post on Zulip Philip Durbin πŸš€ (Apr 02 2025 at 14:49):

:tada:

view this post on Zulip Laura Huis in 't Veld (Apr 02 2025 at 14:58):

Thanks for pointing me in the right direction!

view this post on Zulip Philip Durbin πŸš€ (Apr 02 2025 at 15:00):

Sure. And please feel free to create an issue about the Advanced Search page gobbling up that NOT. :hamburger:

view this post on Zulip Philip Durbin πŸš€ (Apr 02 2025 at 15:02):

It looks like the new front end doesn't have an Advanced Search page yet: https://beta.dataverse.org/spa/

view this post on Zulip Laura Huis in 't Veld (Apr 03 2025 at 08:04):

Ok Round 2 :)

I am trying to recreate this search with the search API:
https://archaeology.datastations.nl/dataverse/root?q=&fq0=dansSpatialCoverageText_ss%3A%22West+Betuwe%22&fq1=authorName_ss%3A%22Portable+Antiquities+of+the+Netherlands%22&types=datasets&page=1&sort=dateSort&order=desc
This link gives us 2403 results.

I have tried this with the search API: https://archaeology.datastations.nl/api/search?&sort=dateSort&order=desc&q=fq0=dansSpatialCoverageText_ss:%22West%20Betuwe%22&fq1=authorName_ss:%22Portable+Antiquities+of+the+Netherlands%22&type=dataset
But it seems to ignore the fq1=authorName_ss:%22Portable+Antiquities+of+the+Netherlands%22 part.

What am I missing here?

view this post on Zulip Laura Huis in 't Veld (Apr 03 2025 at 08:06):

BTW: I was puzzled first by the fact that to filter in the API on datasets only, you have to use type=dataset, instead of 'types=datasets'. As you can see in the examples above.

view this post on Zulip Philip Durbin πŸš€ (Apr 03 2025 at 14:16):

Here you go, 2403: https://archaeology.datastations.nl/api/search?&sort=dateSort&order=desc&q=*&fq=dansSpatialCoverageText_ss:%22West%20Betuwe%22&fq=authorName_ss:%22Portable+Antiquities+of+the+Netherlands%22&type=dataset

view this post on Zulip Philip Durbin πŸš€ (Apr 03 2025 at 14:18):

I changed q= to q=*& an fq0 and fq1 to just fq.

view this post on Zulip Philip Durbin πŸš€ (Apr 03 2025 at 14:18):

The & is just to separate query parameters.

view this post on Zulip Laura Huis in 't Veld (Apr 03 2025 at 14:52):

Thanks for your magic! So the fq does not need numbering at all..
I see now that I got confused comparing the query from the UI with the API query.

view this post on Zulip Philip Durbin πŸš€ (Apr 03 2025 at 14:55):

Interestingly, the SPA does it a third, different way with fqs: https://beta.dataverse.org/spa/collections?fqs=publicationDate%3A2024%2CauthorName_ss%3ADurbin%252C%2520Philip :shrugdog:

I'm not saying it's wrong but I didn't realize until just now it's different than how the API works. It might be nice to line up with with the API so people can more easily copy and paste between the API and SPA. /cc @Ellen K @Cheng Shi @Guillermo Portas @GermΓ‘n Saracca


Last updated: Oct 30 2025 at 06:21 UTC