Stream: community

Topic: delete files using sql


view this post on Zulip Péter Pallinger (Nov 14 2024 at 15:59):

I have a dataset with 460k uploaded files, and now I want to delete it. The problem is, I cannot. The web interface times out, and the API call returns with error after about 1hour. Trying to delete files first also fails: The web interface times out and the API call return an 505 error after about 1 hour.

So I got the wonderful idea that I would delete the files' metadata from the database using SQL, and then delete the files themselves from the S3 bucket.

What tables contain the metadata? So far, I identified the following:
dvobject, datafile, datafiletag, auxiliaryfile, filemetadata, filemetadata_datafilecategory

Are there any more that I should consider?

view this post on Zulip Philip Durbin 🚀 (Nov 14 2024 at 16:28):

Just to be clear, you're ok with deleting (destroying) the entire dataset?

view this post on Zulip Péter Pallinger (Nov 14 2024 at 16:51):

yes!

view this post on Zulip Péter Pallinger (Nov 14 2024 at 16:52):

It is in draft now (never published), so no depublication shenanigans are needed...
As I wrote above, I tried to delete the whole dataset through dataverse, but failed...


Last updated: Nov 01 2025 at 14:11 UTC