Bethany Seeger (Jan 06 2026 at 21:44): Bethany Seeger (Jan 06 2026 at 21:44):Hello,
We are using Dataverse v 6.6 and trying to do a few uploads, one via automated code.
The website is responding normally, and I can browse material, but there are a bunch of files that are still "ingesting" (converting, I think) and I'm not sure how to tell if a part of Dataverse failed.
There are no obvious errors in the log, though I haven't seen the message "Tabular data successfully ingested" in a while - at the time when things seemed to stop getting converted.
What should I check out to see if the system is running as expected and maybe just a little slow? I suspect the files waiting to be convereted are just sitting there.
Thanks!
Bethany Seeger (Jan 06 2026 at 21:45):One error -- probably doesn't stop other file conversions:
[2026-01-06T18:36:54.089+0000] [Payara 6.2023.8] [WARNING] [] [edu.harvard.iq.dataverse.ingest.IngestServiceBean] [tid: _ThreadID=534 _ThreadName=orb-thread-pool-1 (pool #1): worker-4] [timeMillis: 1767724614089] [levelValue: 900] [[
Ingest failure (Exception class java.lang.IllegalArgumentException): The header contains a duplicate name: "Position" in [<feff>Clone, Sequence, Gene_Name, UniProt, Entrez, Exact_HPA_Match?_(Gene,_UniProt,_or_Synonym), HPA_Gene, HPA_Uniprot, HPA_Gene_Synonym, HPA_Ensembl, HPA_Gene_description, Position, Ch ..... ]
[2026-01-06T18:36:54.096+0000] [Payara 6.2023.8] [WARNING] [] [edu.harvard.iq.dataverse.ingest.IngestMessageBean] [tid: _ThreadID=534 _ThreadName=orb-thread-pool-1 (pool #1): worker-4] [timeMillis: 1767724614096] [levelValue: 900] [[
Error occurred during ingest job for file id 27292!]]
BUT -- Could this failure have stopped whatever thread does these conversions?
Bethany Seeger (Jan 06 2026 at 21:47):A few of these:
[2026-01-06T19:06:21.941+0000] [Payara 6.2023.8] [WARNING] [] [] [tid: _ThreadID=106 _ThreadName=http-thread-pool::jk-connector(2)] [timeMillis: 1767726381941] [levelValue: 900] [[
Response has already been committed, and further write operations are not permitted. This may result in an IllegalStateException being triggered by the underlying application. To avoid this situation, consider adding a Rule .when(Direction.isInbound().and(Response.isCommitted())).perform(Lifecycle.abort()), or figure out where the response is being incorrectly committed and correct the bug in the offending code.]]
Bethany Seeger (Jan 06 2026 at 21:49):A few of these:
java.io.IOException: Connection is closed
at org.glassfish.grizzly.nio.NIOConnection.assertOpen(NIOConnection.java:420)
at org.glassfish.grizzly.http.io.OutputBuffer.write(OutputBuffer.java:613)
at org.apache.catalina.connector.OutputBuffer.writeBytes(OutputBuffer.java:340)
at org.apache.catalina.connector.OutputBuffer.write(OutputBuffer.java:327)
at org.apache.catalina.connector.CoyoteOutputStream.write(CoyoteOutputStream.java:158)
at java.base/java.nio.channels.Channels$WritableByteChannelImpl.write(Channels.java:463)
at com.sun.faces.application.resource.ResourceHandlerImpl.handleResourceRequest(ResourceHandlerImpl.java:305)
at jakarta.faces.application.ResourceHandlerWrapper.handleResourceRequest(ResourceHandlerWrapper.java:166)
at org.primefaces.application.resource.PrimeResourceHandler.handleResourceRequest(PrimeResourceHandler.java:87)
at jakarta.faces.application.ResourceHandlerWrapper.handleResourceRequest(ResourceHandlerWrapper.java:166)
at jakarta.faces.application.ResourceHandlerWrapper.handleResourceRequest(ResourceHandlerWrapper.java:166)
at jakarta.faces.application.ResourceHandlerWrapper.handleResourceRequest(ResourceHandlerWrapper.java:166)
at jakarta.faces.application.ResourceHandlerWrapper.handleResourceRequest(ResourceHandlerWrapper.java:166)
at jakarta.faces.webapp.FacesServlet.executeLifecyle(FacesServlet.java:688)
at jakarta.faces.webapp.FacesServlet.service(FacesServlet.java:449)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1554)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:331)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:211)
at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:83)
....
Caused by: java.io.IOException: Connection reset by peer
at java.base/sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at java.base/sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:62)
at java.base/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:132)
at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:76)
at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:53)
at java.base/sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:532)
at org.glassfish.grizzly.nio.transport.TCPNIOUtils.flushByteBuffer(TCPNIOUtils.java:118)
at org.glassfish.grizzly.nio.transport.TCPNIOUtils.writeCompositeBuffer(TCPNIOUtils.java:63)
at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:96)
at org.glassfish.grizzly.nio.transport.TCPNIOAsyncQueueWriter.write0(TCPNIOAsyncQueueWriter.java:77)
at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:211)
at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:137)
at org.glassfish.grizzly.nio.AbstractNIOAsyncQueueWriter.write(AbstractNIOAsyncQueueWriter.java:47)
at org.glassfish.grizzly.nio.transport.TCPNIOTransportFilter.handleWrite(TCPNIOTransportFilter.java:106)
at org.glassfish.grizzly.filterchain.TransportFilter.handleWrite(TransportFilter.java:148)
at org.glassfish.grizzly.filterchain.ExecutorResolver$8.execute(ExecutorResolver.java:81)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:246)
[2026-01-06T20:13:10.947+0000] [Payara 6.2023.8] [WARNING] [faces.application.resource.unable_to_serve] [jakarta.enterprise.resource.webcontainer.faces.application] [tid: _ThreadID=105 _ThreadName=http-thread-pool::jk-connector(1)] [timeMillis: 1767730390947] [levelValue: 900] [[
JSF1064: Unable to find or serve resource, bs/css/bootstrap.min.css.]]
Not sure if that's just a client dropping the connection.
Bethany Seeger (Jan 07 2026 at 14:12):I did restart java and it looks like files are being converted. Perhaps it was just stuck on a large one? If that's the case, would putting more memory on the system help this? The CPU's didn't seem taxed, but the 32 GB of memory was being fully used.
Philip Durbin 🚀 (Jan 07 2026 at 14:22):Interesting. Do you mean you restarted Payara?
Bethany Seeger (Jan 07 2026 at 14:33):Hi Phil - yes, I restarted Payara to see if that would help
Bethany Seeger (Jan 07 2026 at 14:35):We are doing a migration of material into the system, so it's a little under load though the migration waits for the conversions to happen, but it's really the conversions that appear to be taking most of the time (which does make sense).
Philip Durbin 🚀 (Jan 07 2026 at 14:37):Please remind me, have we talked about these docs before? https://guides.dataverse.org/en/6.9/admin/troubleshooting.html#long-running-ingest-jobs-have-exhausted-system-resources
Bethany Seeger (Jan 07 2026 at 14:38):Oh, I don't think we have. I will read through that. Thank you for sharing that!
Philip Durbin 🚀 (Jan 07 2026 at 14:41):Sure, there's also #8954 where @Victoria Lubitch and @Christian Bischof mention slow ingest of SPSS and Stata files.
Also, @Philipp Conzett and @Amber Leahey discussed similar issues at https://groups.google.com/g/dataverse-community/c/fVoClEGg4oU/m/kgYSGiQtCwAJ
Bethany Seeger (Jan 07 2026 at 14:53):This is super helpful. Thank you.
Philip Durbin 🚀 (Jan 07 2026 at 14:53):Sure. At least you know you aren't alone! :sweat_smile:
Philip Durbin 🚀 (Jan 07 2026 at 14:55):As you may know, you can set up rules like "SPSS files over 500 MB should not be ingested".
Bethany Seeger (Jan 07 2026 at 14:55):Yes, that is good to know that. :) Also good to know that sometimes they just take a while and that the server is fine. I was worried that something was going on inside of Java that failed, but not bad enough for the server to stop performing.
Philip Durbin 🚀 (Jan 07 2026 at 14:55):You could even turn off ingest entirely during your migration. And then selectively ingest files afterward.
Bethany Seeger (Jan 07 2026 at 14:56):Do you have a suggestion for a limit, or would that be 500MB? How much does 500MB explode out to in memory during a conversion? I'm not sure how intense the conversion process is.
Philip Durbin 🚀 (Jan 07 2026 at 14:57):I don't, sorry. I made that number up.
Bethany Seeger (Jan 07 2026 at 14:58):I can imagine it depends on what kind of server resources we have.
Philip Durbin 🚀 (Jan 07 2026 at 14:59):These are the limits we have set up for Harvard Dataverse:
Screenshot 2026-01-07 at 9.58.52 AM.png
Bethany Seeger (Jan 07 2026 at 15:05):Does that mean that a tabular file will still be uploaded, but not converted, if greater than 150 MB?
Philip Durbin 🚀 (Jan 07 2026 at 15:07):Yes, exactly. And we can always convert it (ingest it) later via API: https://guides.dataverse.org/en/6.9/api/native-api.html#reingest-a-file
Bethany Seeger (Jan 07 2026 at 15:11):Ahh, okay. I'm begining to understand that "ingest" means "upload and process" for tabular files, when enabled and of the proper size. I was thinking "ingest" meant the upload part alone and then thought of the conversion separately. (mostly getting stuck in terminology here. :) This is very helpful - thank you.
Philip Durbin 🚀 (Jan 07 2026 at 15:13):Yeah, we ask questions like "was the file successfully ingested?" or "was there a failure during the ingest process?". Ingest is only for certain file types like CSV, Excel, Stata, etc. Please see https://guides.dataverse.org/en/6.9/user/tabulardataingest/index.html
Last updated: Jan 09 2026 at 14:18 UTC