Article - CS333440
Scoring a dataset results in error "com.esotericsoftware.kryo.KryoException: Buffer overflow. Available: 0, required: 1" in ThingWorx Analytics
Modified: 15-Oct-2021
Applies To
- ThingWorx Analytics 9.0
Description
- Executing a predictive score either in Builder or using PredictionThing.BatchScore service result in error:
org.apache.spark.SparkException: Kryo serialization failed: Buffer overflow. Available: 2, required: 8
Serialization trace:
rowData (com.thingworx.analytics.ccc.data.utils.DataSetCsvInputStream$CsvRow). To avoid this, increase spark.kryoserializer.buffer.max value.
at org.apache.spark.serializer.KryoSerializerInstance.serialize(KryoSerializer.scala:350)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:393)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.esotericsoftware.kryo.KryoException: Buffer overflow. Available: 2, required: 8
Serialization trace:
rowData (com.thingworx.analytics.ccc.data.utils.DataSetCsvInputStream$CsvRow). To avoid this, increase spark.kryoserializer.buffer.max value.
at org.apache.spark.serializer.KryoSerializerInstance.serialize(KryoSerializer.scala:350)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:393)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.esotericsoftware.kryo.KryoException: Buffer overflow. Available: 2, required: 8
- This happens when scoring very large amount of data
This is a printer-friendly version of Article 333440 and may be out of date. For the latest version click CS333440