1. Home
  2. Support
Article - CS260425

Error "java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set." seen in grid-worker.log in ThingWorx Analytics

Modified: 12-Apr-2017   


Applies To

  • ThingWorx Analytics 52.2

Description

  • Following error is reported when executing a dataset optimize job (DATASET_SNAPSHOT)
[main-EventThread] [org.apache.hadoop.util.Shell] -[Failed to detect a valid hadoop home directory]
java.io.IOException: HADOOP_HOME or hadoop.home.dir are not set.
at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:265)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:290)
at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:76)
at org.apache.hadoop.security.Groups.parseStaticMapping(Groups.java:93)
at org.apache.hadoop.security.Groups.<init>(Groups.java:77)
at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:240)
at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:255)
at org.apache.hadoop.security.UserGroupInformation.ensureInitialized(UserGroupInformation.java:232)
at org.apache.hadoop.security.UserGroupInformation.loginUserFromSubject(UserGroupInformation.java:718)
at org.apache.hadoop.security.UserGroupInformation.getLoginUser(UserGroupInformation.java:703)
at org.apache.hadoop.security.UserGroupInformation.getCurrentUser(UserGroupInformation.java:605)
at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2162)
at org.apache.spark.util.Utils$$anonfun$getCurrentUserName$1.apply(Utils.scala:2162)
at scala.Option.getOrElse(Option.scala:120)
at org.apache.spark.util.Utils$.getCurrentUserName(Utils.scala:2162)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:301)
at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:61)
at com.coldlight.neuron.data.spark.SparkSupplier.sparkctx$lzycompute(SparkSupplier.scala:41)
at com.coldlight.neuron.data.spark.SparkSupplier.sparkctx(SparkSupplier.scala:37)
at com.coldlight.neuron.data.spark.Spark.sc(SparkSupplier.scala:30)
at com.coldlight.neuron.data.PartitionedData.<init>(PartitionedData.scala:28)
at com.coldlight.neuron.data.dataset.DataSet.<init>(DataSet.java:25)
at com.coldlight.neuron.db.impl.dataset.jdbc.JdbcDataSet.<init>(JdbcDataSet.java:139)
at com.coldlight.neuron.db.impl.dataset.jdbc.JdbcDataSet.<init>(JdbcDataSet.java:58)
at com.coldlight.neuron.db.impl.dataset.jdbc.JdbcDataSet$Factory.createDataSet(JdbcDataSet.java:347)
at com.coldlight.neuron.db.impl.dataset.jdbc.JdbcDataSet$Factory.create(JdbcDataSet.java:298)
at com.coldlight.neuron.data.dataset.DataSetBuilder.build(DataSetBuilder.java:75)
at com.coldlight.neuron.job.UriDataSetFactory.createDataSet(UriDataSetFactory.java:69)
at com.coldlight.neuron.job.UriDataSetFactory.createDataSetWithKnownFeatures(UriDataSetFactory.java:123)
at com.coldlight.neuron.job.NeuronJob.inflateDataSet(NeuronJob.java:202)
at com.coldlight.neuron.services.ai.DataSetSnapshotJob.runJob(DataSetSnapshotJob.java:44)
at com.coldlight.neuron.job.NeuronJob.run(NeuronJob.java:123)
at com.coldlight.ccc.job.dempsy.DempsyClusterJobExecutor$DempsyPersistentClusterWatcher.runJobUploadResultsAndCleanup(DempsyClusterJobExecutor.java:334)
at com.coldlight.ccc.job.dempsy.DempsyClusterJobExecutor$DempsyPersistentClusterWatcher.execute(DempsyClusterJobExecutor.java:482)
at com.coldlight.ccc.executor.PersistentTask.executeUntilWorks(PersistentTask.java:92)
at com.coldlight.ccc.executor.PersistentTask.process(PersistentTask.java:58)
at com.nokia.dempsy.cluster.zookeeper.ZookeeperSession$WatcherProxy.process(ZookeeperSession.java:279)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
This is a printer-friendly version of Article 260425 and may be out of date. For the latest version click CS260425