I am trying to export Hive external table data using Oracle Loader for Hadoop in bigdatalite VM and encountered the following error.
java.lang.NoClassDefFoundError: org/apache/hadoop/hive/serde2/objectinspector/ObjectInspector
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.hadoop.conf.Configuration.getClassByNameOrNull(Configuration.java:2138)
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2103)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2197)
at org.apache.hadoop.mapreduce.task.JobContextImpl.getInputFormatClass(JobContextImpl.java:184)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:321)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:199)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1307)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1304)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1304)
at oracle.hadoop.loader.OraLoader.monitorAndPrintJob(OraLoader.java:881)
at oracle.hadoop.loader.OraLoader.run(OraLoader.java:594)
at oracle.hadoop.loader.OraLoader.run(OraLoader.java:425)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at oracle.hadoop.loader.OraLoader.main(OraLoader.java:1187)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 27 more
Below are the steps which I followed
Steps1: Create table in Oracle
CREATE TABLE MovieDemo.Catalog(CatalogId INTEGER PRIMARY KEY, Journal VARCHAR2(25), Publisher VARCHAR2(25), Edition VARCHAR2(25), Title VARCHAR2(45), Author VARCHAR2(25));
Step 2: Create Hive Table
CREATE EXTERNAL TABLE catalog(CATALOGID INT,JOURNAL STRING, PUBLISHER STRING, EDITION STRING,TITLE STRING,AUTHOR STRING)ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' STORED AS TEXTFILE LOCATION '/catalog';
Step 3: Upload data in Hive external table
hdfs dfs -mkdir /catalog
hdfs dfs -put /home/oracle/catalog.txt /catalog/catalog.txt
hdfs dfs -chmod -R 777 /catalog
Step 4: Create OraLoadJobConf.xml
File attached
Step 5 : Run Oracle Loader for Hadoop using the oracle.hadoop.loader.OraLoader class
hadoop jar $OLH_HOME/jlib/oraloader.jar oracle.hadoop.loader.OraLoader -conf /home/oracle/OraLoadJobConf.xml -libjars $OLH_HOME/jlib/oraloader.jar
Hive jars are already in the PATH, CLASSPATH and ODI_ADDITIOANL_PATH
echo $PATH
/usr/lib/hive/lib/:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/lib64/R/bin:/u01/Middleware/jdeveloper/jdev/bin:/usr/java/latest/bin:/u01/app/oracle/product/12.1.0.2/dbhome_1/bin:/home/oracle/scripts:/opt/bin:/u01/sqlcl/bin:/home/oracle/bin:/usr/lib64/R/bin:/u01/Middleware/jdeveloper/jdev/bin:/usr/java/latest/bin:/u01/app/oracle/product/12.1.0.2/dbhome_1/bin:/home/oracle/scripts:/opt/bin:/u01/sqlcl/bin
echo $CLASSPATH
/usr/lib/hive/lib/*:/usr/lib/hive/lib/*:/usr/lib/hive/lib/::/u01/connectors/olh/jlib/*:/usr/lib/hadoop/*:/usr/lib/hadoop/client/*:/u01/nosql/kv-ee/lib/kvstore.jar:.:/u01/connectors/olh/jlib/*:/usr/lib/hadoop/*:/usr/lib/hadoop/client/*:/u01/nosql/kv-ee/lib/kvstore.jar:
echo $ODI_ADDITIONAL_CLASSPATH
/usr/lib/hive/lib/*:/usr/lib/hadoop/client/*:/u01/connectors/osch/jlib/*:/etc/hadoop/conf