执行tez报错,观察YARN日志,Driver端报错为:File does not exist: hdfs://lt-nameservice3.sy/tmp/hive/app/_tez_session_dir/09ff9062-cc3e-4cb3-bc8d-77c275266d94/.tez/application_1528552108294_273009/tez.session.local-resources.pb java.io.FileNotFoundException
此时进入ApplicationMaster的log观察,根本报错内容如下,显然为guava包冲突导致(guava21以上会移除一些method接口不再向前兼容)。
Caused by: java.lang.NoSuchMethodError: com.google.common.base.Objects.toStringHelper(Ljava/lang/Object;)Lcom/google/common/base/Objects$ToStringHelper;
at org.apache.hadoop.metrics2.lib.MetricsRegistry.toString(MetricsRegistry.java:406)
at java.lang.String.valueOf(String.java:2994)
at java.lang.StringBuilder.append(StringBuilder.java:131)
at org.apache.hadoop.ipc.metrics.RpcMetrics.<init>(RpcMetrics.java:74)
at org.apache.hadoop.ipc.metrics.RpcMetrics.create(RpcMetrics.java:80)
at org.apache.hadoop.ipc.Server.<init>(Server.java:2213)
at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:1029)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Server.<init>(ProtobufRpcEngine.java:537)
at org.apache.hadoop.ipc.ProtobufRpcEngine.getServer(ProtobufRpcEngine.java:512)
at org.apache.hadoop.ipc.RPC$Builder.build(RPC.java:874)
at org.apache.tez.dag.api.client.DAGClientServer.createServer(DAGClientServer.java:127)
at org.apache.tez.dag.api.client.DAGClientServer.serviceStart(DAGClientServer.java:79)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1838)
at org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1859)
科普下Driver日志和ApplicationMaster日志查看位置:
如图,从YARN页面点击application id进入的页面为Driver页面,红框内容为Driver端报错信息;点击蓝框才会进入ApplicationMaster日志。
此时,在ApplicationMaster中搜索java.class.path
可以拿到当前am环境下的所有classpath,从url中获取当前am所在节点,把所有jar包拉出来,找下包版本冲突所在jar包即可。如果为自己jar包里的dependency冲突,或shade,或exclude即可解决。
相关命令如下:
jar -tf platform_udf-1.0-SNAPSHOT.jar | grep -i com.google.common
javap -classpath platform_udf-1.0-SNAPSHOT.jar com.google.common.base.Objects