Hi, I'm trying to create an external table in sql 2019 for Cloudera HDFS by using the following code:
-- IDENTITY: the Kerberos user name.
-- SECRET: the Kerberos password
CREATE DATABASE SCOPED CREDENTIAL HadoopUser1
WITH IDENTITY = 'NEAD\tsmdata', Secret = 'SfGgt/ovk83/';
-- LOCATION (Required) : Hadoop Name Node IP address and port.
-- RESOURCE MANAGER LOCATION (Optional): Hadoop Resource Manager location to enable pushdown computation.
-- CREDENTIAL (Optional): the database scoped credential, created above.
CREATE EXTERNAL DATA SOURCE MyHadoopCluster WITH (
TYPE = HADOOP,
-- LOCATION ='hdfs://nameservice1:8020',
LOCATION ='hdfs://ledcb789.frmon.danet:8020',
RESOURCE_MANAGER_LOCATION = 'ledcb788.frmon.danet:8090',
CREDENTIAL = HadoopUser1
);
-- FORMAT TYPE: Type of format in Hadoop (DELIMITEDTEXT, RCFILE, ORC, PARQUET).
CREATE EXTERNAL FILE FORMAT TextFileFormat WITH (
FORMAT_TYPE = DELIMITEDTEXT,
FORMAT_OPTIONS (FIELD_TERMINATOR ='|',
--FIRST_ROW = 2,
USE_TYPE_DEFAULT = TRUE))
-- LOCATION: path to file or directory that contains the data (relative to HDFS root).
CREATE EXTERNAL TABLE [Default].s_lbi_pos (
sku_no varchar(8000),
sku varchar(8000),
brand varchar(8000),
stage varchar(8000),
weight_per_tin float,
barcode float)
WITH (LOCATION='/user/hive/warehouse/s_lbi_pos',
DATA_SOURCE = MyHadoopCluster,
FILE_FORMAT = TextFileFormat
);
This fails with the following error message:
Msg 105036, Level 16, State 1, Line 44Impossible de récupérer les informations relatives à la source de données externe en raison d'une erreur interne Error occurred while accessing HDFS: Une exception de type 'JNI.JavaExceptionCheckException' a été levée. Java exception message:
Call to ledcb788.frmon.danet/10.254.69.141:8090 failed on local exception: java.io.EOFException.
Guillaume Kieffer