Quantcast
Channel: Forum Getting started with SQL Server
Viewing all articles
Browse latest Browse all 7129

SQL 2019 CTP - Polybase External Table for Cloudera HDFS

$
0
0

Hi, I'm trying to create an external table in sql 2019 for Cloudera HDFS by using the following code: 

-- IDENTITY: the Kerberos user name.  
-- SECRET: the Kerberos password  
CREATE DATABASE SCOPED CREDENTIAL HadoopUser1
WITH IDENTITY = 'NEAD\tsmdata', Secret = 'SfGgt/ovk83/';

-- LOCATION (Required) : Hadoop Name Node IP address and port.  
-- RESOURCE MANAGER LOCATION (Optional): Hadoop Resource Manager location to enable pushdown computation.  
-- CREDENTIAL (Optional):  the database scoped credential, created above.  
CREATE EXTERNAL DATA SOURCE MyHadoopCluster WITH (  
      TYPE = HADOOP,
 --     LOCATION ='hdfs://nameservice1:8020',
	  LOCATION ='hdfs://ledcb789.frmon.danet:8020',
      RESOURCE_MANAGER_LOCATION = 'ledcb788.frmon.danet:8090',
      CREDENTIAL = HadoopUser1
);

-- FORMAT TYPE: Type of format in Hadoop (DELIMITEDTEXT,  RCFILE, ORC, PARQUET).
CREATE EXTERNAL FILE FORMAT TextFileFormat WITH (  
      FORMAT_TYPE = DELIMITEDTEXT,
      FORMAT_OPTIONS (FIELD_TERMINATOR ='|',
			--FIRST_ROW = 2, 
            USE_TYPE_DEFAULT = TRUE))

-- LOCATION: path to file or directory that contains the data (relative to HDFS root).  
CREATE EXTERNAL TABLE [Default].s_lbi_pos (
      sku_no varchar(8000),
      sku varchar(8000),
      brand varchar(8000),
      stage varchar(8000),
      weight_per_tin float,
      barcode float)
WITH (LOCATION='/user/hive/warehouse/s_lbi_pos',
      DATA_SOURCE = MyHadoopCluster,  
      FILE_FORMAT = TextFileFormat  
);

This fails with the following error message:

Msg 105036, Level 16, State 1, Line 44
Impossible de récupérer les informations relatives à la source de données externe en raison d'une erreur interne Error occurred while accessing HDFS: Une exception de type 'JNI.JavaExceptionCheckException' a été levée. Java exception message:
Call to ledcb788.frmon.danet/10.254.69.141:8090 failed on local exception: java.io.EOFException.



Guillaume Kieffer


Viewing all articles
Browse latest Browse all 7129

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>