Integrating large-scale data between SAS© Viya™ and a Hadoop cluster is a common operation, but it can sometimes run into subtle configuration problems. One of the most frequent obstacles when defining a Caslib is the occurrence of a timeout error immediately after executing the connection code.
This article details the symptoms of this error and presents a proven solution to stabilize the connection.
When executing a CASLIB statement to connect to Hadoop (often via Hive), the process stops and returns the following error messages in the log:
This issue typically occurs when using an explicit connection string that includes the uri= option, as in the example below:
| 1 | /* Code provoquant l'erreur */ |
| 2 | caslib hdlib datasource=( |
| 3 | srctype="hadoop", |
| 4 | dataTransferMode="serial", |
| 5 | username="{votre_user}", |
| 6 | password="{votre_mdp}", |
| 7 | /* L'option URI ci-dessous est souvent la cause du conflit */ |
| 8 | uri="jdbc:hive2://{host}:2181...;serviceDiscoveryMode=zooKeeper;...", |
| 9 | hadoopjarpath="/sas_mirror/hadoopfiles/lib", |
| 10 | hadoopconfigdir="/sas_mirror/hadoopfiles/conf", |
| 11 | schema="default" |
| 12 | ); |
The java.util.concurrent.TimeoutException error suggests that the SAS© client failed to establish a connection with the cluster within the allotted time. Although one might think of a network issue or cluster unavailability, the root cause often lies in over-configuration.
When you provide both:
The path to the Hadoop configuration files (hadoopconfigdir containing XML files like hive-site.xml, core-site.xml, etc.).
An explicit JDBC connection string (uri=).
A conflict or redundancy can arise in the service discovery method for Zookeeper. SAS© is capable of extracting the Zookeeper configuration directly from your XML files, making the uri option superfluous.
To resolve this issue, the recommended method is to simplify the Caslib declaration. You should remove the uri= option and let SAS© handle the connection by relying solely on the Hadoop configuration files and binaries.
Here is the corrected and functional code:
| 1 | cas mySession sessopts=(caslib=casuser timeout=1800 locale="en_US"); |
| 2 | |
| 3 | caslib hdlib datasource=( |
| 4 | srctype="hadoop", |
| 5 | dataTransferMode="serial", |
| 6 | username="{votre_user}", |
| 7 | password="{votre_mdp}", |
| 8 | /* Suppression de l'option URI */ |
| 9 | hadoopjarpath="/sas_mirror/hadoopfiles/lib", |
| 10 | hadoopconfigdir="/sas_mirror/hadoopfiles/conf", |
| 11 | schema="default" |
| 12 | ); |
| 13 | |
| 14 | PROC CASUTIL; |
| 15 | list files incaslib="hdlib"; |
| 16 | RUN; |
By not specifying a server or URI, SAS© will use the information contained in the directory pointed to by hadoopconfigdir to locate the Zookeeper cluster and establish the connection correctly.
If, after applying the fix above, you still encounter timeout errors (especially on heavily loaded clusters), it is possible that the default timeout is too short.
The default connection timeout for Hadoop is typically 30 seconds. You can increase this value by adding the login_timeout option:
| 1 | caslib hdlib datasource=( |
| 2 | ... |
| 3 | login_timeout=60 /* Augmentation du délai à 60 secondes */ |
| 4 | ... |
| 5 | ); |
Les codes et exemples fournis sur WeAreCAS.eu sont à but pédagogique. Il est impératif de ne pas les copier-coller aveuglément sur vos environnements de production. La meilleure approche consiste à comprendre la logique avant de l'appliquer. Nous vous recommandons vivement de tester ces scripts dans un environnement de test (Sandbox/Dev). WeAreCAS décline toute responsabilité quant aux éventuels impacts ou pertes de données sur vos systèmes.