PROC DATAMETRICS: Advanced Data Quality Analysis

The DATAMETRICS procedure is a powerful tool for assessing data quality. This detailed functional analysis shows how to configure the procedure with a full range of parameters. It covers extracting statistical metrics like the median, determining the most frequent values (frequencies), identifying extreme values (minmax), and using data formats for more relevant analysis. Special attention is given to the integration of the Quality Knowledge Base (QKB) for advanced identity analysis, as well as performance optimization via the 'threads' parameter. The last example demonstrates an execution of the procedure on a table loaded into memory in the CAS-SAS^© Viya^™ environment.

Data Analysis

Type : CREATION_INTERNE

The examples use internally generated data via DATA-Steps with DATALINES, ensuring their autonomy and reproducibility.

1 Code Block

DATA / PROC DATAMETRICS Data

Explanation :
This example initializes a 'my_data' dataset with fictional information (name, address, city, state). Then, the PROC DATAMETRICS procedure is executed with the minimally required parameters to calculate data quality metrics for the 'name' and 'address' variables. The results are saved in the 'basic_metrics' table.

Copied!

1	DATA work.my_data;
2	LENGTH name $30 address $50 city $20 state $2;
3	INPUT name $ address $ city $ state $;
4	DATALINES;
5	"John Doe" "123 Main St" "Anytown" "NY"
6	"Jane Smith" "456 Oak Ave" "Anycity" "CA"
7	"John Doe" "123 Main St" "Anytown" "NY"
8	"Peter Jones" "789 Pine Ln" "Otherville" "TX"
9	"Alice Brown" "101 Maple Dr" "Anytown" "NY"
10	"Bob White" "202 Elm St" "Otherville" "TX"
11	"Charlie Green" "303 Cedar Rd" "Anycity" "CA"
12	"David Black" "404 Birch Ct" "Anytown" "NY"
13	;
14	RUN;
15
16	PROC DATAMETRICS DATA=work.my_data out=work.basic_metrics;
17	variables name address;
18	RUN;
19
20	PROC PRINT DATA=work.basic_metrics;
21	title "Grundlegende Metriken für Name und Adresse";
22	RUN;

2 Code Block

PROC DATAMETRICS

Explanation :
Based on the data from the previous example, this example uses common options: 'frequencies=10' for the 10 most frequent values, 'minmax=5' for 5 minimum and maximum values, and 'median' to calculate the median. The 'identities' statement is used to integrate a specific Quality Knowledge Base (QKB) for the 'ENUSA' locale and the 'Field Content' definition to enhance identity analysis.

Copied!

1	/* Assurez-vous que work.my_data est déjà créé à partir de l'Exemple 1 */
2
3	PROC DATAMETRICS DATA=work.my_data out=work.common_metrics frequencies=10
4	minmax=5 median;
5	identities qkb='/sas/dqc/QKBLoc' locale='ENUSA' def='Field Content';
6	variables name address city;
7	RUN;
8
9	PROC PRINT DATA=work.common_metrics;
10	title "Metriken mit Häufigkeiten, Min/Max, Median und QKB";
11	RUN;

3 Code Block

PROC FORMAT / DATA / PROC DATAMETRICS

Explanation :
This example introduces a custom format for the 'state' variable and then applies this format to a new 'formatted_data' dataset. PROC DATAMETRICS is subsequently executed on this formatted table. The 'frequencies', 'minmax', and 'threads=4' options are used for parallel processing. The 'multiidentity' option in the 'identities' statement allows for the analysis of multiple data quality identities for the specified variables.

Copied!

1	/* Assurez-vous que work.my_data est déjà créé à partir de l'Exemple 1 */
2
3	PROC FORMAT;
4	value $statefmt
5	'NY'='New York'
6	'CA'='California'
7	'TX'='Texas'
8	other='Autre';
9	RUN;
10
11	DATA work.formatted_data;
12	SET work.my_data;
13	FORMAT state $statefmt.;
14	RUN;
15
16	PROC DATAMETRICS DATA=work.formatted_data out=work.advanced_metrics
17	frequencies=20 minmax=10 threads=4 FORMAT;
18	identities qkb='/sas/dqc/QKBLoc' locale='ENUSA'
19	def='Field Content' multiidentity;
20	variables name address city state;
21	RUN;
22
23	PROC PRINT DATA=work.advanced_metrics;
24	title "Erweiterte Metriken mit Formaten, Threads und Multi-Identitäten";
25	RUN;

4 Code Block

CASLIB / PROC CASUTIL / PROC DATAMETRICS

Explanation :
This example demonstrates integration with the SAS Viya Cloud Analytic Services (CAS) environment. The 'my_data' dataset is first loaded into a CAS library ('casuser.my_cas_data') using PROC CASUTIL. Subsequently, PROC DATAMETRICS is executed directly on the in-memory CAS table. Options like 'frequencies', 'minmax', and 'threads' are applied to optimize metric analysis in a distributed environment. The results are also stored in a CAS table.

Copied!

1	/* Assurez-vous que work.my_data est déjà créé à partir de l'Exemple 1 */
2
3	caslib _all_ assign;
4
5	PROC CASUTIL;
6	load DATA=work.my_data outcaslib='casuser' casout='my_cas_data' replace;
7	RUN;
8
9	PROC DATAMETRICS DATA=casuser.my_cas_data out=casuser.cas_metrics
10	frequencies=5 minmax=3 threads=2;
11	identities qkb='/sas/dqc/QKBLoc' locale='ENUSA' def='Field Content';
12	variables name address city;
13	RUN;
14
15	PROC PRINT DATA=casuser.cas_metrics;
16	title "Metriken über DATAMETRICS auf CAS";
17	RUN;

This material is provided "as is" by We Are Cas. There are no warranties, expressed or implied, as to merchantability or fitness for a particular purpose regarding the materials or code contained herein. We Are Cas is not responsible for errors in this material as it now exists or will exist, nor does We Are Cas provide technical support for it.

Retour à la liste