The "Many-to-Many" Trap (MERGE vs. SQL)

A common belief is that the MERGE statement in a DATA step is the exact equivalent of a SQL join (LEFT JOIN or FULL JOIN). While this is true for simple relationships (1-to-1 or 1-to-N), it is completely false for "Many-to-Many" (N-to-N) relationships.

The Observation: Divergent Results

Let's take a simple example where the join key (ID) appears multiple times in both tables:

Table 1: ID 23456 appears 2 times.
Table 2: ID 23456 appears 2 times.

If we try to combine this data, we mathematically expect to get $2 \times 2 = 4$ rows (the Cartesian product for this ID).

Note :
The PROC SQL Approach (Cartesian Product)
The SQL works on set-based logic (relational algebra). It combines each row from table A with every corresponding row from table B.

1	PROC SQL;
2	SELECT * FROM dataset1 t1
3	LEFT JOIN dataset2 t2 ON t1.ID = t2.ID;
4	QUIT;

Result: 4 rows for ID 23456. All possible combinations are created. This is the standard behavior of a relational database.

Note :
The DATA MERGE Approach (Sequential Juxtaposition)
The DATA step operates row by row sequentially. It places the read pointers "side by side".

1	DATA temp;
2	MERGE dataset1 dataset2;
3	BY ID;
4	RUN;

Only 2 rows for ID 23456.

What happened? SAS^© read the 1st occurrence of the ID in Table 1 and matched it with the 1st occurrence in Table 2. Then, it read the 2nd occurrence from Table 1 and matched it with the 2nd from Table 2. Since there is no more data for this ID in either table, it moves on to the next ID.

If Table 1 had 2 rows and Table 2 had 3 rows for the same ID, SAS^© would have produced 3 rows, with the last row of Table 1 being "retained" (values held) to fill the gap opposite the 3rd row of Table 2.

Conclusion: Which One to Choose?

Use PROC SQL if you need a Cartesian product (N-to-N relationship), meaning you want to cross every possible occurrence. This is often the expected result for cross-sectional analyses.
Use DATA MERGE for 1-to-1 or 1-to-N relationships (like a "Look-up" / Data Enrichment). It is much more efficient in terms of computation time on large volumes, as long as the keys are unique in at least one of the two tables.

Technical Note: It is technically possible to simulate a Cartesian product with a DATA step (using two SET statements and explicit loops), but the code becomes unnecessarily complex. In this specific case, the clarity of SQL is unbeatable.

Avertissement important

Les codes et exemples fournis sur WeAreCAS.eu sont à but pédagogique. Il est impératif de ne pas les copier-coller aveuglément sur vos environnements de production. La meilleure approche consiste à comprendre la logique avant de l'appliquer. Nous vous recommandons vivement de tester ces scripts dans un environnement de test (Sandbox/Dev). WeAreCAS décline toute responsabilité quant aux éventuels impacts ou pertes de données sur vos systèmes.

Retour à la liste des articles