mv_jobflow macro test for task orchestration

The script begins by defining a temporary SAS^© program (`testprog`) that generates random data using macro variables (`macrovar1`, `macrovar2`). This program is then used to create two SAS^© jobs (`demo1`, `demo2`) via the `%mv_createjob` macro, simulating independent tasks. A control dataset (`work.inputjobs`) is then constructed. It contains the metadata required by `mv_jobflow` to execute the jobs, including the execution context, flow ID, and specific macro variable values for each job. The `%mv_jobflow` macro is called with parameters to manage concurrency (`maxconcurrency=2`), redirect results to `work.results`, and log detailed information to `myjoblog`. The `raise_err=1` option ensures error propagation. The script then checks the execution log content and uses the `%mp_assert` macro to validate that the system return code (`syscc`) is 0, ensuring that the entire workflow completed without error.

Data Analysis

Type : CREATION_INTERNE

All data used in this script is internally generated or dynamically created. The `testprog` program is written to a temporary file. The `work.inputjobs` dataset is created via a DATA step within the script. The outputs (`work.results`, `myjoblog`) are also products of the script's internal execution.

1 Code Block

DATA STEP Data

Explanation :
This block creates a temporary SAS file named `testprog` and writes a SAS program into it. This internal program is a simple DATA step that generates random numbers and observations based on macro variables (`&macrovar1`, `&macrovar2`) and context information (`&_program`, `&flow_id`). The `testprog` file will serve as the code source for SAS jobs defined later, allowing for parameterizable and reusable execution.

Copied!

1	filename testprog temp;
2	DATA _null_;
3	file testprog;
4	put '%put this is job: &_program;'
5	/ '%put this was run in flow &flow_id;'
6	/ 'data ;'
7	/ ' rval=rand("uniform");'
8	/ ' rand=rval*&macrovar1;'
9	/ ' do x=1 to rand;'
10	/ ' y=rand*&macrovar2;'
11	/ ' output;'
12	/ ' end;'
13	/ 'run;'
14	;
15	RUN;

2 Code Block

Macro mv_createjob

Explanation :
These two calls to the `%mv_createjob` macro register the SAS programs in `testprog` as two distinct executable jobs, named `demo1` and `demo2`, in the `/Public/temp` path. This step is crucial for preparing the tasks that will subsequently be orchestrated by the `mv_jobflow` macro. These jobs are now ready to be referenced and executed within a workflow.

Copied!

1	%mv_createjob(path=/Public/temp,name=demo1,code=testprog)
2	%mv_createjob(path=/Public/temp,name=demo2,code=testprog)

3 Code Block

DATA STEP Data

Explanation :
This DATA step constructs the `work.inputjobs` dataset, which is the input source for the `%mv_jobflow` macro. It defines an execution context ('SAS Job Execution compute context') and specifies details for multiple executions of `demo1` and `demo2` jobs within two flows (`flow_id` 1 and 2). For each job, different values for `macrovar1` and `macrovar2` are set, thus parameterizing the individual job executions in the flow.

Copied!

1	DATA work.inputjobs;
2	_contextName='SAS Job Execution compute context';
3	DO flow_id=1 to 2;
4	DO i=1 to 4;
5	_program='/Public/temp/demo1';
6	macrovar1=10*i;
7	macrovar2=4*i;
8	OUTPUT;
9	i+1;
10	_program='/Public/temp/demo2';
11	macrovar1=40*i;
12	macrovar2=44*i;
13	OUTPUT;
14	END;
15	END;
16	RUN;

4 Code Block

Macro mv_jobflow

Explanation :
This block is the main execution step of the workflow. The `%mv_jobflow` macro is called with `work.inputjobs` as the source of task definitions. It is configured to execute jobs with a maximum of 2 concurrent jobs (`maxconcurrency=2`), store results in the `work.results` dataset, and generate a detailed log file named `myjoblog`. The `raise_err=1` option is enabled so that any job error is reported by the system return code. The `%put` messages display the value of `syscc` before and after the flow execution.

Copied!

1	* Trigger the flow ;
2
3	%put NOTE: &=syscc;
4
5	%mv_jobflow(inds=work.inputjobs
6	,maxconcurrency=2
7	,outds=work.results
8	,outref=myjoblog
9	,raise_err=1
10	,mdebug=1
11	)
12
13	%put NOTE: &=syscc;

5 Code Block

DATA STEP

Explanation :
This DATA step reads the content of the `myjoblog` file, which is generated by the `%mv_jobflow` macro and contains the details of the job flow execution (status, errors, etc.). It then displays each line of this file directly in the SAS log. This allows for a complete post-execution inspection of the workflow's progress and results.

Copied!

1	DATA _null_;
2	INFILE myjoblog;
3	INPUT; put _infile_;
4	RUN;

6 Code Block

Macro mp_assert

Explanation :
This block uses the `%mp_assert` macro to perform a post-execution check. It evaluates the condition `&syscc eq 0`, meaning the system return code should be zero, indicating that the entire job flow executed without error. The `desc` parameter provides a textual description of the assertion, useful for debugging and test reporting.

Copied!

1	%mp_assert(
2	iftrue=(&syscc eq 0),
3	desc=Check that a zero return code is returned IF no called job fails
4	)

This material is provided "as is" by We Are Cas. There are no warranties, expressed or implied, as to merchantability or fitness for a particular purpose regarding the materials or code contained herein. We Are Cas is not responsible for errors in this material as it now exists or will exist, nor does We Are Cas provide technical support for it.

Retour à la liste