How Engines Work with Files

The figure presented illustrates how files are accessed via an engine in the SAS^© Compute Server. Shaded components represent external file types or applications. SAS^© Cloud Analytic Services (CAS) is not shown in this diagram; for more information, see 'SAS^© Cloud Analytic Services: Fundamentals'.
1. A SAS^© dataset or table is stored in one or more physical files, depending on the engine and attributes. If an appropriate SAS^© engine is configured, SAS^© can read and write data created by other applications, such as a DBMS. Base SAS^© can read certain raw data. For example, the DATA step or the IMPORT procedure can read comma-separated data from a text file. The DATA step or procedure provides the data to the V9 engine for output to a SAS^© dataset.
2. When you specify a SAS^© dataset name, the engine locates the stored file(s) to obtain metadata. V9 engine datasets contain metadata (also called descriptor information) in the dataset file. Other file types store metadata in a separate file. While SAS^© can determine metadata for many external file types, additional instructions may be required. Metadata provides information such as variable names and attributes, and whether the file has special processing characteristics such as indexes or compressed observations. Note that multiple engines may be involved in processing. For example, in a DATA step, one engine might be used to read data, and another engine to write data.
3. The engine uses the metadata to organize the data into the standard logical format for SAS^© processing. This standard format is the SAS^© dataset model. A SAS^© dataset consists of data values organized into variables (columns) and observations (rows). Similar to the SAS^© dataset model, the SAS^© library model is a group of datasets and other library members organized in a logical form for processing. When files are accessed as a SAS^© library, SAS^© utilities such as the DATASETS procedure can be used to list their contents and manage them.
4. SAS^© procedures and DATA step statements process data in this logical form, the SAS^© dataset model. During processing, the engine transmits all necessary instructions to open and close physical files and to read and write data. Processing can occur within the SAS^© dataset model without the data ever being physically stored as a SAS^© dataset. If the data is stored in an external application, such as a DBMS, some SAS^© procedures can transfer processing to that application.