Data set (IBM mainframe)

In the context of IBM mainframe computers in the IBM System/360 line and its successors, a data set (IBM preferred) or dataset is a computer file having a record organization. Use of this term began with, e.g., DOS/360 and OS/360, and is still used by their successors, including the current VSE and z/OS. Documentation for these systems historically preferred this term rather than file.

A data set is typically stored on a direct access storage device (DASD) or magnetic tape,<ref>Template:Cite web</ref> however unit record devices, such as punch card readers, card punches, line printers and page printers can provide input/output (I/O) for a data set (file).<ref>Template:Cite web</ref>

Data sets are not unstructured streams of bytes, but rather are organized in various logical record<ref>Template:Cite web</ref> and block structures determined by the DSORG (data set organization), RECFM (record format), and other parameters. These parameters are specified at the time of the data set allocation (creation), for example with Job Control Language DD statements. Within a running program they are stored in the Data Control Block (DCB) or Access Control Block (ACB), which are data structures used to access data sets using access methods.

Records in a data set may be fixed, variable, or “undefined” length.<ref>Template:Cite web</ref>

Data set organization

For OS/360, the DCB's DSORG parameter specifies how the data set is organized. It may be<ref>Template:Cite manual </ref>

CQ

Queued Telecommunications Access Method (QTAM) in Message Control Program (MCP)

CX

Communications line group

DA

Basic Direct Access Method (BDAM)

GS

Graphics device for Graphics Access Method(GAM)

IS

Indexed Sequential Access Method (ISAM)

MQ

QTAM message queue in application

PO

Partitioned Organization

PS

Physical Sequential

among others. Data sets on tape may only be DSORG=PS. The choice of organization depends on how the data is to be accessed, and in particular, how it is to be updated.

Programmers utilize various access methods (such as QSAM or VSAM) in programs for reading and writing data sets. Access method depends on the given data set organization.

Record format (RECFM)

Regardless of organization, the physical structure of each record is essentially the same, and is uniform throughout the data set. This is specified in the DCB RECFM parameter. RECFM=F means that the records are of fixed length, specified via the LRECL parameter. RECFM=V specifies a variable-length record. V records when stored on media are prefixed by a Record Descriptor Word (RDW) containing the integer length of the record in bytes and flag bits. With RECFM=FB and RECFM=VB, multiple logical records are grouped together into a single physical block on tape or DASD. FB and VB are fixed-blocked, and variable-blocked, respectively. RECFM=U (undefined) is also variable length, but the length of the record is determined by the length of the block rather than by a control field.

The BLKSIZE parameter specifies the maximum length of the block. RECFM=FBS<ref>Template:Cite web</ref> could be also specified, meaning fixed-blocked standard, meaning all the blocks except the last one were required to be in full BLKSIZE length. RECFM=VBS, or variable-blocked spanned, means a logical record could be spanned across two or more blocks, with flags in the RDW indicating whether a record segment is continued into the next block and/or was continued from the previous one.

This mechanism eliminates the need for using any "delimiter" byte value to separate records. Thus data can be of any type, including binary integers, floating-point, or characters, without introducing a false end-of-record condition. The data set is an abstraction of a collection of records, in contrast to files as unstructured streams of bytes.

Partitioned data set

Template:AnchorTemplate:AnchorTemplate:Anchor Template:Confused A partitioned data set (PDS)<ref>Template:Cite book</ref> is a data set containing multiple members, each of which holds a separate sub-data set, similar to a directory in other types of file systems. This type of data set is often used to hold load modules (old format bound executable programs), source program libraries (especially Assembler macro definitions), ISPF screen definitions, and Job Control Language. A PDS may be compared to a Zip file or COM Structured Storage.

A Partitioned Data Set can only be allocated on a single volume and have a maximum size of 65,535 tracks.

Besides members, a PDS contains also a directory. Each member can be accessed indirectly via the directory structure. Once a member is located, the data stored in that member are handled in the same manner as a PS (sequential) data set.

Whenever a member is deleted, the space it occupied is unusable for storing other data. Likewise, if a member is re-written, it is stored in a new spot at the back of the PDS and leaves wasted “dead” space in the middle. The only way to recover “dead” space is to perform file compression.<ref name=Stephens>Template:Cite book</ref> Compression, which is done using the IEBCOPY utility,<ref>Template:Cite book</ref> moves all members to the front of the data space and leaves free usable space at the back. (Note that in modern parlance, this kind of operation might be called defragmentation or garbage collection; data compression nowadays refers to a different, more complicated concept.) PDS files can only reside on DASD, not on magnetic tape, in order to use the directory structure to access individual members. Partitioned data sets are most often used for storing multiple job control language files, utility control statements, and executable modules.

An improvement of this scheme is a Partitioned Data Set Extended (PDSE or PDS/E, sometimes just libraries) introduced with DFSMSdfp for MVS/XA and MVS/ESA systems. A PDS/E library can store program objects or other types of members, but not both. BPAM cannot process a PDS/E containing program objects.

PDS/E structure is similar to PDS and is used to store the same types of data. However, PDS/E files have a better directory structure which does not require pre-allocation of directory blocks when the PDS/E is defined (and therefore does not run out of directory blocks if not enough were specified). Also, PDS/E automatically stores members in such a way that compression operation is not needed to reclaim "dead" space.<ref name=Stephens/> PDS/E files can only reside on DASD in order to use the directory structure to access individual members.

Generation Data Group

A Generation Data Group<ref>Template:Cite web</ref> (GDG)<ref>Template:Cite web</ref> is a group of non-VSAM data sets<ref>Template:Cite web</ref> that are successive generations of historically-related data<ref name=G.sets>Template:Cite web</ref> stored on an IBM mainframe (running OS/360 and its successors or DOS/360 and its successors).<ref name=VSE.VSAM>Template:Cite web</ref>

A GDG is usually cataloged.<ref name=G.sets/>

An individual member of the GDG collection is called a "Generation Data Set."<ref name=G.sets/><ref>"A generation data set is one of ...</ref> The latter may be identified by an absolute number, <syntaxhighlight lang="text" class="" style="" inline="1">ACCTG.OURGDG(1234)</syntaxhighlight>, or a relative number: <syntaxhighlight lang="text" class="" style="" inline="1">(-1)</syntaxhighlight> for the previous generation, <syntaxhighlight lang="text" class="" style="" inline="1">(0)</syntaxhighlight> for the current one, and <syntaxhighlight lang="text" class="" style="" inline="1">(+1)</syntaxhighlight> the next generation.<ref>Template:Cite web</ref>

A GDG specifies how many generations of a data set are to be kept and at what age a generation will be deleted. Whenever a new generation is created, the system checks whether one or more obsolete generations are to be deleted.

The purpose of GDGs is to automate archival, using the command language JCL, the data set name given is generic. When DSN appears, the GDG data set appears along with the history number, where

(0) is the most recent version

(-1), (-2), ... are previous generations

(+1) a new generation (see DD)

Another use of GDGs is to be able to address all generations simultaneously within a JCL script without having to know the number of currently available generations. To do this, you have to omit the parentheses and the generation number in the JCL when specifying the dataset.

GDG JCL & features

Generation Data Groups are defined using either the BLDG statement<ref>Template:Cite manual </ref> of the Template:Pslink utility or the <syntaxhighlight lang="text" class="" style="" inline="1">DEFINE GENERATIONGROUP</syntaxhighlight> statement<ref>Template:Cite manual </ref> of the newer IDCAMS utility,<ref name=How2>Template:Cite web</ref> which allows setting various parameters.

• <syntaxhighlight lang="text" class="" style="" inline="1">LIMIT(10)</syntaxhighlight> would limit the number of generations limit to 10.
• <syntaxhighlight lang="text" class="" style="" inline="1">SCRATCH FOR (91)</syntaxhighlight> would retain each member, up to the limited#generations, at least 91 days.

IDCAMS can also delete (and optionally uncatalog) a GDG.<ref>Template:Cite web</ref>

Example

Creation of a standard GDG for five safety scopes, each at least 35 days old: <syntaxhighlight lang="jcl"> //STEP1 EXEC PGM=IDCAMS //SYSPRINT DD SYSOUT=* //SYSIN DD * DEFINE GDG (NAME('DB2.FULLCOPY.DSNDB04.TSTEST') LIMIT(5) SCRATCH FOR(35)) /* </syntaxhighlight>

Delete a standard GDG: <syntaxhighlight lang="jcl"> //STEP3 EXEC PGM=IDCAMS //SYSPRINT DD SYSOUT=* //SYSIN DD * DELETE DB2.FULLCOPY.DSNDB04.TSTEST GDG FORCE /*

</syntaxhighlight>

References

• Introduction to the New Mainframe: z/OS Basics Template:Webarchive, Ch. 5, "Working with data sets", March 29, 2011. Template:ISBN