My input data set has a trailer with a count of the data records and a total of a field in the data records. I want
to remove some data records.
How can I update the count and total in the existing trailer record to reflect the
output data records? For example, my input file has these records:
H 10/12/2010
D key1 0100
D key1 0300
D key2 0200
D key2 0050
D key1 0625
D key1 0300
D key2 3000
T DEPT AXY COUNT=00000007 TOTAL=004575
The trailer record (identified by T in position 1) has the count of the D records in positions 18-25 and the total
of the third field of the D records in positions 33-38. I want to keep only the key1 records and update the
count and total in the existing trailer records accordingly. So the output I want is:
H 10/12/2010
D key1 0100
D key1 0300
D key1 0625
D key1 0300
T DEPT AXY COUNT=00000004 TOTAL=001325
Can I do this with DFSORT?
You can use DFSORT’s IFTRAIL function to do this kind of thing quite easily like this:
//S1 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//SORTIN DD DSN=… input file (FB/40)
//SORTOUT DD DSN=… output file (FB/40)
//SYSIN DD *
OPTION COPY
OUTFIL INCLUDE=(3,4,CH,EQ,C’key1′),
IFTRAIL=(HD=YES,TRLID=(1,1,CH,EQ,C’T’),
TRLUPD=(18:COUNT=(M11,LENGTH=8),
33:TOT=(8,4,ZD,M11,LENGTH=6)))
/*
There are 2 ideas you can do re-order the fields in the output file.
Option -1 is you need to write a COBOL sort program. You need to code the fields how you want and write into output file. Click here to know sample COBOL sort program.
Option-2 is you need to use DFSORT commands in JCL.
Write the fields how you need in OUREC statement. The fields always appear in the order in which you specify them in the OUTREC.
Explanation:
SORT FIELDS=(106,4,CH,A) ==> Just sort the file based on fields specified.
SUM FIELDS=(162,4,BI,166,4,BI) ==> Sum up two fields and add to second field
OUTREC FIELDS=(106,4,166,4,162,4) or
OUTREC BUILD=(106,4,166,4,162,4) ==> Here, you can write fields as you wish. So, you can easily do re-ordering with DFSORT without writing complex COBOL program logic.
The job control language (JCL) you need to do a sort depends on whether you run DFSORT directly or call DFSORT from a program. For now, concentrate on running DFSORT directly.
The below are the list of Parameters to be used in DFSORT job:
Required JCL includes a JOB statement, an EXEC statement, and several DD statements. The statements you need and their exact form depend upon whether you:
Invoke DFSORT with an EXEC statement in the input job stream, or with a system macro instruction within another program
Choose to use EXEC statement cataloged procedures to invoke DFSORT
Choose to specify PARM options on the EXEC statement
Choose to specify PARM options or control statements in a DFSPARM data set
Choose to specify control statements in a SYSIN data set
The JCL statements you need for most jobs are as follows.
//jobname JOB
Signals the beginning of a job. At your site, you might be required to specify information such as your name and account number on the JOB statement.
//stepname EXEC
Signals the beginning of a job step and tells the operating system what program to run. To run DFSORT, write the EXEC statement like this:
//stepname EXEC PGM=SORT
//STEPLIB DD
The DFSORT program would typically be in a library known to the system, so the //STEPLIB DD statement would not be needed. However, if DFSORT is not in a library known to the system, the //STEPLIB DD statement defines the library containing the DFSORT program
//SYSOUT DD
Defines the data set in which DFSORT messages and control statements are listed.
//SORTIN DD
Defines the input data set or concatenated input data sets.
//SORTWKdd DD
Defines a work data set for a sort. Typically not needed, because DFSORT can allocate work data sets for a sort dynamically.
//SORTOUT DD
Defines the output data set.
//SYSIN DD
Precedes or contains the DFSORT program control statements.
WHEN=INIT: Use one or more WHEN=INIT clauses to apply BUILD, FINDREP or OVERLAY items to all of your input records. WHEN=INIT clauses andWHEN=GROUP clauses are processed before any of the other IFTHEN clauses. WHEN=INIT changes (if any) and WHEN=GROUP changes (if any) are applied to input records that do not meet the criteria for any of the WHEN=(logexp) clauses.
WHEN=NONE => It means no other condition satisfied
One IFTHEN clause is satisfied, then skips checking for other IFTHEN statement
DFSORT is a sort utility. If you are aware how to use this utility, you can perform your reporting tasks more effectively and with less time. In all financial projects like Banking, Capital Market and Insurance projects do use DFSORT utility extensively.
INREC IFTHEN=(WHEN=(1,3,CH,EQ,C’HDR’),
OVERLAY=(6:YDDD=(D4/),81:C’0′,82:SEQNUM,2,ZD)),=> IFTHEN works on header record
IFTHEN=(WHEN=(1,3,CH,EQ,C’TRL’),
OVERLAY=(11:YDDD=(D4/),81:C’9′,82:SEQNUM,2,ZD)),=> IFTHEN works on trailer record
IFTHEN=(WHEN=NONE, OVERLAY=(81:C’1′)) => IFTHEN works on details records
SORT FIELDS=(81,1,CH,A,8,5,CH,A) => Sorts input record
OUTFIL REMOVECC, =>REMOVECC removes the ANSI carriage control characters and ensures that the RECFM is FB rather than FBA OMIT=(81,1,SS,EQ,C’0,9′,AND,82,2,ZD,GT,+1), => This Omits records matching to the given criteria OUTREC=(1,80) => It keeps only 80 byte data
Use IFTHEN in sort on input file, that is INREC. That means you need to reformat input record without header and trailer. This way you will get a data set without header and trailer.
INCLUDE Example with DATE4
+————————————————-+
| INCLUDE COND=(1,19,CH,GT,DATE4) |
+————————————————-+
This example illustrates how to include only those records with a C’yyyy-mm-dd-hh.mm.ss’ date value in positions
1-19 greater than the DATE4 character string for the run.
Note: When a field is shorter than the DATE4 character string it’s compared to, DFSORT truncates the DATE4
string on the right. You can take advantage of this to compare a field to only part of the DATE4 timestamp when
appropriate.
For example:
INCLUDE COND=(1,13,CH,GT,DATE4)
would compare the field in positions 1-13 to the truncated DATE4 constant C’yyyy-mm-dd-hh’.
DFSORT -DATE4 Uses
The other uses of DATE4 are you can insert timestamp or you can compare records with current timestamp
Srinimf.com has all the answers for the below complex DFSORT questions.
JOINS in SORT –How to use this feature
Control Statements:
INCLUDE – selects records to be kept based on logical criteria.
OMIT – selects records to be deleted based on logical criteria.
SUM – produces a single record for each unique sort or merge key with optional field totals.
INREC and OUTREC – specifies reformatting, find and replace operations, group operations, parsing, justifying, squeezing, past, current and future timestamps, translation, date editing, date conversion, date field arithmetic,numeric editing, numeric conversion, substitution, arithmetic operations, hexadecimal display, bit display and sequence numbers. Can build records item by item, only overlay specific parts of records, or reformat different records in different ways by specifying how items are to be applied to records that meet given criteria.
OPTION – overrides installation defaults and supplies optional information such as the number of records to skip before sorting.
OUTFIL – specifies multiple output data sets, subsets, sampling, repetition, reports, reformatting, find and replace operations, group operations, parsing, justifying, squeezing, past, current and future timestamps, translation,date editing, date conversion, date field arithmetic, numeric editing, numeric conversion, substitution,arithmetic operations, hexadecimal display, bit display, sequence numbers, VB to FB and FB to VB conversion,and more. Can build records item by item, only overlay specific parts of records, or reformat different records in different ways by specifying how items are to be applied to records that meet given criteria. Can update counts and totals in existing trailer records.
Reformatting Data sets: Reformatting: FINDREP, BUILD, OVERLAY and IFTHEN
This is simple JCL to split input data set into multiple output files.
The OUTFIL operands INCLUDE, OMIT and SAVE can be used to select the records to be included in each output data set.
SAVE can be used to select the records that are not selected for any other subset, eliminating the need to specify complex conditions.
Here’s an example of OUTFIL subsets:
You can use DFSORT for creating an excellent reports. The headers are HEADER1,HEADER2 and HEADER3. Also you can give trailers TRAILER1, TRAILER2 parameters and TRAILER3.
Let you specify multi-line headings with character strings, hexadecimal strings, input fields, the current date, the current time, page numbers and blank lines. In addition, you can include edited or converted record counts, and edited or converted totals, maximums, minimums, and averages for numeric fields, in your trailers.
The following statements create a report with a separate report trailer page containing the overall record count and various overall statistics for the revenue brought in by the branches.
You would use TRAILER1 when you want a report trailer to appear by itself as the last page of the report. This is also handy for creating one record with a count or date.
You would use TRAILER2 when you want a page trailer to appear at the bottom of each page of the report.
You would use TRAILER3 within SECTIONs when you want a section trailer to appear after the last data record of each section.
The JOIN key is a concept used to create new data set from the given data sets by using KEY. This is possible with DFSORT, and can be used with JCL
//JN1 EXEC PGM=SORT
//SYSOUT DD SYSOUT=*
//IN1 DD DSN=DSN=A123456.REGION.IN1,DISP=SHR
//IN2 DD DSN=DSN=A123456.REGION.IN2,DISP=SHR
//SORTOUT DD DSN=A123456.REGION.OUT,DISP=(NEW,CATLG,DELETE),UNIT=3390,
// SPACE=(CYL,(5,5))
//SYSIN DD *
JOINKEYS F1=IN1,FIELDS=(1,5,A) F1 has ddname IN1 and key in 1-5
JOINKEYS F2=IN2,FIELDS=(5,5,A) F2 has ddname IN2 and key in 5-9
REFORMAT FIELDS=(F2:1,4, Office from F2
F1:1,5,21,15, Region and Regional Director from F1
F2:10,4,14,10, Employees and Evaluation from F2
F1:6,15) Headquarters from F1
OPTION COPY Copy joined records
/*
How to understand each step in JOIN Key JCL
Two JOINKEYS statements are required: one for the F1 data set and another for the F2 data set.
In this case, the first JOINKEYS statement identifies IN1 as the ddname for the F1 data set and indicates an ascending key (Region) in positions 1-5 of that data set.
The second JOINKEYS statement identifies IN2 as the ddname for the F2 data set and indicates an ascending key (Region) is at positions 5-9 of that data set.
Each key in the F1 data set must be of the same length and order (ascending or descending) as the corresponding key in the F2 data set, but does not have to be in the same location.
The keys are always treated as unsigned binary (INREC can be used to “normalize” the keys in each data set before the records are joined, if necessary).
The F1 data set will be sorted by the key in positions 1-5. The F2 data set will be sorted by the key in positions 5-9. If the records in a data set are already sorted by the key, you can specify SORTED on the JOINKEYS statement to tell DFSORT to copy the records of that data set rather than sorting them.
Records with the same key in both data sets are joined and constructed as directed by the REFORMAT statement using F1: for fields from the F1 record and F2: for fields from the F2 record.
This REFORMAT statement creates joined records from the following fields:
Output positions 1-4: Office from F2
Output positions 5-9: Region from F1
Output positions 10-24: Regional Director from F1
Output positions 25-28: Employees from F2
Output positions 29-38: Evaluation from F2
Output positions 39-53: Headquarters from F1
The resulting joined records are 53 bytes long and are copied to the SORTOUT data set (REGION.OUT).
SORT FIELDS=COPY
FIELDS - The input data set is copied to the output data set without sorting or merging.
2. How to compare Dates in DFSORT
INCLUDE FORMAT=Y2T,COND=(3,4,GE,Y’9901’,AND,3,4,LE,Y’0312’,OR,3,4,LE,Y’0000’)
This example illustrates how to only include records in which: A C'yymm' date field in bytes 3 through 6 is between January 1999 and December 2003
OR INCLUDE Control Statement
Note that the century window in effect will be used to interpret the Y'9901' and Y'0312' date constants, as well as real dates in the C'yymm' date field. However, the century window will not be used to interpret the Y'0000' special indicator constant or special indicators in the C'yymm' date field.
INCLUDE COND=(2,3,Y2X,LT,36,5,Y2T)
This example illustrates how to only include records in which a P'dddyy' date field in bytes 2 through 4 is less than a Z'yyddd' date field in bytes 36 through 40.
Note that the century window in effect will be used to interpret real dates in the P'dddyy' and Z'yyddd' date fields. However, the century window will not be used to interpret special indicators in the P'dddyy' and Z'yyddd' date fields
3. How to Sort dataset in DFSORT
SORT FIELDS=(7025,4,A,5048,8,A),FORMAT=ZD,EQUALS
FIELDS - The major control field begins on byte 7025 of each record, is 4 bytes long, contains zoned decimal data (FORMAT=ZD), and is to be sorted in ascending sequence. The second control field begins on byte 5048, is 8 bytes long, has the same data format as the first field, and is also to be sorted in ascending order.
FORMAT - FORMAT=ZD is used to supply ZD format for the p,m,s fields and is equivalent to specifying p,m,ZD,s for these fields. With FORMAT=f, you can mix p,m,s and p,m,f,s fields when that's convenient such as when all or most of the fields have the same format (although you can always code p,m,f,s for all fields and not use FORMAT=f, if you prefer). For example, the following are also valid uses of the FORMAT=f parameter:
SORT FORMAT=BI,FIELDS=(21,4,A,5,4,PD,A,31.3,1.4,A,52,20,A)SORT FIELDS=(16,4,A,22,8,BI,D,3,2,A),FORMAT=FI
EQUALS - specifies that the sequence of equal collating records is to be preserved from input to output.
This example illustrates how a sequence number can be generated, how values in one numeric or date format can be converted to another format, and how a lookup table can be used.
The reformatted input records will look as follows:
Position Contents
1-4 A binary sequence number that starts at 1 and increments by 1.
5–7 A PD field containing the converted ZD field from input positions
8 through 12.
8–9 An FI field containing the converted PD field from input positions
31 through 32.
10–14 A P’yyyymmdd’ date field containing the C’yymmdd’ date field
from input positions 15-20 transformed according to the specified
century window of 1985-2084.
15 A BI field containing X’01’, X’02’, X’03’ or X’FF’ as determined by
using a lookup table for the input field in positions 25-27.
The SORT statement can now refer to the “sort” field in the reformatted input records. The OUTREC statement is used to restore the records to their original format.
8. How to remove Duplicates in DFSORT
SUM FIELDS=(5,5,ZD,12,6,PD,21,3,PD,35,7,ZD)
SUM FORMAT=ZD,FIELDS=(5,5,12,6,PD,21,3,PD,35,7)
SUM FIELDS=(5,5,ZD,12,6,21,3,35,7,ZD),FORMAT=PD
SUM FIELDS=NONE ......removes duplicates
The permissible field formats are shown under the description of ‘f’ for fields. Default: None.
FORMAT=f must be specified if any field is specified as p,m rather tha n p,m,f.
Many reformatting features are available with DFSORT’s INREC, OUTREC and OUTFIL statements, making each
useful by itself or in conjunction with the other statements.
INREC, OUTREC and OUTFIL can translate data in several new ways as follows:
– TRAN=ATOE translates ASCII characters anywhere in a specified field to their equivalent EBCDIC characters.
As a simple example, if you specify:
INREC OVERLAY=(11:11,3,TRAN=ATOE)
DFSORT will translate ASCII to EBCDIC in positions 11-13 of each output record. So ASCII aB2
(X’614232′) would be translated to EBCDIC aB2 (X’81C2F2′).
– TRAN=ETOA translates EBCDIC characters anywhere in a specified field to their equivalent ASCII characters.
As a simple example, if you specify:
INREC OVERLAY=(21:21,3,TRAN=ETOA)
DFSORT will translate EBCDIC to ASCII in positions 21-23 of each output record. So EBCDIC aB2
(X’81C2F2′) would be translated to ASCII aB2 (X’614232′).
– TRAN=HEX translates binary values anywhere in a specified field to their equivalent EBCDIC
hexadecimal characters. As a simple example, if you specify:
INREC OVERLAY=(5:51,2,TRAN=HEX)
DFSORT will translate binary to hex in positions 51-52 of each output record. So X’C1F1′ (C’A1′) would
be translated to C’C1F1′.
– TRAN=UNHEX translates EBCDIC hexadecimal characters anywhere in a specified field to their equivalent
binary values. As a simple example, if you specify:
INREC BUILD=(1,4,TRAN=UNHEX)
DFSORT will translate hex to binary in positions 1-4 of each output record. So C’C1F1 would be translated
to X’C1F1′ (C’A1′).
What’s New in DFSORT 13
– TRAN=BIT translates binary values anywhere in a specified field to their equivalent EBCDIC bit values.
As a simple example, if you specify:
INREC OVERLAY=(16:16,2,TRAN=BIT)
DFSORT will translate binary to bits in positions 16-17 of each output record. So X’C1F1′ (C’A1′) would
be translated to C’1100000111110001′.
– TRAN=UNBIT translates EBCDIC bit values anywhere in a specified field to their equivalent binary
values. As a simple example, if you specify:
INREC BUILD=(31:31,16,TRAN=UNBIT)
DFSORT will translate bit to binary in positions 31-46 of each output record. So C’1100000111110001′
would be translated to X’C1F1′ (C’A1′).
– TRAN=LTOU translates lowercase EBCDIC letters anywhere in a specified field to uppercase EBCDIC
letters. As a simple example, if you specify:
INREC OVERLAY=(31:31,11,TRAN=LTOU)
DFSORT will translate lowercase to uppercase in positions 31-41 of each output record. So ‘Vicky-123,x’
would be translated to ‘VICKY-123,X’. letters. As a simple example, if you specify:
OUTFIL BUILD=(1,4,5,TRAN=UTOL)
The OUTFIL operands SPLIT1R, SPLIT and SPLITBY can be used with the operands FNAMES and FILES to split records in various ways among multiple output data sets as follows:
SPLIT1R=n writes the first n output records to the first output data set, the second n output records to the second data set, and so on; when each output data set has n records, any remaining records are written to the last output data, so the records in each output data set will be contiguous.
Here’s an example of OUTFIL SPLIT1R:
OUTFIL FNAMES=(T1,T2,T3,T4),SPLIT1R=50
Records 1-50 are written to the T1 data set, records 51-100 are written to the T2 data set, records 101-150 are written to the T3 data set, and records 151-n (where n is the last record, for example, 220) are written to the T4 data set.
Use SPLIT1R rather than SPLIT or SPLITBY if you want to ensure that each output data set has contiguous records. SPLIT writes the first output record to the first output data set, the second output record to the second output data set, and so on; when each output data set has one record, the rotation starts again with the first output data set, so the records in each output data set will not necessarily be contiguous.
Here’s an example of OUTFIL SPLIT:
OUTFIL FNAMES=(PIPE1,PIPE2,PIPE3),SPLIT
The first record is written to the PIPE1 pipe, the second record is written to the PIPE2 pipe, the third record is written to the PIPE3 pipe, the fourth record is written to the PIPE1 pipe, and so on.
dfsort examples
SPLIT can also be used with / in OUTFIL OUTREC to put pieces of each input record into different data sets. Here’s an example:
OUTFIL FNAMES=(PART1,PART2,PART3),SPLIT,
BUILD=(1,20,/,21,20,/,41,20)
Positions 1-20 of each input record are written to PART1. Positions 21-40 of each input record are written to PART2. Positions 41-60 of each input record are written to PART3.
SPLITBY=n writes the first n output records to the first output data set, the second n output records to the second data set, and so on; when each output data set has n records, the rotation starts again with the first output data set, so the records in each output data set will not necessarily be contiguous.
Here’s an example of OUTFIL SPLITBY:
OUTFIL FNAMES=(OUT1,OUT2),SPLITBY=50
Records 1-50 are written to the OUT1 data set, records 51-100 are written to the OUT2 data set, records 101-150 are written to the OUT1 data set, records 151-200 are written to the OUT2 data set, and so on. (Ref: IBM)
The best DFSORT sort to use readily in JCL to create reports in your project given here. This example is to get matching timestamp records into your output file.
Sorting and report generation is most common in mainframe JCL. In almost all projects DFSORT being used to generate different reports depending on requirement. The list of key areas given here mostly interviewer ask questions.