IEFU29 - SMF Dataset Switch Exit

 

If you have SMF (System Management Facility) set up to collect statistical information about the tasks running on your MVS system, you will eventually see the message:

IEE362A   SMF ENTER DUMP FOR SYS1.MANx ON volser

when the active dataset fills up.  This message is also issued when MVS is shut down (with the Z EOD operator command) or when the SMF dataset is switched in response to the operator command:  I SMF).

There is a user exit that may be installed - IEFU29 - that will intercept the IEE362A message and start a task to dump and clear the filled SMF dataset automatically.  The tasks involved in setting up the exit are:

  1. create and catalog a procedure in SYS1.PROCLIB to (optionally) dump and clear the filled dataset, and 

  2. install the exit to intercept the IEE362A message.

Even if you don't want to install the exit, creating the procedure can simplify responding to the message manually.  The jobstreams to do all the work are contained in the archive - iefu29.tgz [MD5: 12a129c405148b2a5873776ccf530fd4]:

  1. smfgdg.jcl - defines a Generation Data Group to hold the dumped SMF data

  2. smfproc.jcl - catalogs the SMFDUMP procedure in SYS1.PROCLIB and the parameter cards for it in SYS2.CONTROL

  3. iefu29.jcl - installs the IEFU29 exit.

 

13 September 2020 Update:  It was pointed out to me that my recent revision of my instructions/tutorial for installing MVS 3.8j instituted a renaming of the user modification that applies the IEFU29 exit with that installation, so in order to be in sync I have now corrected the documentation on this page (and in the jobstreams contained in the archive) to reflect the name for the user modification as of the September 2020 revision.  Another change that was made to this material since its original creation is the location of the generation datasets and high level qualifier of the dataset name for the generation datasets that will contain the offloaded SMF data.  That change was documented in an addition to this page a number of years ago and, as such, is left in the archive as it was originally written.  

 

Creating a SYS1.MANx Dump Procedure

By using a symbolic parameter, it is possible to create a single set of JCL to process either of the SMF datasets.  When the procedure is started, whether by executing the procedure from a jobstream, or by starting the procedure with a command from the console operator or by the IEFU29 exit, a value is passed to the procedure which is substituted for the symbolic parameter, thus referring to either of the two SMF datasets.  There is no need to maintain two sets of JCL.  Here is the procedure:

//*-------------------------------------------------------------------*
//*                 SMF DATASET DUMP/CLEAR PROCEDURE                  *
//*-------------------------------------------------------------------*
//SMFDUMP PROC CLASS=X,ID=                                             
//DUMP    EXEC PGM=IFASMFDP,REGION=4096K                               
//SYSPRINT  DD SYSOUT=&CLASS                                           
//DUMPIN    DD DSN=SYS1.MAN&ID,DISP=SHR                                
//DUMPOUT   DD DSN=SYS2.SMF.DATA(+1),DISP=(NEW,CATLG),                 
//             UNIT=3350,VOL=SER=SMP001,SPACE=(CYL,(5,1),RLSE)         
//SYSIN     DD DSN=SYS2.CONTROL(SMFDUMP),DISP=SHR                      
//*  INDD(DUMPIN,OPTIONS(DUMP))                                        
//*-------------------------------------------------------------------*
//CLEAR   EXEC PGM=IFASMFDP,REGION=4096K,COND=(0,NE,DUMP)              
//SYSPRINT  DD SYSOUT=&CLASS                                           
//DUMPIN    DD DSN=SYS1.MAN&ID,DISP=SHR                                
//DUMPOUT   DD DUMMY                                                   
//SYSIN     DD DSN=SYS2.CONTROL(SMFCLEAR),DISP=SHR                     
//*  INDD(DUMPIN,OPTIONS(CLEAR))                                       
//*-------------------------------------------------------------------*

There are actually two symbolic parameters, CLASS and ID, but CLASS has a default value and I largely included it to have some flexibility over the disposition of the SYSOUT during testing.  The value passed to the JCL via the ID symbolic parameter will be appended to the end of the coded value for the DSN parameter on the DUMPIN DD statements, thereby yielding either SYS1.MANX or SYS1.MANY depending upon whether the value of ID passed is X or Y.

The IFASMFDP program, which is supplied with MVS to dump and clear the SMF datasets requires a parameter card via SYSIN.  Because this is a catalogued procedure, the SYSIN data cannot be placed inline with the JCL, so I have placed the two parameter cards in the SYS2.CONTROL dataset.  If you don't have a SYS2.CONTROL dataset, you can create one by using this job:

//IEFBR14  JOB (001),'ALLOCATE SYS2.CONTROL',
//             CLASS=A,MSGCLASS=A,MSGLEVEL=(1,1)
//IEFBR14  EXEC PGM=IEFBR14
//CONTROL  DD  DSN=SYS2.CONTROL,
//             UNIT=3350,VOL=SER=MVSRES,DISP=(,CATLG),
//             SPACE=(TRK,(30,1,20)),
//             DCB=(RECFM=FB,LRECL=80,BLKSIZE=3120)
//

Notice that for documentation of the SMFDUMP procedure, I place the value of the respective parameter cards that are pointed to by the SYSIN DD statements as comments following the SYSIN DD statements in the procedure.

When this procedure runs, the first step will dump the contents of the SMF dataset to another dataset, then, if that step doesn't terminate abnormally, the second step of the procedure will clear the dataset.  

You can see that if you really just wanted to have the IEFU29 exit automatically clear the filled SMF dataset, you could simply eliminate the first step of the procedure.

 

Retaining a Group of SMF Datasets

Notice that in the procedure, I have specified a Generation Data Group dataset name on the DUMPOUT DD statement to receive the contents of the filled SMF dataset.  In order to do any meaningful analysis on SMF data, it is necessary to have available a collection of data that spans some period of time.  Using a Generation Data Group is an easy way to let the system manage how much data I retain.  If you prefer to use a static dataset, either on tape or disk, you can modify the first step of the procedure accordingly and forego running the supplied jobstream that defines the Generation Data Group and Model DSCB..

 

Manually Starting the Procedure

Once the procedure has been catalogued in SYS1.PROCLIB, you can manually start the procedure when ever the IEE362A message is issued, by using the START operator command:

start smfdump,id=x        or        start smfdump,id=y

depending upon which dataset needs dumping.  If you don't care to install the IEFU29 exit, just setting up the catalogued procedure can save you some effort when the time comes to clear an SMF dataset.

 

Installing the Exit

Activating the exit requires an Assembler program that MVS will automatically invokes whenever an SMF dataset becomes filled and the IEE362A message is about to be displayed.  The program runs in supervisor state and is passed the name of the dataset that needs dumping.  The functions performed by the program are simple:

The IEFU29 exit is packaged as an SMP4 User Modification - JLM0002 - and is installed, assembled and link edited under the control of SMP4.  Once installed, an IPL with the CLPA option needs to be performed to rebuild the load modules in the LPA which will activate the exit.  When the SMP job is run to install the exit, the return codes to be expected are:

IEF142I IEFU29 CLEANUP - STEP WAS EXECUTED - COND CODE 0000
IEF142I IEFU29 ADD - STEP WAS EXECUTED - COND CODE 0000
IEF142I IEFU29 ASM ASM - STEP WAS EXECUTED - COND CODE 0000
IEF142I IEFU29 HMASMP RECAPP - STEP WAS EXECUTED - COND CODE 0012

The condition code 0012 on the final step is expected, because SMP first tries to delete any existing version of IEFU29 and there is not one to delete.

 

Expected Error at MVS Shutdown

When the IEFU29 exit is activated, at MVS shutdown (the Z EOD command is issued by the operator), the IEFU29 exit will issue the START command.  However, since JES2 has already been stopped, an error message will be issued by MVS:

*IEE362A SMF ENTER DUMP FOR SYS1.MANX ON MVSRES
START SMFDUMP,ID=X
IEFU29 HAS ISSUED COMMAND 'START SMFDUMP,ID=X '
IEE360I SMF NOW RECORDING ON SYS1.MANY ON MVSRES TIME=22.29.25
IEE334I HALT     EOD SUCCESSFUL
IEC141I 013-C8,IGG0193K,MSTRJCL,JES2,STCINRDR
IEF170I 1 MSTRJCL  IEC141I 013-C8,IGG0193K,MSTRJCL,JES2,STCINRDR

At the next IPL, SMF will resume recording in the same dataset that was in use - SYS1.MANX in the case of the messages above - and no data will be lost.  

Update 08/2004 - Following a discussion on bit.listserv.ibm-main, I added code posted by Gary DiPillo to test for JES2 being active when the exit is started.  If JES2 is not running, which it is not during shutdown, IEFU29 simply exits without issuing the Start command.  This will result in the system displaying the IEE326A message, which can safely be ignored.  SMF will resume using the last active dataset at the next IPL of MVS.  The code added is identified by comments in the source.

 

Manually Switching the SMF Dataset

You can direct SMF to switch from the active dataset to the alternate by issuing the operator command:

i smf

If you have installed the SMFDUMP procedure and IEFU29 exit, the exit will issue the START command as described above to dump and clear the previously active SMF dataset.

 

Analyzing SMF Data

After you have transferred the collected SMF data into your own datasets, whether you have chosen to use a Generation Data Group managed dataset or a scheme of your own choosing, you may then wonder what to do with it.  There are several SMF extraction and reporting programs on the CBT tape, notable among them are those found in files #94 (SMF Dataset Audit Facility) and #523 (General SMF Record Selection Program).

 

Determining the Active SMF Dataset

There is a useful program on CBT that may be run as either a TSO command or a started task that will display information about the SMF datasets allocated in the system, including noting which is currently active (receiving output from SMF).  You can download the installation jobstream from my site at CBTWare/showsmf.

 

Considerations / Implications of Password Protected Master Catalog

A post to the main Hercules discussion group pointed out that if you follow these instructions to set up the SMF datasets with a high level qualifier of SYS2 and have password protected your Master Catalog, you will be prompted for the Master Catalog password whenever the SMFDUMP procedure executes, either when invoked manually via the i smf console command or by the IEFU29 exit.  When I initially set this up on my own system, I did not have the Master Catalog password protected and, even after I added password protection to my Master Catalog, I did not see this as much of an inconvenience.  Never the less, it is probably a factor that you need to consider if you follow my instructions.

If you have not yet begun, you simply need to change the SYS2 high level qualifier in the smfgdg and smfproc jobstreams discussed above to one that is related to a user catalog on your own system.  Remember that the model DSCB must reside on the same volume as the catalog into which the SMF dataset generations will be cataloged.  The datasets themselves need not reside on that volume, but the catalog and model DSCB must be kept together.

If you have already followed the instructions above and wish to modify your system to move the SMF datasets into a user catalog, I have created a jobstream that will be useful to you in accomplishing that:

//SMFALTER JOB (001),SMFALTER,CLASS=A,MSGCLASS=A
//STEP01   EXEC PGM=IEHPROGM                    
//********************************************************************  
//* UNCATALOG EXISTING SMF DATASETS             
//********************************************************************  
//SYSPRINT  DD SYSOUT=*                         
//SYSIN     DD *                                
  UNCATLG DSNAME=SYS2.SMF.DATA.G0004V00         
  UNCATLG DSNAME=SYS2.SMF.DATA.G0003V00         
  UNCATLG DSNAME=SYS2.SMF.DATA.G0002V00         
  UNCATLG DSNAME=SYS2.SMF.DATA.G0001V00         
/*                                              **********************  
//STEP02   EXEC PGM=IDCAMS,REGION=4096K         
//********************************************************************  
//* REPLACE MODEL DSCB AND GDG BASE             
//********************************************************************  
//STEPCAT   DD DSN=UCMVSOPS,DISP=SHR            
//MODELOLD  DD DSN=SYS2.SMF.DATA,DISP=(OLD,DELETE), 
//             UNIT=3350,VOL=SER=MVSRES             
//MODELNEW  DD DSN=SYSO.SMF.DATA,DISP=(NEW,KEEP),   
//             UNIT=3350,VOL=SER=WORK02,SPACE=(TRK,0), 
//             DCB=(RECFM=VBS,LRECL=32756,BLKSIZE=32760)
//SYSPRINT  DD SYSOUT=*                                 
//SYSIN     DD *                                        
  DELETE SYS2.SMF.DATA GENERATIONDATAGROUP              
  DEFINE GENERATIONDATAGROUP (                              -
               NAME(SYSO.SMF.DATA)                          -
               LIMIT(5)                                     -
               SCRATCH )                                     
                                                             
  IF LASTCC = 0 THEN                                        -
        LISTCAT ALL ENTRIES(SYSO.SMF.DATA)                   
                                                             
/*                                              **********************  
//STEP03   EXEC PGM=IEHPROGM                                 
//STEPCAT   DD DSN=UCMVSOPS,DISP=SHR                         
//********************************************************************  
//* RENAME AND RECATALOG EXISTING SMF DATASETS              
//********************************************************************  
//SYSPRINT  DD SYSOUT=*                                     
//SMP001    DD UNIT=3350,VOL=SER=SMP001,DISP=OLD            
//SYSIN     DD *                                            
  RENAME  DSNAME=SYS2.SMF.DATA.G0004V00,VOL=3350=SMP001,               X
               NEWNAME=SYSO.SMF.DATA.G0004V00                           
  RENAME  DSNAME=SYS2.SMF.DATA.G0003V00,VOL=3350=SMP001,               X
               NEWNAME=SYSO.SMF.DATA.G0003V00                           
  RENAME  DSNAME=SYS2.SMF.DATA.G0002V00,VOL=3350=SMP001,               X
               NEWNAME=SYSO.SMF.DATA.G0002V00                           
  RENAME  DSNAME=SYS2.SMF.DATA.G0001V00,VOL=3350=SMP001,               X
               NEWNAME=SYSO.SMF.DATA.G0001V00                           
  CATLG   DSNAME=SYSO.SMF.DATA.G0004V00,VOL=3350=SMP001                 
  CATLG   DSNAME=SYSO.SMF.DATA.G0003V00,VOL=3350=SMP001                 
  CATLG   DSNAME=SYSO.SMF.DATA.G0002V00,VOL=3350=SMP001                 
  CATLG   DSNAME=SYSO.SMF.DATA.G0001V00,VOL=3350=SMP001                 
/*                                              **********************  
//STEP04   EXEC PGM=IKJEFT01,REGION=1024K,DYNAMNBR=50                   
//********************************************************************  
//* MODIFY CATALOGED PROCEDURE                                          
//********************************************************************  
//SYSPRINT DD  SYSOUT=*                                                 
//SYSTSPRT DD  SYSOUT=*                                                 
//SYSTERM  DD  SYSOUT=*                                                 
//SYSTSIN  DD  *                                                        
  EDIT 'SYS1.PROCLIB(SMFDUMP)' CNTL                                     
  F /SYS2/                                                              
  C * /SYS2/SYSO/                                                       
  SAVE                                                                  
  END                                                                   
/*                                              **********************  
//                                                                      

Step 1 of the jobstream uncatalogs existing SMF dataset generations.  When I built the jobstream on my system, there were four generations, so you may need to modify the control statements following the SYSIN statement to match the conditions of your own system.

Step 2 of the jobstream removes the GDG base (SYS2.SMF.DATA) from the Master Catalog, removes the matching model DSCB from MVSRES, creates a new GDG base (SYSO.SMF.DATA) in the User Catalog UCMVSOPS, and a matching model DSCB on the WORK02 volume.  On my system, I already had a User Catalog - UCMVSOPS - in which I catalog miscellaneous operations related datasets.  The catalog resides on my WORK02 volume and is related to the high level qualifier SYSO.  You will either need to create a User Catalog specifically for SMF or select an existing User Catalog on your system.  Remember that the model DSCB must reside in the VTOC on the same volume where the User Catalog resides.  So there are three items to keep in alignment as you tailor this jobstream for your system - placement of the model DSCB, location of the User Catalog, and high level qualifier of the SMF.DATA must be related to the selected User Catalog.  Note that this step has a STEPCAT pointing to the target User Catalog.

Step 3 of the jobstream renames the existing SMF datasets changing the high level qualifier - SYS2 to SYSO.  As you can see, the datasets remain on volume SMP001 on my system; only the catalog in which the datasets are cataloged and the location of the model DSCB are affected.  Then the newly renamed datasets are cataloged in the user catalog.  As with Step 1, I had four datasets to rename/recatalog on my system; you may have fewer or more, so change the control statements as required for your system as it exists at the time you submit the jobstream.  Again note that this step must have a STEPCAT pointing to the target User Catalog.

Step 4 of the jobstream utilizes TSO execute in batch to edit the catalogued procedure in SYS1.PROCLIB to alter the high level qualifier of the SMF datasets.  

The jobstream only needs to be run once and it should go without saying that you should back up your system before submitting the job.  You will be required to enter the Master Catalog password for steps one and two of the jobstream as alterations are being made to entries in the Master Catalog.  Once this jobstream is run, you will no longer be required to enter the Master Catalog password when the SMFDUMP procedure executes.


I hope that you have found my instructions useful.  If you have questions that I can answer to help expand upon my explanations and examples shown here, please don't hesitate to send them to me:


Return to Site Home Page 


This page was last updated on September 13, 2020 .