Continuous mode processing

Continuous mode processing is an optional way to run Log Analysis Tool general reports. In continuous mode, you run reports continuously over time, each new report beginning where the previous report ended. In this manner, you can track updates to specific objects on a daily basis.

You indicate that you want to run in continuous mode by typing name of the continuous mode file in the Continuous mode file field on the general report options panel. Prior to running Log Analysis Tool in continuous mode, you must have created the continuous mode file using the Defaults option on the main menu. For more information on creating the continuous mode file, see Create the continuous mode data set.

Once you have created the continuous mode file, follow these steps to run Log Analysis Tool in continuous mode:

On the general report option panel, specify a start date and time. The first run of the JCL uses this specified date and time.
Specify an end date and time. The first run of the JCL uses this specified date and time. If you do not specify an end date and time, the end date and time is set to the current date and time when the job runs. The job then processes the log continuously over time.
Type the name of the continuous mode file in the Continuous mode file field.
Set all of the other options for the general report.
Generate the JCL.
Submit the job.

This initial run of the JCL logs the start date and time for the next run in the continuous mode file. Log Analysis Tool also logs uncommitted units of recovery, so that on a subsequent run, any URIDs with an uncommitted status will likely be resolved (committed or rolled back). Without continuous mode processing, any uncommitted records are lost if the end date and time occurs prior to a commit.

You can put the JCL in a job scheduler for execution every night. Each run will find data in the continuous mode file and use it to determine the needed start date and time as well as handling uncommitted URIDs. If you put this JCL in a job scheduler, do not include an end date or time, because each subsequent run will always end at that specified end date and time. In this case, leave end date and time fields blank, or at their default values (for example, YYYY/MM/DD).

Note: For each scheduled job you want to run in continuous mode, you must have a unique continuous mode file. For example, if you want to run a continuous mode process against two distinct sets of filters, you must create two separate continuous mode files from the Defaults option and generate the continuous mode JCL for each process. The continuous mode file is associated with DD statement UNCRE in the general report JCL.

Attention: When using continuous mode processing, it is possible that units of work (UOW) will be in the Continuous Mode File that do not match your filters. Most of the time these UOWs are uncommitted during the specified timeframe and must be tracked in case additional log records come through that match the filter criteria the next time the job runs.

In certain circumstances, SYSLGRNX may have eliminated a Db2 log file that contained the COMMIT or ABORT record so this UOW remains in a pending state. In this case, the UOW will be tracked for an internally specified period to make sure no subsequent activity comes in. After this internally specified period, with no activity that matches the filter, this UOW will go away.

If you use MISC FLAGS=X, all the logs for the time period are read and the pending UOW will not be included in the continuous mode file – just the ones that are truly uncommitted. This does not cause any problems other than it may appear that more activity is being tracked / reported on than was requested.

Using continuous mode

Continuous mode is an option for a general report run. A panel field on the general report options determines if running in continuous mode. If in continuous mode (herein referred to as CM), the start date and time is determined from the contents of the CM file. The CM file must first be created within the "Defaults" option off the main menu; failure to create the specified continuous mode file will result in a JCL error at run time.

CM means that you want to run reports continuously over time. For example, you want to track updates to a specific object every day. You can generate the CM JCL and place it in a job scheduler for execution every night. Each run of the job will track the needed start date and time for a subsequent run. It will also track uncommitted units of recovery, so that on a subsequent run, the uncommitted status will likely be resolved (commit or rollback). Without CM, uncommitted records are lost if the end date and time occurs prior to commit.

Note: Because continuous mode runs continuously over time, some files are used as communication mechanisms from one run to the next. For restart purposes, this product now saves a backup copy of these files before reading and modifying them for the current run. The files involved are associated with the DDs UNCRE and EXTFILE. The product now saves these automatically into backup files associated with the DDs UNCREB and EXTFILB. The product does not use these backup files in anyway; it just populates them to allow for easier restart capabilities.

To start the process, generate the JCL with a start date and time as desired. The first run will use this start date and time, as the CM file will currently be empty. That initial run will then log to the CM file any uncommitted URIDs and keep track of the next needed start date and time. Each subsequent run will find data in the CM file and use it to determine the needed start date and time (as well as handling uncommitted URIDs).

The end date and time can either be omitted (spaces), entered as normal end date/time, or use a relative end date/time. If omitted, the end date/time of the CM run is the current date and time. This way, the job processes the log continuously over time. If end date/time is included or a relative end date/time is used the continuous mode will stop at the specified or calculated end date/time, and this will be used as the next start date/time. This allows you to run continuously, but at time increments deemed appropriate by you. Using the relative end date/time specified as -060, would allow you to run each job from the time the previous job ended to 1 hour before run time. If you place this JCL in a job scheduler, the end date/time can either be omitted to always run to current or the relative end date/time can be used. However, if a specified end date/time is used, the job will never run past the specified end date/time.

After an initial run of CM, subsequent runs will actually use internally stored LRSNs as the true starting time. This is done because data in Db2's log buffers may not be externalized at the actual end time, but could have timestamps (LRSNs) that are lower than the end time. Using the last returned LRSN (by member, in a data sharing environment) ensures that this data will not be lost. Please note, therefore, that it is possible to report on log activity that is earlier than the specified start date/time when running in continuous mode. The specified start date/time is reported in the general report heading, and based only on actual dates and times, not LRSNs.

Note that for each scheduled job you want to run as CM, a unique CM file is needed. For example, if you want to run a CM process against 2 distinct set of filters (for example, 1-tablespace and plan name; 2-table name), you need to create 2 separate CM files from the "Defaults" option and generate the CM JCL for each of these. The CM file is associated with DD statement "UNCRE" in the general report JCL.

Restarting the JCL in continuous mode

All references to the term 'copy' below mean 'copy with replace'. An ISPF 3.3 copy will copy with replace, but be sure the 'Sequential Disposition = Old'. You do not want to add the records from the backup files to the originals; you want to completely replace them. IEBGENER can also be used to copy with replace.

If the "User Settings" panel field entitled "Restart assist on continuous mode" is set to Y, this means you are asking the product to track continuous mode jobs over time, and the product will automatically determine whether a job is being restarted, or if it is a new job. A restarted job occurs when the product determines the previous run of the job did not complete successfully. This type of job tracking can only be accomplished if the job JCL is static, that is, the job is not continuously recreated using the product panels. In general, this tracking is intended for JCL setup in a job scheduler, so that a third party operator can more easily restart a failed job without manual intervention. See the conditions for restart in a section below. The manual intervention necessary when this flag is set to N is described next.

If your jobs fails in continuous mode, the following rules apply:

If the job fails after STEP2, restart from whatever step is appropriate after STEP2. Do not restart from the beginning of the job as the continuous mode communication files have already been modified. Also, restarting from the beginning in this case is not necessary, so doing so just causes more overhead in the process. If, for whatever reason, you do restart from the beginning of the job (that is, from STEP1), you must first copy the backup files (DDs UNCREB and EXTFILB) into their associated original files (DDs UNCRE and EXTFILE). If you fail to perform this copy, your restart job will overlay the contents of these backups with the previously updated file information. This means you've permanently lost the contents of these communication files as they existed at the time of start of the originally failing run. Restart becomes more difficult in this case, so be sure to follow this copy procedure whenever restarting from the beginning.
If the job fails during STEP2, the copy procedure from the backups to the original files outlined above must be followed. This presumes the job has failed during program execution, and not, for example, prior to program execution by a JCL error. If you're unsure of where in the process the job has failed. look at the informational messages generated in the job log by the product. If you see this: 'ALAA156I: CMODE FILES UPDATED', then you must copy the backup files (DDs UNCREB and EXTFILB) into their associated original files (DDs UNCRE and EXTFILE). If you fail to perform this copy, your restart job will overlay the contents of these backups with the previously updated file information. This means you've permanently lost the contents of these communication files as they existed at the time of start of the originally failing run. Restart becomes more difficult in this case, so be sure to follow this copy procedure whenever restarting from the beginning.

A final note on this backup process is that GDGs may be used to ensure that you never accidentally lose the ability to restart using continuous mode. For example, if you wish to use GDGs, you can remove the UNCREB and EXTFILB DDs from STEP1, and replace these DDs in STEP2 with the needed GDGs changing disposition from SHARE to (NEW,CATLG,DELETE). In this way, an accidental restart without first copying the backups into the originals would mean a wasted generation, but you could still go back and copy the previous generation of the backup into the original. See IBM document "DFSMS/MVS Using Data Sets" for more information on GDGs.

Conditions for restart assist in continuous mode

If the "User Settings" panel field entitled "Restart assist on continuous mode" is set to Y, this means you are asking the product to track continuous mode jobs over time, and the product will automatically determine whether a job is being restarted, or if it a new job. It does this by tracking the status of the previously executed job within the MODEFILE. So, this process can only work if the MODEFILE is not re-generated between jobs. Re-generation of the MODEFILE occurs when JCL for a job is created via the product panels. Therefore, this process is intended for JCL in a job scheduler, or any saved off JCL for repeated execution.

It is important to understand the conditions that will cause the product to assume a restart, versus a new job run. Restart is considered in effect if the previous job did not end successfully. This would include such things as an abend, or a return code that ceases job execution. For example, assume a job is run that terminates with return code=8. The next submission of this job will be considered a restart operation by the product, and it will automatically restore the backup files into their original counterparts. (This is the purpose of restart assist; it removes the manual intervention of this file restore process allowing an operator to merely resubmit the job without other external actions). Obviously, the situation causing the return code=8 must be resolved first, or it will most likely recur. But the issue here is that the job can then be restarted without further actions.

Conditions that place the job in restart mode are as follows:

any abend, such as a S0C4, or a User abend 55, etc.
a return code which causes job termination in an unsuccessful state. Such return codes are: return code=8 or higher, or return code=4 which is a warning, but the "User Settings" panel field "Continue processing after warnings" is set to N, which causes the job to terminate.

Conditions that place the job in successful mode (that is, no restart) are as follows:

a return code which causes job termination in a successful state. Such return codes are: return code=0,2, or return code=4 which is a warning, but the "User Settings" panel field "Continue processing after warnings" is set to Y, which allows the job to successfully complete with warnings.

Therefore, it is important to understand that although the product tracks these states, if your required/desired actions are not in line with these conditions, manual intervention will still be required. For example, assume you get a return code=8, but do not wish to restart that particular job. The product automatically handles the internal restart mechanism and copies the backup files into their originals, and this isn't your intention. Therefore, you would have to take some action to handle that, such as re-generating the MODEFILE and JCL via the panels. Modifying the "User Settings" field which handles restart assist will not solely handle such a case as the MODEFILE will still need to be re-generated.

It is recommended that this restart assist be used with the full understanding of its implications and conditions, as stated above. If a continuous mode job is run after failure (as defined above) or success (as defined above), and you desire to take a different path than that assumed by the product (as defined above), then some sort of manual intervention will be required depending on your actions. If you do not wish the product to track your jobs this way, simply set the "User Settings" field for restart assist to N.

Note: Restart assist in continuous mode does not depend on the existence of, or lack of, a "RESTART=STEPn" parameter on your job card. Once restart assist is set to Y, it tracks these jobs automatically. But it is still mandatory that you restart on the proper job step. In one-phase mode, a RESTART parameter would only be needed if you have additional steps in the job, for example, if LOAD options are present, because a basic one-phase mode job typically only has one (primary) step. But if running in two-phase mode where there are several job steps, you must restart from the failing step. For example, if the job fails in some step after the step where program ALAGEN1 was executed, and you restart from the step where program ALAGEN1 executes, the files will not be properly copied because the product has determined that step already completed successfully. So it is imperative that you restart from the failing step. If you still wish to restart from the very beginning of the job for whatever reason in two-phase mode, you will have to manually do the copies yourself, as was always the case in prior releases.

Continuous mode file report

The CM files store information about activity that occurred during a run's timeframe, but was not reported because the unit of work had not completed. This information may be of interest and can be report on by entering 'C' in the general report panel's "Output flags" field. This report examines the contents of the CM file at the end of a run and provides some of the details, such as information about the unit of work, the tables involved in the unit of work, the actions performed on each table, and the number of times each action was performed.