Tuesday, December 2, 2014

Scheduling FINAL (Best Practices)

One of the most important process when setting a TWSd environment is when and how to extend the plan and there are several options in optman that are related to this process.

I already presented this topic during the ASAP conference in 2011, but due to the important, and many users that are not yet familiar with that, I think this is a good argument to start the blog.


First decision to make is when to run the FINAL job stream to extend the plan. The decision have to take in consideration several elements like:

  • At what time most of the batch workload is complete and the plan extension have the minimum impact?
  • When are the administrators available in case there is any issue with the plan extension?
  • Are the administrators available in the weekends and holidays?
  • How many hours of buffer I want to keep in order to be able to fix extension problems without impacting the productions?

The result of this decision could be for example that I want to schedule FINAL only on working days, running at 7 AM and with 5 hours of buffer.
That means we need a FINAL scheduled at 7 AM on working days that extends the plan until the 12 PM of the next working day.



To schedule FINAL and FINALPOSTREPORTS jobstreams we just need to change the runcycle, from TDWC just change the runcycle and select workdays, from composer use this ICalendar for the runcycle:
FREQ=DAILY;INTERVAL=1;BYWORKDAY

To extend the plan until the next working day we need to calculate the next end of the plan, we will use datecalc, and since we calculates working days, we need calendars in the symphony file setting ignoreCals=no in optman.

At this point we can change our Makeplan job definition to use datecalc, change the task string in this way:
/opt/IBM/TWA/TWS/MakePlan -to `datecalc today+1 workday pic MM/DD/YYYY` 1200 timezone America/Chicago
Of course you need to use the timezone you prefer instead of America Chicago.
This command calculates then next working day and the run MakePlan to create the new plan up to 12 PM of that day.

We will also have part of the plan (from 7 AM to 12 PM) that is usually executed after the next JnextPlan, this is our buffer in case of issues, but we need to assure that this is not removed from the plan. For this reason, and to allow to have a long plan over freedays, we need to assure that:

  • enCarryForward=ALL in optman, this assures that all job streams are carry forwarded and we don't loose instances scheduled in our buffer hours.
  • Every job stream without a follows dependency should be time dependent, we cannot let them run right after JnextPlan, otherwise during weekend plans we will have Friday, Saturday and Sunday instances running together on Friday morning when the multi-day plan is generated. 
Finally, you may have noticed that I've not talked about the value of startOfDay to set in optman.
This is not a case and the reason is that this actually not related to how we run FINAL, and this is true since TWS 8.3. I'll dedicate a different post on how startOfDay works, for now I'll just say that my suggestion is to set it to 0000, that makes definitions more intuitive.

If you like this article and you find it useful, please share it on social media so other people may take advantage of it.

2 comments:

  1. Hi Franco,
    Might be a simple query, but need your help.

    In our environment, we are thinking of changing "SwitchPlan" run time, do we need to change the "StartofDay" time also.

    Can "startofday" be at 1700 Hrs while "SwitchPlan" runs at "1600" hours.

    What would be the impact ?

    Thanks in advance.

    ReplyDelete
    Replies
    1. Simple answer: you can leave StartOfDay unchanged.
      The impact is that if you have jobs or jobstreams without follows or time dependencies, these jobs can now start at 16 instead at 17. To keep them at 17 you need to explicitly set an AT dependency at 17 for those jobstreams to avoid they start before that time. You can also consider to run only MakePlan at 16 and leaver SwitchPlan at 17.

      The more complete answer is that since TWS 8.3 StartOfDay, the end of the plan and the time you run Final can be at different times. E.g. You can have StartOfDay at 0000, run Final at 16 and extend the plan to 22 of the next day.
      See also this article about StartOfDay: http://runtws.blogspot.it/2015/01/startofday.html

      Delete