Monday, January 26, 2015

"Start Of Day" in Tivoli Workload Scheduler

While writing the article about Scheduling FINAL I've realized the need to write a specific article about the meaning of the "Start Of Day" and how it works.

Most of the customers are still running TWS using Start Of Day set to 0600 (the 6 in the morning), or they are scheduling FINAL 1 minute before the Start Of Day. This is no more required since TWS 8.3, but changing this setting in an existing production environment is not easy, and just since 8.6 the default value for a fresh installation has been changed to 0000.


"Start Of Days" setting is stored with optman in the database, with the keys startOfDay and its value is set in the format HHMM.

The meaning and usage of "Start Of Days" has slightly changed during the history of the product and since 8.3 FP1 depends on the setting of the enLegacyStartOfDayEvaluation optman option.
In any case since 8.3 this is no more related to when the plan is created or extended (when FINAL runs), TWS is able to generate correctly the plan even if a day is split in 2 different plans, and is able to create dependencies between jobs created in the 2 plans.

When enLegacyStartOfDayEvaluation is NO, the default setting for fresh installations, the meaning of Start Of Day is the time of the day we consider as the start of the day.

If the Start Of Day is 0000 (midnight) the day starts at midnight as the calendar day, this is the default and the more natural way to consider a day. Any time specified in TWS definition is just the time you have set, if you specify that a job have to run every Monday at 2 AM, it will run on Monday at 2 AM.



The question at this point is: why I have to consider a different behavior?
Let's start describing what's the behavior with a different Start of Day.
A value different from 0000 for Start Of Day moves the start of the day at that time, e.g. if we set Start Of Day at 06:00, TWS will consider the day as starting at 6 AM and completing right before 6 AM the next day, so Monday for TWS is now the time windows from Monday at 6 AM to Tuesday at 5:59:59 AM.

As consequence, if we specify that a job has to run on Monday at 4 AM, the job will start when on the wall clock it's Tuesday at 4AM. That's probably the hardest and less intuitive part of using a different value for Start Of Day.
On the other side this can be useful if you consider the workload run in the night as closure of the previous day and for example you consider the night between Sunday and Monday as part of Sunday. This can be important especially when you want to apply free day calendar to decide if that night is free or working time.
So the usage of Start Of Day still has some area of application, but the drawback is that every time you have to set a time between midnight and the start of day you have to think what's day it is.

Above I was saying that Start Of Day behavior depends on enLegacyStartOfDayEvaluation option, but I have not yet specified how.
When enLegacyStartOfDayEvaluation is set to NO, the same behavior applies to any timezone, regardless of what's the master and the agent timezone, if you are in a single timezone or enLegacyStartOfDayEvaluation is NO the behavior is the one described above, leave enLegacyStartOfDayEvaluation set to NO and don't care about it.

The purpose of enLegacyStartOfDayEvaluation set to YES is to assure backward compatibility if you are coming from a TWS 8.2.x or previous release, you use multiple timezones and you have enTimeZone set to YES. This behavior is coming from the limitations present in TWS until release 8.2.1 where the product was only able to create plans of 24 hour where it was including all workload scheduled on a specific date.
But for every specific date, for 49 hours there is a place on the planet where the current date is that date. Consider for example Jan 1st and how long it takes from when Jan 1st starts in New Zealand and when if ends in the Hawaii, and there are some small island in the pacific sea that makes it even longer.
The legacy behavior that was until 8.2.1 was introduced to fit a date in the plan, considering a day (e.g. Monday) as the time window from Start Of Day on the master timezone, to the Start Of Day of the next day on the master timezone.
In the picture for example, we have Start Of Day set to 6 AM, the master in New York and the agent in Los Angeles. The legacy Start Of Day behavior consider 3 AM as the Start Of Day for times in the Los Angeles timezone. This makes definitions very complex since the Start Of Days varies in every timezone, this is even more complex if you have timezones with different DST rules making the Start Of Day different during the year.
That's the reason I suggest to not use enLegacyStartOfDayEvaluation and leave it set to NO, but of course this requires to change the scheduling definitions if coming from releases before 8.3 with multiple timezones.

The need to include completely a date in a plan and the inability to have dependencies between plan, is also the reason that until 8.2 the FINAL had to run 1 minute before Start Of Day, as said above these limitation as been removed since 8.3 and you can now run FINAL when you want as described in the Scheduling FINAL article.

If you like this article and you find it useful, please share it on social media so other people may take advantage of it.

9 comments:

  1. Hi,

    Sorry to bother you.

    First of all I don't know much TWS (I am a SAP consultant).
    I was wondering one thing about TWS and I would appreciate if you could answer my interrogation.

    My client, a french company, is using TWS to manage all the jobs of our SAP systems.

    We have 2 environnements containing several SAP systems , both environnements on the same IBM P595 server (installed on LPARs) : pre-production and production environnements. I think they have one TWS server managing both.

    So to prepare our night workload scheduled they develop the plan on DEV environnement then put it on the pre-production one to test it and then push it to the production.
    As they don't want to make any change to the plan, the pre-production plan and the production plan are both working on the P595 at the same time every night,
    consuming all the P595 CPUs making the workload on production taking more time than it should take.
    My question :
    Is there any option on TWS to have an exact same plan for the pre-production and the production environnements (SAP) but to make only the production one starting at 8pm and the pre-production one starting at 8am ?

    Many thanks and best regards for your help.

    ReplyDelete
    Replies
    1. If I understood correctly your question you have the same definitions on the 2 environments (prod and pre-prod) generating the same plan and running the same jobs at the same time and you would like to shift the time on the pre-prod environment to distribute the workload on different time windows.

      The best way I see to do that is to use different time zones on the pre-production environment.
      If your production environment is in French, you could setup the pre-prod as running in New Zealand for example.
      If you are not currently specifying timezones on workstation and job streams, you will just need to set the new timezone on the master (and backup master) workstations.

      Delete
  2. Hi Franco,

    I am new to TWS and i want to know job scheduling in tws in Mainframes. Could you please suggest links or any tws material to read.

    ReplyDelete
    Replies
    1. Hi,
      there are several sources of information, the main one are:
      - the official manual: https://goo.gl/ZHwKXt
      - local user groups, especially the ASAP conference on May ( http://twsuser.org/ )
      - IBM education (online or in classroom) : https://goo.gl/hBPEYq

      Delete
  3. Hi Franco,

    I have a Small Question,

    what are the Major Differences Between TWS Versions 8.2 ,8.4,9.1 and 9.3 and

    Is there any link where I can check for TWS 9.3 Online,

    Apart from Websphere
    Can You Please Suggest

    ReplyDelete
    Replies
    1. The question is small, but the answer should be huge, you are asking to summarize about 10 years of new features in a small answer, consider that we are currently releasing new functionalities every 6 months via new releases of functional fixpacks.

      For the enhancements of the last years your can visit this link: https://goo.gl/hjUVIU there you can also go back to previous releases changing the release in the dropbox.

      Just to summarize the main enhancements since 8.2, these are the major improvements:
      - Storage of workload definition on RDBMS and referential integrity
      - Improved planning capabilities that had surpassed many limits present in 8.2, e.g. cross-day dependencies, multiple instances, advanced runcycles.
      - Forecast and trial plans
      - J2EE and REST APIs
      - Reporting
      - Web User Interface
      - Event Management for triggering, notifications and automation
      - Dynamic Scheduling
      - Conditional Dependencies
      - Workload Service Assurance and Critical path management
      - Analytics and improved job duration forecast
      - What-if
      - Min and Max duration for jobs
      - Advanced Job plugins to schedule without using scripts in many situations
      - Version control
      - Centralized auditing
      - Variable passing
      - New role based security model
      - Zero downtime agent installation
      - Agent installation remotely from the UI
      - NOP jobs
      and many others

      Delete
  4. Hi Franco,

    We have UTC time zone at TWS master OS level, In TWS Master .profile America/newyork Tz has been given,

    The Local timezone of the FTA under this master is UTC, but in CPU definition , America/newyork TZ has been given

    Startofday of this master is 1300 America/newyork

    Final scheduled on 1259 America/newyork

    For this EnlegacyStartDayOfEvaluation has been set to yes.

    With this setting we upgraded TWS master from TWS 8.6 to TWS 9.3 , But after the upgrade all the job got stuck in ready state and start time has been changed 5 hours before

    We tried release the jobs manually , but job still stcuked in ready it didnt moved to exec state.. what is the possible cause of the issue?, is this due to TZ mismatch between startofday and other FTA's defined under this master?

    Finally we restored the upgrade from TWS 9.3 to TWS 8.6

    What is your recommendation steps needs to be taken before 9.3 upgrade for this kind of environment setting?

    Note: we have done similar master upgrade to 9.3 in development environment, that was successful, so its very confused how to proceed with production upgrade

    Waiting for your suggestion

    ReplyDelete
    Replies
    1. My recommendation is to migrate to enlegacyStartDayOfEvaluation=no.
      The current configuration is very fragile because it relies on every TWS process to run with TZ environment variable correctly set.
      You can try to shut down any process, including netman, assure TZ is set correctly and then restart TWS.
      Consider that .profile is loaded only at logon, and not when each process starts.

      Delete