How to manage chained jobs with dependencies ?
Working in IT application development world, we always think that we have to solve the problems ourselves. Having the power of development ability forces us to solve the management problems with development of local applications.
Sometimes it is nice to have the key of all the locks, but sometimes it comes with too much overhead when we try to solve generic problems with local approach.
I am sure that all of us, as developers, sometimes need to execute some kind of applications with sequential way or with some result oriented or time based dependencies. In the world of IT automation, batch scheduling or with the todays common name Workload Automation, this job management pattern is in the center of architecture of applications that are solving the problem.
By the nature of the automation world, you have some kind of processes or jobs that may run together concurrently or that have to dependently chained to each other and should be executed as some time or result dependent job chains.
Nearly five years ago, after 10 years of server side java development, I really dived in to this job scheduling world with little operational background. It was really hard to get in to the dynamics of this new world. Problem looks like as it is very simple. It has taken sometime but at the and realized that it is not true !
After so much work and discussion, we accepted that it is worth to work on scheduling solution and produced Tlos Lite as our first job scheduling product.
Let us come to our question, how to manage chained jobs with dependencies ?
If we have group of jobs that has to run according to some kind of dependency definitions with different execution intervals, the answer is ¨use a job scheduler¨.
Dependency management is the key concept of the job scheduling. The ideal scheduler should provide easy configurable dependency definition between the related jobs. Scheduling application executes the jobs and waits for the started job to complete its duty and gets the process result code provided to the operating system and decide what to do according to the definition of relation. If it is successfully completed the scheduler may start the following job or jobs and so on.
These kind of jobs may run several times a day or on the specific days of week or may be defined on more complex execution calendars.
As I mentioned on the first paragraph, this can be done with some development with any programming languages for some special cases but it is really better to use the solution that is ready on the market and specifically focused on the business of scheduling. And leave your teams to work on their real problems focus on their own businesses.
Serkan Taş
Tlos Schedulers
Sometimes it is nice to have the key of all the locks, but sometimes it comes with too much overhead when we try to solve generic problems with local approach.
I am sure that all of us, as developers, sometimes need to execute some kind of applications with sequential way or with some result oriented or time based dependencies. In the world of IT automation, batch scheduling or with the todays common name Workload Automation, this job management pattern is in the center of architecture of applications that are solving the problem.
By the nature of the automation world, you have some kind of processes or jobs that may run together concurrently or that have to dependently chained to each other and should be executed as some time or result dependent job chains.
Nearly five years ago, after 10 years of server side java development, I really dived in to this job scheduling world with little operational background. It was really hard to get in to the dynamics of this new world. Problem looks like as it is very simple. It has taken sometime but at the and realized that it is not true !
After so much work and discussion, we accepted that it is worth to work on scheduling solution and produced Tlos Lite as our first job scheduling product.
Let us come to our question, how to manage chained jobs with dependencies ?
If we have group of jobs that has to run according to some kind of dependency definitions with different execution intervals, the answer is ¨use a job scheduler¨.
Dependency management is the key concept of the job scheduling. The ideal scheduler should provide easy configurable dependency definition between the related jobs. Scheduling application executes the jobs and waits for the started job to complete its duty and gets the process result code provided to the operating system and decide what to do according to the definition of relation. If it is successfully completed the scheduler may start the following job or jobs and so on.
These kind of jobs may run several times a day or on the specific days of week or may be defined on more complex execution calendars.
As I mentioned on the first paragraph, this can be done with some development with any programming languages for some special cases but it is really better to use the solution that is ready on the market and specifically focused on the business of scheduling. And leave your teams to work on their real problems focus on their own businesses.
Serkan Taş
Tlos Schedulers
Comments
Post a Comment