Distributed Order Processing

A JobScheduler cluster can be used to distribute orders for processing on more than one node. This feature can be used for load balancing - reducing processing times by using more hardware to process an order.

All the JobSchedulers in a cluster indicate their availability by sending out heartbeats (to the database). At the same time they check the availability of all the other JobSchedulers in the cluster. Should a JobScheduler determine that the heartbeat of another JobScheduler has been missing for a longer period of time (approx. 1-2 minutes), then it takes over the processing of the orders started by the missing JobScheduler.

The conditions under which a JobScheduler cluster can be used for distributed orders are schematically represented on the following diagram and described in the next section.

Distributed orders can be processed from different JobSchedulers on their way along a job chain. The handing over of the processing of an order to another job node on another JobScheduler is schematically illustrated in the following diagram:

The Requirement for using Distributed Orders

• All the JobSchedulers in the cluster must use the same database.
• The JobSchedulers must all use the same configuration file or an exact copy.
• All the JobSchedulers must be started with the same JobScheduler ID.
• The JobSchedulers must all be started with the option.
• All the JobSchedulers must have access to any resources required by jobs run on a distributed job chain - for example, monitored directories. Directories which are used by all the JobSchedulers must be mounted (or linked) on the same path on all systems.

Starting a JobScheduler Cluster for Distributed Orders

The JobSchedulers making up the cluster can be started in any order. Any JobScheduler belonging to the cluster can be removed or added as required whilst the cluster is in operation. When a JobScheduler is removed from a cluster it should, where possible, be stopped (terminated) normally, to allow any orders which are being processed at the time to be completed.

Generation of Distributed Orders

Distributed Orders can either be generated by the command or by directory monitoring. A persistent order can not be used as a distributed order.

Distributed Orders Using add_order

Either the eAPI command or the iAPI function can be used. Neither the JobScheduler to which the command is sent to nor the JobScheduler on which the command is carried out is important - the order will be made available to the whole cluster for processing.

Distributed File Orders

Distributed File Orders are configured in a distributed job chain using . Each JobScheduler in the cluster monitors the relevant directory or directories and can create file orders. A file order can be processed by a different JobScheduler to the one that created the order.

Stand-Alone Jobs with Distributed JobSchedulers

Independent jobs which do not process orders are processed in a JobScheduler cluster by the JobSchedulers with which they were configured.

Load balancing methods

The method used for load balancing can be set using the scheduler.order.distributed.balanced global parameter:

• Overload balancing:

  With this method, an additional JobScheduler of the cluster takes over the execution of a distributed order when the JobScheduler initially processing the order is working at maximum capacity - i.e. the allowed number of processes reaches a maximum value. This is the standard balancing method used by the JobScheduler.

  see also and .

  
  
• Equal distribution:

  With this method, distributed order tasks are allocated equally to the JobSchedulers in a cluster.