Scheduler

Scheduler combines multiple sequential jobs into one logical unit of work.

Scheduler 2018-03-14T14:03:10+00:00

What Scheduler Does?

 It enables application schedulers to build a pipeline or a workflow from a sequence of jobs developed in designer and provides greater control over the jobs and makes it easier to repeat the jobs at predetermined intervals.

The Scheduler provides the following features for a quick and easy workflow implementation.

Create and Monitor

Easily create jobs, collaborate, schedule and manage batch business services. Manage workflows using a simple drag ­and ­drop interface with a single point of control for all workloads. The tool provides a real­time monitoring interface with colour indicators.

Conditional flow

The scheduler provides a set of components to decide whether to run a job or not based on the up stream processes outcome.

big data engineering data lake temenos scheduler components

Parallel execution

The tool not only combines a set of jobs but also provides a component to run the independent jobs in parallel.

Notification

This component ensures that the right people are automatically notified when something goes wrong or after successful completion of the jobs.

Configure the execution mode

The module allows users to pick the data pipe line to be executed on. User can orchestrate and pipeline data flows on YARN, Apache Spark cluster or Spark local.

Memory Allocation

No matter you run on YARN, Spark or local each data pipeline in the scheduler flow can be assigned to run on a user defined memory allocated by the user.