Introduction to Jobs¶
A job in Zuar Runner represents a unit of work. There are a variety of different types of jobs, most of which can be found on the Add Jobs page within the UI.
Each job performs a specific function or set of functions. Jobs can be run, scheduled, sequenced, tagged, and searched for.
The most common job types are IO, which handle the input and output of data, and Sequences which organize, order, and run jobs in series.
Timeouts¶
A timeout can be set for your job/sequence which will cause the job to be killed if it exceeds a certain length of time. This can be useful for any job type once an average runtime has been established. Setting the timeout around or shortly after the average runtime can help safeguard the job from issues that may cause increased runtime. Pairing this with a failure notification can help monitor the health of jobs.
To set a timeout, simply click the pencil in the TIMEOUT section of the job/sequence page. Click the carat on the modal which appears and select a time from the dropdown. Available times include Never, which is the default, and Custom, which allows a time to be entered in minutes.
Notifications¶
Zuar Runner can send notifications on the status of a job/sequence.
Navigate to Settings on the left hand panel. Learn more about the Settings page.
Click on Configure.
The Messaging section is where you set your notifications.
State: This is where you can either enable or disable notifications.
Recipient(s): Enter in the email address of the person(s) that will be receiving these notifications. Separate different email addresses with a comma. Any email listed in here will receive a notification.
Default level: Choose either None, All, or Errors Only for when you will be notified. With None, you will not be notified at all for any status of your job. All is the exact opposite. Choosing this will notify you of any status. Errors Only will notify you when there is an error in your job run. This is turned off by default so you only get notifications when you specify it on jobs.
Templates: Shows how your notification message is encoded. This is an advanced feature. Do not modify this unless specified.
The section below the Messaging section is the Navigation section. Learn more about custom navigation.
Webhooks¶
Zuar Runner webhooks come in two flavors: Incoming Webhooks and Outgoing Webhooks. Incoming Webhooks are new as of Runner version 3.3.0 and can serve as endpoints to be used by a webhook initiated from an external system to start, stop. Outgoing webhooks have been around for a while and can be used to send data to any URL when a job starts, completes, succeeds, or fails. Outgoing webhooks can also be used to start a job when any of these things occur.
Both types of Webhooks can be added to any individual job or sequence in Zuar Runner.
Add an Incoming Webhook to a Job¶
Click the plus button in the Incoming Webhooks section of any job. The Runner will create an incoming webhook which is valid for that Runner. This is an example from stage-mitto3.zuarbases.net.
Once the incoming webhook has been created, simply click the copy button to the right of the URL and use that URL in your remote webhook or CURL.
Add an Outgoing Webhook to a Job¶
Click the edit button in the Outoing Webhooks section of any job.
Fill in the form.
URL¶
The URL to send the GET or POST request to.
Event¶
Outgoing webhooks can be initiated from four different job event types:
JOB_START - initiate the webhook when the job starts
JOB_COMPLETE - initiate the webhook when the job completes
JOB_SUCCESS - initiate the webhook if the job succeeds
JOB_FAILURE - initiate the webhook if the job fails
Method¶
GET - request data from the URL
POST - send data to the URL
Typically, you want to send data to another application via a POST.
Content Type¶
This is the format of the body of the request.
application/json
application/x-www-form-urlencoded
text/plain
The choice here depends on the data format the URL is expecting.
Enable¶
Check to enable this outgoing webhook or uncheck to disable it. The webhook will be enabled by default.
Test¶
You can test whether the webhook is functional by clicking Test.
Editing an Existing Outgoing Webhook¶
Click the edit icon next to any existing outgoing webhook to edit the webhook.
Multiple Outgoing Webhooks on the Same Job¶
Multiple outgoing webhooks can be added to any job in Zuar Runner, enabling complex use cases and conditional logic.
Outgoing Webhooks Data¶
Data from the Zuar Runner API can be sent in the body of a webhook.
You can see the available data in the Zuar Runner API endpoints:
Endpoint: {runner_url}/api/job/{job_id}
Endpoint: {runner_url}/api/system
Simple Examples:¶
${job['title']}translates to “Webhooks!”${system['fqdn']}translates to “stage-mitto3.zuarbase.net”
Complete Example:¶
This example sends text to a Slack channel.
{
"text": "Job Complete!: ${system['fqdn']} - ${job['title']}\nhttps://${system['fqdn']}/#!/job/${job['id']}"
}
This outgoing webhook body translates to:
Job Complete!: stage-mitto3.zuarbase.net - Webhooks!
https://stage-mitto3.zuarbase.net/#!/job/12984
Outgoing Webhooks that Interact with Zuar Runner¶
You can create outgoing webhooks that interact directly with Zuar Runner itself.
For example, one job’s outgoing webhook can start another job:
Notice the URL:
http://webapp:7127/v2/jobs/<job_id>/:actions
And the body:
{
"action": "START"
}
Outgoing Webhooks that Interact with Microsoft Teams¶
The previous examples have utilized Slack as the receiving application. Outgoing webhooks can be sent to most chat application. You can create outgoing webhooks that interact directly with Microsoft Teams .
The URL will be the outgoing webhook URL obtained from the Teams Channel configuration.
Use that URL in the Zuar Runner outgoing webhook config:
The body of the notification should be structured like the following example in order for Microsoft Teams to accept the alert:
{
"text": "Job Failed!: ${system['fqdn']} - ${job['title']}\nhttps://${system['fqdn']}/#!/job/${job['id']}"
}
Scheduling¶
Every Zuar Runner job can be scheduled to run at specific times.
On any individual job page, click on the edit button under SCHEDULE to open a modal and set a schedule.
Jobs can be scheduled to to run daily at the specified time, hourly on the specified minute (00, 15, 30, 45), end of a specific time (week, month, quarter), as soon as it finishes (on the next minute) AKA continuously, or to run on any custom schedule using CRON logic. To test different CRON schedules, use this helpful tool: https://crontab.guru/
For example, run the job at 12:00 AM every Tuesday.
Zuar Runner will attempt to translate your custom schedule on the job page.
Jobs are not scheduled by default when they are created.
After you have made your selection, click Save. Congratulations! You have just scheduled your job.
Zuar Runner Scheduling Best Practices¶
Set job schedules that make sense both from a source system perspective (e.g. consider API limits, etc.) and end user needs (e.g. do your users really need data refreshed continuously?).
Sequences¶
Generally speaking, you should schedule sequences of jobs instead of individual jobs. This helps you to ensure that your jobs run in the correct order and to see the whole sequence more clearly.
Once a job/sequence is added to a sequence, it will appear in the Parent Sequences section on the job/sequence page.
Tags¶
Tags are short keywords or keyword phrases associated with jobs. They help organize jobs in the Zuar Runner UI.
Jobs can have zero, one, or multiple tags.
There are several benefits for tagging jobs in Zuar Runner.
Organization with Tags¶
Clicking on a tag shows a page with all jobs with that tag.
For example, clicking on the tag curl navigates to a page ending in
/#!/tag/curl:
This /#!/tag/{tag} page can be added a custom navigation item:
Learn more about adding custom navigation in Zuar Runner.
The Stage is a specific example
of this tag and page interaction. Any jobs tagged with stage end up
on the Zuar Runner’s Stage.
How to Add a Tag to a Job¶
Tags can be added to jobs in a number of ways:
On an individual job’s page
On certain pages where there is a list of jobs (e.g. jobs, stage, any “tags” page)