# Introduction

"Subscriptions NG" is a Runner connector that creates artifacts (e.g., a PDF document from
a Zuar Portal dashboard) and emails the artifact to one or more recipients according to
a schedule.

Subscriptions NG requires the "Python Job" connector for running artibtrary Python code.
A Runner job built using this Python Job connector uses Playwright to create PDF exports
of Zuar Portal dashboards.

An understanding of the Runner Python Job and the Portal Export Job will be helpful in
undertanding and configurint Subscriptions NG.

Subscriptions are defined in the `subscriptions_ng` database table located on Zuar Portal:
```
create table public.subscriptions_ng
(
    id         uuid                     not null
        primary key,
    user_id    uuid                     not null,
    email      varchar(512)             not null,
    schedule   varchar(8)               not null,
    day_of     integer                  not null,
    send_at    time                     not null,
    tz         varchar(256)             not null,
    source     text                     not null,
    json_data  text                     not null,
    updated_at timestamp with time zone not null,
    created_at timestamp with time zone not null
);
```

The `json_data` field contains additional data that can be used to filter modify the
subscription.

A `subscription_status` table exists on Runner.  It contains the current status of each
subscription in the `subscriptions` table.  The DDL for the table is:

```
create table public.subscription_status_ng (
  id uuid primary key not null,
  last_checked timestamp with time zone,
  last_run timestamp with time zone
);

ALTER TABLE ONLY public.subscription_status_ng
    ADD CONSTRAINT subscription_status_pkey PRIMARY KEY (id);
```

A subscriptions job (run via `job_subscriptions_ng.py`) is scheduled to run regularly
(e.g. hourly) on Runner.  Each time the job is run, it uses the two tables to determine
which, if any, subscriptions should be run.  If a subscription is due to run, the
specifics of the subscription are obtained from the `subscriptions` table.

Creation of the file to be exported is implemented by the `job_portal_export.py` -- with
the data specified by the subscription record.  As `job_subscriptions_ng.py` processes a
subscription, it uses the underlying implementation of `job_portal_export.py` to create and
download the export file to Runner.

Once the export file is downloaded, `job_subscriptions.py` creates the email message,
attaches the export file, and sends the email to the recipient.  Finally, the
`subscription_status_ng` table is updated to reflect that the subscription has been
processed.

## Quickstart

Configuring Subscriptions NG isn't difficult, but it does require a fair amount of
effort and many parameters.  We'll simplify this quickstart by breaking it into two
distincts parts.  The first is to create a working standalone Portal Export job.  This
will confirm that your export parameters are correct and that you are getting the PDF
document you expect.  The second is to create a Subscriptions NG job that uses the
Portal Export job to create attachments for outgoing subscriptions.

### Create a Standalone Portal Export Job

Let's create our standalone Portal Export job.

* [Portal Export Job configuration reference](schemas/subscriptions_ng/jobs/job_portal_export/index.html)

1. Login to Zuar Portal as an admin user. 

1. Use the Zuar Portal OpenAPI documentation at
   `/auth/docs#/default/create_api_key_api_keys_post`
   to create an API Key to be used when impersonating a Zuar Portal user
   and exporting a dashboard as a PDF document.
   We'll assume the API_KEY is `TWbMjCSa8_uPEW5Cijslielk8984AiYzRDpYweDrUo0`.

1. Create a new generic job using one of the job configs below.

   1. For use on a production Runner:
   
	  ```
	  {
		export: {
		  source_url: https://portal-automation.zuarbase.net
		  source_credentials: TWbMjCSa8_uPEW5CijRabaSwPVHCAiYzRDpYweDrUo0
		  file_path: file-exported-from-portal.pdf
		}
		subscription: {
		  source: /p/runner-automation
		  user_id: e8041bb9-00dc-46f1-8397-ee0ecf312fa3
		  json_data: {
			export_type: pdf
			window_size: 1280x1024
		  }
		}
		python_job_config: {
		  venv_dir: /var/mitto/data/venvs/portal_export_job/venv
		  requirements_file_path: /var/mitto/data/venvs/portal_export_job/requirements.txt
		  requirements_body: /var/mitto/plugin/subscriptions_ng/export/requirements.txt
		  python_file_path: /var/mitto/data/py_files/portal_export_job/job_file.py
		  python_file_body: /var/mitto/plugin/subscriptions_ng/export/portal_export.py
		  env_vars: {
		  }
		}
	  }
	  ```

   1. For use locally with `mitto-dev-runtime`:

	  ```
	  {
		  "export": {
			  "source_url": "https://custom-portal-test.zuarbase.net",
			  "source_credentials": "TWbMjCSa8_uPEW5Cijslielk8984AiYzRDpYweDrUo0",
			  "file_path": "daily-operating-metrics.pdf"
		  },
		  "subscription": {
			  "source": "/p/daily-operating-metrics?GLOBAL_START_DATE:range=Yesterday",
			  "user_id": "3ee5e4fb-5b2b-4c52-a3b6-f54ea1d24395",
			  "json_data": {
				  "export_type": "pdf",
				  "window_size": "1280x1024"
			  },
		  },
		  "python_job_config": {
			  "venv_dir": "/var/mitto/data/venvs/portal_export_job/venv",
			  "requirements_file_path": "/var/mitto/data/venvs/portal_export_job/requirements.txt",
			  "requirements_body": "/app/plugins/mitto-plugin-subscriptions-ng/subscriptions_ng/export/requirements.txt",
			  "python_file_path": "/var/mitto/data/py_files/portal_export_job/job_file.py",
			  "python_file_body": "/app/plugins/mitto-plugin-subscriptions-ng/subscriptions_ng/export/portal_export.py",
			  "env_vars": {}
		  }
	  }
	  ```

1. Edit the job config:
   1. `export` section
      1. Set `source_url` to the base URL of the Poral.
      1. Set `source_credentials` to the Portal's API_KEY created in step 1.
      1. `file_path` should be a fully qualified path to the file that the export will be
         placed in.
   1. `subscription` section
      1. Set `source` to the path to the dashboard to be exported.  This path should
         begin with `/`.  The path should not include the base  URL provided in `source_url`.
      1. Set `user_id` to the Portal `user_id` to impersonate when performing the export.
      1. Set `export_type` to `pdf`.
      1. Set `window_size` to the desired size.
   1. Set the job type to `job_portal_export`.

1. Run the job.

1. The export should be in the location you specified in `file_path`.
   

### Create a Subscription and a Subscriptions NG Job

Now that we have a Portal Export job that successfully creates PDF exports from a Zuar
Portal, let's finish up by creating a subscription in Zuar Portal and configuring a
Subscriptions NG job in Runner to process subscriptions every hour.  The Subscriptions
NG job will:

1. Check the `subscriprtions_ng` (on Zuar Portal) `subscription_status_ng` (on Zuar
   Runner) tables every hour, looking for subscriptions that are due to run at that
   time.
   
1. When it finds a subscription to run, it will:
   1. Use the subscription info to modify the Portal Export job created in the last
      section so that it creates a PDF with the desired content.
   1. Send an email to the subscription's recipients and attach the exported PDF.

#### Create a Subscription

* ['subscriptions_ng' table reference](schemas/subscriptions_ng/subscriptions/config/index.html)

We'll assume you have created a subscription in Zuar Portal with the following details:

* `user_id` : `571048a0-5de3-42a0-a910-380890c6de03`
* `email` : `Steve@greif.com`
* `schedule` : `DAILY`
* `day_of` : `0`
* `send_at` : `08:00`
* `tz` : `America/Chicago`
* `source` : `/p/daily-operating-metrics?GLOBAL_START_DATE:range=Yesterday`
* `json_data` : `{"export_type": "pdf", "window_size": "1920x945"}`

#### Create a Subscriptions NG Job

* [Subscriptions NG Job configuration reference](schemas/subscriptions_ng/jobs/job_subscriptions_ng/index.html)

1. Use Runner's Generic Job wizard to create a new Subscriptions NG job using the following
   job configuration:
   ```
   {
	 subscriptions: {
	   subscriptions_input: {
		 dbo: postgresql://<user>:<pass>@custom-portal-test.zuarbase.net/portal
		 schema_name: public
		 table_name: subscriptions_ng
	   }
	   subscription_status_input: {
		 dbo: postgresql://<user>:<pass>@db/mitto
		 schema_name: public
		 table_name: subscription_status_ng
	   }
	 }
	 export: {
	   source_url: https://custom-portal-test.zuarbase.net
	   source_credentials: TWbMjCSa8_uPEW5Cijslielk8984AiYzRDpYweDrUo0
	   file_path: daily-operating-metrics.pdf
	   job_type: portal_export
	   job_config_path: /var/mitto/data/example_portal_export_job_config.json
	 }
	 mail: {
	   server: in-v3.mailjet.com
	   port: 587
	   require_tls: true
	   credentials: {
		 username: 83acd7b55a240a02f79abdffccf5474a
		 password: 6cfdae93dabf6835453326cfdaed2b7d
	   }
	   timeout: 30
	   mail_subject: Daily Operating Metrics Subscription
	   mail_from: portal-admin-3425@zuar.net
	   text: Attached is your Daily Operating Metrics dashboard
	   html: <html>Attached is your Daily Operating Metrics dashboard</html>
	 }
   }
   ```

1. Configure `mail` for your SMTP server of choice.

1. Change the job type to `subscriptions_ng`.

1. Schedule the job to run hourly.

1. When the job runs, it will look at each subscription and its status.
   When a subscription is due to run, the following happens:
   
   1. Values in the `subscriptions_ng` record are used to create a temporary Portal
      Export job configuration for the specific export to be created.
   1. A Portal Export job is run using the temporary configuration.
   1. Values in the `subscriptions_ng` record are used to create an email, attach the
      export to the email, and send the email to the recipient(s).