Deploy Custom Webhooks to Launch Workflows#

In this example, we use Union Serving to deploy a custom webhook that can trigger any of your registered workflows. We will learn how to use a simple cURL command with authentication to easily start your workflows!

Run on Union BYOC

Once you have a Union account, install union:

pip install union

Export the following environment variable to build and push images to your own container registry:

# replace with your registry name
export IMAGE_SPEC_REGISTRY="<your-container-registry>"

Then run the following commands to run the workflow:

git clone https://github.com/unionai/unionai-examples
cd unionai-examples/tutorials/serving_webhook
union create secret --name WEBHOOK_API_KEY
union create api-key admin --name admin-union-api-key
# Copy the UNION_API_KEY and create a secret with that value
union create secret --name MY_UNION_API_KEY
union deploy apps app.py fastapi-webhook

The source code for this tutorial can be found here .

Defining the Application Configuration#

First, we define the image spec for the runtime image. We will use fastapi for defining the serving API.

import os
from union import ImageSpec, Resources, Secret
from union.app import App

image_spec = ImageSpec(
    name="webhook-serving",
    packages=["union-runtime>=0.1.11", "fastapi[standard]==0.115.11", "union>=0.1.150"],
    registry=os.getenv("IMAGE_SPEC_REGISTRY"),
)

We define the configuration for for the webhook. It includes ./main.py which defines the FastAPI application. There are also two secrets WEBHOOK_API_KEY and MY_UNION_API_KEY. The WEBHOOK_API_KEY is used by the FastAPI app to authenticate the webhook and MY_UNION_API_KEY is used to authenticate UnionRemote with Union.

app = App(
    name="fastapi-webhook",
    container_image=image_spec,
    limits=Resources(cpu="1", mem="1Gi"),
    port=8080,
    include=["./main.py"],
    args="fastapi run --port 8080",
    secrets=[
        Secret(key="WEBHOOK_API_KEY", env_var="WEBHOOK_API_KEY"),
        Secret(key="MY_UNION_API_KEY", env_var="UNION_API_KEY"),
    ],
    requires_auth=False,
)

With requires_auth=False, the endpoint can be reached without going through Union’s authentication, which is okay since we are rolling our own WEBHOOK_API_KEY. Before we can deploy the app, we create the secrets required by the application:

union create secret --name WEBHOOK_API_KEY

For this example, we’ll assume that WEBHOOK_API_KEY is defined in your shell. Next, to create the MY_UNION_API_KEY secret, we need to first create a admin api-key:

union create api-key admin --name admin-union-api-key

You will see a export UNION_API_KEY=<api-key>, copy the api key and create a secret with it:

union create secret --name MY_UNION_API_KEY

Finally, you can now deploy the application:

union deploy apps app.py fastapi-webhook

Deploying the application will stream the status:

Image ghcr.io/thomasjpfan/webhook-serving:KXwIrIyoU_Decb0wgPy23A found. Skip building.
✨ Deploying Application: fastapi-webhook
🔎 Console URL: https://<union-tenant>/console/projects/thomasjpfan/domains/development/apps/fastapi-webhook
[Status] Pending: App is pending deployment
[Status] Pending: RevisionMissing: Configuration "fastapi-webhook" is waiting for a Revision to become ready.
[Status] Pending: IngressNotConfigured: Ingress has not yet been reconciled.
[Status] Pending: Uninitialized: Waiting for load balancer to be ready
[Status] Started: Service is ready

🚀 Deployed Endpoint: https://rough-meadow-97cf5.apps.<union-tenant>

Save the deployed endpoint for the next section.

Launching the Workflow with Webhook#

For this demo, we’ll register a simple workflow add_one:

union register wf.py

This command outputs the version of the workflow:

Computed version is Kh8OaZYZzsiLipGwTS18rw
Serializing and registering 3 flyte entities
[✔] Task: wf.add_one
[✔] Workflow: wf.add_one_wf
[✔] Launch Plan: wf.add_one_wf
Successfully registered 3 entities

Finally, we can launch the workflow with our application with curl:

export WEBHOOK_API_KEY=...  # use your custom api key
export APP_ENDPOINT=... # Use your app endpoint
export WF_VERSION=... # Use your workflow version

curl -X 'POST' \
  'https://$APP_ENDPOINT/run-workflow/thomasjpfan/development/wf.add_one_wf/$WF_VERSION' \
  -H 'accept: application/json' \
  -H "Authorization: Bearer $WEBHOOK_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"x": 2}'

The response will contain the URL of the execution:

{"url": "https://<union-tenant>/..."}

You can modify main.py to make adjustments to the FastAPI webhook for your use case.

FastAPI code#

The following is the main.py defining the FastAPI app. With the above App configuration the WEBHOOK_API_KEY environment variable is injected into the FastAPI runtime.