Development cycle#

This section covers developing production-ready workflows for Union.

Setting up a project

Create a project on Union and initialize a workflow directory on your local machine.

Local dependencies

Install the required dependencies locally.

Remote dependencies with ImageSpec

Use ImageSpec to specify the dependencies needed in the containers that will run your tasks on Union.

Running your code

Use different deploy and run commands for different steps in the development cycle.

Overriding parameters

Use with_overrides to change parameters at execution time.

Details of union run

Programmatically perform Union operations in Python.

Debugging with interactive tasks

Inspect and debug live task code directly in the Union console.

Managing secrets

Create and manage secrets to connect to third-party services.

Managing apps

Create applications to allow external systems to run compute on Union.

Accessing AWS S3 buckets

Access data in AWS S3 Buckets from Union.

Task resource validation

How Union handles workflows with unsatisfiable resource requests.

UnionRemote

Programmatically perform Union operations in Python.