ImageSpec#

During the development cycle you will want to be able to run your workflows both locally on your machine and remotely on Union, so you will need to ensure that the required dependencies are installed in both environments.

Here we will explain how to set up the dependencies for your workflow to run remotely on Union. For information on how to make your dependencies available locally, see Local dependencies.

ImageSpec#

When a workflow is deployed to Union, each task is set up to run in its own container in the Kubernetes cluster. You specify the dependencies as part of the definition of the container image to be used for each task using the ImageSpec class. For example::

import union

image_spec = union.ImageSpec(
    builder="union",
    name="say-hello-image",
    requirements="uv.lock",
)

@union.task(container_image=image_spec)
def say_hello(name: str) -> str:
    return f"Hello, {name}!"

@union.workflow
def hello_world_wf(name: str = "world") -> str:
    greeting = say_hello(name=name)
    return greeting

Here, the ImageSpec class is used to specify the container image to be used for the say_hello task.

  • The builder parameter specifies how the image should be built. The value union means that the image will be built using Union’s built-in cloud builder. In some cases you may want to build the image locally on your machine and push it to a container registry. In that case, you would remove the builder parameter (or set it to envd) and add a registry parameter with the URL of the registry to push the image to. See below for more details.

  • The name parameter specifies the name of the image. This name will be used to identify the image in the container registry.

  • The requirements parameter specifies the path to a file (relative to the directory in which the union run or union register command is invoked) that specifies the dependencies to be installed in the image. The file may be:

    • A requirements.txt file.

    • A uv.lock file generated by the uv sync command.

    • A poetry.lock file generated by the poetry install command.

    • A pyproject.toml file.

When you execute the union run or union register command, Union will build the container image defined in ImageSpec block (as well as registering the tasks and workflows defined in your code).

Union cloud image builder#

If you have specified builder="union" in the ImageSpec, Union will build the image using its ImageBuilder service in the cloud and registered the image in Union’s own container registry. From there it will be pulled and installed in the task container when it speins up. All this is done transparently and does not require any set up by the user.

Local image builder#

Local image build in BYOC

In Union BYOC, you can build images from ImageSpec either using the Union cloud image builder (by specifying builder="union") or on your local machine (by omitting the builder parameter or specifying builder=”envd”). In Union Serverless, images defined by ImageSpec are always built using the Union cloud image builder. Local image building is not supported in Serverless.

If you have not specified a builder or have specified builder="envd", Union will build the image locally on your machine and push it to the registry you specify. This also requires that you specify a registry parameter in the ImageSpec. For example:

image_spec = union.ImageSpec(
    builder="envd",
    name="say-hello-image",
    requirements="uv.lock",
    registry="https://ghcr.io/<my-github-org>",
)

Here we assume you are using GitHub’s GHCR, and that you substitute your GitHub organization name for <my-github-org>.

Local container engine#

To enable local image building you must have an OCI-compatible container engine, like Docker, installed and running locally. Other options include Podman, LXD, or Containerd.

Access to a container registry#

You will also need access to a container registry. You must specify the URL of the registry in the registry parameter of the ImageSpec.

Above we used the GitHub Container Registry (GHCR) that comes as part of your GitHub account. For more information, see Working with the Container registry.

You may use another container registry if you prefer, such as Docker Hub, Amazon Elastic Container Registry (ECR), or Google Artifact Registry (GAR).

You will need to set up your local Docker client to authenticate to GHCR in order for union to be able to push the image built according to the ImageSpec to GHCR.

Follow the directions in Working with the Container registry > Authenticating to the Container registry.

Make your image accessible to Union#

In addition to making sure your registry is accessible from your local machine, you will need to ensure that the specific image, once pushed to the registry, is itself publicly accessible.

Make your image public

Note that in the case of our example registry (GHCR), making the image public can only be done once the image has been pushed. This means that you will need to register your workflow first, then make the image public and then run the workflow from the Union UI. If you try to run the workflow before making the image public (for example by doing a union run which both registers and runs immediately) the workflow execution will fail with an ImagePullBackOff error.

In the GitHub Container Registry, switch the visibility of your container image to Public. For more information, see Configuring a package’s access control and visibility.

At this point, you can run the workflow from the Union interface.