Customizing task resources ​

When defining a task function, you can specify resource requirements for the pod that runs the task. Union Cloud will take this into account to ensure that the task pod is scheduled to run on a Kubernetes node that meets the specified resource profile.

Resources are specified in the @task decorator. Here is an example:

python
@task(
    requests=Resources(mem="120Gi", cpu="44", gpu="8", storage="100Gi", ephemeral_storage="100Gi"),
    limits=Resources(mem="200Gi", cpu="100", gpu="12", storage="200Gi", ephemeral_storage="200Gi")
)
def my_task()
    ...
@task(
    requests=Resources(mem="120Gi", cpu="44", gpu="8", storage="100Gi", ephemeral_storage="100Gi"),
    limits=Resources(mem="200Gi", cpu="100", gpu="12", storage="200Gi", ephemeral_storage="200Gi")
)
def my_task()
    ...

As you can see tasks have two separate resource-related settings:

  • requests
  • limits

Each of these takes a Resource object, which itself has five possible attributes:

  • cpu: Number of CPU cores (in whole numbers or millicores (m)).
  • gpu: Number of GPU cores (in whole numbers or millicores (m)).
  • mem: Main memory (in Mi, Gi, etc.).
  • storage: Storage (in Mi, Gi etc.).
  • ephemeral_storage: Ephemeral storage (in Mi, Gi etc.).

Note that CPU and GPU allocations can be specified either as whole numbers or in millicores (m). For example cpu="2" means 2 CPU cores and gpu="3500m", meaning three and a half GPU cores.

The requests setting tells the system that the task requires at least the resources specified and therefore the pod running this task should be scheduled only on a node that meets or exceeds the resource profile specified.

The limits setting serves as a hard upper bound on the resource profile of nodes to be scheduled to run the task. The task will not be scheduled on a node that exceeds the resource profile specified (in any of the specified attributes).

GPUs take only limits

GPUs should only be specified in the limits section of the task decorator:

  • You should specify GPU requirements only in limits, not in requests, because Kubernetes will use the limits value as the requests value anyway.

  • You can specify GPU in both limits and requests but the two values must be equal.

  • You cannot specify GPU requests without specifying limits.

Specify all significant resources requirements ​

Tasks can only rely on resources if they have been explicitly requested. If you know that a task requires significant resources, be sure to specify these in the decorator. A task can only rely on a given level of a resource if it has been explicitly requested in the task decorator, otherwise, the system cannot guarantee that the task will be allocated sufficient resources and the task might fail.

See also Customizing Task Resources in the Flyte OSS docs.

The with_overrides method ​

When resource requests or limits are specified in the @task decorator, they apply every time that a task is invoked from a workflow. In some cases, you may wish to change the resources specified from one invocation to another. To do that, use the with_overrides method of the task function. For example:

python
@task
def my_task(ff: FlyteFile):
    ...

@workflow
def my_workflow():
    my_task(ff=smallFile)
    my_task(ff=bigFile).withoverrides(requests=Resources(mem="120Gi", cpu="10"))
@task
def my_task(ff: FlyteFile):
    ...

@workflow
def my_workflow():
    my_task(ff=smallFile)
    my_task(ff=bigFile).withoverrides(requests=Resources(mem="120Gi", cpu="10"))