Pickle type#

Union enforces type safety by utilizing type information for compiling tasks and workflows, enabling various features such as static analysis and conditional branching.

However, we also strive to offer flexibility to end-users, so they don’t have to invest heavily in understanding their data structures upfront before experiencing the value Union has to offer.

Union supports the FlytePickle transformer, which converts any unrecognized type hint into FlytePickle, enabling the serialization/deserialization of Python values to/from a pickle file.

Important

Pickle can only be used to send objects between the exact same Python version. For optimal performance, it’s advisable to either employ Python types that are supported by Union or register a custom transformer, as using pickle types can result in lower performance.

This example demonstrates how you can utilize custom objects without registering a transformer.

import union

Superhero represents a user-defined complex type that can be serialized to a pickle file by Union and transferred between tasks as both input and output data.

Note

Alternatively, you can turn this object into a dataclass for improved performance. We have used a simple object here for demonstration purposes.

class Superhero:
    def __init__(self, name, power):
        self.name = name
        self.power = power


@union.task
def welcome_superhero(name: str, power: str) -> Superhero:
    return Superhero(name, power)


@union.task
def greet_superhero(superhero: Superhero) -> str:
    return f"👋 Hello {superhero.name}! Your superpower is {superhero.power}."


@union.workflow
def superhero_wf(name: str = "Thor", power: str = "Flight") -> str:
    superhero = welcome_superhero(name=name, power=power)
    return greet_superhero(superhero=superhero)