Apache Airflow agent#

Note

To use the Airflow agent, you must pip install flytekitplugins-airflow.

class flytekitplugins.airflow.AirflowTask(*args, **kwargs)#

This python task is used to wrap an Airflow task. It is used to run an Airflow task in Flyte agent. The airflow task module, name and parameters are stored in the task config. We run the Airflow task in the agent.

get_custom(settings)#

Return additional plugin-specific custom data (if any) as a serializable dictionary.

Parameters:

settings (SerializationSettings)

Return type:

Dict[str, Any]

class flytekitplugins.airflow.agent.AirflowMetadata(airflow_operator, airflow_trigger=None, airflow_trigger_callback=None, job_id=None)#

This class is used to store the Airflow task configuration. It is serialized and returned to FlytePropeller.

Parameters:

  • airflow_operator (AirflowObj)

  • airflow_trigger (AirflowObj)

  • airflow_trigger_callback (str)

  • job_id (str | None)

classmethod decode(data)#

Decode the resource meta from bytes.

Parameters:

data (bytes)

Return type:

AirflowMetadata

encode()#

Encode the resource meta to bytes.

Return type:

bytes