Build Bash/shell commands into your pipeline
About this integration
The Dagster shell library provides utilities and op factories for the execution of inline shell scripts or script files.
It provides functions for constructing ops to execute a shell command, or execute a shell command read from a script file. You can also execute a shell command from within a Dagster op (or from Python in general), on either op input data, or any data generated within a generic python op.
pip install dagster-shell
from dagster_shell import create_shell_script_op from dagster import file_relative_path, job @job def my_job(): a = create_shell_script_op(file_relative_path(__file__, "hello_world.sh"), name="a") a()
A shell is a computer program that presents a command line interface which allows you to control your computer using commands entered with a keyboard instead of controlling graphical user interfaces (GUIs) with a mouse/keyboard/touchscreen combination.