Submit a script

This example submits a shell script. It installs a few additional libraries, downloads a notebook from GitHub, and executes it with papermill.

It requires a SAS token to upload the executed notebook to Azure Blob Storage, which is provided as an environment variable. You can customize the job using a YAML file:

$ kbatch job submit --file=shell-script.yaml --env="{\"SAS_TOKEN\": \"$SAS_TOKEN\"}"
...

or arguments to the CLI

$ kbatch job submit --name=my-job \
  --image=mcr.microsoft.com/planetary-computer/python:latest \
  --command='["sh", "shell-script.sh"]' \
  --code="shell-script.sh" \
  --env="{\"SAS_TOKEN\": \"$SAS_TOKEN\"}"
...

The job configuration file:

name: "my-job"
code: "shell-script.sh"
command:
  - sh
  - shell-script.sh
image: "mcr.microsoft.com/planetary-computer/python:latest"

And the code file is

#!/usr/bin/env bash

set -eux

wget https://raw.githubusercontent.com/microsoft/PlanetaryComputerExamples/main/datasets/sentinel-2-l2a/sentinel-2-l2a-example.ipynb

/srv/conda/envs/notebook/bin/python -m pip install papermill
# workaround https://github.com/intake/filesystem_spec/pull/784
/srv/conda/envs/notebook/bin/python -m pip uninstall -y gcsfs

/srv/conda/envs/notebook/bin/papermill \
    --no-request-save-on-cell-execute \
    sentinel-2-l2a-example.ipynb \
    abs://kbatchtest.blob.core.windows.net/kbatch/output/out.ipynb?${SAS_TOKEN}