Discovered that there is a relatively new feature in GitHub Actions which let's you configure matrix jobs: https://docs.github.com/en/actions/using-jobs/using-a-matrix-for-your-jobs#expanding-or-adding-matrix-configurations

It allows me to apply settings depending on which matrix job is run: https://github.com/NordicPlayground/nrf-docker/blob/0788ddf1ebff31ff3c83989ea25467dead0cfbbd/.github/workflows/publish-docker.yaml#L12-L72