Merge pull request #9 from joergschultzelutter/trusted-workflows

Trusted Publisher migration

Author: joergschultzelutter

README.md

@@ -2,26 +2,20 @@
 
 [![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0) [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black) [![CodeQL](https://github.com/joergschultzelutter/pypi-publish-workflow/actions/workflows/codeql.yml/badge.svg)](https://github.com/joergschultzelutter/pypi-publish-workflow/actions/workflows/codeql.yml)
 
-This is a __Github Actions__ workflow for automatic publications to PyPi. Version data from a python file is extracted and then used by the PyPi setup process which will publish the package to PyPi Test and Prod. 
+This is a __Github Actions__ workflow for automatic publications to PyPi. Version data from a python file is extracted and then used by the PyPi setup process which will publish the package to PyPi Test and Prod, following PyPi's [Trusted Publishing](https://docs.pypi.org/trusted-publishers/) model. 
 
 The workflow will only be triggered for the publication of new repo releases / prereleases for the 'master' repo branch.
 
-## Installation instructions
-
-### Setup Github Secrets
-
-- Create token secrets for both [PyPi Test](https://test.pypi.org/) and [PyPi Prod](https://www.pypi.org/) (``Account Settings`` > ``API Tokens`` > ``Add API token``). 
-- In your Github project, goto ``Settings`` > ``Secrets and Variables`` > ``Actions``
-- Create two `Secrets` keys (`New repository secret`) named ``TEST_PYPI_API_TOKEN`` and ``PROD_PYPI_API_TOKEN`` and assign the previously created token secrets to these keys
-
-### Overview on config files
+## Overview on config files
 
 This repo contains three files that you may need to amend and copy to your Github repository:
 
 - ``setup.py``: this is a regular Python ``setup.py`` file; amend the file content with your package information and then save the file in your repo's root directory
 - ``publish-to-pypi.yml``: Edit this file, amend the configuration settings (see next chapter) and then save the file in your repo's Github Actions directory (``.github/workflows``). You may also need to activate the new workflow once you have installed it - see [documentation on Github](https://docs.github.com/en/actions).
 
-### `publish-to-pypi.yml` configuration
+## Configuration file instructions
+
+### `publish-to-pypi.yml`
 
 Open the file. You will notice a section which looks like this:
 
@@ -98,20 +92,42 @@ Necessary steps for a manual usage:
 - open `setup.py` and assign a version number to the `GITHUB_PROGRAM_VERSION` variable
 - `pip install git+https://github.com/my-repository-name@my-branch#egg=my-package-name`
 
+## Installation instructions
+
+The workflow uses PyPi's [Trusted Publisher](https://docs.pypi.org/trusted-publishers/) model. For a new project on PyPi, follow [these instructions](https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/) for setting up a trusted publisher. For an existing project which you may want to migrate from a secret-based workflow to the new trusted workflows, use [these instructions](https://docs.pypi.org/trusted-publishers/adding-a-publisher/).
+
+### Step 1: set up a Github environment
+
+- In your GitHub project, go to **Settings > Environments** and create a new environment called `pypi`.
+- Configure `Required reviewers` or other settings, if necessary. You do NOT need to configure any secrets here.
+
+### Step 2: Deploy the workflow and the setup file
+
+- You need to configure the files prior to deployment, see previous chapter **Configuration file instructions**
+- `setup.py` goes into your repository's root directory
+- `publish-to-pypi.yml` goes into your repository's `.github/workflows` directory (or add as a new GitHub action)
+
+### Step 3: Trusted Publisher Setup
+
+- Log on to your PyPi Test & Prod accounts
+- Follow the instructions on how to set up a [Trusted Publisher](https://docs.pypi.org/trusted-publishers/) on both Test and Prod environments:
+  - Set the `Workflow Name` to `publish-to-pypi.yml`
+  - Set the `Environment` to `pypi`
+
 ## Running the Github Action
 
 This Github action will do the following __whenever a new release/pre-release is published for the 'master' branch__:
 
 - Read the Python file and extract the version information, based on the given Regex. Abort job if no match was found.
 - Check if the Github ``ref_type`` has the value ``tag``. This is only the case when you drafted a new release. Otherwise, this value is likely set to ``master``. Abort job in case of a mismatch.
 - Check if the Github ``ref_name`` is equal to the extracted version from you Python file. Abort job in case of a mismatch. This will prevent issues where there is a mismatch between your Github release version (tag) and the one in the Python file.
-- Build the PyPi package. Deploy it to PyPi Test and (if successful AND not a pre-release) PyPi Prod.
+- Build the PyPi package. Deploy it to PyPi Test and (if successful AND not a pre-release) PyPi Prod. Note: This is done as a separate workflow step, see [this issue](https://github.com/pypa/gh-action-pypi-publish/issues/319) for technical details.
 
 This job will be triggered for releases AND prereleases in 'created' state (read: you tag a (pre)release in Github). Releases will be pushed to both PyPi Test and Prod whereas prereleases will only be pushed to PyPi Test.
 
 ## Test your work flow
 
-In case you want to become acquainted with this work flow: The safest way to test the work flow is to create both Github secret entries ``TEST_PYPI_API_TOKEN`` and ``PROD_PYPI_API_TOKEN`` but assign an invalid token to them. When you run the workflow for a new 'master' branch prerelease, the job will try to push it to PyPi Test and will fail because of the invalid token.
+- Publish your package as a prerelease. This should deploy your code only to PyPi Test.
 
 ## Workflow
 A basic workflow diagram of this Github Action can be found [here](docs/workflow.jpg)

publish-to-pypi.yml

@@ -84,6 +84,7 @@ jobs:
   # to next Github Actions job in line.
   get-python-version-info:
     runs-on: ubuntu-latest
+    environment: pypi
     permissions: write-all
 
     # Output which is passed to the PyPi publication job
@@ -150,12 +151,21 @@ jobs:
   # 
   # BEGIN of Job 2
   #
-  # This section will create the PyPi package and first deploy it to PyPi test.
-  # If successful, it will also try to issue a PyPi Prod deployment afterwards
+  # This section will create the PyPi package and deploy it to PyPi test.
+  # in case a pre-release was selected
   #
-  deploy-to-pypi:
+  # NOTE: When using PyPi's 'Trusted Publishing", one cannot use the same
+  # OIDC token for both Test and Prod - otherwise, error
+  # "Attestation generation failure: The following distributions already have publish attestations"
+  # will be thrown. See https://github.com/pypa/gh-action-pypi-publish/issues/319 for details
+  # This is the only reason for Test and Prod not sharing the same work flow
+  #
+  deploy-to-pypi-test:
     runs-on: ubuntu-latest
+    environment: pypi
     needs: get-python-version-info
+    permissions:
+      id-token: write
 
     steps:
 
@@ -187,17 +197,65 @@ jobs:
       if: github.event_name == 'release' && startsWith(github.ref, 'refs/tags')
       uses: pypa/gh-action-pypi-publish@release/v1
       with:
-        user: __token__
-        password: ${{ secrets.TEST_PYPI_API_TOKEN }}
+        #user: __token__
+        #password: ${{ secrets.TEST_PYPI_API_TOKEN }}
         repository-url: https://test.pypi.org/legacy/
 
+  #
+  # END of Job 2
+  #
+  #
+  # BEGIN of Job 3
+  #
+  # This section will create the PyPi package and fdeploy it to PyPi Prod.
+  #
+  # NOTE: When using PyPi's 'Trusted Publishing", one cannot use the same
+  # OIDC token for both Test and Prod - otherwise, error
+  # "Attestation generation failure: The following distributions already have publish attestations"
+  # will be thrown. See https://github.com/pypa/gh-action-pypi-publish/issues/319 for details
+  # This is the only reason for Test and Prod not sharing the same work flow
+  #
+  deploy-to-pypi-prod:
+    runs-on: ubuntu-latest
+    environment: pypi
+    needs: [get-python-version-info, deploy-to-pypi-test]
+    permissions:
+      id-token: write
+
+    steps:
+
+    - uses: actions/checkout@v5
+    # Set up Python environment
+    - name: Set up Python
+      if: github.event_name == 'release' && startsWith(github.ref, 'refs/tags') && !github.event.release.prerelease
+      uses: actions/setup-python@v6
+      with:
+        python-version: '${{ env.PYTHON_VERSION }}'
+
+    # Install all dependencies
+    - name: Install dependencies
+      if: github.event_name == 'release' && startsWith(github.ref, 'refs/tags') && !github.event.release.prerelease
+      run: |
+        python -m pip install --upgrade pip
+        pip install build
+
+    # Export the program version; content will be picked up by the setup.py script
+    - name: Export program version
+      if: github.event_name == 'release' && startsWith(github.ref, 'refs/tags') && !github.event.release.prerelease
+      run: echo GITHUB_PROGRAM_VERSION='${{ needs.get-python-version-info.outputs.my-program-version }}' >> $GITHUB_ENV
+
+    # Build the package. The export MUST be part of THIS step
+    # Otherwise, the Python setup job will not see this information
+    - name: Build package
+      if: github.event_name == 'release' && startsWith(github.ref, 'refs/tags') && !github.event.release.prerelease
+      run: python -m build
+
     # Publish everything to Prod PyPi but only if it is not a prerelease
     - name: Publish package to Prod PyPi
       if: github.event_name == 'release' && startsWith(github.ref, 'refs/tags') && !github.event.release.prerelease
       uses: pypa/gh-action-pypi-publish@release/v1
-      with:
-        user: __token__
-        password: ${{ secrets.PROD_PYPI_API_TOKEN }}
-  #
-  # END of Job 2
+      #with:
+        #user: __token__
+        #password: ${{ secrets.PROD_PYPI_API_TOKEN }}
   #
+  # END of Job 3