Skip to content

Commit 0c26089

Browse files
sierra-moxonsierra-moxon
authored
Merge branch 'main' into csv-test
2 parents 7f7cd5d + 905789c commit 0c26089

File tree

229 files changed

+47356
-10189
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

229 files changed

+47356
-10189
lines changed

‎.github/workflows/check-dependencies.yaml‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
name: Build and test linkml-runtime with latest dependency versions
2+
3+
on:
4+
schedule:
5+
- cron: '0 5 * * 1' # once per week on Monday at 05:00 UTC
6+
workflow_dispatch:
7+
# Allows you to run this workflow manually from the Actions tab
8+
types: trigger-run-check-dependencies
9+
10+
jobs:
11+
test:
12+
strategy:
13+
fail-fast: false
14+
matrix:
15+
os: [ubuntu-latest, windows-latest]
16+
python-version: ["3.8", "3.9", "3.10"]
17+
exclude:
18+
- os: windows-latest
19+
python-version: "3.8"
20+
21+
runs-on: ${{ matrix.os }}
22+
23+
steps:
24+
25+
#----------------------------------------------
26+
# install poetry
27+
#----------------------------------------------
28+
- name: Install Poetry
29+
# Pin to 1.3.2 to workaround https://github.com/python-poetry/poetry/issues/7611
30+
run: pipx install poetry==1.3.2
31+
32+
#----------------------------------------------
33+
# check-out repo and set-up python
34+
#----------------------------------------------
35+
- name: Check out repository
36+
uses: actions/checkout@v3
37+
38+
- name: Set up Python ${{ matrix.python-version }}
39+
uses: actions/setup-python@v4
40+
with:
41+
python-version: ${{ matrix.python-version }}
42+
cache: 'poetry'
43+
44+
#----------------------------------------------
45+
# install your root project, if required
46+
#----------------------------------------------
47+
- name: Install library
48+
run: poetry install --no-interaction
49+
50+
# this step we remove and rebuild the poetry.lock file to ensure that the tests that follow can be run
51+
# with the latest dependencies
52+
53+
#----------------------------------------------
54+
# Remove and Rebuild the poetry.lock File
55+
#----------------------------------------------
56+
- name: Remove poetry.lock (Unix)
57+
if: runner.os != 'Windows'
58+
run: rm -rf poetry.lock
59+
60+
- name: Remove poetry.lock (Windows)
61+
if: runner.os == 'Windows'
62+
run: Remove-Item poetry.lock -Force
63+
64+
- name: Run tests
65+
run: poetry run python -m unittest discover

‎.github/workflows/main.yaml‎

Lines changed: 16 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -4,31 +4,35 @@
44

55
name: Build and test linkml-runtime
66

7-
on: [pull_request]
7+
on:
8+
push:
9+
branches:
10+
- main
11+
pull_request:
12+
workflow_dispatch:
813

914
jobs:
1015
test:
1116
strategy:
1217
fail-fast: false
1318
matrix:
1419
os: [ubuntu-latest, windows-latest]
15-
python-version: ["3.7", "3.8", "3.9", "3.10"]
20+
python-version: ["3.9", "3.10", "3.12", "3.13"]
1621
exclude:
1722
- os: windows-latest
18-
python-version: "3.7"
23+
python-version: "3.10"
1924
- os: windows-latest
20-
python-version: "3.8"
25+
python-version: "3.12"
2126

2227
runs-on: ${{ matrix.os }}
23-
28+
2429
steps:
2530

2631
#----------------------------------------------
2732
# install poetry
2833
#----------------------------------------------
2934
- name: Install Poetry
30-
# Pin to 1.3.2 to workaround https://github.com/python-poetry/poetry/issues/7611
31-
run: pipx install poetry==1.3.2
35+
run: pipx install poetry==1.4.0
3236

3337
#----------------------------------------------
3438
# check-out repo and set-up python
@@ -37,7 +41,7 @@ jobs:
3741
uses: actions/checkout@v3
3842

3943
- name: Set up Python ${{ matrix.python-version }}
40-
uses: actions/setup-python@v4
44+
uses: actions/setup-python@v5
4145
with:
4246
python-version: ${{ matrix.python-version }}
4347
cache: 'poetry'
@@ -48,18 +52,14 @@ jobs:
4852
- name: Install library
4953
run: poetry install --no-interaction
5054

51-
#----------------------------------------------
52-
# run test suite
53-
#----------------------------------------------
54-
- name: Run tests
55-
run: poetry run python -m unittest discover
56-
5755
#----------------------------------------------
5856
# coverage report
5957
#----------------------------------------------
6058
- name: Generate coverage results
59+
# Set bash shell to fail correctly on Windows https://github.com/actions/runner-images/issues/6668
60+
shell: bash
6161
run: |
62-
poetry run coverage run -m unittest discover
62+
poetry run coverage run -m pytest
6363
poetry run coverage xml
6464
poetry run coverage report -m
6565
@@ -72,4 +72,4 @@ jobs:
7272
name: codecov-results-${{ matrix.os }}-${{ matrix.python-version }}
7373
token: ${{ secrets.CODECOV_TOKEN }}
7474
file: coverage.xml
75-
fail_ci_if_error: true
75+
fail_ci_if_error: false

‎.github/workflows/pypi-publish.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ jobs:
1515
- name: Set up Python
1616
uses: actions/setup-python@v2.2.2
1717
with:
18-
python-version: 3.8
18+
python-version: 3.12
1919

2020
- name: Install Poetry
2121
run: pipx install poetry

‎.github/workflows/test-upstream.yaml‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
name: Test with upstream linkml
2+
on:
3+
pull_request:
4+
branches:
5+
- main
6+
workflow_dispatch:
7+
inputs:
8+
upstream_repo:
9+
description: "Upstream linkml repository to test against"
10+
required: true
11+
default: "linkml/linkml"
12+
upstream_branch:
13+
description: "Upstream linkml branch to test against"
14+
required: true
15+
default: "main"
16+
17+
jobs:
18+
test_upstream:
19+
strategy:
20+
fail-fast: false
21+
matrix:
22+
os: [ ubuntu-latest, windows-latest ]
23+
python-version: [ "3.9", "3.10", "3.11", "3.12", "3.13" ]
24+
exclude:
25+
- os: windows-latest
26+
python-version: "3.10"
27+
- os: windows-latest
28+
python-version: "3.11"
29+
- os: windows-latest
30+
python-version: "3.12"
31+
runs-on: ${{ matrix.os }}
32+
env:
33+
POETRY_VIRTUALENVS_IN_PROJECT: true
34+
35+
steps:
36+
- name: Get upstream branch and repo from first lines of PR Body
37+
if: github.event_name == 'pull_request'
38+
shell: bash
39+
env:
40+
PR_BODY: ${{ github.event.pull_request.body }}
41+
run: |
42+
set +e
43+
set +o pipefail
44+
45+
UPSTREAM_BRANCH=$( \
46+
echo "$PR_BODY" | \
47+
head -2 | \
48+
grep upstream_branch | \
49+
cut -d ":" -f 2 | \
50+
awk '{$1=$1};1' \
51+
)
52+
echo "Got upstream branch:"
53+
echo $UPSTREAM_BRANCH
54+
55+
if [[ -z "$UPSTREAM_BRANCH" ]]; then
56+
echo "Using main as default"
57+
UPSTREAM_BRANCH="main"
58+
fi
59+
60+
UPSTREAM_REPO=$( \
61+
echo "$PR_BODY" | \
62+
head -2 | \
63+
grep upstream_repo | \
64+
cut -d ":" -f 2 | \
65+
awk '{$1=$1};1' \
66+
)
67+
echo "Got upstream repo:"
68+
echo $UPSTREAM_REPO
69+
70+
if [[ -z "$UPSTREAM_REPO" ]]; then
71+
echo "Using linkml/linkml as default"
72+
UPSTREAM_REPO="linkml/linkml"
73+
fi
74+
75+
echo "upstream_branch=$UPSTREAM_BRANCH" >> "$GITHUB_ENV"
76+
echo "upstream_repo=$UPSTREAM_REPO" >> "$GITHUB_ENV"
77+
78+
- name: Get upstream branch from workflow dispatch
79+
if: github.event_name == 'workflow_dispatch'
80+
shell: bash
81+
run: |
82+
echo "upstream_branch=${{ inputs.upstream_branch }}" >> "$GITHUB_ENV"
83+
echo "upstream_repo=${{ inputs.upstream_repo }}" >> "$GITHUB_ENV"
84+
85+
- name: checkout upstream
86+
uses: actions/checkout@v4
87+
with:
88+
repository: "${{ env.upstream_repo }}"
89+
path: linkml
90+
ref: "${{ env.upstream_branch }}"
91+
fetch-depth: 0
92+
93+
- name: checkout linkml-runtime
94+
uses: actions/checkout@v4
95+
with:
96+
# don't specify repository like this or else we won't get pull request branches correctly
97+
# repository: linkml/linkml-runtime
98+
path: linkml-runtime
99+
fetch-depth: 0
100+
101+
- name: Ensure tags for linkml upstream if a different one than linkml/linkml is specified
102+
if: "${{ env.upstream_repo != 'linkml/linkml' }}"
103+
working-directory: linkml
104+
run: |
105+
git remote add upstream https://github.com/linkml/linkml
106+
git fetch upstream --tags
107+
108+
- name: Ensure tags for linkml-runtime if not run from main repo
109+
if: github.repository != 'linkml/linkml-runtime'
110+
working-directory: linkml-runtime
111+
run: |
112+
git remote add upstream https://github.com/linkml/linkml-runtime
113+
git fetch upstream --tags
114+
115+
- name: set up python
116+
uses: actions/setup-python@v5
117+
with:
118+
python-version: ${{ matrix.python-version }}
119+
120+
- name: Install poetry
121+
run: pipx install poetry==1.*
122+
123+
- name: Check Poetry version
124+
run: poetry --version
125+
126+
- name: Install dynamic versioning plugin
127+
run: poetry self add "poetry-dynamic-versioning[plugin]"
128+
129+
- name: Load cached venv
130+
id: cached-poetry-dependencies
131+
uses: actions/cache@v3
132+
with:
133+
path: linkml/.venv
134+
key: venv-${{ matrix.python-version }}-${{ runner.os }}-${{ hashFiles('**/poetry.lock') }}
135+
136+
# make extra sure we're removing any old version of linkml-runtime that exists
137+
- name: uninstall potentially cached linkml-runtime
138+
working-directory: linkml
139+
run: poetry run pip uninstall -y linkml-runtime
140+
141+
# we are not using linkml-runtime's lockfile, but simulating what will happen
142+
# when we merge this and update linkml's lockfile
143+
- name: add linkml-runtime to lockfile
144+
working-directory: linkml
145+
run: poetry add ../linkml-runtime
146+
147+
# note that we run the installation step always, even if we restore a venv,
148+
# the cache will restore the old version of linkml-runtime, but the lockfile
149+
# will only store the directory dependency (and thus will reinstall it)
150+
# the cache will still speedup the rest of the installation
151+
- name: install linkml
152+
working-directory: linkml
153+
run: poetry install --no-interaction -E tests
154+
155+
- name: print linkml-runtime version
156+
working-directory: linkml
157+
run: poetry run python -c 'import linkml_runtime; from importlib.metadata import version; print(linkml_runtime.__file__); print(version("linkml_runtime"))'
158+
159+
- name: run tests
160+
working-directory: linkml
161+
run: poetry run python -m pytest --with-slow
162+
163+
164+

‎CONTRIBUTING.md‎

Lines changed: 48 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,39 @@ ALSO NOTE: github.com lets you create a pull request from the main branch, autom
3131

3232
> A code review (which happens with both the contributor and the reviewer present) is required for contributing.
3333
34+
## Pull Requests
35+
36+
### Upstream Testing
37+
38+
`linkml-runtime` is tightly coupled to upstream `linkml`,
39+
so all pull requests have their changes tested by running the upstream tests
40+
against the PR version of `linkml-runtime`.
41+
42+
In some circumstances, paired changes need to be made against *both*
43+
`linkml` and `linkml-runtime`, where testing against the `main` branch
44+
of `linkml` is insufficient.
45+
46+
When opening a pull request, you can specify that your PR needs to be
47+
tested against a specific upstream branch and repository by specifying it
48+
in the first two lines of your pull request like this:
49+
50+
> upstream_repo: my-cool-username/linkml
51+
> upstream_branch: some-complicated-feature
52+
>
53+
> Hey everyone what up it's me your boy MC spongebob here with another banger
54+
> ... (PR continues)
55+
56+
The order of the `upstream_repo` and `upstream_branch` tags doesn't matter,
57+
but they must be on the first two lines of the pull request comment and separated with a colon.
58+
59+
Maintainers can also specify upstream branches to test against when
60+
dispatching the `test-upstream` workflow manually via the GUI prompt.
61+
62+
Testing against an unverified upstream branch is not necessarily dangerous,
63+
since the [input is stored as a variable first and not executed as untrusted code](https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions#using-an-intermediate-environment-variable),
64+
but maintainers should take care to verify that the upstream branch and repo
65+
are correct and expected given the context of the PR.
66+
3467
## Development environment setup
3568

3669
1. Install [poetry](https://python-poetry.org/docs/#installation).
@@ -70,7 +103,21 @@ All code added to the linkml-runtime source must have tests. The repo uses the n
70103
You can run the test suite in the following way:
71104

72105
```
73-
poetry run python -m unittest discover
106+
poetry run python -m pytest
107+
```
108+
109+
### Upstream Testing
110+
111+
To run the upstream `linkml` tests against your branch,
112+
install `linkml` locally with poetry, and then manually install your
113+
local copy of `linkml-runtime`
114+
115+
```shell
116+
git clone https://github.com/linkml/linkml
117+
cd linkml
118+
poetry install --all-extras --with tests
119+
poetry run pip install -e ~/location/of/linkml-runtime
120+
poetry run pytest
74121
```
75122

76123
## Code style

‎Makefile‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,8 @@ update_model:
1010
cp -pr $(MODEL_DIR)/* linkml_runtime/linkml_model
1111

1212
test:
13-
poetry run python -m unittest discover
13+
poetry run pytest
14+
1415

1516
# temporary measure until linkml-model is synced
1617
linkml_runtime/processing/validation_datamodel.py: linkml_runtime/processing/validation_datamodel.yaml

0 commit comments

Comments
 (0)