Skip to content

Conversation

@davidlehn
Copy link
Member

No description provided.

@davidlehn davidlehn force-pushed the add-github-workflow branch 3 times, most recently from 6423f32 to ca21290 Compare November 3, 2023 22:24
@BigBlueHat BigBlueHat added this to the v2.0.4 milestone Feb 5, 2024
@BigBlueHat BigBlueHat modified the milestones: v2.0.4, v2.0.5 Feb 6, 2024
@BigBlueHat
Copy link
Contributor

I'm in favor of getting this merged sooner than later--and it can happen now (vs. as part of a release) since it only affects the master branch and nothing about the running code (or even the tests themselves).

@anatoly-scherbakov we should separate the rdf-canon introduction from #182, so both can move forward at different speeds.

@ff137

This comment was marked as off-topic.

@davidlehn davidlehn force-pushed the add-github-workflow branch 2 times, most recently from 603c1b8 to 8383625 Compare October 29, 2025 21:01
@davidlehn
Copy link
Member Author

  • Trying to update this change to it can be merged.
  • Too many other issues getting in the way, they can be handled on top of this with other PRs.
  • This was hanging out so long that 2 python versions hit EOL (or are set to this month).
  • 3.14 has a new event loop issue that causes aiohttp to fail. Anyone know a small non-invasive fix?

@mielvds
Copy link
Collaborator

mielvds commented Nov 4, 2025

Can we not skip 3.14 for now or use xfail in tests?
Seems like aiohttp is used in an optional documentloader to fetch remote JSON-LD documents over HTTP(S) using an asynchronous HTTP client (aiohttp). Remote document loading might be something we want to offload to rdflib eventually.

- Only test with supported Python versions.
- Drop testing of 3.8 - 3.9.
- Require >= 3.10.
- Add testing of 3.13 - 3.14.
- Lint with 3.14.
- Update aiohttp document loader to work with Python 3.14.
  - Minimize async related changes to library code in this release.
  - In sync environment use `asyncio.run`.
  - In async environment use background thread.
- AI pair programming used for this patch. This seemed to be the best of
  a few alternatives to fix Python 3.14 support while minimizing other
  changes. Future work could enable better concurrency.
@davidlehn davidlehn force-pushed the add-github-workflow branch from 01fe0f8 to f9772ea Compare November 7, 2025 15:14
@mielvds
Copy link
Collaborator

mielvds commented Nov 12, 2025

@davidlehn @anatoly-scherbakov @BigBlueHat This looks ready to merge! Great work.

One question about the python versions and versioning though: as it stands, we're doing 2.0.5 here. Are the breaking changes absolutely necessary here? I have a slight preference for doing some patch/minor releases first that setup testing (2.0.5) and reach spec compliance (2.1.0). I based this off on the milestones that were already there: https://github.com/digitalbazaar/pyld/milestones

But bundling everything in a new major release is also fine (just more complex to manage the issues).

@davidlehn
Copy link
Member Author

@mielvds:

  • What happened with the merge of Update aiohttp support for Python 3.14. #208? Confusing why it's listing all the commits here twice now on top of the aiohttp commit.
  • Digital Bazaar has been a rebase merge organization vs having merge commits. That is of course a topic of debate, but that's what we've been doing. I'm guessing that's related to the above somehow.
  • Should I do some git surgery here to fix up whatever happened?

One question about the python versions and versioning though: as it stands, we're doing 2.0.5 here. Are the breaking changes absolutely necessary here? I have a slight preference for doing some patch/minor releases first that setup testing (2.0.5) and reach spec compliance (2.1.0). I based this off on the milestones that were already there: https://github.com/digitalbazaar/pyld/milestones

But bundling everything in a new major release is also fine (just more complex to manage the issues).

It's been so long since a release that I think I once had trouble making testing even run on the old versions that were previously supported. I had figured we'd drop saying old versions were supported, mark it a major version to avoid any potential issues there, and move on with other changes. I look at that from the js semver policy though. Not sure if python community does it different.

I'm not going to look into if those aiohttp changes work with old versions, and certainly not bother to fix compat issues. Maybe that alone is enough of a breaking change?

Pardon if I missed an overall plan. I had kind of thought a major version with the workflow testing and that dropped old python support would be the next release. Other stuff could be done afterwards. But maybe some other path is better.

@davidlehn
Copy link
Member Author

Can we not skip 3.14 for now or use xfail in tests? Seems like aiohttp is used in an optional documentloader to fetch remote JSON-LD documents over HTTP(S) using an asynchronous HTTP client (aiohttp). Remote document loading might be something we want to offload to rdflib eventually.

I am really not up-to-date on how more advanced async code works in Python. My attempts at fixing aiohttp were headed towards needing to refactor the whole pyld api to be async. That's what JS code did. It might be the way to allow more parallelism. Not quite sure what the story is there with aiohttp. In any case, that's a future problem if that patch works.

It's certainly possible to use other document loaders. That's why that interface exists. Static files, caching, other methods, other libraries, etc. It does get tricky when async is mixed into the story.

@anatoly-scherbakov
Copy link
Collaborator

needing to refactor the whole pyld api to be async

That would be a very great breaking change for basically all client code, I guess 🤔 And not all such code is async/await itself so they will have trouble migrating, having to insert asyncio.run() everywhere.

I guess another set of functions could be feasible, like

from pyld.jsonld import async_expand

await async_expand(…)

@mielvds
Copy link
Collaborator

mielvds commented Nov 13, 2025

@mielvds:

* What happened with the merge of [Update aiohttp support for Python 3.14. #208](https://github.com/digitalbazaar/pyld/pull/208)?  Confusing why it's listing all the commits here twice now on top of the aiohttp commit.

oh jeez, that's not right. My bad! I guess I wasn't sure what the strategy was. There were a couple of force-pushes to the branch and that complicated the rebasing, so I went for a merge in the end. I think I might have ended up merging both branches.

* Digital Bazaar has been a rebase merge organization vs having merge commits.  That is of course a topic of debate, but that's what we've been doing.  I'm guessing that's related to the above somehow.

Yeah that's what I prefer too, I just got confused by git and github. I'll be more careful. Sorry!

* Should I do some git surgery here to fix up whatever happened?

Please do. I can try reverting the branch, but I think that better leave this to you as you know what's supposed to happen

One question about the python versions and versioning though: as it stands, we're doing 2.0.5 here. Are the breaking changes absolutely necessary here? I have a slight preference for doing some patch/minor releases first that setup testing (2.0.5) and reach spec compliance (2.1.0). I based this off on the milestones that were already there: https://github.com/digitalbazaar/pyld/milestones
But bundling everything in a new major release is also fine (just more complex to manage the issues).

It's been so long since a release that I think I once had trouble making testing even run on the old versions that were previously supported. I had figured we'd drop saying old versions were supported, mark it a major version to avoid any potential issues there, and move on with other changes. I look at that from the js semver policy though. Not sure if python community does it different.

Yeah makes sense. 3.0.0 is probably best; I suggest I rename the 2.0.5 milestone to v3.0.0b-testing (or v3.0.0 or something like that) and merge the remaining two in v3.0.0 (or v3.1.0)

I'm not going to look into if those aiohttp changes work with old versions, and certainly not bother to fix compat issues. Maybe that alone is enough of a breaking change?

Pardon if I missed an overall plan. I had kind of thought a major version with the workflow testing and that dropped old python support would be the next release. Other stuff could be done afterwards. But maybe some other path is better.

No it's perfectly fine, I just tried to make sense of what was already there. It's not that different either, v2.0.5 now becomes v3.0.0. I just wonder if it make sense to have a new major release with failing spec tests. A 3.0.0 with spec compliance would communicate so much better.

@mielvds
Copy link
Collaborator

mielvds commented Nov 13, 2025

needing to refactor the whole pyld api to be async

That would be a very great breaking change for basically all client code, I guess 🤔 And not all such code is async/await itself so they will have trouble migrating, having to insert asyncio.run() everywhere.

I guess another set of functions could be feasible, like

from pyld.jsonld import async_expand

await async_expand(…)

I agree. Async is not that obvious in Python as it is in Javascript. Anyway, let's focus on a working sync version for now :)

@davidlehn
Copy link
Member Author

I agree. Async is not that obvious in Python as it is in Javascript. Anyway, let's focus on a working sync version for now :)

Yes, that was the plan. The new 3.14 APIs are tricky though. The patch to that should, in theory, allow both sync code and newer async code that has a top level async loop to work. Yet another reason to have a major release. In case this is somehow broken, the 2.x series will keep working as it did before.

- Remove unused loader env.
- Remove matrix and use direct python version.
@davidlehn davidlehn merged commit a7b3261 into master Nov 14, 2025
15 checks passed
@davidlehn davidlehn deleted the add-github-workflow branch November 14, 2025 23:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants