Migrate to renovate

[ulgens]

Unfortunately, dependabot's development has been stagnant for a while. It works well enough with pip / requirements.txt, but its uv support has many issues that weren't resolved, even though they announced it ~1.5 years ago.

Comments on details of this process and active issues can be found in this issue's comments: dependabot/dependabot-core#10478

The migration is a blocker for #1901

One small extra benefit is pre-commit hooks support: https://docs.renovatebot.com/modules/manager/pre-commit/

---

Why do we need this?

The core idea is to keep the automation working for dependency updates. Currently, we have dependabot in place (integrated to GitHub) but...

• It has multiple issues with modern Python tooling (https://github.com/dependabot/dependabot-core/issues?q=is%3Aissue%20state%3Aopen%20label%3A%22L%3A%20python%3Auv%22)
• It doesn't support all the ecosystems we use (missing pre-commit support).
• This is more of a personal experience but I've been a contributor of dependabot since 2020; the Python ecosystem was always suffering from a lack of maintenance effort. The Python-related modules were not in a usable state without the volunteer effort. The situation got even worse after uv's initial release. I've been in discussions with different GitHub employees / Dependabot team members over different channels for the last 1.5 years and it saddens me to tell that that effort amounted into nothing in the end. The current state of Dependabot's Python ecosystem tools poses a risk to supply chain security (Support updating uv.lock dependabot/dependabot-core#10478 (comment))

Can't we just do it manually?

We can, but I believe that we shouldn't. It requires a not-so-small amount of effort. To ensure consistent security for the project, we need to be actively watching for all 3rd party updates and create the PRs manually. Right now, I'm handling this for a tiny portion of our dependencies. It's repetitive, and it requires the maintainer to stay vigilant. It's a perfect task for automation.

A new integration

Thanks to the benefits it provides, I don't see Renovate as an optional 3rd party - what it does is vital for any software project in this day and age. I'd be happier if Dependabot (an integrated GitHub product) were still an option, but unfortunately, that's not the case. Also, with Renovate's pre-commit support, we won't need pre-commit.ci integration for hook updates.

Required permissions

Renovate creates issues and PRs, and uses security warnings to do its job.

• The feature that needs issue creation can be disabled, but in my experience it's a useful feature: https://docs.renovatebot.com/configuration-options/#dependencydashboard
• The feature that needs to create PRs can be disabled, and we can create PRs ourselves (I wouldn't prefer that): https://docs.renovatebot.com/configuration-options/#prcreation
• The feature that needs access to vulnerability alerts can be disabled too, but I think having it enabled is a good and useful thing: https://docs.renovatebot.com/configuration-options/#vulnerabilityalerts

Please let me know if there is any detail/concern that I missed and I'll try my best to address that.

[ulgens]

A nice reading about Renovate's position in its area: https://www.jvt.me/posts/2024/04/12/use-renovate/

[jacobtylerwalls]

Hi @ulgens. I chatted with @bmispelon and @nessita about this at our monthly Ops team checkin. We agreed we can't grant the renovate bot's requested perms at this time. To flesh out what was already alluded to on Slack:

• the Django org includes private repos managing sensitive infrastructure and handling security reports, so we're much more careful in this regard when a bot requests org-level perms.
• we're actually going through a period of formalizing matters like who has the GitHub ownership to grant these kinds of requests. Even I don't have access to grant this kind of request.

We're not against tooling modernization, and that part is not even our call, the website WG can decide what tools it wants to use. I hear you that Dependabot has a lot of gaps supporting uv. I think that's something the website WG can decide on: the roadmap for adopting uv lockfiles vs. staying on dependabot vs. some other automation solution vs. wait and see. I hope that makes sense.

[ulgens]

@jacobtylerwalls Thanks for taking the time to discuss this, but I'm sorry that the main concern here is just a misunderstanding.

• Renovate doesn't need org-level umbrella permissions that gives access to everything that org owns. The request I sent (and yes it was premature, sorry for that) was for only djangoproject.com and code.djangoproject.com repos. I tried to explain this to @nessita in Slack and also in this issue, but it seems it wasn't enough to clarify. I believe we already have multiple integrations with similar permissions, like Sentry and pre-commit.ci. If there is anything I'm missing, the best way to move forward is to discuss it together.
• A formalization would be nice. Right now, I'm confused about how to move forward with this discussion. On one hand, I do understand that there should be an approval consensus in place and it shouldn't be a blind approval. On the other hand, the discussion already caused a waste of time on your end because of this permission misunderstanding - the goal wasn't to integrate it into any resource that the Website WG doesn't already own. If we say that "when an integration is needed, the WG group that will use it needs to decide on it officialy (in a meeting and or with a voting)", I would support that decision, but I also think that WGs deserve a level of autonomy for the resources they own so I'd expect that voting to be final and shouldn't require another set of voting/discussion in the org level.

think that's something the website WG can decide on: the roadmap for adopting uv lockfiles vs. staying on dependabot vs. some other automation solution vs. wait and see.

We already have an issue to discuss the adaptation of the modern standard: #1901. Under that issue, we have a supporting comment from @tobiasmcnulty (here) and no opposition. By the nature of the process, we will need approvals before a PR like #2321 got merged, but I'm not sure there will be an actual roadmap because this is not even a controversial change (and if there is any controversy, I still think that issue is a good place to share concerns). Apart from the pyproject and lock file, I can and will present the topic of dependency updates in a website WG meeting.

All in all, I'm happy we are having this conversation and pointing out the missing org-level policies/permissions (like the officialization of ownership for these types of requests), but it's also sad because a simple misunderstanding led to a misguided concern. For the future, I think it's much better to include the owner of the request in the discussion because otherwise the communication process is too fragmented and things can go wrong (as it's happening now)