CIHelper: offer to pre-populate the upstream branches

GitGitGadget's code has auto-magic logic to fetch those, but even in a
partial clone that would result in a ~100MB fetch. Maybe that's okay,
and I worry too much about bandwidth? Inside hosted Actions runners,
download speeds of ~30MB/sec are not unheard of, after all... But then,
I do think that too few people are careful to waste less resources than
is ethical, so I want to be mindful.

Note that we use a "funny" `--filter=blob:limit=1g` option here instead
of the more natural `--no-filter`: Git introduced a subtle bug (that I
have no time nor patience to pursue any further) in v2.48.0-rc0~58^2
(index-pack: repack local links into promisor packs, 2024-11-01) that
would occasionally result in this strange error when fetching in the
partial clone after an occasional `--no-filter` fetch:

  git -C git.git fetch upstream
  remote: Enumerating objects: 2, done.
  remote: Counting objects: 100% (2/2), done.
  remote: Compressing objects: 100% (2/2), done.
  remote: Total 2 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
  Receiving objects: 100% (2/2), 14.66 KiB | 7.33 MiB/s, done.
  BUG: builtin/pack-objects.c:4320: should_include_obj should only be called on existing objects
  error: pack-objects died of signal 6
  fatal: could not finish pack-objects to repack local links
  fatal: index-pack failed

The most likely reason is that the logic needs to improved to
accommodate for situations where commits are fetched into a non-promisor
pack as part of fetch that notices that some of the commits reachable
from that pack are present locally (but they are in promisor packs,
which the naïve `pack-objects` call fails to anticipate). But again, I
do not have time to pursue this any further. Keeping out non-promisor
packs avoids the bug, and that's that.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>

Author: dscho

lib/ci-helper.ts

@@ -67,7 +67,10 @@ export class CIHelper {
         this.urlRepo = `${this.urlBase}${this.config.repo.name}/`;
     }
 
-    public async setupGitHubAction(setupOptions?: { needsMailingListMirror?: boolean }): Promise<void> {
+    public async setupGitHubAction(setupOptions?: {
+        needsMailingListMirror?: boolean;
+        needsUpstreamBranches?: boolean;
+    }): Promise<void> {
         // help dugite realize where `git` is...
         const gitExecutable = os.type() === "Windows_NT" ? "git.exe" : "git";
         const stripSuffix = `bin${path.sep}${gitExecutable}`;
@@ -123,6 +126,9 @@ export class CIHelper {
             ["remote.origin.url", `https://github.com/${this.config.repo.owner}/${this.config.repo.name}`],
             ["remote.origin.promisor", "true"],
             ["remote.origin.partialCloneFilter", "blob:none"],
+            ["remote.upstream.url", `https://github.com/${this.config.repo.baseOwner}/${this.config.repo.name}`],
+            ["remote.upstream.promisor", "true"],
+            ["remote.upstream.partialCloneFilter", "blob:none"],
         ]) {
             await git(["config", key, value], { workDir: this.workDir });
         }
@@ -142,6 +148,41 @@ export class CIHelper {
         );
         console.timeEnd("fetch Git notes");
         this.gggNotesUpdated = true;
+        if (setupOptions?.needsUpstreamBranches) {
+            console.time("fetch upstream branches");
+            await git(
+                [
+                    "fetch",
+                    "origin",
+                    "--no-tags",
+                    "--depth=500",
+                    "--filter=blob:limit=1g",
+                    ...this.config.repo.trackingBranches.map(
+                        (name) => `+refs/heads/${name}:refs/remotes/upstream/${name}`,
+                    ),
+                ],
+                {
+                    workDir: this.workDir,
+                },
+            );
+            console.timeEnd("fetch upstream branches");
+            console.time("get open PR head commits");
+            const openPRCommits = (
+                await Promise.all(
+                    this.config.repo.owners.map(async (repositoryOwner) => {
+                        return await this.github.getOpenPRs(repositoryOwner);
+                    }),
+                )
+            )
+                .flat()
+                .map((pr) => pr.headCommit);
+            console.timeEnd("get open PR head commits");
+            console.time("fetch open PR head commits");
+            await git(["fetch", "--no-tags", "origin", "--filter=blob:limit=1g", ...openPRCommits], {
+                workDir: this.workDir,
+            });
+            console.timeEnd("fetch open PR head commits");
+        }
         // "Unshallow" the refs by fetching the shallow commits with a tree-less filter.
         // This is needed because Git will otherwise fall over left and right when trying
         // to determine merge bases with really old branches.