Skip to content

Commit 4f82389

Browse files
alan707github-actions[bot]
alan707
and
github-actions[bot]
authored
Issue #65: Automatically convert new markdown files in bazel.build after submodule update (#86)
* push files to the branch that will be merged in * create scripts to run github actions workflow locally and debug * fixed copy-upstream-docs.sh and cleaup-mdx.sh * git pull first * test new strategy * don't use rebase * chore: update documentation from upstream Bazel repo Generated by GitHub Actions workflow from upstream Bazel repository. This commit includes transformed documentation files ready for Mintlify deployment. * attempt to re-trigger workflow after commit is auto-pushed * remove go.mod, go.sum and binary from repo * now ignore them * fix those files later * chore: update documentation from upstream Bazel repo Generated by GitHub Actions workflow from upstream Bazel repository. This commit includes transformed documentation files ready for Mintlify deployment. * revert docs.json.update.sh * fix some files * chore: update documentation from upstream Bazel repo Generated by GitHub Actions workflow from upstream Bazel repository. This commit includes transformed documentation files ready for Mintlify deployment. * add more broken files * chore: update documentation from upstream Bazel repo Generated by GitHub Actions workflow from upstream Bazel repository. This commit includes transformed documentation files ready for Mintlify deployment. * chore: update documentation from upstream Bazel repo Generated by GitHub Actions workflow from upstream Bazel repository. This commit includes transformed documentation files ready for Mintlify deployment. * ignore more broken files * clean upstream before pushing changes * chore: update documentation from upstream Bazel repo Generated by GitHub Actions workflow from upstream Bazel repository. This commit includes transformed documentation files ready for Mintlify deployment. * make pull from bazel build a callable workflow * fix typo * renove packages --------- Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
1 parent a9af3e5 commit 4f82389

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

77 files changed

+6639
-20149
lines changed

‎.github/workflows/dependabot-automerge.yml‎

Lines changed: 28 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -13,27 +13,45 @@ concurrency:
1313
cancel-in-progress: false
1414

1515
jobs:
16-
automerge:
17-
timeout-minutes: 10
18-
# Only run on Dependabot PRs targeting main
16+
check-dependabot:
17+
timeout-minutes: 5
1918
if: >
2019
github.event.pull_request.user.login == 'dependabot[bot]' &&
2120
github.event.pull_request.base.ref == 'main'
2221
runs-on: ubuntu-latest
22+
outputs:
23+
should_process: ${{ steps.check.outputs.should_process }}
2324
steps:
2425
- name: Fetch Dependabot metadata
2526
id: meta
2627
uses: dependabot/fetch-metadata@v2
2728
with:
28-
# already added to repo
2929
github-token: ${{ secrets.GH_AUTOMERGE_PAT }}
3030

31-
# Gate: only submodules updates; only the "upstream" submodule path
31+
- name: Check if upstream submodule
32+
id: check
33+
run: |
34+
if [[ "${{ steps.meta.outputs.package-ecosystem }}" == "submodules" ]] && \
35+
[[ "${{ steps.meta.outputs.dependency-names }}" == *"upstream"* ]]; then
36+
echo "should_process=true" >> $GITHUB_OUTPUT
37+
else
38+
echo "should_process=false" >> $GITHUB_OUTPUT
39+
fi
40+
41+
# Generate docs after dependabot updates the submodule
42+
generate-docs:
43+
needs: check-dependabot
44+
if: needs.check-dependabot.outputs.should_process == 'true'
45+
uses: ./.github/workflows/pull-from-bazel-build.yml
46+
secrets: inherit
47+
48+
# Auto-merge after docs are generated
49+
enable-automerge:
50+
needs: [check-dependabot, generate-docs]
51+
if: needs.check-dependabot.outputs.should_process == 'true'
52+
runs-on: ubuntu-latest
53+
steps:
3254
- name: Enable auto-merge (squash)
33-
if: >
34-
steps.meta.outputs.package-ecosystem == 'submodules' &&
35-
contains(steps.meta.outputs.dependency-names, 'upstream')
3655
run: gh pr merge --auto --squash "${{ github.event.pull_request.html_url }}"
3756
env:
38-
# already added to repo
39-
GH_TOKEN: ${{ secrets.GH_AUTOMERGE_PAT }}
57+
GH_TOKEN: ${{ secrets.GH_AUTOMERGE_PAT }}

‎.github/workflows/generate-docs.yml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
name: Generate Documentation on PR
2+
3+
on:
4+
pull_request:
5+
types: [opened, reopened, synchronize, ready_for_review]
6+
7+
permissions:
8+
contents: write
9+
pull-requests: write
10+
11+
concurrency:
12+
group: generate-docs-${{ github.event.pull_request.number }}
13+
cancel-in-progress: true
14+
15+
jobs:
16+
generate-docs:
17+
# Skip Dependabot PRs (handled by auto-merge workflow)
18+
if: github.event.pull_request.user.login != 'dependabot[bot]'
19+
uses: ./.github/workflows/pull-from-bazel-build.yml
20+
secrets: inherit

‎.github/workflows/pull-from-bazel-build.yml‎

Lines changed: 112 additions & 83 deletions
Original file line numberDiff line numberDiff line change
@@ -1,107 +1,136 @@
1-
name: Pull Bazel Build Upstream Repo
1+
name: Pull Fresh Upstream Documentation
22

33
on:
4-
pull_request:
5-
types: [opened, synchronize, reopened]
6-
push:
7-
branches: [main]
8-
# called from trigger-from-bazel-repo.yml
9-
workflow_dispatch:
10-
inputs:
11-
bazelCommitHash:
12-
description: 'The commit hash of the Bazel repo to use'
13-
type: string
14-
default: origin/main
15-
# allow debugging by triggering from the GitHub UI
164
workflow_call:
175
inputs:
18-
bazelCommitHash:
19-
description: 'The commit hash of the Bazel repo to use'
20-
type: string
21-
default: origin/main
6+
bazelCommitHash:
7+
description: 'Specific Bazel commit hash to checkout (optional)'
8+
required: false
9+
type: string
10+
default: ''
11+
2212
jobs:
2313
pull-fresh-upstream:
24-
# don't run on dependabot PRs
25-
if: ${{ github.event.pull_request.user.login != 'dependabot[bot]' }}
2614
runs-on: ubuntu-latest
2715
permissions:
28-
contents: read
29-
packages: write
16+
contents: write
17+
pull-requests: write
3018

3119
steps:
32-
- uses: actions/checkout@v5
33-
with:
34-
# Don't auto-init submodules
35-
submodules: false
20+
- name: Checkout repository
21+
uses: actions/checkout@v5
22+
with:
23+
# Don't auto-init submodules
24+
submodules: false
25+
# This token is necessary so that the action can push into the repo
26+
# and trigger the required check (this workflow). Otherwise, the
27+
# default GITHUB_TOKEN cannot be considered a 'synchronize' event
28+
token: ${{ secrets.GH_AUTOMERGE_PAT }}
3629

37-
- name: Checkout submodules
38-
run: git submodule update --init -- upstream
30+
- name: Checkout submodules
31+
run: git submodule update --init -- upstream
3932

40-
- name: Checkout commit of Bazel Build submodule
41-
if: ${{ inputs.bazelCommitHash != '' }}
42-
working-directory: upstream
43-
run: git checkout '${{ inputs.bazelCommitHash }}'
33+
- name: Checkout commit of Bazel Build submodule
34+
if: ${{ inputs.bazelCommitHash != '' }}
35+
working-directory: upstream
36+
run: git checkout '${{ inputs.bazelCommitHash }}'
4437

45-
- name: Setup Bazel
46-
uses: bazel-contrib/setup-bazel@0.15.0
47-
with:
48-
bazelisk-cache: true
49-
repository-cache: true
38+
- name: Setup Bazel
39+
uses: bazel-contrib/setup-bazel@0.15.0
40+
with:
41+
bazelisk-cache: true
42+
repository-cache: true
5043

51-
- name: Build reference documentation
52-
working-directory: upstream
53-
run: >
54-
bazel build
55-
--config=docs
56-
--build_metadata=ROLE=DOCS
57-
--remote_header=x-buildbuddy-api-key=${{ secrets.BUILDBUDDY_ORG_API_KEY }}
58-
--bes_results_url=https://app.buildbuddy.io/invocation/
59-
--bes_backend=grpcs://remote.buildbuddy.io
60-
--remote_cache=grpcs://remote.buildbuddy.io
61-
--remote_timeout=10m
62-
//src/main/java/com/google/devtools/build/lib:gen_reference_docs
44+
- name: Build reference documentation
45+
working-directory: upstream
46+
run: >
47+
bazel build
48+
--config=docs
49+
--build_metadata=ROLE=DOCS
50+
--remote_header=x-buildbuddy-api-key=${{ secrets.BUILDBUDDY_ORG_API_KEY }}
51+
--bes_results_url=https://app.buildbuddy.io/invocation/
52+
--bes_backend=grpcs://remote.buildbuddy.io
53+
--remote_cache=grpcs://remote.buildbuddy.io
54+
--remote_timeout=10m
55+
//src/main/java/com/google/devtools/build/lib:gen_reference_docs
6356
64-
# Upload reference-docs.zip as an artifact for debugging purposes
65-
- name: Upload reference docs artifact
66-
if: ${{ github.ref != 'refs/heads/main' }}
67-
uses: actions/upload-artifact@v4.6.2
68-
with:
69-
name: reference-docs
70-
path: upstream/bazel-bin/src/main/java/com/google/devtools/build/lib/reference-docs.zip
71-
retention-days: 7
57+
- name: Upload reference docs artifact
58+
if: ${{ github.ref != 'refs/heads/main' }}
59+
uses: actions/upload-artifact@v4.6.2
60+
with:
61+
name: reference-docs
62+
path: upstream/bazel-bin/src/main/java/com/google/devtools/build/lib/reference-docs.zip
63+
retention-days: 7
7264

73-
- name: Clean up mdx files
74-
run: ./cleanup-mdx.sh
65+
- name: Clean up mdx files
66+
run: ./cleanup-mdx.sh
7567

76-
- name: Set up Go
77-
uses: actions/setup-go@v6
78-
with:
79-
go-version: '1.25.2'
68+
- name: Set up Go
69+
uses: actions/setup-go@v6
70+
with:
71+
go-version: '1.25.2'
8072

81-
- name: Initialize Go module for converter
82-
run: |
83-
cd html2md_converter
84-
go mod init html-to-md-converter
85-
go get github.com/JohannesKaufmann/html-to-markdown
73+
- name: Initialize Go module for converter
74+
run: |
75+
cd html2md_converter
76+
go mod init html-to-md-converter
77+
go get github.com/JohannesKaufmann/html-to-markdown
8678
87-
- name: Build HTML to Markdown converter
88-
run: |
89-
cd html2md_converter
90-
go build -o html-to-md main.go
79+
- name: Build HTML to Markdown converter
80+
run: |
81+
cd html2md_converter
82+
go build -o html-to-md main.go
9183
92-
- name: Convert reference documentation HTML to Markdown
93-
run: |
94-
# Extract and convert HTML reference docs to Markdown
95-
./html2md_converter/html-to-md \
96-
-zip upstream/bazel-bin/src/main/java/com/google/devtools/build/lib/reference-docs.zip \
97-
-output reference-docs-temp
84+
- name: Convert reference documentation HTML to Markdown
85+
run: |
86+
# Extract and convert HTML reference docs to Markdown
87+
./html2md_converter/html-to-md \
88+
-zip upstream/bazel-bin/src/main/java/com/google/devtools/build/lib/reference-docs.zip \
89+
-output reference-docs-temp
9890
99-
- name: Transform upstream docs to mdx
100-
run: ./copy-upstream-docs.sh
91+
- name: Transform upstream docs to mdx
92+
run: ./copy-upstream-docs.sh
10193

102-
- name: Create versioned navigation
103-
run: ./docs.json.update.sh
94+
- name: Create versioned navigation
95+
run: ./docs.json.update.sh
10496

105-
- name: Clean up temporary files
106-
run: rm -rf reference-docs-temp
97+
- name: Clean up temporary files
98+
run: rm -rf reference-docs-temp
99+
100+
- name: Configure Git
101+
run: |
102+
git config user.name "github-actions[bot]"
103+
git config user.email "github-actions[bot]@users.noreply.github.com"
107104
105+
- name: Commit and push changes
106+
env:
107+
BRANCH: ${{ github.head_ref || github.ref_name }}
108+
run: |
109+
set -euo pipefail
110+
111+
# Ensure local branch points at origin/BRANCH and is checked out
112+
git fetch origin "$BRANCH"
113+
git switch -C "$BRANCH" "origin/$BRANCH"
114+
git branch --set-upstream-to="origin/$BRANCH" "$BRANCH"
115+
116+
# Rebase onto latest remote before creating a new commit
117+
git pull
118+
119+
# Clean up any modified content in submodules (keep pointer changes)
120+
# This handles the "modified content" issue while preserving submodule commit updates
121+
cd upstream
122+
git reset --hard
123+
git clean -fd
124+
cd ..
125+
126+
if [[ -n "$(git status --porcelain)" ]]; then
127+
echo "Changes detected, committing and pushing..."
128+
git add -A
129+
git commit -m $'chore: update documentation from upstream Bazel repo\n\nGenerated by GitHub Actions workflow from upstream Bazel repository.\nThis commit includes transformed documentation files ready for Mintlify deployment.'
130+
131+
# Push back to the same branch
132+
git push origin "HEAD:$BRANCH"
133+
echo "Changes committed and pushed successfully"
134+
else
135+
echo "No changes detected, skipping commit"
136+
fi

‎.gitignore‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,7 @@ __pycache__/
44
.venv/
55
.hugo_build.lock
66
node_modules
7+
# ignore go build files
8+
html2md_converter/go.sum
9+
html2md_converter/go.mod
10+
html2md_converter/html-to-md

‎basics/hermeticity.mdx‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ non-hermetic actions.
8787
rich enough to allow arbitrary processing to happen in the process. You can
8888
get a log of some potentially non-hermetic actions in Bazel workspace rules by
8989
adding the flag
90-
`--experimental_workspace_rules_log_file={{ '<var>' }}PATH{{ '</var>' }}` to
90+
`--experimental_workspace_rules_log_file=<var>PATH</var>` to
9191
your Bazel command.
9292

9393
Note: Make your build fully hermetic when mixing remote and local execution,

‎cleanup-mdx.sh‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,11 @@ LOCAL_FILES="
1212

1313
echo "Cleaning up .mdx files..."
1414

15-
# Find all .mdx files in the repo (excluding upstream, work, .github, .cursor directories)
16-
find . -name "*.mdx" -type f | while read -r mdx_file; do
15+
# Find all .mdx files in the repo, excluding versioned directories
16+
# This excludes any directory starting with a digit (e.g., 6.5.0, 7.6.1, 8.0.1)
17+
find . -name "*.mdx" -type f \
18+
-not -path "./[0-9]*/*" \
19+
-not -path "./.github/*" | while read -r mdx_file; do
1720

1821
# Check if this exact file path is in LOCAL_FILES
1922
if echo "$LOCAL_FILES" | grep -q "^\s*$mdx_file\s*$"; then

0 commit comments

Comments
 (0)