add robust upstream md->mdx transformer tooling #114
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
adit-chandra
force-pushed
the
adit/mdx-transform-tooling
branch
from
5cb8ae4 to
492a978
Compare
Contributor
|
📝 Documentation updates detected! New suggestion: Document new MDX transformation tooling in README |
Collaborator
|
@adit-chandra I think you will need to rebase on top of |
adit-chandra
force-pushed
the
adit/mdx-transform-tooling
branch
from
86f5474 to
b344f2c
Compare
Author
rebased! |
Collaborator
|
@adit-chandra the reason this is failing is because the secrets inherited by your PR are not accepted by the |
Collaborator
|
once #125 lands, you should be able to merge |
adit-chandra
force-pushed
the
adit/mdx-transform-tooling
branch
from
b344f2c to
49974ed
Compare
Co-authored-by: promptless[bot] <promptless[bot]@users.noreply.github.com>
Co-authored-by: promptless[bot] <promptless[bot]@users.noreply.github.com>
Co-authored-by: promptless[bot] <promptless[bot]@users.noreply.github.com>
This commit introduces a comprehensive Node.js-based MDX transformation tool
that successfully processes all previously broken documentation files from the
upstream Bazel repository.
## What Changed
- Created `tools/mdx-transform/` with a new Node.js transformer
- Replaces the legacy `transform-docs.awk` script
- Uses unified/remark/rehype ecosystem for robust parsing
- Handles complex HTML, liquid templates, and MDX edge cases
- Updated `copy-upstream-docs.sh` to use the new transformer
- Cleared BROKEN_FILES list (30 → 0 files)
- All files now transform successfully
- Updated CI workflow to install Node.js dependencies
## Key Features
The new transformer handles:
- Liquid template syntax removal ({% %}, {{ }})
- Curly brace escaping for MDX
- Compare callouts → Mintlify Callout components
- Navigation tables → clean link navigation
- Material icons → arrow symbols
- HTML entity preservation in tables
- Email and URL autolink conversion
- Frontmatter extraction from H1 titles
## Test Results
✅ All unit tests passing (3/3)
✅ 156 files transformed successfully
✅ 19/19 previously broken files now working
✅ Zero Mintlify validation errors on transformed files
Co-authored-by: promptless[bot] <179508745+promptless[bot]@users.noreply.github.com>
adit-chandra
force-pushed
the
adit/mdx-transform-tooling
branch
from
49974ed to
86ef3a8
Compare
Collaborator
|
help is on the way, I will create a Github App to fix this issue |
Collaborator
|
@adit-chandra if you rebase now it should work - I've added a Github App with #135 and fixed with #137 |
Collaborator
|
confirmed the dependabot upgrade works with #136 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
transform-docs.awkwithtransform.mjs.transform-docs.awklacks AST functionality. lack of AST-support in pure awk made edge case handling and residual-html parsing exception unwieldy and intractable.transform.mjsis more easily extensible and maintainable tooling moving forward.BROKEN_FILESlist incopy-upstream-docs.shdown to 0git submodule update --init --depth 1 -- upstreamcd tools/mdx-transform && npm cinpm testcd ../../ && copy-upstream-docs.sh /tmp/mdx-checkmint dev --no-opentransform.mjsexpected behavior