From 9bd8a0df08ccd59594877f3746851be7e5b6afb6 Mon Sep 17 00:00:00 2001 From: Jason Yundt Date: Mon, 28 Jul 2025 15:34:52 -0400 Subject: [PATCH 1/4] Use .yaml instead of .yml for YAML files MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This change makes it so that the names for all of the YAML files in this repository end with “.yaml” instead of “.yml”. According to the IANA Media Types list, “yaml” is the preferred file extension for YAML files. “yml” is still a valid file extension, but it’s not preferred [1]. Additionally, there are some situations where you need to use “yaml” instead of “yml”. Specifically, if you want to create a pre-commit [2] hook repository, then you need to create a file named “.pre-commit-hooks.yaml”. If you create a file named “.pre-commit-hooks.yml”, then pre-commit will fail with this error [3]: > An error has occurred: InvalidManifestError: > =====> /home/jayman/.cache/pre-commit/repo5e3r89vk/.pre-commit-hooks.yaml is not a file > Check the log at /home/jayman/.cache/pre-commit/pre-commit.log The main motivation behind this change is to prepare for a future commit. That future commit will add a .pre-commit-hooks.yaml file. It would be inconsistent if some YAML files ended with “.yaml” and others ended with “.yml”. This commit ensures that we end up using the same file extension for every YAML file in this repository even after a .pre-commit-hooks.yaml file is added. --- When making this change, I was concerned that renaming all of these files might break something so I did some research. I tried to find sources that would confirm that each of the affect files is allowed to use “.yaml” instead of “.yml”. • .github/dependabot.yaml: • .github/workflows/*: • .goreleaser.yaml: • codecov.yaml: I couldn’t find a citation for this one, so I decided to follow this codecov tutorial [4][5]. When I got to the part of the tutorial that told me to create a “codecov.yml” file, I decided to create a “codecov.yaml” file instead. Everything seemed to work fine when using “codecov.yaml” instead of “codecov.yml”. I was able to verify that codecov.yaml was being read by Codecov by logging in to Codecov, selecting the repository, going to the “Configuration” tab and going to the “Yaml” section. Whenever I changed codecov.yaml in the repository’s main branch, the contents of the Configuration tab’s Yaml section were updated. • docker-compose.yaml: --- This change was created using this Bash script: #!/usr/bin/env bash set -o errexit -o nounset -o pipefail git ls-files -z | while read -rd '' path do if [[ "$path" == *.yml ]] then git mv -- "$path" "${path/%.yml/.yaml}" fi done After running that script, I ran “git grep .yml” and manually updated anything that referenced one of the old file paths. --- [1]: [2]: [3]: [4]: [5]: --- .github/{dependabot.yml => dependabot.yaml} | 0 .github/workflows/{build.yml => build.yaml} | 4 ++-- .github/workflows/{release.yml => release.yaml} | 0 .goreleaser.yml => .goreleaser.yaml | 0 codecov.yml => codecov.yaml | 0 docker-compose.yml => docker-compose.yaml | 0 6 files changed, 2 insertions(+), 2 deletions(-) rename .github/{dependabot.yml => dependabot.yaml} (100%) rename .github/workflows/{build.yml => build.yaml} (93%) rename .github/workflows/{release.yml => release.yaml} (100%) rename .goreleaser.yml => .goreleaser.yaml (100%) rename codecov.yml => codecov.yaml (100%) rename docker-compose.yml => docker-compose.yaml (100%) diff --git a/.github/dependabot.yml b/.github/dependabot.yaml similarity index 100% rename from .github/dependabot.yml rename to .github/dependabot.yaml diff --git a/.github/workflows/build.yml b/.github/workflows/build.yaml similarity index 93% rename from .github/workflows/build.yml rename to .github/workflows/build.yaml index b53c4c0..75cb88a 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yaml @@ -5,12 +5,12 @@ on: paths: - '**.go' - 'go.mod' - - 'build.yml' + - 'build.yaml' pull_request: paths: - '**.go' - 'go.mod' - - 'build.yml' + - 'build.yaml' jobs: build: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yaml similarity index 100% rename from .github/workflows/release.yml rename to .github/workflows/release.yaml diff --git a/.goreleaser.yml b/.goreleaser.yaml similarity index 100% rename from .goreleaser.yml rename to .goreleaser.yaml diff --git a/codecov.yml b/codecov.yaml similarity index 100% rename from codecov.yml rename to codecov.yaml diff --git a/docker-compose.yml b/docker-compose.yaml similarity index 100% rename from docker-compose.yml rename to docker-compose.yaml From 6d3cb83490d2ec07244e832431f6fb8dfbc7d1ca Mon Sep 17 00:00:00 2001 From: Jason Yundt Date: Tue, 29 Jul 2025 14:34:40 -0400 Subject: [PATCH 2/4] Add --overwrite option MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The main motivation behind this change is to prepare for a future commit. That future commit will make it possible to use xq as a pre-commit [1] hook. The idea is that the xq pre-commit hook will fail if any XML files in the repository aren’t formatted the way that xq would format them. In order for a pre-commit hook to report a failure, it must do the following [2]: > The hook must exit nonzero on failure or modify files. Before this change, xq did neither of those things. This change makes it so that xq will modify unformatted files if the --overwrite option is used. I could have added an option that made xq exit with a nonzero exit status, but I thought that making xq modify files would be more convenient and make more sense. [1]: [2]: --- cmd/root.go | 32 ++++++++++++++++++++++++++++---- docs/xq.man | 5 +++++ 2 files changed, 33 insertions(+), 4 deletions(-) diff --git a/cmd/root.go b/cmd/root.go index 3786447..dce9fea 100644 --- a/cmd/root.go +++ b/cmd/root.go @@ -30,7 +30,9 @@ func NewRootCmd() *cobra.Command { var err error var reader io.Reader var indent string + var path string + overwrite, _ := cmd.Flags().GetBool("overwrite") if indent, err = getIndent(cmd.Flags()); err != nil { return err } @@ -42,17 +44,27 @@ func NewRootCmd() *cobra.Command { return nil } + if overwrite { + return errors.New("--overwrite was used but no filenames were specified") + } + reader = os.Stdin } else { var err error - if reader, err = os.Open(args[len(args)-1]); err != nil { + path = args[len(args)-1] + if reader, err = os.Open(path); err != nil { return err } } xPathQuery, singleNode := getXpathQuery(cmd.Flags()) withTags, _ := cmd.Flags().GetBool("node") - colors := getColorMode(cmd.Flags()) + var colors int + if overwrite { + colors = utils.ColorsDisabled + } else { + colors = getColorMode(cmd.Flags()) + } options := utils.QueryOptions{ WithTags: withTags, @@ -101,8 +113,18 @@ func NewRootCmd() *cobra.Command { errChan <- err }() - if err := utils.PagerPrint(pr, cmd.OutOrStdout()); err != nil { - return err + if overwrite { + var allData []byte + if allData, err = io.ReadAll(pr); err != nil { + return err + } + if err = os.WriteFile(path, allData, 0666); err != nil { + return err + } + } else { + if err = utils.PagerPrint(pr, cmd.OutOrStdout()); err != nil { + return err + } } return <-errChan @@ -140,6 +162,7 @@ func InitFlags(cmd *cobra.Command) { cmd.PersistentFlags().BoolP("json", "j", false, "Output the result as JSON") cmd.PersistentFlags().Bool("compact", false, "Compact JSON output (no indentation)") cmd.PersistentFlags().IntP("depth", "d", -1, "Maximum nesting depth for JSON output (-1 for unlimited)") + cmd.PersistentFlags().Bool("overwrite", false, "Instead of printing the formatted file, replace the original with the formatted version") } func Execute() { @@ -209,6 +232,7 @@ func detectFormat(flags *pflag.FlagSet, origReader io.Reader) (utils.ContentType buf := make([]byte, 10) length, err := origReader.Read(buf) if err != nil { + print(err.Error()) return utils.ContentText, origReader } diff --git a/docs/xq.man b/docs/xq.man index 71b1b61..0a20b01 100644 --- a/docs/xq.man +++ b/docs/xq.man @@ -68,6 +68,11 @@ Uses HTML formatter instead of XML. .RS 4 Returns the node content instead of text. .RE +.PP +\fB--overwrite\fR +.RS 4 +Instead of printing the formatted file, replace the original with the formatted version. +.RE .SH EXAMPLES .PP Format an XML file and highlight the syntax: From ab1a28701510d1a23810f7022408f060c96381bb Mon Sep 17 00:00:00 2001 From: Jason Yundt Date: Wed, 30 Jul 2025 10:23:23 -0400 Subject: [PATCH 3/4] Allow multiple files to be passed via the command-line The main motivation behind this change is prepare for a future commit. That future commit will make it so that you can use xq as a pre-commit [1] hook. When pre-commit executes a pre-commit hook, it passes each file that the hook should be run on as a command-line argument. In order for xq to work properly as a pre-commit hook, xq needs to support passing multiple files as command-line arguments. [1]: --- cmd/root.go | 168 ++++++++++++++++++++++++++++++++++------------------ docs/xq.man | 2 +- 2 files changed, 113 insertions(+), 57 deletions(-) diff --git a/cmd/root.go b/cmd/root.go index dce9fea..97e23da 100644 --- a/cmd/root.go +++ b/cmd/root.go @@ -9,6 +9,7 @@ import ( "os" "path" "strings" + "sync" "github.com/antchfx/xmlquery" "github.com/sibprogrammer/xq/internal/utils" @@ -28,35 +29,12 @@ func NewRootCmd() *cobra.Command { SilenceUsage: true, RunE: func(cmd *cobra.Command, args []string) error { var err error - var reader io.Reader var indent string - var path string overwrite, _ := cmd.Flags().GetBool("overwrite") if indent, err = getIndent(cmd.Flags()); err != nil { return err } - if len(args) == 0 { - fileInfo, _ := os.Stdin.Stat() - - if (fileInfo.Mode() & os.ModeCharDevice) != 0 { - _ = cmd.Help() - return nil - } - - if overwrite { - return errors.New("--overwrite was used but no filenames were specified") - } - - reader = os.Stdin - } else { - var err error - path = args[len(args)-1] - if reader, err = os.Open(path); err != nil { - return err - } - } - xPathQuery, singleNode := getXpathQuery(cmd.Flags()) withTags, _ := cmd.Flags().GetBool("node") var colors int @@ -79,55 +57,133 @@ func NewRootCmd() *cobra.Command { } jsonOutputMode, _ := cmd.Flags().GetBool("json") - pr, pw := io.Pipe() - errChan := make(chan error, 1) + var pagerPR *io.PipeReader + var pagerPW *io.PipeWriter + if !overwrite { + pagerPR, pagerPW = io.Pipe() + } - go func() { - defer close(errChan) - defer pw.Close() - - var err error - if xPathQuery != "" { - err = utils.XPathQuery(reader, pw, xPathQuery, singleNode, options) - } else if cssQuery != "" { - err = utils.CSSQuery(reader, pw, cssQuery, cssAttr, options) + var totalFiles int + if len(args) == 0 { + totalFiles = 1 + } else { + totalFiles = len(args) + } + // The “totalFiles * 2” part comes from the fact that there’s two goroutines + // inside the for loop and the fact that those two goroutines send a maximum + // of one value to errChan. The “+ 1” part comes from the fact that there’s + // one goroutine outside of the for loop and the fact that that goroutine + // sends a maximum of one value to errChan. + errChan := make(chan error, totalFiles * 2 + 1) + var wg sync.WaitGroup + for i := 0; i < totalFiles; i++ { + var path string + var reader io.Reader + if len(args) == 0 { + fileInfo, _ := os.Stdin.Stat() + + if (fileInfo.Mode() & os.ModeCharDevice) != 0 { + _ = cmd.Help() + return nil + } + + if overwrite { + return errors.New("--overwrite was used but no filenames were specified") + } + + reader = os.Stdin } else { - var contentType utils.ContentType - contentType, reader = detectFormat(cmd.Flags(), reader) - if jsonOutputMode { - err = processAsJSON(cmd.Flags(), reader, pw, contentType) + path = args[i] + if reader, err = os.Open(path); err != nil { + return err + } + } + + formattedPR, formattedPW := io.Pipe() + + wg.Add(1) + go func(reader io.Reader, formattedPW *io.PipeWriter) { + defer wg.Done() + defer formattedPW.Close() + + var err error + if xPathQuery != "" { + err = utils.XPathQuery(reader, formattedPW, xPathQuery, singleNode, options) + } else if cssQuery != "" { + err = utils.CSSQuery(reader, formattedPW, cssQuery, cssAttr, options) } else { - switch contentType { - case utils.ContentHtml: - err = utils.FormatHtml(reader, pw, indent, colors) - case utils.ContentXml: - err = utils.FormatXml(reader, pw, indent, colors) - case utils.ContentJson: - err = utils.FormatJson(reader, pw, indent, colors) - default: - err = fmt.Errorf("unknown content type: %v", contentType) + var contentType utils.ContentType + contentType, reader = detectFormat(cmd.Flags(), reader) + if jsonOutputMode { + err = processAsJSON(cmd.Flags(), reader, formattedPW, contentType) + } else { + switch contentType { + case utils.ContentHtml: + err = utils.FormatHtml(reader, formattedPW, indent, colors) + case utils.ContentXml: + err = utils.FormatXml(reader, formattedPW, indent, colors) + case utils.ContentJson: + err = utils.FormatJson(reader, formattedPW, indent, colors) + default: + err = fmt.Errorf("unknown content type: %v", contentType) + } } } - } - errChan <- err + errChan <- err + }(reader, formattedPW) + + wg.Add(1) + go func(formattedPR *io.PipeReader, path string) { + defer wg.Done() + defer formattedPR.Close() + + var err error + var allData []byte + if allData, err = io.ReadAll(formattedPR); err != nil { + errChan <- err + return + } + if overwrite { + if err = os.WriteFile(path, allData, 0666); err != nil { + errChan <- err + return + } + } else { + if _, err = pagerPW.Write(allData); err != nil { + errChan <- err + return + } + } + }(formattedPR, path) + } + + go func() { + wg.Wait() + if !overwrite { + if err := pagerPW.Close(); err != nil { + errChan <- err + } + } + close(errChan) }() - if overwrite { - var allData []byte - if allData, err = io.ReadAll(pr); err != nil { + if !overwrite { + if err = utils.PagerPrint(pagerPR, cmd.OutOrStdout()); err != nil { return err } - if err = os.WriteFile(path, allData, 0666); err != nil { + if err = pagerPR.Close(); err != nil { return err } - } else { - if err = utils.PagerPrint(pr, cmd.OutOrStdout()); err != nil { + } + + for err = range errChan { + if err != nil { return err } } - return <-errChan + return nil }, } } diff --git a/docs/xq.man b/docs/xq.man index 0a20b01..31b1e07 100644 --- a/docs/xq.man +++ b/docs/xq.man @@ -3,7 +3,7 @@ .SH NAME xq - command-line XML and HTML beautifier and content extractor .SH SYNOPSIS -xq [\fIoptions...\fR] [\fIfile\fR] +xq [\fIoptions...\fR] [\fIfiles...\fR] .SH DESCRIPTION Formats the provided \fIfile\fR and outputs it in the colorful mode. The file can be provided as an argument or via stdin. From 1bf86f7d5ba0569f4062617fc862d2c321d6007e Mon Sep 17 00:00:00 2001 From: Jason Yundt Date: Tue, 29 Jul 2025 14:35:41 -0400 Subject: [PATCH 4/4] Create xq pre-commit hook This changes makes it so that you can use pre-commit [1] to help make sure that any XML files in your Git repositories are always formatted nicely. [1]: --- .pre-commit-hooks.yaml | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 .pre-commit-hooks.yaml diff --git a/.pre-commit-hooks.yaml b/.pre-commit-hooks.yaml new file mode 100644 index 0000000..181c9a5 --- /dev/null +++ b/.pre-commit-hooks.yaml @@ -0,0 +1,5 @@ +- id: xq + name: xq + entry: xq --overwrite + language: golang + types: [ "xml" ]