diff --git a/content/packer/.github/CONTRIBUTING.md b/content/packer/.github/CONTRIBUTING.md new file mode 100644 index 0000000000..2696178f86 --- /dev/null +++ b/content/packer/.github/CONTRIBUTING.md @@ -0,0 +1,639 @@ +# Contributing to Packer + +**First:** if you're unsure or afraid of _anything_, just ask or submit the +issue or pull request anyway. You won't be yelled at for giving your best +effort. The worst that can happen is that you'll be politely asked to change +something. We appreciate any sort of contributions, and don't want a wall of +rules to get in the way of that. + +However, for those individuals who want a bit more guidance on the best way to +contribute to the project, read on. This document will cover what we're looking +for. By addressing all the points we're looking for, it raises the chances we +can quickly merge or address your contributions. + +When contributing in any way to the Packer project (new issue, PR, etc), please +be aware that our team identifies with many gender pronouns. Please remember to +use nonbinary pronouns (they/them) and gender neutral language ("Hello folks") +when addressing our team. For more reading on our code of conduct, please see the +[HashiCorp community guidelines](https://www.hashicorp.com/community-guidelines). + +## Issues + +### Reporting an Issue + +- Make sure you test against the latest released version. It is possible we + already fixed the bug you're experiencing. + +- Run the command with debug output with the environment variable `PACKER_LOG`. + For example: `PACKER_LOG=1 packer build template.pkr.hcl`. Take the _entire_ + output and create a [gist](https://gist.github.com) for linking to in your + issue. Packer should strip sensitive keys from the output, but take a look + through just in case. + +- Provide a reproducible test case. If a contributor can't reproduce an issue, + then it dramatically lowers the chances it'll get fixed. And in some cases, + the issue will eventually be closed. + +- Respond promptly to any questions made by the Packer team to your issue. Stale + issues will be closed. + +### Issue Lifecycle + +1. The issue is reported. + +2. The issue is verified and categorized by a Packer collaborator. + Categorization is done via tags. For example, bugs are marked as "bugs" and + simple fixes are marked as "good first issue". + +3. Unless it is critical, the issue is left for a period of time (sometimes many + weeks), giving outside contributors a chance to address the issue. + +4. The issue is addressed in a pull request or commit. The issue will be + referenced in the commit message so that the code that fixes it is clearly + linked. + +5. Sometimes, if you have a specialized environment or use case, the maintainers + may ask for your help to test the patch. You are able to download an + experimental binary of Packer containing the Pull Request's patch via from + the Pull Request page on GitHub. You can do this by scrolling to the + "checks" section on GitHub, and clicking "details" on the + "store_artifacts" check. This will take you to Packer's Circle CI page for + the build, and you will be able to click a tab named "Artifacts" which will + contain zipped Packer binaries for each major OS architecture. + +6. The issue is closed. + +## Setting up Go + +If you have never worked with Go before, you will have to install its +runtime in order to build packer. + +1. This project always releases from the latest version of golang. +[Install go](https://golang.org/doc/install#install) To properly build from +source, you need to have golang >= v1.20 + +## Setting up Packer for dev + +If/when you have go installed you can already clone packer and `make` in +order to compile and test Packer. These instructions target +POSIX-like environments (macOS, Linux, Cygwin, etc.) so you may need to +adjust them for Windows or other shells. + + +1. Create a directory in your GOPATH for the code `mkdir -p $(go env GOPATH)/src/github.com/hashicorp && cd $_` +and clone the packer repository from GitHub into your GOPATH `git clone https://github.com/hashicorp/packer.git` +then change into the packer directory `cd packer` + +2. When working on Packer, first `cd $GOPATH/src/github.com/hashicorp/packer` + so you can run `make` and easily access other files. Run `make help` to get + information about make targets. + +3. Make your changes to the Packer source. You can run `make` in + `$GOPATH/src/github.com/hashicorp/packer` to run tests and build the Packer + binary. Any compilation errors will be shown when the binaries are + rebuilding. If you don't have `make` you can simply run + `go build -o bin/packer .` from the project root. + +4. After running building Packer successfully, use + `$GOPATH/src/github.com/hashicorp/packer/bin/packer` to build a machine and + verify your changes work. For instance: + `$GOPATH/src/github.com/hashicorp/packer/bin/packer build template.pkr.hcl`. + +5. If everything works well and the tests pass, run `go fmt` on your code before + submitting a pull-request. + +### Windows Systems + +On windows systems you need at least the [MinGW Tools](http://www.mingw.org/), e.g. install via [choco](https://chocolatey.org/): + +``` +choco install mingw -y +``` + +This installs the GCC compiler, as well as a `mingw32-make` which can be used wherever +this documentation mentions `make` + +when building using `go` you also need to mention the windows +executable extension + +``` +go build -o bin/packer.exe +``` + +### Opening a Pull Request + +Thank you for contributing! When you are ready to open a pull-request, you will +need to [fork +Packer](https://github.com/hashicorp/packer#fork-destination-box), push your +changes to your fork, and then open a pull-request. + +For example, my GitHub username is `cbednarski`, so I would do the following: + +``` +git checkout -b f-my-feature +# Develop a patch. +git push https://github.com/cbednarski/Packer f-my-feature +``` + +From there, open your fork in your browser to open a new pull-request. + +**Note:** Go infers package names from their file paths. This means `go build` +will break if you `git clone` your fork instead of using `go get` on the main +Packer project. + +**Note:** See '[Working with +forks](https://help.github.com/articles/working-with-forks/)' for a better way +to use `git push ...`. + +### Pull Request Lifecycle + +1. You are welcome to submit your pull request for commentary or review before + it is fully completed. Please prefix the title of your pull request with + "[WIP]" to indicate this. It's also a good idea to include specific questions + or items you'd like feedback on. + +2. Once you believe your pull request is ready to be merged, you can remove any + "[WIP]" prefix from the title and a core team member will review. + +3. One of Packer's core team members will look over your contribution and + either merge, or provide comments letting you know if there is anything left + to do. We do our best to provide feedback in a timely manner, but it may take + some time for us to respond. We may also have questions that we need answered + about the code, either because something doesn't make sense to us or because + we want to understand your thought process. + +4. If we have requested changes, you can either make those changes or, if you + disagree with the suggested changes, we can have a conversation about our + reasoning and agree on a path forward. This may be a multi-step process. Our + view is that pull requests are a chance to collaborate, and we welcome + conversations about how to do things better. It is the contributor's + responsibility to address any changes requested. While reviewers are happy to + give guidance, it is unsustainable for us to perform the coding work necessary + to get a PR into a mergeable state. + +5. Once all outstanding comments and checklist items have been addressed, your + contribution will be merged! Merged PRs will be included in the next + Packer release. The core team takes care of updating the + [CHANGELOG.md](../CHANGELOG.md) as they merge. + +6. In rare cases, we might decide that a PR should be closed without merging. + We'll make sure to provide clear reasoning when this happens. + +### Tips for Working on Packer + +#### Getting Your Pull Requests Merged Faster + +It is much easier to review pull requests that are: + +1. Well-documented: Try to explain in the pull request comments what your + change does, why you have made the change, and provide instructions for how + to produce the new behavior introduced in the pull request. If you can, + provide screen captures or terminal output to show what the changes look + like. This helps the reviewers understand and test the change. + +2. Small: Try to only make one change per pull request. If you found two bugs + and want to fix them both, that's _awesome_, but it's still best to submit + the fixes as separate pull requests. This makes it much easier for reviewers + to keep in their heads all of the implications of individual code changes, + and that means the PR takes less effort and energy to merge. In general, the + smaller the pull request, the sooner reviewers will be able to make time to + review it. + +3. Passing Tests: Based on how much time we have, we may not review pull + requests which aren't passing our tests. (Look below for advice on how to + run unit tests). If you need help figuring out why tests are failing, please + feel free to ask, but while we're happy to give guidance it is generally + your responsibility to make sure that tests are passing. If your pull request + changes an interface or invalidates an assumption that causes a bunch of + tests to fail, then you need to fix those tests before we can merge your PR. + +If we request changes, try to make those changes in a timely manner. Otherwise, +PRs can go stale and be a lot more work for all of us to merge in the future. + +Even with everyone making their best effort to be responsive, it can be +time-consuming to get a PR merged. It can be frustrating to deal with +the back-and-forth as we make sure that we understand the changes fully. Please +bear with us, and please know that we appreciate the time and energy you put +into the project. + +### PR Checks + +The following checks run when a PR is opened: + +- Contributor License Agreement (CLA): If this is your first contribution to Packer you will be asked to sign the CLA. +- Tests: tests include unit tests, documentation checks, and code formatting checks, and all checks must pass before a PR can be merged. + +#### Working on forks + +The easiest way to work on a fork is to set it as a remote of the Packer +project. After following the steps in "Setting up Go to work on Packer": + +1. Navigate to the code: + + `cd $GOPATH/src/github.com/hashicorp/packer` + +2. Add the remote by running: + + `git remote add ` + + For example: + + `git remote add mwhooker https://github.com/mwhooker/packer.git` + +3. Checkout a feature branch: + + `git checkout -b new-feature` + +4. Make changes. +5. (Optional) Push your changes to the fork: + + `git push -u new-feature` + +This way you can push to your fork to create a PR, but the code on disk still +lives in the spot where the go cli tools are expecting to find it. + +#### Go modules & go vendor + +If you are submitting a change that requires new or updated dependencies, +please include them in `go.mod`/`go.sum` and in the `vendor/` folder. This +helps everything get tested properly in CI. + +Note that you will need to use [go +mod](https://github.com/golang/go/wiki/Modules) to do this. This step is +recommended but not required. + +Use `go get ` to add dependencies to the project and `go mod vendor` +to make vendored copy of dependencies. See [go mod quick +start](https://github.com/golang/go/wiki/Modules#quick-start) for examples. + +Please only apply the minimal vendor changes to get your PR to work. Packer +does not attempt to track the latest version for each dependency. + +#### Code generation + +Packer relies on `go generate` to generate a [peg parser for boot +commands](https://github.com/hashicorp/packer/blob/master/packer-plugin-sdk/bootcommand/boot_command.go), +[docs](https://github.com/hashicorp/packer/blob/master/website/pages/partials/builder/amazon/chroot/_Config-not-required.mdx) +and HCL2's bridging code. Packer's testing suite will run `make generate-check` +to check that all the generated files Packer needs are what they should be. +`make generate` re-generates all these file and can take a while depending on +your machine's performances. To make it faster it is recommended to run +localized code generation. Say you are working on the Amazon builder: running +`go generate ./builder/amazon/...` will do that for you. Make sure that the +latest code generation tool is installed by running `make install-gen-deps`. + +#### Code linting + +Packer relies on [golangci-lint](https://github.com/golangci/golangci-lint) for linting its Go code base, excluding any generated code created by `go generate`. Linting is executed on new files during Travis builds via `make ci`; the linting of existing code base is only executed when running `make lint`. Linting a large project like Packer is an iterative process so existing code base will have issues that are actively being fixed; pull-requests that fix existing linting issues are always welcomed :smile:. + +The main configuration for golangci-lint is the `.golangci.yml` in the project root. See `golangci-lint --help` for a list of flags that can be used to override the default configuration. + +Run golangci-lint on the entire Packer code base. + +``` +make lint +``` + +Run golangci-lint on a single pkg or directory; PKG_NAME expands to /builder/amazon/... + +``` +make lint PKG_NAME=builder/amazon +``` + +Note: linting on Travis uses the `--new-from-rev` flag to only lint new files added within a branch or pull-request. To run this check locally you can use the `ci-lint` make target. See [golangci-lint in CI](https://github.com/golangci/golangci-lint#faq) for more information. + +``` +make ci-lint +``` + +#### Running Unit Tests + +You can run tests for individual packages using commands like this: + +``` +make test TEST=./builder/amazon/... +``` + +#### Running Builder Acceptance Tests + +Packer has [acceptance tests](https://en.wikipedia.org/wiki/Acceptance_testing) +for various builders. These typically require an API key (AWS, GCE), or +additional software to be installed on your computer (VirtualBox, VMware). + +If you're working on a new builder or builder feature and want to verify it is +functioning (and also hasn't broken anything else), we recommend creating or +running the acceptance tests. + +**Warning:** The acceptance tests create/destroy/modify _real resources_, which +may incur costs for real money. In the presence of a bug, it is possible that +resources may be left behind, which can cost money even though you were not +using them. We recommend running tests in an account used only for that purpose +so it is easy to see if there are any dangling resources, and so production +resources are not accidentally destroyed or overwritten during testing. + +To run the acceptance tests, invoke `make testacc`: + +``` +make testacc TEST=./builder/amazon/ebs +... +``` + +The `TEST` variable lets you narrow the scope of the acceptance tests to a +specific package / folder. The `TESTARGS` variable is recommended to filter down +to a specific resource to test, since testing all of them at once can sometimes +take a very long time. + +To run only a specific test, use the `-run` argument: + +``` +make testacc TEST=./builder/amazon/ebs TESTARGS="-run TestBuilderAcc_forceDeleteSnapshot" +``` + +Acceptance tests typically require other environment variables to be set for +things such as API tokens and keys. Each test should error and tell you which +credentials are missing, so those are not documented here. + +#### Running Provisioner Acceptance Tests + +**Warning:** The acceptance tests create/destroy/modify _real resources_, which +may incur costs for real money. In the presence of a bug, it is possible that +resources may be left behind, which can cost money even though you were not +using them. We recommend running tests in an account used only for that purpose +so it is easy to see if there are any dangling resources, and so production +resources are not accidentally destroyed or overwritten during testing. +Also, these typically require an API key (AWS, GCE), or additional software +to be installed on your computer (VirtualBox, VMware). + +To run the Provisioners Acceptance Tests you should use the +**ACC_TEST_BUILDERS** environment variable to tell the tests which builder the +test should be run against. + +Examples of usage: + +- Run the Shell provisioner acceptance tests against the Amazon EBS builder. + ``` + ACC_TEST_BUILDERS=amazon-ebs go test ./provisioner/shell/... -v -timeout=1h + ``` +- Do the same but using the Makefile + ``` + ACC_TEST_BUILDERS=amazon-ebs make provisioners-acctest TEST=./provisioner/shell + ``` +- Run all provisioner acceptance tests against the Amazon EBS builder. + ``` + ACC_TEST_BUILDERS=amazon-ebs make provisioners-acctest TEST=./... + ``` +- Run all provisioner acceptance tests against all builders whenever they are compatible. + ``` + ACC_TEST_BUILDERS=all make provisioners-acctest TEST=./... + ``` + +The **ACC_TEST_BUILDERS** env variable accepts a list of builders separated by +commas. (e.g. `ACC_TEST_BUILDERS=amazon-ebs,virtualbox-iso`) + + +#### Writing Provisioner Acceptance Tests + +Packer has implemented a `ProvisionerTestCase` structure to help write +provisioner acceptance tests. + +```go +type ProvisionerTestCase struct { + // Check is called after this step is executed in order to test that + // the step executed successfully. If this is not set, then the next + // step will be called + Check func(*exec.Cmd, string) error + // IsCompatible checks whether a provisioner is able to run against a + // given builder type and guest operating system, and returns a boolean. + // if it returns true, the test combination is okay to run. If false, the + // test combination is not okay to run. + IsCompatible func(builderType string, BuilderGuestOS string) bool + // Name is the name of the test case. Be simple but unique and descriptive. + Name string + // Setup, if non-nil, will be called once before the test case + // runs. This can be used for some setup like setting environment + // variables, or for validation prior to the + // test running. For example, you can use this to make sure certain + // binaries are installed, or text fixtures are in place. + Setup func() error + // Teardown will be called before the test case is over regardless + // of if the test succeeded or failed. This should return an error + // in the case that the test can't guarantee all resources were + // properly cleaned up. + Teardown builderT.TestTeardownFunc + // Template is the provisioner template to use. + // The provisioner template fragment must be a json-formatted string + // containing the provisioner definition but no other portions of a packer + // template. For + // example: + // + // ```json + // { + // "type": "shell-local", + // "inline", ["echo hello world"] + // } + //``` + // + // is a valid entry for "template" here, but the complete Packer template: + // + // ```json + // { + // "provisioners": [ + // { + // "type": "shell-local", + // "inline", ["echo hello world"] + // } + // ] + // } + // ``` + // + // is invalid as input. + // + // You may provide multiple provisioners in the same template. For example: + // ```json + // { + // "type": "shell-local", + // "inline", ["echo hello world"] + // }, + // { + // "type": "shell-local", + // "inline", ["echo hello world 2"] + // } + // ``` + Template string + // Type is the type of provisioner. + Type string +} + +``` + +To start writing a new provisioner acceptance test, you should add a test file +named `provisioner_acc_test.go` in the same folder as your provisioner is +defined. Create a test case by implementing the above struct, and run it +by calling `provisioneracc.TestProvisionersAgainstBuilders(testCase, t)` + +The following example has been adapted from a shell-local provisioner test: + +``` +import ( + "github.com/hashicorp/packer-plugin-sdk/acctest/provisioneracc" + "github.com/hashicorp/packer-plugin-sdk/acctest/testutils" +) + +// ... + +func TestAccShellProvisioner_basic(t *testing.T) { + // Create a json template fragment containing just the provisioners you want + // to run. + templateString := `{ + "type": "shell-local", + "script": "test-fixtures/script.sh", + "max_retries" : 5 +}` + + // instantiate a test case. + testCase := &provisioneracc.ProvisionerTestCase{ + IsCompatible: func() bool {return true}, + Name: "shell-local-provisioner-basic", + Teardown: func() error { + testutils.CleanupFiles("test-fixtures/file.txt") + return nil + }, + Template: templateString, + Type: "shell-local", + Check: func(buildcommand *exec.Cmd, logfile string) error { + if buildcommand.ProcessState != nil { + if buildcommand.ProcessState.ExitCode() != 0 { + return fmt.Errorf("Bad exit code. Logfile: %s", logfile) + } + } + filecontents, err := loadFile("file.txt") + if err != nil { + return err + } + if !strings.Contains(filecontents, "hello") { + return fmt.Errorf("file contents were wrong: %s", filecontents) + } + return nil + }, + } + + provisioneracc.TestProvisionersAgainstBuilders(testCase, t) +} + +``` + + +After writing the struct and implementing the interface, now is time to write the test that will run all +of this code you wrote. Your test should be like: + +```go +func TestShellProvisioner(t *testing.T) { + acc.TestProvisionersPreCheck("shell", t) + acc.TestProvisionersAgainstBuilders(new(ShellProvisionerAccTest), t) +} +``` + +The method `TestProvisionersAgainstBuilders` will run the provisioner against +all available and compatible builders. If there are not builders compatible with +the test you want to run, you can add a builder using the following steps: + +Create a subdirectory in provisioneracc/test-fixtures for the type of builder +you are adding. In this subdirectory, add one json file containing a single +builder fragment. For example, one of our amazon-ebs builders is defined in +provisioneracc/test-fixtures/amazon-ebs/amazon-ebs.txt and contains: + +```json +{ + "type": "amazon-ebs", + "ami_name": "packer-acc-test", + "instance_type": "t2.micro", + "region": "us-east-1", + "ssh_username": "ubuntu", + "source_ami_filter": { + "filters": { + "virtualization-type": "hvm", + "name": "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*", + "root-device-type": "ebs" + }, + "owners": ["099720109477"], + "most_recent": true + }, + "force_deregister" : true, + "tags": { + "packer-test": "true" + } +} +``` + +note that this fragment does not contain anything other than a single builder +definition. The testing framework will combine this with the provisioner +fragment to create a working json template. + +In order to tell the testing framework how to use this builder fragment, you +need to implement a `BuilderFixture` struct: + +```go +type BuilderFixture struct { + // Name is the name of the builder fixture. + // Be simple and descriptive. + Name string + // Setup creates necessary extra test fixtures, and renders their values + // into the BuilderFixture.Template. + Setup func() + // Template is the path to a builder template fragment. + // The builder template fragment must be a json-formatted file containing + // the builder definition but no other portions of a packer template. For + // example: + // + // ```json + // { + // "type": "null", + // "communicator", "none" + // } + //``` + // + // is a valid entry for "template" here, but the complete Packer template: + // + // ```json + // { + // "builders": [ + // "type": "null", + // "communicator": "none" + // ] + // } + // ``` + // + // is invalid as input. + // + // Only provide one builder template fragment per file. + TemplatePath string + + // GuestOS says what guest os type the builder template fragment creates. + // Valid values are "windows", "linux" or "darwin" guests. + GuestOS string + + // HostOS says what host os type the builder is capable of running on. + // Valid values are "any", windows", or "posix". If you set "posix", then + // this builder can run on a "linux" or "darwin" platform. If you set + // "any", then this builder can be used on any platform. + HostOS string + + Teardown builderT.TestTeardownFunc +} +``` +Implement this struct to the file "provisioneracc/builders.go", then add +the new implementation to the `BuildersAccTest` map in +`provisioneracc/provisioners.go` + +Once you finish these steps, you should be ready to run your new provisioner +acceptance test by setting the name used in the BuildersAccTest map as your +`ACC_TEST_BUILDERS` environment variable. + +#### Debugging Plugins + +Each packer plugin runs in a separate process and communicates via RPC over a +socket therefore using a debugger will not work (be complicated at least). + +But most of the Packer code is really simple and easy to follow with PACKER_LOG +turned on. If that doesn't work adding some extra debug print outs when you have +homed in on the problem is usually enough. diff --git a/content/packer/.github/ISSUE_TEMPLATE/bug_report.md b/content/packer/.github/ISSUE_TEMPLATE/bug_report.md new file mode 100644 index 0000000000..b02fc4b911 --- /dev/null +++ b/content/packer/.github/ISSUE_TEMPLATE/bug_report.md @@ -0,0 +1,50 @@ +--- +name: Bug Report +about: You're experiencing an issue with Packer that is different from the documented behavior. +labels: bug +--- + + + +#### Community Note + +* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request +* Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request +* If you are interested in working on this issue or have submitted a pull request, please leave a comment + + + +When filing a bug, please include the following headings if possible. Any +example text in this template can be deleted. + +#### Overview of the Issue + +A paragraph or two about the issue you're experiencing. + +#### Reproduction Steps + +Steps to reproduce this issue + +### Packer version + +From `packer version` + +### Simplified Packer Template + +If the file is longer than a few dozen lines, please include the URL to the +[gist](https://gist.github.com/) of the log or use the [Github detailed +format](https://gist.github.com/ericclemmons/b146fe5da72ca1f706b2ef72a20ac39d) +instead of posting it directly in the issue. + +### Operating system and Environment details + +OS, Architecture, and any other information you can provide about the +environment. + +### Log Fragments and crash.log files + +Include appropriate log fragments. If the log is longer than a few dozen lines, +please include the URL to the [gist](https://gist.github.com/) of the log or +use the [Github detailed format](https://gist.github.com/ericclemmons/b146fe5da72ca1f706b2ef72a20ac39d) instead of posting it directly in the issue. + +Set the env var `PACKER_LOG=1` for maximum log detail. diff --git a/content/packer/.github/ISSUE_TEMPLATE/config.yml b/content/packer/.github/ISSUE_TEMPLATE/config.yml new file mode 100644 index 0000000000..b79897d595 --- /dev/null +++ b/content/packer/.github/ISSUE_TEMPLATE/config.yml @@ -0,0 +1,8 @@ +# Copyright (c) HashiCorp, Inc. +# SPDX-License-Identifier: BUSL-1.1 + +blank_issues_enabled: false +contact_links: + - name: Packer Community Support + url: https://discuss.hashicorp.com/c/packer/23 + about: If you have a question, or are looking for advice, please post on our Discuss forum! The community loves to chime in to help. Happy Coding! diff --git a/content/packer/.github/ISSUE_TEMPLATE/feature_requests.md b/content/packer/.github/ISSUE_TEMPLATE/feature_requests.md new file mode 100644 index 0000000000..1d73353840 --- /dev/null +++ b/content/packer/.github/ISSUE_TEMPLATE/feature_requests.md @@ -0,0 +1,35 @@ +--- +name: Feature Request +about: If you have something you think Packer could improve or add support for. +labels: enhancement +--- + + + +#### Community Note + +* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request +* Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request +* If you are interested in working on this issue or have submitted a pull request, please leave a comment + + + +Please search the existing issues for relevant feature requests, and use the +reaction feature +(https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) +to add upvotes to pre-existing requests. + +#### Description + +A written overview of the feature. + +#### Use Case(s) + +Any relevant use-cases that you see. + +#### Potential configuration + +``` +``` + +#### Potential References diff --git a/content/packer/.github/ISSUE_TEMPLATE/plugin_integration.md b/content/packer/.github/ISSUE_TEMPLATE/plugin_integration.md new file mode 100644 index 0000000000..540e437fe6 --- /dev/null +++ b/content/packer/.github/ISSUE_TEMPLATE/plugin_integration.md @@ -0,0 +1,21 @@ +--- +name: Plugin Integration Request +about: Open request to add your plugin as a Packer integration (https://developer.hashicorp.com/packer/integrations) +labels: integration-request +--- + +#### Description + +A written description of your plugin along with a link to the plugin repository. + +#### Integration Tier + + +#### Checklist +- [ ] Has valid [`metadata.hcl`](https://github.com/hashicorp/integration-template) file in plugin repository. +- [ ] Has added integration scripts [packer-plugin-scaffolding](https://github.com/hashicorp/packer-plugin-scoffolding) to plugin repository. +- [ ] Has added top-level integration README.md file to plugin `docs` directory. +- [ ] All plugins components have one README.md describing their usage. +- [ ] Has a fully synced `.web-docs` directory ready for publishing to the integrations portal. + diff --git a/content/packer/.github/ISSUE_TEMPLATE/question.md b/content/packer/.github/ISSUE_TEMPLATE/question.md new file mode 100644 index 0000000000..83a7fd0ee2 --- /dev/null +++ b/content/packer/.github/ISSUE_TEMPLATE/question.md @@ -0,0 +1,15 @@ +--- +name: Question +about: If you have a question, please check out our other community resources instead of opening an issue. +labels: question +--- + +Issues on GitHub are intended to be related to bugs or feature requests, so we +recommend using our other community resources instead of asking here if you +have a question. + +- Packer Guides: https://www.packer.io/guides +- Discussion List: https://groups.google.com/group/packer-tool +- Any other questions can be sent to the packer section of the HashiCorp + forum: https://discuss.hashicorp.com/c/packer +- Packer community links: https://www.packer.io/community diff --git a/content/packer/.github/ISSUE_TEMPLATE/ssh_or_winrm_times_out.md b/content/packer/.github/ISSUE_TEMPLATE/ssh_or_winrm_times_out.md new file mode 100644 index 0000000000..1206ae0c4b --- /dev/null +++ b/content/packer/.github/ISSUE_TEMPLATE/ssh_or_winrm_times_out.md @@ -0,0 +1,23 @@ +--- +name: SSH or WinRM times out +about: I have a waiting SSH or WinRM error. +labels: communicator-question +--- + +Got one of the following errors ? See if the related guides can help. + +- `Waiting for WinRM to become available` ? + + - See our basic WinRm Packer guide: https://www.packer.io/guides/automatic-operating-system-installs/autounattend_windows + +- `Waiting for SSH to become available` ? + + - See our basic SSH Packer guide: https://www.packer.io/guides/automatic-operating-system-installs/preseed_ubuntu + +Issues on GitHub are intended to be related to bugs or feature requests, so we recommend using our other community resources instead of asking here if you have a question. + +- Packer Guides: https://www.packer.io/guides +- Discussion List: https://groups.google.com/group/packer-tool +- Any other questions can be sent to the packer section of the HashiCorp + forum: https://discuss.hashicorp.com/c/packer +- Packer community links: https://www.packer.io/community diff --git a/content/packer/.github/PULL_REQUEST_TEMPLATE.md b/content/packer/.github/PULL_REQUEST_TEMPLATE.md new file mode 100644 index 0000000000..cd88455f67 --- /dev/null +++ b/content/packer/.github/PULL_REQUEST_TEMPLATE.md @@ -0,0 +1,31 @@ +**DELETE THIS PART BEFORE SUBMITTING** + +In order to have a good experience with our community, we recommend that you +read the contributing guidelines for making a PR, and understand the lifecycle +of a Packer Plugin PR: +- https://github.com/hashicorp/$REPO_NAME/blob/main/.github/CONTRIBUTING.md#opening-an-pull-request + +Please include tests. Check out these examples: + +- https://github.com/hashicorp/packer/blob/master/builder/parallels/common/ssh_config_test.go#L34 +- https://github.com/hashicorp/packer/blob/master/post-processor/compress/post-processor_test.go#L153-L182 + +---- + +### Description +What code changed, and why? + +### Resolved Issues +If your PR resolves any open issue(s), please indicate them like this so they +will be closed when your PR is merged: +Closes #xxx +Closes #xxx + + +### Rollback Plan +If a change needs to be reverted, we will roll out an update to the code within +7 days. + +### Changes to Security Controls +Are there any changes to security controls (access controls, encryption, logging) +in this pull request? If so, explain. diff --git a/content/packer/.github/dependabot.yml b/content/packer/.github/dependabot.yml new file mode 100644 index 0000000000..bbf1c27f6c --- /dev/null +++ b/content/packer/.github/dependabot.yml @@ -0,0 +1,9 @@ +version: 2 +updates: +- package-ecosystem: gomod + directory: / + schedule: + interval: daily + allow: + - dependency-name: "github.com/hashicorp/packer-plugin-sdk" + - dependency-name: "github.com/hashicorp/hcp-sdk-go" diff --git a/content/packer/.github/labeler-issue-triage.yml b/content/packer/.github/labeler-issue-triage.yml new file mode 100644 index 0000000000..6d9ef6d547 --- /dev/null +++ b/content/packer/.github/labeler-issue-triage.yml @@ -0,0 +1,8 @@ +# Copyright (c) HashiCorp, Inc. +# SPDX-License-Identifier: BUSL-1.1 + +bug: + - 'panic:' +crash: + - 'panic:' + diff --git a/content/packer/.github/workflows/acceptance-test.yml b/content/packer/.github/workflows/acceptance-test.yml new file mode 100644 index 0000000000..50e92ecfbf --- /dev/null +++ b/content/packer/.github/workflows/acceptance-test.yml @@ -0,0 +1,100 @@ +# +# This GitHub action runs Packer's acceptance tests every night. +# Failures are reported to slack. +# + +name: "Acceptance Test" + +on: + # workflow_dispatch allows manual triggering of the workflow + workflow_dispatch: + schedule: + # Runs against the default branch every day at midnight + - cron: "0 0 * * *" + +permissions: + contents: read + +jobs: + get-go-version: + runs-on: ubuntu-latest + outputs: + go-version: ${{ steps.get-go-version.outputs.go-version }} + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - name: 'Determine Go version' + id: get-go-version + # We use .go-version as our source of truth for current Go + # version, because "goenv" can react to it automatically. + run: | + echo "Building with Go $(cat .go-version)" + echo "go-version=$(cat .go-version)" >> $GITHUB_OUTPUT + acceptance-test: + runs-on: ubuntu-latest + name: Acceptance Test + needs: get-go-version + env: + # AWS Creds for Assume Role + AWS_ROLE_ARN: ${{ secrets.AWS_ROLE_ARN }} + AWS_ACCESS_KEY_ID: ${{ secrets.TESTACC_AWS_ACCESS_KEY_ID }} + AWS_SECRET_ACCESS_KEY: ${{ secrets.TESTACC_AWS_SECRET_ACCESS_KEY }} + AWS_REGION: ${{ secrets.TESTACC_AWS_REGION }} + # Packer GH Token for API Rate Limiting + PACKER_GITHUB_API_TOKEN: ${{ secrets.GITHUB_TOKEN }} + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - uses: actions/setup-go@f111f3307d8850f501ac008e886eec1fd1932a34 # v5.3.0 + with: + go-version: ${{ needs.get-go-version.outputs.go-version }} + - name: IAM Assume Role + uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2 + with: + role-to-assume: ${{ env.AWS_ROLE_ARN }} + aws-region: ${{ env.AWS_REGION }} + aws-access-key-id: ${{ env.AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ env.AWS_SECRET_ACCESS_KEY }} + role-duration-seconds: 3600 + - name: Install gotestsum + run: go install gotest.tools/gotestsum@latest + + # we set the ACC_TEST_BUILDERS="amazon-ebs" since we want to test against the amazon-ebs tests (e.g. Powershell + # provisioner acceptance tests) + - name: Run acceptance tests per module + run: | + mkdir -p /tmp/test-results + make dev + ACC_TEST_BUILDERS="amazon-ebs" PACKER_ACC=1 gotestsum --format=short-verbose --junitfile /tmp/test-results/gotestsum-report.xml -- -timeout=120m -p 2 $(go list ./... | grep -v inspec | grep -v profitbricks | grep -v oneandone) + # Send a slack notification if either job defined above fails + slack-notify: + permissions: + contents: none + needs: + - get-go-version + - acceptance-test + if: always() && (needs.get-go-version.result == 'failure' || needs.acceptance-test.result == 'failure') + runs-on: ubuntu-latest + steps: + - name: Send slack notification on failure + uses: slackapi/slack-github-action@37ebaef184d7626c5f204ab8d3baff4262dd30f0 # v1.27.0 + with: + payload: | + { + "text": ":alert: Packer Nightly Acceptance Tests *FAILED* :alert:", + "attachments": [ + { + "color": "#C41E3A", + "blocks": [ + { + "type": "section", + "text": { + "type": "mrkdwn", + "text": "Branch: `${{ github.ref_name }}`\nRef: ${{ github.event.pull_request.html_url || github.event.head_commit.url }}\nWorkflow: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}" + } + } + ] + } + ] + } + env: + SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }} + SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK diff --git a/content/packer/.github/workflows/auto-close-stale-issues.yml b/content/packer/.github/workflows/auto-close-stale-issues.yml new file mode 100644 index 0000000000..d272c8b1aa --- /dev/null +++ b/content/packer/.github/workflows/auto-close-stale-issues.yml @@ -0,0 +1,29 @@ +name: Close Stale Issues + +on: + schedule: + - cron: '0 0 * * *' + +permissions: + contents: read + +jobs: + stale-bot: + permissions: + issues: write # for actions/stale to close stale issues + pull-requests: write # for actions/stale to close stale PRs + runs-on: ubuntu-latest + steps: + - uses: actions/stale@5bef64f19d7facfb25b37b414482c7164d639639 # v9.1.0 + with: + repo-token: ${{ secrets.GITHUB_TOKEN }} + days-before-issue-stale: 23 + days-before-pr-stale: -1 # deactivate the action for PRs + days-before-close: 7 + stale-issue-message: | + Hi, + + This issue has not received any update in the last 3 weeks, and will automatically be closed in 7 days if it does not receive any activity by then. + + If you find the [community forum](https://discuss.hashicorp.com/c/packer) to be more helpful or if you've found the answer to your question elsewhere please feel free to post a response and close the issue. + only-labels: needs-reply diff --git a/content/packer/.github/workflows/backport.yml b/content/packer/.github/workflows/backport.yml new file mode 100644 index 0000000000..c04e3ce838 --- /dev/null +++ b/content/packer/.github/workflows/backport.yml @@ -0,0 +1,52 @@ +--- +name: Backport Assistant Runner + +on: + pull_request_target: + types: + - closed + - labeled + +permissions: + contents: read + +jobs: + backport: + permissions: + contents: none + if: github.event.pull_request.merged + runs-on: ubuntu-latest + container: hashicorpdev/backport-assistant:v0.5.8 + steps: + - name: Backport changes to stable-website + run: | + backport-assistant backport -automerge + env: + BACKPORT_LABEL_REGEXP: "backport/(?Pwebsite)" + BACKPORT_TARGET_TEMPLATE: "stable-{{.target}}" + GITHUB_TOKEN: ${{ secrets.ELEVATED_GITHUB_TOKEN }} + - name: Backport changes to latest release branch + run: | + resp=$(curl -f -s "https://api.github.com/repos/$GITHUB_REPOSITORY/labels?per_page=100") + ret="$?" + if [[ "$ret" -ne 0 ]]; then + echo "The GitHub API returned $ret" + exit $ret + fi + # get the latest backport label excluding any website labels, ex: `backport/0.3.x` and not `backport/website` + latest_backport_label=$(echo "$resp" | jq -r '.[] | select(.name | (startswith("backport/") and (contains("website") | not))) | .name' | sort -rV | head -n1) + echo "Latest backport label: $latest_backport_label" + # set BACKPORT_TARGET_TEMPLATE for backport-assistant + # trims backport/ from the beginning with parameter substitution + export BACKPORT_TARGET_TEMPLATE="release/${latest_backport_label#backport/}" + backport-assistant backport -automerge + env: + BACKPORT_LABEL_REGEXP: "backport/(?Pwebsite)" + GITHUB_TOKEN: ${{ secrets.ELEVATED_GITHUB_TOKEN }} + - name: Backport changes to targeted release branch + run: | + backport-assistant backport -automerge + env: + BACKPORT_LABEL_REGEXP: "backport/(?P\\d+\\.\\d+\\.\\w+)" + BACKPORT_TARGET_TEMPLATE: "release/{{.target}}" + GITHUB_TOKEN: ${{ secrets.ELEVATED_GITHUB_TOKEN }} diff --git a/content/packer/.github/workflows/build.yml b/content/packer/.github/workflows/build.yml new file mode 100644 index 0000000000..b5f8bdf85b --- /dev/null +++ b/content/packer/.github/workflows/build.yml @@ -0,0 +1,292 @@ +# +# This GitHub action builds Packer binaries, linux packages, +# and Docker images from source, and uploads them to GitHub artifacts. +# Note that artifacts available via GitHub Artifacts are not codesigned or notarized. +# + +name: build + +on: + workflow_dispatch: + workflow_call: + push: + branches: + - main + - release/** + - feature/** + +env: + REPO_NAME: "packer" + +permissions: + contents: read + +jobs: + get-go-version: + runs-on: ubuntu-latest + outputs: + go-version: ${{ steps.get-go-version.outputs.go-version }} + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - name: 'Determine Go version' + id: get-go-version + # We use .go-version as our source of truth for current Go + # version, because "goenv" can react to it automatically. + run: | + echo "Building with Go $(cat .go-version)" + echo "go-version=$(cat .go-version)" >> $GITHUB_OUTPUT + + set-product-version: + runs-on: ubuntu-latest + outputs: + product-version: ${{ steps.set-product-version.outputs.product-version }} + base-product-version: ${{ steps.set-product-version.outputs.base-product-version }} + product-date: ${{ steps.set-product-version.outputs.product-date }} + product-prerelease-version: ${{ steps.set-product-version.outputs.prerelease-product-version }} + set-ld-flags: ${{ steps.set-ld-flags.outputs.set-ld-flags }} + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - name: set product version + id: set-product-version + uses: hashicorp/actions-set-product-version@v1 + - name: set-ld-flags + id: set-ld-flags + run: | + T="github.com/hashicorp/packer/version" + echo "set-ld-flags=-s -w -B gobuildid -X ${T}.GitCommit=${GITHUB_SHA::8} -X ${T}.GitDescribe=${{ steps.set-product-version.outputs.product-version }} -X ${T}.Version=${{ steps.set-product-version.outputs.base-product-version }} -X ${T}.VersionPrerelease=${{ steps.set-product-version.outputs.prerelease-product-version }} -X ${T}.VersionMetadata=" >> $GITHUB_OUTPUT + - name: validate outputs + run: | + echo "Product Version: ${{ steps.set-product-version.outputs.product-version }}" + echo "Base Product Version: ${{ steps.set-product-version.outputs.base-product-version }}" + echo "Prerelease Version: ${{ steps.set-product-version.outputs.prerelease-product-version }}" + echo "ldflags: ${{ steps.set-ld-flags.outputs.set-ld-flags }}" + + generate-metadata-file: + needs: set-product-version + runs-on: ubuntu-latest + outputs: + filepath: ${{ steps.generate-metadata-file.outputs.filepath }} + steps: + - name: 'Checkout directory' + uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - name: Generate metadata file + id: generate-metadata-file + uses: hashicorp/actions-generate-metadata@main + with: + version: ${{ needs.set-product-version.outputs.product-version }} + product: ${{ env.REPO_NAME }} + + - uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0 + with: + name: metadata.json + path: ${{ steps.generate-metadata-file.outputs.filepath }} + + build-other: + needs: + - set-product-version + - get-go-version + runs-on: ubuntu-latest + strategy: + matrix: + goos: [ freebsd, windows, netbsd, openbsd, solaris ] + goarch: [ "386", "amd64", "arm"] + go: [ "${{ needs.get-go-version.outputs.go-version }}" ] + exclude: + - goos: solaris + goarch: 386 + - goos: solaris + goarch: arm + - goos: windows + goarch: arm + fail-fast: true + + name: Go ${{ matrix.go }} ${{ matrix.goos }} ${{ matrix.goarch }} build + + env: + GOPRIVATE: "github.com/hashicorp" + GO111MODULE: on + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - name: Go Build + env: + PRODUCT_VERSION: ${{ needs.set-product-version.outputs.product-version }} + PRERELEASE_VERSION: ${{ needs.set-product-version.outputs.product-prerelease-version }} + LD_FLAGS: "${{ needs.set-product-version.outputs.set-ld-flags}}" + CGO_ENABLED: "0" + uses: hashicorp/actions-go-build@v1 + with: + product_name: ${{ env.REPO_NAME }} + product_version: ${{ needs.set-product-version.outputs.product-version }} + go_version: ${{ matrix.go }} + os: ${{ matrix.goos }} + arch: ${{ matrix.goarch }} + reproducible: report + instructions: |- + cp LICENSE "$TARGET_DIR/LICENSE.txt" + go build -o "$BIN_PATH" -ldflags="$LD_FLAGS" -trimpath -buildvcs=false + + build-linux: + needs: + - set-product-version + - get-go-version + runs-on: ubuntu-latest + strategy: + matrix: + goos: [ linux ] + goarch: [ "arm", "arm64", "386", "amd64", "ppc64le"] + go: [ "${{ needs.get-go-version.outputs.go-version }}" ] + fail-fast: true + + name: Go ${{ matrix.go }} ${{ matrix.goos }} ${{ matrix.goarch }} build + + env: + GOPRIVATE: "github.com/hashicorp" + GO111MODULE: on + + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - name: Go Build + env: + PRODUCT_VERSION: ${{ needs.set-product-version.outputs.product-version }} + PRERELEASE_VERSION: ${{ needs.set-product-version.outputs.product-prerelease-version }} + LD_FLAGS: "${{ needs.set-product-version.outputs.set-ld-flags}}" + CGO_ENABLED: "0" + uses: hashicorp/actions-go-build@v1 + with: + product_name: ${{ env.REPO_NAME }} + product_version: ${{ needs.set-product-version.outputs.product-version }} + go_version: ${{ matrix.go }} + os: ${{ matrix.goos }} + arch: ${{ matrix.goarch }} + reproducible: report + instructions: |- + cp LICENSE "$TARGET_DIR/LICENSE.txt" + go build -o "$BIN_PATH" -ldflags="$LD_FLAGS" -trimpath -buildvcs=false + + - name: Copy license file to config_dir + if: ${{ matrix.goos == 'linux' }} + env: + LICENSE_DIR: ".release/linux/package/usr/share/doc/${{ env.REPO_NAME }}" + run: | + mkdir -p "$LICENSE_DIR" && cp LICENSE "$LICENSE_DIR/LICENSE.txt" + - name: Linux Packaging + uses: hashicorp/actions-packaging-linux@v1 + with: + name: ${{ env.REPO_NAME }} + description: "HashiCorp Packer - A tool for creating identical machine images for multiple platforms from a single source configuration" + arch: ${{ matrix.goarch }} + version: ${{ needs.set-product-version.outputs.product-version }} + maintainer: "HashiCorp" + homepage: "https://www.packer.io/docs" + license: "BUSL-1.1" + binary: "dist/${{ env.REPO_NAME }}" + deb_depends: "openssl" + rpm_depends: "openssl" + config_dir: ".release/linux/package/" + - name: Add Linux Package names to env + run: | + echo "RPM_PACKAGE=$(basename out/*.rpm)" >> $GITHUB_ENV + echo "DEB_PACKAGE=$(basename out/*.deb)" >> $GITHUB_ENV + - uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0 + with: + name: ${{ env.RPM_PACKAGE }} + path: out/${{ env.RPM_PACKAGE }} + - uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0 + with: + name: ${{ env.DEB_PACKAGE }} + path: out/${{ env.DEB_PACKAGE }} + + build-darwin: + needs: + - set-product-version + - get-go-version + runs-on: macos-latest + strategy: + matrix: + goos: [ darwin ] + goarch: [ "amd64", "arm64" ] + go: [ "${{ needs.get-go-version.outputs.go-version }}" ] + fail-fast: true + name: Go ${{ matrix.go }} ${{ matrix.goos }} ${{ matrix.goarch }} build + + env: + GOPRIVATE: "github.com/hashicorp" + GO111MODULE: on + + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - name: Go Build + env: + PRODUCT_VERSION: ${{ needs.set-product-version.outputs.product-version }} + PRERELEASE_VERSION: ${{ needs.set-product-version.outputs.product-prerelease-version }} + LD_FLAGS: "${{ needs.set-product-version.outputs.set-ld-flags}}" + CGO_ENABLED: "0" + uses: hashicorp/actions-go-build@v1 + with: + product_name: ${{ env.REPO_NAME }} + product_version: ${{ needs.set-product-version.outputs.product-version }} + go_version: ${{ matrix.go }} + os: ${{ matrix.goos }} + arch: ${{ matrix.goarch }} + reproducible: report + instructions: |- + cp LICENSE "$TARGET_DIR/LICENSE.txt" + go build -o "$BIN_PATH" -ldflags="$LD_FLAGS" -tags netcgo -trimpath -buildvcs=false + + build-docker-light: + name: Docker light ${{ matrix.arch }} build + needs: + - set-product-version + - build-linux + runs-on: ubuntu-latest + strategy: + matrix: + arch: [ "arm", "arm64", "386", "amd64" ] + env: + version: ${{ needs.set-product-version.outputs.product-version }} + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - name: Docker Build (Action) + uses: hashicorp/actions-docker-build@v2 + with: + version: ${{ env.version }} + target: release-light + arch: ${{ matrix.arch }} + tags: | + ${{ steps.set-product-version.output.prerelease-product-version == '' && format('docker.io/hashicorp/{0}:light', env.REPO_NAME) }} + docker.io/hashicorp/${{ env.REPO_NAME }}:light-${{ env.version }} + docker.io/hashicorp/${{ env.REPO_NAME }}:${{ env.version }} + ${{ steps.set-product-version.output.prerelease-product-version == '' && format('public.ecr.aws/hashicorp/{0}:light', env.REPO_NAME) }} + public.ecr.aws/hashicorp/${{ env.REPO_NAME }}:light-${{ env.version }} + public.ecr.aws/hashicorp/${{ env.REPO_NAME }}:${{ env.version }} + dev_tags: | + docker.io/hashicorppreview/${{ env.REPO_NAME }}:${{ env.version }} + docker.io/hashicorppreview/${{ env.REPO_NAME }}:${{ env.version }}-${{ github.sha }} + + build-docker-full: + name: Docker full ${{ matrix.arch }} build + needs: + - set-product-version + - build-linux + runs-on: ubuntu-latest + strategy: + matrix: + arch: [ "arm", "arm64", "386", "amd64" ] + env: + version: ${{ needs.set-product-version.outputs.product-version }} + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - name: Docker Build (Action) + uses: hashicorp/actions-docker-build@v2 + with: + version: ${{ env.version }} + target: release-full + arch: ${{ matrix.arch }} + tags: | + ${{ steps.set-product-version.output.prerelease-product-version == '' && format('docker.io/hashicorp/{0}:full', env.REPO_NAME) }} + docker.io/hashicorp/${{ env.REPO_NAME }}:full-${{ env.version }} + ${{ steps.set-product-version.output.prerelease-product-version == '' && format('public.ecr.aws/hashicorp/{0}:full', env.REPO_NAME) }} + public.ecr.aws/hashicorp/${{ env.REPO_NAME }}:full-${{ env.version }} + dev_tags: | + docker.io/hashicorppreview/${{ env.REPO_NAME }}:full-${{ env.version }} + docker.io/hashicorppreview/${{ env.REPO_NAME }}:full-${{ env.version }}-${{ github.sha }} diff --git a/content/packer/.github/workflows/create-release-branch.yml b/content/packer/.github/workflows/create-release-branch.yml new file mode 100644 index 0000000000..1428c92b6b --- /dev/null +++ b/content/packer/.github/workflows/create-release-branch.yml @@ -0,0 +1,10 @@ +name: Create a release branch +on: [workflow_dispatch] +jobs: + create-branch: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - uses: hashicorp/actions-create-release-branch@v1 + with: + token: ${{ secrets.ELEVATED_GITHUB_TOKEN }} diff --git a/content/packer/.github/workflows/go-test.yml b/content/packer/.github/workflows/go-test.yml new file mode 100644 index 0000000000..0a610d51c9 --- /dev/null +++ b/content/packer/.github/workflows/go-test.yml @@ -0,0 +1,66 @@ +# +# This GitHub action runs Packer go tests across +# Windows, Linux, and MacOS runners. +# + +name: "Go Test" + +on: + push: + branches: + - 'main' + - release/** + - feature/** + pull_request: + +env: + PACKER_GITHUB_API_TOKEN: ${{ secrets.GITHUB_TOKEN }} + +permissions: + contents: read + +jobs: + get-go-version: + runs-on: ubuntu-latest + outputs: + go-version: ${{ steps.get-go-version.outputs.go-version }} + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - name: 'Determine Go version' + id: get-go-version + run: | + echo "Found Go $(cat .go-version)" + echo "go-version=$(cat .go-version)" >> $GITHUB_OUTPUT + linux-go-tests: + needs: + - get-go-version + runs-on: ubuntu-latest + name: Linux go tests + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - uses: actions/setup-go@f111f3307d8850f501ac008e886eec1fd1932a34 # v5.3.0 + with: + go-version: ${{ needs.get-go-version.outputs.go-version }} + - run: TESTARGS="-coverprofile=coverage.txt -covermode=atomic" make ci + darwin-go-tests: + needs: + - get-go-version + runs-on: macos-latest + name: Darwin go tests + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - uses: actions/setup-go@f111f3307d8850f501ac008e886eec1fd1932a34 # v5.3.0 + with: + go-version: ${{ needs.get-go-version.outputs.go-version }} + - run: go test ./... -coverprofile=coverage.txt -covermode=atomic + windows-go-tests: + needs: + - get-go-version + runs-on: windows-latest + name: Windows go tests + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - uses: actions/setup-go@f111f3307d8850f501ac008e886eec1fd1932a34 # v5.3.0 + with: + go-version: ${{ needs.get-go-version.outputs.go-version }} + - run: go test ./... -coverprofile=coverage.txt -covermode=atomic diff --git a/content/packer/.github/workflows/go-validate.yml b/content/packer/.github/workflows/go-validate.yml new file mode 100644 index 0000000000..efc8ee604e --- /dev/null +++ b/content/packer/.github/workflows/go-validate.yml @@ -0,0 +1,78 @@ +# +# This GitHub action runs basic linting checks for Packer. +# + +name: "Go Validate" + +on: + push: + branches: + - 'main' + - release/** + - feature/** + pull_request: + +permissions: + contents: read + +jobs: + get-go-version: + runs-on: ubuntu-latest + outputs: + go-version: ${{ steps.get-go-version.outputs.go-version }} + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - name: 'Determine Go version' + id: get-go-version + run: | + echo "Found Go $(cat .go-version)" + echo "go-version=$(cat .go-version)" >> $GITHUB_OUTPUT + check-mod-tidy: + needs: + - get-go-version + runs-on: ubuntu-latest + name: Go Mod Tidy + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - uses: actions/setup-go@f111f3307d8850f501ac008e886eec1fd1932a34 # v5.3.0 + with: + go-version: ${{ needs.get-go-version.outputs.go-version }} + - run: go mod tidy + check-lint: + needs: + - get-go-version + runs-on: ubuntu-latest + name: Lint + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + with: + fetch-depth: 0 + - uses: actions/setup-go@f111f3307d8850f501ac008e886eec1fd1932a34 # v5.3.0 + with: + go-version: ${{ needs.get-go-version.outputs.go-version }} + - run: echo "$GITHUB_SHA" + - run: git fetch --all + - run: echo $(git merge-base origin/main $GITHUB_SHA) + - run: make ci-lint + check-fmt: + needs: + - get-go-version + runs-on: ubuntu-latest + name: Fmt check + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - uses: actions/setup-go@f111f3307d8850f501ac008e886eec1fd1932a34 # v5.3.0 + with: + go-version: ${{ needs.get-go-version.outputs.go-version }} + - run: make fmt-check + check-generate: + needs: + - get-go-version + runs-on: ubuntu-latest + name: Generate check + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - uses: actions/setup-go@f111f3307d8850f501ac008e886eec1fd1932a34 # v5.3.0 + with: + go-version: ${{ needs.get-go-version.outputs.go-version }} + - run: make generate-check diff --git a/content/packer/.github/workflows/issue-comment-created.yml b/content/packer/.github/workflows/issue-comment-created.yml new file mode 100644 index 0000000000..e74be499b1 --- /dev/null +++ b/content/packer/.github/workflows/issue-comment-created.yml @@ -0,0 +1,24 @@ +name: Issue Comment Created Triage + +on: + issue_comment: + types: [created] + +permissions: + contents: read + +jobs: + issue_comment_triage: + permissions: + contents: read # for actions/checkout to fetch code + issues: write # for actions-ecosystem/action-remove-labels to remove issue labels + if: ${{contains(github.event.issue.labels.*.name, 'waiting-reply') || contains(github.event.issue.labels.*.name, 'stale')}} + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - uses: actions-ecosystem/action-remove-labels@2ce5d41b4b6aa8503e285553f75ed56e0a40bae0 # v1.3.0 + with: + github_token: "${{ secrets.GITHUB_TOKEN }}" + labels: | + stale + waiting-reply diff --git a/content/packer/.github/workflows/issue-migrator.yml b/content/packer/.github/workflows/issue-migrator.yml new file mode 100644 index 0000000000..719cfa44f2 --- /dev/null +++ b/content/packer/.github/workflows/issue-migrator.yml @@ -0,0 +1,27 @@ +name: Migrate Remote Plugin Issues + +on: + issues: + types: [labeled] + +permissions: + issues: write + +jobs: + PluginSplit: + if: startsWith(github.event.label.name, 'remote-plugin/') + runs-on: ubuntu-latest + steps: + - uses: bflad/action-migrate-issue@2e049d822b08e67d25b938343227051518f14d9a # v1.0.2 + with: + source-issue-comment: | + This issue has been migrated to {target-issue-url} due to the [Packer Plugin split](https://github.com/hashicorp/packer/issues/8610#issuecomment-770034737). + + Please follow the new issue for updates. + source-issue-skip-label-names: | + core + new-plugin-contribution + website + target-repository-name-prefix: 'packer-plugin-' + target-repository-name-prefix-label-prefix: 'remote-plugin/' + target-repository-github-token: ${{ secrets.PLUGIN_REPO_GITHUB_TOKEN }} diff --git a/content/packer/.github/workflows/issues-opened.yml b/content/packer/.github/workflows/issues-opened.yml new file mode 100644 index 0000000000..03587e747a --- /dev/null +++ b/content/packer/.github/workflows/issues-opened.yml @@ -0,0 +1,57 @@ +on: + issues: + types: [opened] + +permissions: + contents: read + +jobs: + issue_triage: + name: Issue Opened Triage + permissions: + contents: read + issues: write # for github/issue-labeler to create or remove labels + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - uses: github/issue-labeler@c1b0f9f52a63158c4adc09425e858e87b32e9685 # v3.4 + with: + repo-token: "${{ secrets.GITHUB_TOKEN }}" + configuration-path: .github/labeler-issue-triage.yml + enable-versioned-regex: 0 + label-question: + name: Redirect to discuss + permissions: + contents: read + issues: write # for actions/github-script to create comments and add labels + if: contains(github.event.issue.labels.*.name, 'question') + runs-on: ubuntu-latest + steps: + - uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1 + with: + script: | + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: ` + Hi 👋 thanks for reaching out. + + For general questions we recommend reaching out to the [community forum](https://discuss.hashicorp.com/c/packer) for greater visibility. + As the GitHub issue tracker is only watched by a small subset of maintainers and is really reserved for bugs and enhancements, you'll have a better chance of finding someone who can help you in the forum. + We'll mark this issue as needs-reply to help inform maintainers that this question is awaiting a response. + If no activity is taken on this question within 30 days it will be automatically closed. + + If you find the forum to be more helpful or if you've found the answer to your question elsewhere please feel free to post a response and close the issue.` + }) + + - uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1 + with: + script: | + github.rest.issues.addLabels({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + labels: ['needs-reply'] + }) + diff --git a/content/packer/.github/workflows/jira.yml b/content/packer/.github/workflows/jira.yml new file mode 100644 index 0000000000..4225f5f4aa --- /dev/null +++ b/content/packer/.github/workflows/jira.yml @@ -0,0 +1,86 @@ +on: + issues: + types: [labeled] + +permissions: + contents: read + +name: JIRA Sync + +jobs: + sync: + name: Sync to JIRA + permissions: + issues: write # for actions/github-script to create comments + runs-on: ubuntu-latest + steps: + - name: Login + uses: atlassian/gajira-login@45fd029b9f1d6d8926c6f04175aa80c0e42c9026 # v3.0.1 + env: + JIRA_BASE_URL: ${{ secrets.JIRA_BASE_URL }} + JIRA_USER_EMAIL: ${{ secrets.JIRA_USER_EMAIL }} + JIRA_API_TOKEN: ${{ secrets.JIRA_API_TOKEN }} + + - name: Search + if: github.event.action == 'labeled' + id: search + uses: tomhjp/gh-action-jira-search@04700b457f317c3e341ce90da5a3ff4ce058f2fa # v0.2.2 + with: + # cf[10089] is Issue Link (use JIRA API to retrieve) + jql: 'project = "HPR" AND cf[10089] = "${{ github.event.issue.html_url }}"' + + + - name: Set type + id: set-ticket-type + run: | + # Questions are not tracked in JIRA at this time. + if [[ "${{ contains(github.event.issue.labels.*.name, 'question') }}" == "true" ]]; then + echo "type=Invalid" >> $GITHUB_OUTPUT + else + # Properly labeled GH issues are assigned the standard "GH Issue" type upon creation. + echo "type=GH Issue" >> $GITHUB_OUTPUT + fi + + - name: Set labels + id: set-ticket-labels + run: | + if [[ "${{ contains(github.event.issue.labels.*.name, 'bug') }}" == "true" ]]; then + echo "labels=[\"bug\"]" >> $GITHUB_OUTPUT + elif [[ "${{ contains(github.event.issue.labels.*.name, 'enhancement') }}" == "true" ]]; then + echo "labels=[\"enhancement\"]" >> $GITHUB_OUTPUT + else + echo "labels=[]" >> $GITHUB_OUTPUT + fi + + - name: Validate ticket + if: steps.set-ticket-type.outputs.type == 'Invalid' + run: | + echo "Questions are not being synced to JIRA at this time." + echo "If the issue is a bug or an enhancement please remove the question label and reapply the 'sync to jira' label." + + - name: Create ticket + id: create-ticket + if: steps.search.outputs.issue == '' && github.event.label.name == 'sync to jira' && steps.set-ticket-type.outputs.type != 'Invalid' + uses: atlassian/gajira-create@59e177c4f6451399df5b4911c2211104f171e669 # v3.0.1 + with: + project: HPR + issuetype: "${{ steps.set-ticket-type.outputs.type }}" + summary: "${{ github.event.repository.name }}: ${{ github.event.issue.title }}" + description: "${{ github.event.issue.body }}\n\n_Created from GitHub by ${{ github.actor }}._" + # The field customfield_10089 refers to the Issue Link field in JIRA. + fields: '{ "customfield_10089": "${{ github.event.issue.html_url }}", "components": [{ "name": "core" }], "labels": ${{ steps.set-ticket-labels.outputs.labels }} }' + + - name: Add tracking comment + if: steps.create-ticket.outputs.issue != '' && steps.set-ticket-type.outputs.type != 'Invalid' + uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1 + with: + script: | + github.rest.issues.createComment({ + issue_number: context.issue.number, + owner: context.repo.owner, + repo: context.repo.repo, + body: ` + This issue has been synced to JIRA for planning. + JIRA ID: [${{ steps.create-ticket.outputs.issue }}](https://hashicorp.atlassian.net/browse/${{steps.create-ticket.outputs.issue}})` + }) + diff --git a/content/packer/.github/workflows/lock.yml b/content/packer/.github/workflows/lock.yml new file mode 100644 index 0000000000..7cbfe6c6c3 --- /dev/null +++ b/content/packer/.github/workflows/lock.yml @@ -0,0 +1,35 @@ +name: 'Lock Threads' + +on: + schedule: + - cron: '50 1 * * *' + +# Only 50 issues will be handled during a given run. +permissions: + contents: read + +jobs: + lock: + permissions: + issues: write # for dessant/lock-threads to lock issues + pull-requests: write # for dessant/lock-threads to lock PRs + runs-on: ubuntu-latest + steps: + - uses: dessant/lock-threads@1bf7ec25051fe7c00bdd17e6a7cf3d7bfb7dc771 # v5.0.1 + with: + github-token: ${{ github.token }} + issue-comment: > + I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues. + + If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further. + issue-inactive-days: '30' + # Issues older than 180 days ago should be ignored + exclude-issue-created-before: '2020-11-01' + pr-comment: > + I'm going to lock this pull request because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues. + + If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further. + pr-inactive-days: '30' + # Issues older than 180 days ago should be ignored + exclude-pr-created-before: '2020-11-01' + diff --git a/content/packer/.github/workflows/nightly-release-readme.md b/content/packer/.github/workflows/nightly-release-readme.md new file mode 100644 index 0000000000..4f92515642 --- /dev/null +++ b/content/packer/.github/workflows/nightly-release-readme.md @@ -0,0 +1,5 @@ +Nightly releases are snapshots of the development activity on the Packer project that may include new features and bug fixes scheduled for upcoming [releases](https://github.com/hashicorp/packer/milestones). These releases are made available to make it easier for users to test their existing build configurations against the latest Packer code base for potential issues or to experiment with new features, with a chance to provide feedback on ways to improve the changes before being released. + +As these releases are snapshots of the latest code, you may encounter an issue compared to the latest stable release. Users are encouraged to run nightly releases in a non production environment. If you encounter an issue, please check our [issue tracker](https://github.com/hashicorp/packer/issues) to see if the issue has already been reported; if a report hasn't been made, please report it so we can review the issue and make any needed fixes. + +**Note**: Nightly releases are only available via GitHub Releases, and artifacts are not codesigned or notarized. Distribution via other [Release Channels](https://www.hashicorp.com/official-release-channels) such as the Releases Site or Homebrew is not yet supported. diff --git a/content/packer/.github/workflows/nightly-release.yml b/content/packer/.github/workflows/nightly-release.yml new file mode 100644 index 0000000000..52113c9ebe --- /dev/null +++ b/content/packer/.github/workflows/nightly-release.yml @@ -0,0 +1,142 @@ +# +# This GitHub action triggers a fresh set of Packer builds +# and publishes them to GitHub Releases under the `nightly` tag. +# Note that artifacts available via GitHub Releases are not codesigned or notarized. +# Failures are reported to slack. +# + +name: Nightly Release + +on: + schedule: + # Runs against the default branch every day at midnight + - cron: "0 0 * * *" + workflow_dispatch: + +permissions: + contents: write + +jobs: + # Build a fresh set of artifacts + build-artifacts: + uses: hashicorp/packer/.github/workflows/build.yml@main + github-release: + needs: build-artifacts + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 + - name: Download built artifacts + uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8 + with: + path: out/ + # Set BUILD_OUTPUT_LIST to out\-.\*,out\... + # This is needed to attach the build artifacts to the GitHub Release + - name: Set BUILD_OUTPUT_LIST + run: | + echo "$(ls -xm1 out/)" > tmp.txt + cat tmp.txt | sed 's:.*:out/&/*:' > tmp2.txt + echo "BUILD_OUTPUT_LIST=$(cat tmp2.txt | tr '\n' ',' | perl -ple 'chop')" >> $GITHUB_ENV + rm -rf tmp.txt && rm -rf tmp2.txt + - name: Advance nightly tag + uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + try { + await github.rest.git.deleteRef({ + owner: context.repo.owner, + repo: context.repo.repo, + ref: "tags/nightly" + }) + } catch (e) { + console.log("Warning: The nightly tag doesn't exist yet, so there's nothing to do. Trace: " + e) + } + await github.rest.git.createRef({ + owner: context.repo.owner, + repo: context.repo.repo, + ref: "refs/tags/nightly", + sha: context.sha + }) + # This will create a new GitHub Release called `nightly` + # If a release with this name already exists, it will overwrite the existing data + - name: Create a nightly GitHub prerelease + id: create_prerelease + continue-on-error: true + uses: ncipollo/release-action@cdcc88a9acf3ca41c16c37bb7d21b9ad48560d87 # v1.15.0 + with: + name: nightly + artifacts: "${{ env.BUILD_OUTPUT_LIST }}" + tag: nightly + bodyFile: ".github/workflows/nightly-release-readme.md" + prerelease: true + allowUpdates: true + removeArtifacts: true + draft: false + token: ${{ secrets.GITHUB_TOKEN }} + - name: Store GitHub Release ID + if: steps.create_prerelease.outcome == 'success' + run: | + echo "prerelease_id=${{ steps.create_prerelease.outputs.id }}" >> $GITHUB_ENV + - name: Sleep before retry + id: sleep_before_retry + if: steps.create_prerelease.outcome == 'failure' + run : sleep 30m + shell: bash + - name: Retry failed nightly GitHub prerelease + id: create_prerelease_retry + if: steps.create_prerelease.outcome == 'failure' + uses: ncipollo/release-action@cdcc88a9acf3ca41c16c37bb7d21b9ad48560d87 # v1.15.0 + with: + name: nightly + artifacts: "${{ env.BUILD_OUTPUT_LIST }}" + tag: nightly + bodyFile: ".github/workflows/nightly-release-readme.md" + prerelease: true + allowUpdates: true + removeArtifacts: true + draft: false + token: ${{ secrets.GITHUB_TOKEN }} + - name: Store Updated GitHub Release ID + if: steps.create_prerelease_retry.outcome == 'success' + run: | + echo "prerelease_id=${{ steps.create_prerelease_retry.outputs.id }}" >> $GITHUB_ENV + - name: Publish nightly GitHub prerelease + uses: eregon/publish-release@01df127f5e9a3c26935118e22e738d95b59d10ce # v1.0.6 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + with: + release_id: ${{ env.prerelease_id }} + # Send a slack notification if either job defined above fails + slack-notify: + permissions: + contents: none + needs: + - build-artifacts + - github-release + if: always() && (needs.build-artifacts.result == 'failure' || needs.github-release.result == 'failure') + runs-on: ubuntu-latest + steps: + - name: Send slack notification on failure + uses: slackapi/slack-github-action@37ebaef184d7626c5f204ab8d3baff4262dd30f0 # v1.27.0 + with: + payload: | + { + "text": ":alert: Packer Nightly Release *FAILED* :alert:", + "attachments": [ + { + "color": "#C41E3A", + "blocks": [ + { + "type": "section", + "text": { + "type": "mrkdwn", + "text": "Branch: `${{ github.ref_name }}`\nRef: ${{ github.sha }}\nWorkflow: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}" + } + } + ] + } + ] + } + env: + SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }} + SLACK_WEBHOOK_TYPE: INCOMING_WEBHOOK diff --git a/content/packer/CHANGELOG.md b/content/packer/CHANGELOG.md new file mode 100644 index 0000000000..b5d615375e --- /dev/null +++ b/content/packer/CHANGELOG.md @@ -0,0 +1,6598 @@ +# 📦 Changelog +## 1.14.2 (September 9, 2025) +## ✨ Features +- **HCP Certificate Authentication Support** – by @JenGoldstrich ([#13435](https://github.com/hashicorp/packer/pull/13435)) + Adds support for the `HCP_CRED_FILE` environment variable and removes restrictions on `HCP_CLIENT_ID` and `HCP_CLIENT_SECRET` when connecting builds to an HCP Packer registry. + +- **Upgrade Node.js to v22** – by @LeahMarieBush ([#13450](https://github.com/hashicorp/packer/pull/13450)) + Updates the Node.js version used for Packer website builds. + +--- + +## 🐛 Bug Fixes +- **fix(winrm): catch cmd err from winrm** – by @anurag5sh in ([#298](https://github.com/hashicorp/packer-plugin-sdk/pull/298)) + Improved reliability by catching WinRM remote shell failures during provisioning +- **PowerShell wrapper cleanup** – by @kp2099 ([#13451](https://github.com/hashicorp/packer/pull/13451)) + Removed the unused `$result` variable from the wrapper string. +- **fix tests for shell and shell-local** – by @kp2099 in ([#300](https://github.com/hashicorp/packer-plugin-sdk/pull/300)) + Acceptance test fixes for shell and shell-local + +--- + +## 🛠 Improvements +- Added workflow-dispatch and set `PACKER_ACC_BUILDERS` for acceptance tests – by @kp2099 ([#13444](https://github.com/hashicorp/packer/pull/13444)) +- Improved spacing in `hcl2template` error messages – by @sbraz ([#13453](https://github.com/hashicorp/packer/pull/13453)) +- Added callouts for HashiCorp-maintained plugins moving to [releases.hashicorp.com](https://releases.hashicorp.com) – by @BrianMMcClain ([#13438](https://github.com/hashicorp/packer/pull/13438)) + +--- + +## 📦 Dependencies +- Bump `github.com/ulikunitz/xz` from **0.5.10 → 0.5.14** – by @dependabot ([#13459](https://github.com/hashicorp/packer/pull/13459)) +- Bump `golang.org/x/oauth2` from **0.13.0 → 0.27.0** – by @dependabot ([#13460](https://github.com/hashicorp/packer/pull/13460)) +- Bump `github.com/ulikunitz/xz` from **0.5.10 → 0.5.15** – by @kp2099 ([#13461](https://github.com/hashicorp/packer/pull/13461)) +- Bump `github.com/hashicorp/packer-plugin-sdk` from **0.6.2 → 0.6.3** – by @kp2099 ([#13462](https://github.com/hashicorp/packer/pull/13462)) + +--- + +## 👩‍💻 New Contributors +- @LeahMarieBush made their first contribution in [#13450](https://github.com/hashicorp/packer/pull/13450) 🎉 + + + +## 1.14.1 (August 5, 2025) + +### BUG FIXES: + +* core: Changes on the packer plugin installation process to ensure that the plugin is installed correctly. + The releases binary from official site releases.hashicorp.com was not getting installed from a proxy server, which was addressed in this PR. + [GH-13439](https://github.com/hashicorp/packer/pull/13439) + +## 1.14.0 (July 30, 2025) + +### IMPROVEMENTS: + +* core: Changes to pull official packer plugins binaries from official site (releases.hashicorp.com). + This change allows Packer to automatically download and install official plugins from the HashiCorp official release site. + This change standardizes our release process and ensures a more secure and reliable pipeline for plugin delivery. + [GH-13431](https://github.com/hashicorp/packer/pull/13431) + +* core: Updated the docs related to packer provisioners + [GH-13412](https://github.com/hashicorp/packer/pull/13412) + +* core/HCP: This change corrects the example docs to move the registry block to the top level of the template file + [GH-13427](https://github.com/hashicorp/packer/pull/13427) + +* core: This update adds a new section explaining conditional expressions in HCL, including syntax, usage examples, and best practices for clarity and maintainability. + The documentation covers how to use conditionals for default values, dynamic argument omission, and recommends using locals for complex logic. + [GH-13429](https://github.com/hashicorp/packer/pull/13429) + +### BUG FIXES: + +* core: Fixes the default execute command for script and scripts to its original behavior from v1.12.0. + The recent change to use the -File operator for script execution required adjustments that are addressed in this PR. + [GH-13422](https://github.com/hashicorp/packer/pull/13422) + + +### SECURITY: + +* Bump go-git from v5.13.0 to v5.16.2 + [GH-13420](https://github.com/hashicorp/packer/pull/13420) + + +## 1.13.0 (June 2, 2025) + +### FEATURES: + +* core: Enhanced the existing inline field in the shell provisioner to support any script with its own shebang (#!). + Allows users to provide scripts directly in the inline field, making it possible to use scripts generated by functions like templatefile. + Ensures that the shell provisioner executes the given script as intended, honoring the script's shebang line. + [GH-13313](https://github.com/hashicorp/packer/pull/13313) + +* core: Added new built-in functions to Packer: + sum + startswith + endswith + These new functions expand the capabilities for template logic and variable handling in Packer builds. + [GH-13349](https://github.com/hashicorp/packer/pull/13349) + +### IMPROVEMENTS: + +* hcl2: Added the ability for the packer fmt command to format multiple HCL files at once. + This enhancement allows users to format several files in a single command invocation. + [GH-13362](https://github.com/hashicorp/packer/pull/13362) + +* hcl2: Added top level hcp_packer_registry block (this configuration was previously nested inside the build block and is still valid but deprecated) + [GH-13314](https://github.com/hashicorp/packer/pull/13314) + + + +### BUG FIXES: + +* core: Updated the documentation to explicitly state that a trailing slash is required when uploading directories, regardless of the guest OS. + This clarification is intended to ensure consistent behavior and prevent confusion, especially for Windows users. + [GH-13309](https://github.com/hashicorp/packer/pull/13309) + +* core: Fixed an error occurring when using the --use-sequential-evaluation flag with packer console. + Updated the FlagSets to correctly include this flag, resolving the error. + [GH-13316](https://github.com/hashicorp/packer/pull/13316) + +* core: added a wrapper around inline PowerShell commands executed by the provisioner to catch errors and ensure the Packer build fails if any command results in an error. + [GH-13334](https://github.com/hashicorp/packer/pull/13334) + +* hcl2: Ensures that Packer now preserves the user-defined order of variable files as input by the user. + This change maintains the intended precedence of variables, honoring the order specified on the command line. + [GH-13350](https://github.com/hashicorp/packer/pull/13350) + +### SECURITY: + +* Bump to go-crypto v0.36.0 + [GH-13369](https://github.com/hashicorp/packer/pull/13369) + + +## 1.12.0 (January 22, 2025) + +### FEATURES: + +* core: add support for a DAG-based evaluation on locals and datasources. + A long-standing odditiy of Packer has been the order of evaluation for + locals and data sources. In previous versions of Packer, the + data sources were evaluated first, then the local variables were, making + it impossible to have a datasource that referenced a local variable as + part of its configuration. + This change introduces a Directed Acyclic Graph (DAG) to evaluate those + resources, instead of the phased approach of old, which makes the order + of evaluation not dependent on the type of resource, but instead of the + detected dependencies between them. + **Note**: While we are confident this should be robust enough for general + use, we do recognise that it is possible some users might encounter issues. + To give those users a way to continue using the old evaluation method, we + introduced a `-use-sequential-evaluation` command-line flag to the build, + validate, console and inspect subcommands, to force using the sequential + evaluation approach for those entities. + [GH-13155](https://github.com/hashicorp/packer/pull/13155) + +* core/hcp: support for uploading SBOMs to HCP Packer. + Software Bill of Materials (SBOM) are a standardised way to export the various + software packages linked to an artifact. As some users have expressed a + need to produce and access those for images they build, we now add the + feature to Packer itself. + While the generation of the SBOM itself is not done directly by + Packer, instead we recommend using known scanners to produce them, we add + the capacity to upload this SBOM file to HCP Packer, and link it to a + build artifact. + [GH-13171](https://github.com/hashicorp/packer/pull/13171) + +* core: support for alternate serialisation formats for plugin communication. + Packer relies on plugins to do most of the actual workload related to + building and provisioing artifacts, while Packer is mostly an orchestrator + for those plugins to perform their work. + This separation of concerns implies that both entities have to + communicate on multiple occasions during the course of a build. + Before v1.12.0 of Packer, and v0.6.0 of the plugin SDK, we used Gob to + do most of the serialisation for those steps. + This is however a bit of a problem recently, as go-cty, the library we + use for dynamic objects lifted from HCL templates, dropped support for + this a while back. + Therefore now, we introduce an alternative: protobuf/msgpack, which are + both usable and maintained by the projects around Packer, so we can begin + our transition away from gob with this change. + **Note**: as with the introduction of the DAG for locals/datasources, this + is a feature that we are reasonably confident you will not encounter bugs + with, however we cannot rule-out this possibility, therefore we introduce + a new environment variable: `PACKER_FORCE_GOB`, which if set to '1', forces + the use of Gob instead of protobuf/msgpack. + [GH-13120](https://github.com/hashicorp/packer/pull/13120) + +### IMPROVEMENTS: + +* hcl2/json: add `aws_secretsmanager_raw` funcion. + When using the AWS secretsmanager function with a non-text secret, one could + only get a secret once at a time. + This could get cumbersome if wanting to get multiple through one request, + which led people to encode their JSON/Object secrets as a big base64 + encoded string that they could get once, and then manipulate through JSON + functions. + While the workaround works, it is one extra layer of manipulations to do so, + therefore a new function to always get the raw textual version of a secret + is now added to Packer. + [GH-13242](https://github.com/hashicorp/packer/pull/13242) +* hcl2: add `alltrue` and `anytrue` functions. + As with Terraform, Packer now supports the HCL functions `alltrue` and + `anytrue`, which returns whether or not a collection only consists of + `true` values, or if any is. + [GH-13237](https://github.com/hashicorp/packer/pull/13237) +* hcl2: add `strcontains` function. + As with Terraform, Packer now supports the HCL function `strcontains`, + which returns whether or not a string contains a substring within it. + [GH-13217](https://github.com/hashicorp/packer/pull/13217) + [GH-13222](https://github.com/hashicorp/packer/pull/13222) +* datasource/http: Support other methods than GET. + The HTTP datasource used to always use GET requests for getting data + from a remote HTTP server, which was not always enough since some endpoints + may only support other methods. This change allows for most of the HTTP + methods to perform those requests. + [GH-13190](https://github.com/hashicorp/packer/pull/13190) +* hcl2: add `base64gzip` function. + In some cases, small blobs may need to be kept in memory, and injected in + a template somewhere else, but if the blob needs to be minimised, the + base64gzip function can be invoked to compress the blob and expose it + as a valid HCL2 string for use later. + [GH-13142](https://github.com/hashicorp/packer/pull/13142) + +### BUG FIXES: + +* hcl2: Fix duplicate error messages on top-level HCL violations. + A parsing quirk for HCL templates caused Packer to produce the same parsing + error multiple times if the error was caused by a top-level violation. + [GH-13245](https://github.com/hashicorp/packer/pull/13245) +* build: Include LC_UUID in Darwin binaries. + A change in how Apple authorises a plugin to access the network caused + Packer to break on recent (14.7 and above) macOS versions, as Packer uses + the local network to communicate with plugins. + The fix is to include an additional UUID into the metadata of the produced + binary, so it is authorised by macOS to use the local network, which prompts + an update to the version of Go used for building Packer (1.22.9), as it is + when this addition is supported by an LDFLAG. + [GH-13214](https://github.com/hashicorp/packer/pull/13214) +* hcl2: Don't error on empty bucker slug. + As reported by members of our community, using a hcp_packer_registry + block without a bucket slug, even if provided by external means, would cause + Packer to fail with an invalid bucket slug error. This is most definitely + a bug, which is addressed in this release. + [GH-13210](https://github.com/hashicorp/packer/pull/13210) +* hcp: fix bug when trying to extract HEAD SHA from empty Git repo. + [GH-13165](https://github.com/hashicorp/packer/pull/13165) + +### SECURITY: + +* Bump to go-crypto v0.31.0 + [GH-13233](https://github.com/hashicorp/packer/pull/13233) + +### NOTES: + +* docs: fix UUIDv4 example. A community user has found discrepancies in the + UUIDv4 examples which were used in our docs, as they do not match the + standard. + [GH-13229](https://github.com/hashicorp/packer/pull/13229) +* hcl2: fix slice initialisation method during variable evaluation phase. + [GH-13175](https://github.com/hashicorp/packer/pull/13175) + +## 1.11.2 (July 30, 2024) + +### FEATURES + +* core/hcp: export Packer options, OS, CI and VCS metadata for a build. + Following up on the introduction of metadata for builds in Packer 1.11.0, + this version introduces more metadata. In addition to the version of Packer + core, and the plugins used, we now capture CI-specific environment variables + (gitlab-ci and github-actions for now), Git-specific information, OS details + like architecture and kernel version, and the command-line options passed + to packer build. + +## 1.11.1 (July 1, 2024) + +### NOTES: +* Future Scaffolding: This release contains additional changes that allow + Packer core to validate access a HCP Packer bucket before trying to + publish to it. If the bucket does not exist and the associated service + principle does not have permission to create the bucket Packer will fail + the build.[GH-13059](https://github.com/hashicorp/packer/pull/13059) + +### SECURITY: +* core: Bump github.com/hashicorp/go-retryablehttp to address + CVE-2024-6104.[GH-13081](https://github.com/hashicorp/packer/pull/13081) + +### IMPROVEMENTS: +* core/hcl2: The issue is that local variables in templates are evaluated in a + non-deterministic order, leading to inconsistent behavior. To fix this, + local variables will now build a list of direct dependencies, similar to + datasources, and evaluate these dependencies recursively. A caveat is that + there's a recursion cap of 10 to prevent infinite recursion; if this limit + is reached, an error is returned, prompting the user to fix their template. + [GH-13039](https://github.com/hashicorp/packer/pull/13039) +* core: bump github.com/hashicorp/hcp-sdk-go from 0.96.0 to 0.99.0 + [GH-13063](https://github.com/hashicorp/packer/pull/13063) +* core: bump github.com/hashicorp/packer-plugin-sdk from 0.5.3 to 0.5.4 + [GH-13061](https://github.com/hashicorp/packer/pull/13061) + +### BUG FIXES: +* core/hcp: Change UpsertBucket to call GetBucket to address unauthorized error + from ustream API. + [GH-13059](https://github.com/hashicorp/packer/pull/13059) + +## 1.11.0 (2024-05-31) + +### NOTES: +* A LICENSE.txt file has been added to the Packer release artifacts. + [GH-12931](https://github.com/hashicorp/packer/pull/12931) + [GH-12940](https://github.com/hashicorp/packer/pull/12940) + +* **Breaking Change**: Support for loading single-component plugins has been removed from Packer. [GH-12785](https://github.com/hashicorp/packer/pull/12785) + +* **Breaking Change**: Support for loading plugin binaries following + the naming convention of packer-plugin-name has been dropped. Packer will now only load + plugins stored under the [Packer plugin directory](https://developer.hashicorp.com/packer/docs/configure#packer-s-plugin-directory) using the expected namespaced + directory and CHECKSUM files. This change drops support for loading plugin + binaries in Packer's executable directory or a template's current working + directory. [GH-12828](https://github.com/hashicorp/packer/pull/12828) + +```shell +/Users/dev/.packer.d/plugins +└── github.com + └── hashicorp + └── happycloud + ├── packer-plugin-happycloud_v0.0.1_x5.0_darwin_arm64 + └── packer-plugin-happycloud_v0.0.1_x5.0_darwin_arm64_SHA256SUM +``` + +### FEATURES: +* core: Add `-ignore-prerelease-plugins` flag to disable the use of development + plugin binaries for the `build` and `validate` commands development plugin + binaries. [GH-12828](https://github.com/hashicorp/packer/pull/12828) + [GH-12882](https://github.com/hashicorp/packer/pull/12882) +* Packer users can now track Packer version and plugin versions used for each + build artifact in HCP Packer. + [GH-12866](https://github.com/hashicorp/packer/pull/12866) +* hcl2: add textencodebase64 and textdecodebase64 funcs + For feature parity with Terraform, and since having access to strings + encoded in something that is not UTF-8 is required in some cases, we add + both the textencodebase64 and textdecodebase64 functions to HCL2 templates. + Please note these functions return base64 encoded byte slices because of how + cty/hcl defines strings (NFC normalised, UTF-8 encoded). + [GH-12997](https://github.com/hashicorp/packer/pull/12997) + +### SECURITY: +* Bump github.com/go-jose/go-jose/v3 to address GHSA-c5q2-7r4c-mv6g. + [GH-12880](https://github.com/hashicorp/packer/pull/12880) +* Bump golang.org/x/net to v0.24.0 to address GO-2024-2687. + [GH-12924](https://github.com/hashicorp/packer/pull/12924) + +### IMPROVEMENTS: +* core: Bump github.com/hashicorp/hcp-sdk-go from 0.90.0 to 0.96.0. + [GH-12935](https://github.com/hashicorp/packer/pull/12935) + [GH-12942](https://github.com/hashicorp/packer/pull/12942) + [GH-12960](https://github.com/hashicorp/packer/pull/12960) + [GH-12979](https://github.com/hashicorp/packer/pull/12979) +* core: Bump github.com/hashicorp/packer-plugin-sdk from 0.5.2 to 0.5.3 + [GH-12932](https://github.com/hashicorp/packer/pull/12932) +* core: Bump go-getter/v2 from 2.2.1 to v2.2.2 + [GH-12988](https://github.com/hashicorp/packer/pull/12988) +* datasource/http: don't error on 2xx code + Previous versions of Packer only supported 200 as a success case for the http + datasource. This change makes any status code from 200 to 299 successful. + [GH-12989](https://github.com/hashicorp/packer/pull/12989) +* core: Move to predictable plugin loading schema - Packer will now only load + plugins stored under the [Packer plugin directory](https://developer.hashicorp.com/packer/docs/configure#packer-s-plugin-directory) using the expected namespaced + directory and CHECKSUM files. + [GH-12828](https://github.com/hashicorp/packer/pull/12828) +* core: Remove support loading single-component plugins. + [GH-12785](https://github.com/hashicorp/packer/pull/12785) +* core: Rename internal `packer plugin` command to `packer execute` to avoid user confusion with + the plugins subcommand. + [GH-12865](https://github.com/hashicorp/packer/pull/12865) +* core: Packer now considers development binaries when evaluating plugin + version constraints. This work allows users to use binaries with versions + reported as "x.y.z-dev" to be used with the Packer `required_plugins` + block. [GH-12828](https://github.com/hashicorp/packer/pull/12828) +* core: Packer now supports local paths to plugins for the `packer plugins remove` + command. This addition makes it possible to pipe commands like + `packer plugins installed` with it for speedy cleanup of installed plugins. + [GH-12886](https://github.com/hashicorp/packer/pull/12886) +* core: Relax Packer source address URIs within the `required_plugins` block to + support the installation of local plugin binary using a custom or internal + source address (e.g. mycompany.com/plugins/happyorg/happycloud). Remote + installation using `packer init` or `packer plugins install` does not + support non-GitHub source URIs. Users using alternative hosts must + install plugins manually using `packer plugins install --path`. + [GH-12911](https://github.com/hashicorp/packer/pull/12911), [GH-12962] (https://github.com/hashicorp/packer/pull/12962) +* core: Remote plugins installed containing an internal version number that + differs from the version number within the binary name can lead to + confusion when tracking Packer plugin version information. To help track + such discrepancies in the plugin version, `packer init` and `packer plugin + install` have been updated to reject installation of such plugins. + 1.0.0-dev). Users are encouraged to notify plugin maintainers of any + version mismatches. + [GH-12915](https://github.com/hashicorp/packer/pull/12915), [GH-12953](https://github.com/hashicorp/packer/pull/12953), [GH-12972](https://github.com/hashicorp/packer/pull/12972) +* core: don't load plugins with metadata in name + To avoid confusion with multiple plugins that report the same effective version, + plugins installed need to have no metadata in their name. + When installed through Packer commands, the metadata is scrubbed from the name of the + installed plugin binary, but manually it may still be possible, so we enforce + that scrubbing at load-time as well. + [GH-12980](https://github.com/hashicorp/packer/pull/12980) +* core: Error when multiple paths are specified for PACKER_PLUGIN_PATH + Since Packer 1.11 removed the capability for PACKER_PLUGIN_PATH to specify + multiple directories separated by `:` or `;` (depending on the platform), we + are explicitly erroring when this is discovered, with suggestions as to how + to fix the problem. + [GH-12967](https://github.com/hashicorp/packer/pull/12967) +* core: Version metadata support for plugins. Plugins may now formally have metadata + in their versions, Packer supports it, and applies the semver recommendations on + them, i.e. they are ignored for comparison/sorting purposes, but allowed for + adding extra information about a plugin. + [GH-12888](https://github.com/hashicorp/packer/pull/12888) + +Given the specified version constraint only versions greater than or equal to 1.1.0 will be considered. + +```hcl +amazon = { + source = "github.com/hashicorp/amazon" + version = ">= 1.1.0" +} +``` +If a development binary is installed, Packer will use it if: + +1. It is the highest compatible version installed. +2. There is no final plugin version with the same version number installed alongside it. + +```shell +/Users/dev/.packer.d/plugins +└─ github.com + └─ hashicorp + └── amazon + ├── packer-plugin-amazon_v1.1.0_x5.0_darwin_arm64 + ├── packer-plugin-amazon_v1.1.0_x5.0_darwin_arm64_SHA256SUM + ├── packer-plugin-amazon_v1.1.1-dev_x5.0_darwin_arm64 + └── packer-plugin-amazon_v1.1.1-dev_x5.0_darwin_arm64_SHA256SUM +``` + +Version 1.1.1-dev of the Amazon plugin will match the specified version constraint and be used for executing the Packer build. + +If, however, a 1.1.1 release version of the plugin is available, it will have precedence over the development binary. + +```shell +/Users/dev/.packer.d/plugins +└─ github.com + └─ hashicorp + └── amazon + ├── packer-plugin-amazon_v1.1.1-dev_x5.0_darwin_arm64 + ├── packer-plugin-amazon_v1.1.1-dev_x5.0_darwin_arm64_SHA256SUM + ├── packer-plugin-amazon_v1.1.1_x5.0_darwin_arm64 + └── packer-plugin-amazon_v1.1.1_x5.0_darwin_arm64_SHA256SUM +``` + +### BUG FIXES: +* core: fix plugin version ordering to not be lexicographic. This fixes an issue + with how plugins are discovered by Packer, and ensures proper version ordering. + This means that with this change, versions that are semantically greater, + but lexicographically inferior will be loaded. + Ex: 1.0.9 vs. 1.0.10; `1.0.9 > 1.0.10` lexicographically, but semantically + `1.0.10 > 1.0.9` +* core/hcp: fix potential race condition when storing plugin details to the HCP + Packer metadata storage map. + [GH-12936](https://github.com/hashicorp/packer/pull/12936) +* core: fix plugin listing on Windows + This fix addresses bugs present in the alpha releases of 1.11, where + the discovery of Windows binaries were not matching against the + filename extension (.exe). + [GH-12981](https://github.com/hashicorp/packer/pull/12981) + +## 1.10.3 (April 22, 2024) + +### NOTES +* A LICENSE.txt file has been added to the Packer release artifacts. + [GH-12931](https://github.com/hashicorp/packer/pull/12931) + +### FEATURES +* Packer users can now track Packer version and plugin versions used for each + build artifact in HCP Packer. + [GH-12891](https://github.com/hashicorp/packer/pull/12891) + [GH-12910](https://github.com/hashicorp/packer/pull/12910) + [GH-12923](https://github.com/hashicorp/packer/pull/12923) + [GH-12926](https://github.com/hashicorp/packer/pull/12926) + +### SECURITY +* Bump github.com/go-jose/go-jose/v3 to address GHSA-c5q2-7r4c-mv6g. + [GH-12928](https://github.com/hashicorp/packer/pull/12928) +* Bump golang.org/x/net to v0.24.0 to address GO-2024-2687. + [GH-12924](https://github.com/hashicorp/packer/pull/12924) + +### BUG FIXES +* cmd/fmt: Display information error when Packer `fmt` fails due to HCL2 + parsing error. [GH-12870](https://github.com/hashicorp/packer/pull/12870) + + +## 1.10.2 (March 6, 2024) + +### NOTES: +* Continuing the work in in Packer v1.10.0 we introduced the ability to install + a locally sourced plugin using packer `plugins install --path`, this + release extends support to development plugin binaries - binaries that + report "dev" as part of their plugin version. Instead of manually placing a + downloaded binary into the executable or current working directory we + encourage you to run the command `packer plugins install –path github.com/hashicorp/happycloud` to install + the binary into a Packer compatible path. + [GH-12855](https://github.com/hashicorp/packer/pull/12855) + +### IMPROVEMENTS: +* cmd/plugins: Add support for installing local development binaries to `packer plugins install`. + [GH-12855](https://github.com/hashicorp/packer/pull/12855) +* core: Validate bucket name when using `hcp_packer_registry` block. + [GH-12820](https://github.com/hashicorp/packer/pull/12820) +* core: Update github.com/hashicorp/hcp-sdk-go from 0.83.0 to 0.85.0. + [GH-12850](https://github.com/hashicorp/packer/pull/12850) + [GH-12827](https://github.com/hashicorp/packer/pull/12827) + +### BUG FIXES: +* core/hcp: HCP Packer build failures properly distinguish between incompatible + plugins and general publishing errors. + [GH-12854](https://github.com/hashicorp/packer/pull/12854) + [GH-12835](https://github.com/hashicorp/packer/pull/12835) + +## 1.10.1 (January 30, 2024) + +### NOTES: +* **HCP Packer** has been updated to follow an artifact-agnostic approach for + publishing and consuming built artifacts. To best support the new approach, HCP Packer components within Packer have + been updated to use the v2 nomenclature when publishing build metadata to + HCP Packer. These changes include data source components for consuming HCP + Packer stored artifacts. The data source components [hcp-packer-version](https://developer.hashicorp.com/packer/docs/datasources/hcp/hcp-packer-version) + and [hcp-packer-artifact](https://developer.hashicorp.com/packer/docs/datasources/hcp/hcp-packer-artifact) respectively replace, the now deprecated, + [hcp-packer-iteration](https://developer.hashicorp.com/packer/docs/datasources/hcp/hcp-packer-iteration) + and [hcp-packer-image](https://developer.hashicorp.com/packer/docs/datasources/hcp/hcp-packer-image) + component data sources. The added changes are compatible with all existing HCP Packer workflows and do not warrant any + immediate template changes by the user. However, we encourage all HCP Packer users to visit the updated + HCP Packer [CHANGELOG](https://developer.hashicorp.com/hcp/docs/changelog) to familiarize themselves with the nomenclature changes. + [GH-12794](https://github.com/hashicorp/packer/pull/12794) [GH-12799](https://github.com/hashicorp/packer/pull/12799) +* The `hcp-packer-image` data source has been deprecated, please use HCP Packer + Artifact data source instead. + [GH-12794](https://github.com/hashicorp/packer/pull/12794) +* The `hcp-packer-iteration` data source has been deprecated, please use HCP + Packer Version data source instead. + [GH-12794](https://github.com/hashicorp/packer/pull/12794) +* The contextual variable `packer.iterationID` has been deprecated, please use + the new contextual variable `packer.versionFingerprint`. The iterationID + represents the HCP Packer iteration that build metadata has been publish + to, with this release an Iteration is now referred to as a Version and the + ID has been replaced by the Version Fingerprint. + [GH-12803](https://github.com/hashicorp/packer/pull/12803) + +### FEATURES: +* **New Data source** `hcp-packer-artifact` The HCP Packer Artifact Data Source + retrieves information about an artifact from the HCP Packer Registry. + [GH-12794](https://github.com/hashicorp/packer/pull/12794) +* **New Data source** `hcp-packer-version` The HCP Packer Version Data Source + retrieves information about HCP Packer Version from the HCP Packer. + Registry. [GH-12794](https://github.com/hashicorp/packer/pull/12794) + +### SECURITY: +* core: Bump Go dependencies to address various CVEs. [GH-12777](https://github.com/hashicorp/packer/pull/12777) +* core: Bump github.com/cloudflare/circl to address GHSA-9763-4f94-gfch. [GH-12781](https://github.com/hashicorp/packer/pull/12781) + +### IMPROVEMENTS: +* core: Bump github.com/hashicorp/hcp-sdk-go from 0.79.0 to 0.81.0 + [GH-12792](https://github.com/hashicorp/packer/pull/12792) + [GH-12764](https://github.com/hashicorp/packer/pull/12764) +* core: Update error messaging for HCP incompatible plugin builds. + [GH-12800](https://github.com/hashicorp/packer/pull/12800) +* core: Update HCP Packer integration to use HCP Packer v2 nomenclature, with + this change HCP Packer iterations are now referred to as Versions, and + Images are now referred to as Artifacts. The data sources for interacting + with HCP Packer have been respectively renamed to align with the terminology + of Version and Artifact. + [GH-12794](https://github.com/hashicorp/packer/pull/12794) + [GH-12799](https://github.com/hashicorp/packer/pull/12799) +* core: Update spacing for flags within Help text. + [GH-12742](https://github.com/hashicorp/packer/pull/12742) + [GH-12743](https://github.com/hashicorp/packer/pull/12743) + + +## 1.10.0 (December 5, 2023) + +### NOTES: +This release contains breaking changes for practitioner relying on plugins + previously bundled into Packer. As specified in the [Bundled Plugins Removal announcement](https://www.hashicorp.com/blog/announcing-the-removal-of-bundled-plugins-in-hashicorp-packer) + HashiCorp maintained plugins will no longer be shipped with Packer. + Users are encouraged to use `packer init` or `packer plugins [...]` when managing required plugins + needed for executing their builds. + [GH-12660](https://github.com/hashicorp/packer/pull/12660) + +The affected plugins include: +* [packer-plugin-amazon](https://github.com/hashicorp/packer-plugin-amazon) +* [packer-plugin-ansible](https://github.com/hashicorp/packer-plugin-ansible) +* [packer-plugin-azure](https://github.com/hashicorp/packer-plugin-azure) +* [packer-plugin-docker](https://github.com/hashicorp/packer-plugin-docker) +* [packer-plugin-googlecompute](https://github.com/hashicorp/packer-plugin-googlecompute) +* [packer-plugin-qemu](https://github.com/hashicorp/packer-plugin-qemu) +* [packer-plugin-vagrant](https://github.com/hashicorp/packer-plugin-vagrant) +* [packer-plugin-virtualbox](https://github.com/hashicorp/packer-plugin-virtualbox) +* [packer-plugin-vmware](https://github.com/hashicorp/packer-plugin-vmware) +* [packer-plugin-vsphere](https://github.com/hashicorp/packer-plugin-vsphere) + +### SECURITY: +* Bump github.com/go-jose/go-jose/v3 to address GO-2023-2334. There have been + no reported issues with Packer but we are bumping given that it is a + secondary + dependency. [GH-12723](https://github.com/hashicorp/packer/pull/12723) +* Bump Go to 1.20.10 to Address CVE-2023-44487 / CVE-2023-39325. + [GH-12661](https://github.com/hashicorp/packer/pull/12661) +* Bump Go to 1.20.11 CVE-2023-45283. There have been no reported issues with + Packer but we are bumping given its usage of the path/filepath pkg. + [GH-12690](https://github.com/hashicorp/packer/pull/12690) +* Bump Packer Plugin SDK to v0.5.2. + [GH-12717](https://github.com/hashicorp/packer/pull/12717) + +### PLUGINS: + +* Removed all HashiCorp vendored plugins from being bundled into the Packer binary. +[GH-12660](https://github.com/hashicorp/packer/pull/12660), [GH-12720](https://github.com/hashicorp/packer/pull/12720) +* packer-plugin-hcloud: The Hetzner Cloud plugin has been handed over to the + Hetzner integrations team. New releases for this plugin are available at + https://github.com/hetznercloud/packer-plugin-hcloud. Existing references + to the plugin will continue to work but users are advised to update the + `required_plugins` block to use the new plugin source address. + ``` + required_plugins { + parallels = { + source = "github.com/hetznercloud/hcloud" + version = "~> 1" + } + } + ``` +### IMPROVEMENTS: +* cmd/init: Warn users running `packer init` on configuration templates with a + missing `required_plugins` blocks. + [GH-12638](https://github.com/hashicorp/packer/pull/12638) +* cmd/plugins: Add `--path` flag to `packer plugins install` subcommand to + support installation of plugins from a local binary rather than from + GitHub. [GH-12643](https://github.com/hashicorp/packer/pull/12643) +* cmd/plugins: Install SHA256SUM file with 0644 perms. + [GH-12665](https://github.com/hashicorp/packer/pull/12665) +* cmd/plugins: Remove SHA256SUM file on plugin removal. + [GH-12666](https://github.com/hashicorp/packer/pull/12666) +* cmd/plugins: Remove will error if it fails to find the plugin being selected + for removal. [GH-12669](https://github.com/hashicorp/packer/pull/12669) +* core/hcl2: Remove empty source file reference on unset variable errors. + [GH-12712](https://github.com/hashicorp/packer/pull/12712) +* core/hcl: Improve the recursive execution of data sources with other data + source dependencies. + [GH-12608](https://github.com/hashicorp/packer/pull/12608) +* core: Bump github.com/hashicorp/hcp-sdk-go from 0.73.0 to 0.74.0 - + [GH-12704](https://github.com/hashicorp/packer/pull/12704) +* core: Bump Packer Plugin SDK to v0.5.2. + [GH-12717](https://github.com/hashicorp/packer/pull/12717) +* core: Provide integration link as hint in error message if a plugin component + is unknown. [GH-12705](https://github.com/hashicorp/packer/pull/12705) +* core: Update version output for `packer -v` and `packer --version` to match + `packer version`. + [GH-12569](https://github.com/hashicorp/packer/pull/12569) +* docs: Update plugin installation guides with supported methods for 1.10.0 and + above. [GH-12713](https://github.com/hashicorp/packer/pull/12713) + +### BUG FIXES: + +* cmd/plugins: Update `packer plugin installed` to show all installed plugin + binaries, including any duplicate versions or binaries copied from a + different source URI. + [GH-12731](https://github.com/hashicorp/packer/pull/12731) + +## 1.9.5 (December 4, 2023) + +### SECURITY + +* Bump github.com/go-jose/go-jose/v3 to address GO-2023-2334. + [GH-12723](https://github.com/hashicorp/packer/pull/12723) + +### BUG FIXES: + +* Add VirtualBox as known plugin prefix to prevent endless bundled plugin warning. + [GH-12719](https://github.com/hashicorp/packer/pull/12719) + +## 1.9.4 (August 18, 2023) + +### BUG FIXES: + +* core: When invoking Packer with the CHECKPOINT_DISABLE environment variable the telemetry + reporter is left uninitialized in order to disable telemetry reporting. + Any method calls on the nil reporter is expected to check if the reporter is active or in + NOOP mode. The SetBundledUsage function, introduced in Packer 1.9.2, failed to perform a nil + check before attempting to modify an attribute, causing Packer to fail when telemetry is + disabled. This release fixes this issue by introducing such a check. + +## 1.9.3 (August 17, 2023) + +### NOTES: +* **New Docker Image**: As part of the bundled plugin removal effort, a new + Docker target called `release-full` has been added to the Packer release + artifacts. The release-full image includes Packer and all the official + plugins pre-installed in its environment. This image is being offered as an + alternative option for those users who may still be relying on the plugin + versions bundled into the Packer binary. + [GH-12532](https://github.com/hashicorp/packer/pull/12532) + +### IMPROVEMENTS: +* core/docs: Clarify the expected usage of the `packer init` command for HCL2 + template builds.[GH-12535](https://github.com/hashicorp/packer/pull/12535) +* core/hcp: Add support for project-level service principals. A user connecting + with a project level service principals must provide a valid HCP_PROJECT_ID + in order to connect. + [GH-12520](https://github.com/hashicorp/packer/pull/12520) + [GH-12576](https://github.com/hashicorp/packer/pull/12576) +* core: A new Docker image `packer:release-full` has been added for all + supported architectures. The release-full image includes Packer and all the + official plugins pre-installed in its environment. + [GH-12532](https://github.com/hashicorp/packer/pull/12532) +* core: Add enhanced support to Packer telemetry for bundle plugins usage. + [GH-12536](https://github.com/hashicorp/packer/pull/12536) + +### BUG FIXES: + +* core: Bump golang.org/x/net to v0.13.0 to address CVE GO-2023-1988. Packer + itself is not vulnerable to the CVE as we don't render web pages, but + security checks do point it as an issue that needs to be addressed. + [GH-12561](https://github.com/hashicorp/packer/pull/12561) +* core: Fix custom plugin loading in current working directory regression. + [GH-12544](https://github.com/hashicorp/packer/pull/12544) + + +## 1.9.2 (July 19, 2023) + +### NOTES: + +* Vendored plugins within Packer have not been updated. Plugin releases occur on + a regular basis to address issues and feature requests. + Please note that in an upcoming version of Packer, we will remove the last + bundled plugins from Packer. Users are encouraged to use `packer init` for HCL2 templates or + `packer plugins install` with legacy JSON templates for installing external + plugins. + +* Packer will now warn when using bundled plugins. This feature will be removed in + a future version of the tool, so this warning is meant to bring awareness of the + upcoming change, and help users update their templates. + [GH-12495](https://github.com/hashicorp/packer/pull/12495) + +### BUG FIXES: + +* Fixed a bug with how Packer was discovering plugins: in order to load + plugins, Packer would recursively scan all the known directories in which + we could have installed plugins. This caused unnecessary directory + walks and slowdowns upon invocation. Packer will now only check + for nested plugins within the directories used by commands such as `packer + init`, or `packer plugins install`, or as defined in PACKER_PLUGIN_PATH. + Refer to + [Packer's plugin directory documentation](https://developer.hashicorp.com/packer/docs/configure#packer-s-plugin-directory) + for details on how loading works. + [GH-12414](https://github.com/hashicorp/packer/pull/12414) + +* The `packer init` subcommand now bundles all the missing installed plugins into one + condensed warning, as opposed to one warning per missing plugin. + [GH-12506](https://github.com/hashicorp/packer/pull/12506) + +### PLUGINS: + +* packer-plugin-parallels: The Parallels plugin has been handed over to the Parallels + team. New releases for this plugin are available at + https://github.com/parallels/packer-plugin-parallels. This plugin is no longer + being bundled in the Packer binary release. Existing references to the + plugin will continue to work but users are advised to update the + `required_plugins` block to use the new plugin source address. + [GH-12476](https://github.com/hashicorp/packer/pull/12476) + ``` + required_plugins { + parallels = { + source = "github.com/parallels/parallels" + version = "~> 1" + } + } + ``` + +### IMPROVEMENTS: + +* The `hcl2_upgrade` sub-command will now add `required_plugins` to the template + generated from JSON for [official plugins](https://developer.hashicorp.com/packer/plugins#tiers-and-namespaces). + [GH-12504](https://github.com/hashicorp/packer/pull/12504) + +## 1.9.1 (June 1, 2023) + +### BUG FIXES: + +* On May 16th 2023, HCP introduced multi-project support to the platform. + In order to use multiple projects in your organization, you will need to update Packer + to version 1.9.1 or above. Starting with 1.9.1, you may specify a project ID to push + builds to with the `HCP_PROJECT_ID` environment variable. If no project ID is specified, + Packer will pick the project with the oldest creation date. Older versions of Packer are + incompatible with multi-project support on HCP, and builds will fail for HCP + organizations with multiple projects on versions before 1.9.1. + [GH-12453](https://github.com/hashicorp/packer/pull/12453) + +## 1.9.0 (May 31, 2023) + +### NOTES: + +* **Breaking Change**: Iteration fingerprints used to be computed from the Git SHA of the + repository where the template is located when running packer build. This + changes with this release, and now fingerprints are automatically generated + as a ULID. This implies that continuing an existing iteration will require + users to define the fingerprint in the environment manually in order to + adopt this behaviour, otherwise, by default, a new iteration will be + created. This does not impact workflows where the fingerprint was defined + through the `HCP_PACKER_ITERATION_FINGERPRINT` environment variable, and + these builds will work exactly as they did before. + [GH-12172](https://github.com/hashicorp/packer/pull/12172) + +* **Breaking Change**: Community-maintained plugins bundled with the Packer binary have been removed. + These external plugin components are released independently of Packer core and can be installed + directly by the user. Users relying on the external plugin components listed below should refer + to the `packer plugins` sub-command and, if using HCL2, a `required_plugins` block to define a + list of plugins for building a template. + +### PLUGINS + +* Remove provisioner plugins for Chef, Converge, Puppet, Salt, and Inspec as + vendored plugins. These plugins have been previously archived and not + updated in release since being archived. These plugins can be installed + using `packer init` or with the Packer plugins sub-command `packer plugins install github.com/hashicorp/chef`. + [GH-12374](https://github.com/hashicorp/packer/pull/12374) + +* The following community plugins won't be bundled with Packer anymore: + + * [Alicloud](https://github.com/hashicorp/packer-plugin-alicloud) + * [CloudStack](https://github.com/hashicorp/packer-plugin-cloudstack) + * [HCloud](https://github.com/hashicorp/packer-plugin-hcloud) + * [HyperOne](https://github.com/hashicorp/packer-plugin-hyperone) + * [Hyper-V](https://github.com/hashicorp/packer-plugin-hyperv) + * [JDCloud](https://github.com/hashicorp/packer-plugin-jdcloud) + * [LXC](https://github.com/hashicorp/packer-plugin-lxc) + * [LXD](https://github.com/hashicorp/packer-plugin-lxd) + * [NCloud](https://github.com/hashicorp/packer-plugin-ncloud) + * [OpenStack](https://github.com/hashicorp/packer-plugin-openstack) + * [Proxmox](https://github.com/hashicorp/packer-plugin-proxmox) + * [TencentCloud](https://github.com/hashicorp/packer-plugin-tencentcloud) + * [Triton](https://github.com/hashicorp/packer-plugin-triton) + * [Yandex](https://github.com/hashicorp/packer-plugin-yandex) + + [GH-12436](https://github.com/hashicorp/packer/pull/12436) + +Users relying on these external plugin components should refer to the `packer plugins` sub-command and, +if using HCL2, a `required_plugins` block to define a list of plugins to use for building a template. + +### IMPROVEMENTS: + +* core/hcp: Now, fingerprints used by HCP Packer are randomly generated ULIDs + instead of a Git SHA, and a new one is always generated, unless one is + specified in the environment. + [GH-12172](https://github.com/hashicorp/packer/pull/12172) + +### BUG FIXES: + +* Fix LDFLAGS for release pipelines: Between Packer 1.8.5 and Packer 1.8.7, changes + to the LDFLAGS in use for building the binaries for Packer had mistakenly + removed some compilation flags, leading to the final binaries not being stripped. + This change raised the size of the built binaries by as much as 45%. + In this release, we fixed the LDFLAGS during compilation, yielding leaner binaries. + +* Bumped gopsutil to v3. This fixes a macOS intermittent crash reported by the community + [GH-12430](https://github.com/hashicorp/packer/issues/12430) + +### HCP Packer -- Multi project support advisory + +If using HCP Packer to store metadata on your builds, please be aware that Packer +does not support this feature yet, and as such, having multiple projects in your +organization will make it impossible to use HCP Packer in this release. + +We are actively working on supporting this feature, which will be available in a +subsequent release. + +## 1.8.7 (May 4, 2023) + +### NOTES: +* Vendored plugins within Packer have not been updated. Plugin releases occur on + a regular basis to address issues and feature requests. Users are encouraged + to use `packer init` for HCL2 templates or `packer plugins install` with + legacy JSON templates for installing external plugins. + +* packer-plugin-digitalocean: The Digital Ocean Packer plugin has been handed + over to the Digital Ocean team. New releases for this plugin are available + at https://github.com/digitalocean/packer-plugin-digitalocean. + ``` + required_plugins { + digitalocean = { + source = "github.com/digitalocean/digitalocean" + version = ">=1.1.1" + } + } + ``` +* packer-plugin-linode: The Linode plugin has been handed over to the Linode + team. New releases for this plugin are available at + https://github.com/linode/packer-plugin-linode. This plugin is is no longer + being bundled in the Packer binary release. Existing references to the + plugin will continue to work but users are advised to update the + `required_plugins` block to use the new plugin source address. + [GH-12329](https://github.com/hashicorp/packer/pull/12329) + ``` + required_plugins { + linode = { + source = "github.com/linode/linode" + version = ">=1.0.5" + } + } + ``` +* packer-plugin-ucloud: The UCloud plugin has been handed over to the UCloud + team. New releases for this plugin are available at + https://github.com/ucloud/packer-plugin-ucloud. This plugin is is no longer + being bundled in the Packer binary release. Existing references to the + plugin will continue to work but users are advised to update the + `required_plugins` block to use the new plugin source address. + [GH-12335](https://github.com/hashicorp/packer/pull/12335) + ``` + required_plugins { + ucloud = { + source = "github.com/ucloud/ucloud" + version = ">=1.0.8" + } + } + ``` +* packer-plugin-profitbricks: The Profitbricks plugin has been removed as a + bundled plugin in Packer. New releases for this plugin are available at + https://github.com/hashicorp/packer-plugin-profitbricks. This plugin is is + no longer being bundled in the Packer binary release. Existing references + to the plugin will continue to work but users are advised to update the + `required_plugins` block to use the new plugin source address. + [GH-12385](https://github.com/hashicorp/packer/pull/12385) + ``` + required_plugins { + ucloud = { + source = "github.com/hashicorp/profitbricks" + version = ">=1.0.2" + } + } + ``` + +### PLUGINS +* core: Migrate external Linode plugin to linode/packer-plugin-linode. + [GH-12329](https://github.com/hashicorp/packer/pull/12329) +* core: Migrate external UCloud plugin to ucloud/packer-plugin-ucloud. + [GH-12335](https://github.com/hashicorp/packer/pull/12335) +* core: Remove external plugin for Digital Ocean as a vendored plugin. + [GH-12376](https://github.com/hashicorp/packer/pull/12376) +* core: Remove external plugins for Profitbricks and 1&1 as vendored plugins. + [GH-12385](https://github.com/hashicorp/packer/pull/12385) +* docs: Add HCP Ready label to Oracle builder components. + [GH-12217](https://github.com/hashicorp/packer/pull/12217) + + +### IMPROVEMENTS +* cmd/console: Add config-type flag to command help. + [GH-12360](https://github.com/hashicorp/packer/pull/12360) +* core: Add enhanced support to Packer telemetry for HCL2. + [GH-12319](https://github.com/hashicorp/packer/pull/12319) +* Enhance zsh completion for the Packer command. + [GH-12356](https://github.com/hashicorp/packer/pull/12356), + [GH-12366](https://github.com/hashicorp/packer/pull/12366) + +### BUG FIXES +* cmd/hcl2_upgrade: Fix a crash when running the hcl2_upgrade command against a + legacy JSON template containing user variables with an undefined variables + block. [GH-12257](https://github.com/hashicorp/packer/pull/12257) +* core: Bump github.com/hashicorp/hcp-sdk-go to 0.36.0. + [GH-12292](https://github.com/hashicorp/packer/pull/12292) +* core: Bump github.com/hashicorp/packer-plugin-sdk to 0.4.0 to address + CVE-2023-0475, CVE-2022-41723. + [GH-12306](https://github.com/hashicorp/packer/pull/12306) +* core: Bump Go module version to 1.20 + [GH-12380](https://github.com/hashicorp/packer/pull/12380) +* core: Fix regression introduced in 1.8.6, where legacy JSON templates with + custom builder names are outputted to STDOUT as uninterpolated user + variables. [GH-12290](https://github.com/hashicorp/packer/pull/12290) + + +## 1.8.6 (February 15, 2023) + +### NOTES: +* core: Users will see some changes in how names are displayed during a Packer + build for JSON templates. Previously only the builder type or the builder + name, if it was set, would be displayed. Now for named builders + (`"name":"mybuilder"`) the builder's type and name will be displayed (i.e + ".mybuilder". This does not impact the behavior of options such as + only or except, they will continue to work as they did before.) + +### PLUGINS: +The following external plugins have been updated and pinned to address open + issues. Please see their respective changelogs for details on plugin + specific bug fixes and improvements. + +* alicloud@v1.0.7 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-alicloud/releases/tag/v1.0.7) +* amazon@v1.2.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-amazon/releases/tag/v1.2.1) +* azure@v1.4.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-azure/releases/tag/v1.4.0) +* googlecompute@v1.1.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-googlecompute/releases/tag/v1.1.0) +* proxmox@v1.1.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-proxmox/releases/tag/v1.1.1) +* tencentcloud@v1.0.6 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-tencentcloud/releases/tag/v1.0.6) +* qemu@v1.0.9 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-qemu/releases/tag/v1.0.9) + + +### IMPROVEMENTS: +* Bump bundled plugins to latest available version. + [GH-12274](https://github.com/hashicorp/packer/pull/12274) +* bump github.com/hashicorp/hcp-sdk-go from 0.28.0 to 0.29.0 + [GH-12163](https://github.com/hashicorp/packer/pull/12163) +* Bump github.com/hashicorp/hcp-sdk-go from 0.33.0 to 0.34.0 + [GH-12275](https://github.com/hashicorp/packer/pull/12275) +* core/hcl2: Packer will no longer warn on excluded post-processors when using + `-only/exlude` filters for running select builds. + [GH-12187](https://github.com/hashicorp/packer/pull/12187) + +### BUG FIXES: +* cmd/hcl2_upgrade: Fix crash when variables block is undefined. + [GH-12250](https://github.com/hashicorp/packer/pull/12250) +* core/hcl2: Templates with build blocks referencing an unknown source block + would display an empty string for the template filename at line 0, which + made it difficult to identify the broken build block. Packer has been + updated to display the proper filename and line number where the unknown + reference resides. + [GH-12167](https://github.com/hashicorp/packer/pull/12167) +* core: Linux packages now have vendor label and set the default label to + HashiCorp. This fix is implemented for any future releases, but will not be + updated for historical releases. + [GH-12252](https://github.com/hashicorp/packer/pull/12252) +* core/hcp: The bucket's description was not properly set in the bucket + object sent to HCP Packer leaving all newly created buckets with an empty + description. [GH-12235] + [GH-12235](https://github.com/hashicorp/packer/pull/12235) + +## 1.8.5 (December 12, 2022) + +### NOTES: +* data/packer-image-iteration has been removed. This was an undocumented and + unusable data source that was built for experimentation but not released. It + should not affect users in any way but is being mentioned for visibility + purposes. + [GH-12111](https://github.com/hashicorp/packer/pull/12111) + +### FEATURES: +* core: Metadata for capturing template types such as JSON or HCL2 has been added to the + HCP Packer registry metadata. Upon running a `packer build` the type of + template used during execution will be sent along to the registry as + additional build metadata.[GH-12132](https://github.com/hashicorp/packer/pull/12132) + +### PLUGINS: +The following external plugins have been updated and pinned to address open + issues. Please see their respective changelogs for details on plugin + specific bug fixes and improvements. + +* alicloud@v1.0.5 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-alicloud/releases/tag/v1.0.5) +* amazon@v1.1.6 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-amazon/releases/tag/v1.1.6) +* proxmox@v1.1.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-proxmox/releases/tag/v1.1.0) +* vsphere@v1.1.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-vsphere/releases/tag/v1.1.1) +* qemu@v1.0.8 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-qemu/releases/tag/v1.0.8) + +### IMPROVEMENTS: +* cmd/hcl2_upgrade: Generate variable block for all referenced user input + variables. [GH-12136](https://github.com/hashicorp/packer/pull/12136) +* cmd/validate: Add support for the `-evaluate-datasources` flag to evaluate + the data sources from a template during validation time. + [GH-12106](https://github.com/hashicorp/packer/pull/12106) + [GH-12152](https://github.com/hashicorp/packer/pull/12152) +* core/hcl2: Variable definition files containing undeclared variables within + an HCL2 template will no longer warn during build execution. Warnings will + be displayed to a user during template validation, which can be disabled by + passing the `-no-warn-undeclared-var` flag to the validate command. + [GH-12104](https://github.com/hashicorp/packer/pull/12104) + [GH-12109](https://github.com/hashicorp/packer/pull/12109) +* core: Docker images have been updated to include the `xorriso` package for + supporting the creation of ISO files.[GH-12081](https://github.com/hashicorp/packer/pull/12081) +* core: Split HCP Packer publishing components into a separate internal-only + module. [GH-11](https://github.com/hashicorp/packer/pull/11967) + [GH-12116](https://github.com/hashicorp/packer/pull/12116) + +### BUG FIXES: +* cmd/init: The init command will try to fallback to the next available version + for a plugin, if the most recent version is not available or has a missing + checksum file. This should prevent Packer from trying to install versions + that have a GitHub tag but no actual assets. + [GH-12103](https://github.com/hashicorp/packer/pull/12103) +* cmd/plugins: The `plugins install` sub-command will try to fallback to the + next available version for a plugin, if the most recent version is not + available or has a missing checksum file. This should prevent Packer from + trying to install versions that have a GitHub tag but no actual assets. + [GH-12103](https://github.com/hashicorp/packer/pull/12103) +* core: Bump Go version to 1.18.9 to address vulnerability GO-2022-1144, which + concerns the net/http and golang.org/x/net + packages.[GH-12153](https://github.com/hashicorp/packer/pull/12153) + [GH-12158](https://github.com/hashicorp/packer/pull/12158) +* core: Fix the registration of deleted input artifacts in HCP Packer when + setting `keep_input_artifacts` to + false.[GH-11462](https://github.com/hashicorp/packer/pull/11967) +* core: Using different template types when building a HCP Packer + image iteration will now trigger a runtime build error due to the + mixing of template types. + [GH-12132](https://github.com/hashicorp/packer/pull/12132) + +## 1.8.4 (October 28 2022) + +### NOTES: + +* Packer user documentation has been moved to the new HashiCorp developer + documentation portal. The main Packer site (https://www.packer.io) will + continue to be the home for Packer but all requests for general + documentation and binary downloads will be redirected to + https://developer.hashicorp.com/packer. Users are encouraged to visit the + developer documentation portal for access to all Packer related + documentation; including integrations with HCP Packer. + +* The Oracle builder and post-processor are no longer vendored with Packer + core. Users of the Oracle plugin should use `packer init` to install the + latest version of the plugin. See the [Oracle Plugin + Documentation](https://github.com/hashicorp/packer-plugin-oracle) for more + information. [GH-11983](https://github.com/hashicorp/packer/pull/11983) + +* HCP Packer environment variables: The behavior of some HCP Packer-specific + environment variables have changed slightly. Refer to [HCP Packer](https://developer.hashicorp.com/packer/docs/hcp) + in the Packer documentation for a full list of HCP Packer environment variables. [GH-12059](https://github.com/hashicorp/packer/pull/12059) + - For JSON templates, the `HCP_PACKER_REGISTRY` environment variable was + previously required to enable the HCP Packer integration. In this release, + the environment variable is now optional, and can be used for disabling the + publishing of metadata for any HCP Packer enabled configuration template. + - For HCL2 templates, the `HCP_PACKER_REGISTRY` environment variable can be used + to disable publishing to a HCP Packer registry even if the template defines a + `hcp_packer_registry` block. This can be useful for testing that a template + works as intended prior to pushing metadata to HCP Packer. + - The `HCP_PACKER_BUCKET_NAME` environment variable is now the only + requirement to push metadata to a HCP Packer registry, in both JSON + and HCL2 templates without a `hcp_packer_registry` block. + +### FEATURES: + +* provisioner/powershell: Add `use_pwsh` configuration argument to support pwsh + in powershell provisioner. [GH-11950](https://github.com/hashicorp/packer/pull/11950) + +### PLUGINS: + +* builder/oracle: Remove Oracle plugin from the list of vendored + plugins. [GH-11983](https://github.com/hashicorp/packer/pull/11983) + +### IMPROVEMENTS: + +* command/hcl2_upgrade: Has been updated to persist all possible template + engine options that were supported by the legacy JSON templates. While the + upgrade command copies the template engine options as is support for the + template options may not actually work with HCL2 templates; indicated by an + error similar to `fieldname type is invalid`. Before executing a + build with the upgraded template you are encouraged to run `packer validate` + against the template and fix any invalid `` references. + [GH-12068](https://github.com/hashicorp/packer/pull/12068) +* core/hcl2: Packer will now report an error when executing a build with no + sources selected for execution. + [GH-12016](https://github.com/hashicorp/packer/pull/12016) +* core/hcp: Configuration errors for HCP Packer enabled builds have been + consolidated into a single report to help users address all potential + issues before retrying their build. + [GH-12031](https://github.com/hashicorp/packer/pull/12031) +* core/hcp: Named builds within a legacy JSON template are now published to a + HCP Packer registry using its full build name (e.g `happycloud.windows-srv-2019)`, + as opposed to just the build name field (e.g `"name"="windows-srv-2019"`). + Builders with no defined name will continue to publish build + metadate using the builder type as the build name (e.g `happycloud`). + [GH-12059](https://github.com/hashicorp/packer/pull/12059) +* core:hcl2: When a variable is set in a variables definitions file (i.e + \*.pkrvars.hcl), but isn't defined with the template files (i.e + \*.pkr.hcl), the outputted error message will now include an example of + variable block that can be added to the build template to remedy the issue. + [GH-12020](https://github.com/hashicorp/packer/pull/12020) +* core: Add ppc64le to binary releases for Linux. + [GH-11966](https://github.com/hashicorp/packer/pull/11966) +* core: Bump github.com/hashicorp/packer-plugin-sdk from 0.3.1 to 0.3.2. + [GH-11981](https://github.com/hashicorp/packer/pull/11981) +* core: Bump supported Go version to 1.18. + [GH-11927](https://github.com/hashicorp/packer/pull/11927) + +### BUG FIXES: +* command/hcl2_upgrade: special case: Azure `shared_image_gallery` fix. + [GH-12087](https://github.com/hashicorp/packer/pull/12087) +* core: Bump golang.org/x/sys to address CVE-2022-29526. + [GH-11953](https://github.com/hashicorp/packer/pull/11953) +* core: Bump golang.org/x/text to v0.3.8. + [GH-12047](https://github.com/hashicorp/packer/pull/12047) +* core: Update dependency to resolve GO-2022-0969. + [GH-12009](https://github.com/hashicorp/packer/pull/12009) + +## 1.8.3 (August 2, 2022) + +### NOTES: +* There's been a change in the way the `ssh_timeout` and the + `ssh_handshake_attempts` configuration arguments work together. The + behaviour is unchanged if both or none are specified. However, if only one + of the two is set the other won't have a default value anymore and will be + ignored. See [Packer Plugin SDK change](https://github.com/hashicorp/packer-plugin-sdk/pull/116) for details + +* packer-plugin-digitalocean: The Digital Ocean Packer plugin has been handed over + to the Digital Ocean team. New releases for this plugin are available at + https://github.com/digitalocean/packer-plugin-digitalocean. This plugin is + still being bundled in the Packer binary but will be removed in a future + release. Existing references to the plugin will continue to work but + users are advised to update the required_plugins block to use the new + plugin source address. + [GH-11912](https://github.com/hashicorp/packer/pull/11912) +``` +required_plugins { + digitalocean = { + source = "github.com/digitalocean/digitalocean" + version = ">=1.0.8" + } +} +``` +* packer-plugin-outscale: The Outscale Packer plugin managed by the Outscale + team, since Packer 1.7.9, has been removed from the Packer binary. Users are + advised to install the latest version of the plugin by running + `packer plugins install github.com/outscale/outscale`. [GH-11912](https://github.com/hashicorp/packer/pull/11912) + +* packer-plugin-scaleway: The Scaleway Packer plugin managed by the Scaleway + team, since Packer 1.7.7, has been removed from the Packer binary. Users are + advised to install the latest version of the plugin by running + `packer plugins install github.com/scaleway/scaleway`. [GH-11912](https://github.com/hashicorp/packer/pull/11912) + +### FEATURES: +* Future Scaffolding: This release contains additional changes that allow + Packer core to validate that a newly built image is a direct child of a HCP + Packer registry source image. This feature is only available for HCP Packer + enabled builds using the `hcp_packer_image` and `hcp_packer_iteration` data + source for setting a builder's source image. + [GH-11861](https://github.com/hashicorp/packer/pull/11861) + +### PLUGINS: + +The following external plugins have been updated and pinned to address open + issues. Please see their respective changelogs for details on plugin + specific bug fixes and improvements. + +* amazon@v1.1.2 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-amazon/releases/tag/v1.1.2) +* ansible@v1.0.3 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-ansible/releases/tag/v1.0.3) +* azure@v1.3.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-azure/releases/tag/v1.3.0) +* docker@v1.0.7 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-docker/releases/tag/v1.0.7) +* googlecompute@v1.0.14 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-googlecompute/releases/tag/v1.0.14) +* lxc@v1.0.2 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-lxc/releases/tag/v1.0.2) +* triton@v1.0.2 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-triton/releases/tag/v1.0.2) +* vsphere@v1.0.7 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-vsphere/releases/tag/v1.0.7) +* yandex@v1.1.2 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-yandex/releases/tag/v1.1.2) + + +### IMPROVEMENTS: +* Bump packer-plugin-sdk to v0.3.1 to fix inconsistencies between `ssh_timeout` + and `ssh_handshake_attempts` configuration arguments in the SSH + communicator. [GH-11909](https://github.com/hashicorp/packer/pull/11909) +* core: During long running builds the HCP Packer registry will mark a build as + timed out if it has not posted an update after a certain number of minutes. + For HCP Packer enabled builds a status update will now be sent every 2 + minutes to the registry to prevent long builds from being marked as timed + out. [GH-11846](https://github.com/hashicorp/packer/pull/11846) +* data/hcp_packer_image: Add `component_type` configuration argument to + support specifying an exact build image when multiple images exist in the + same provider and region for a given HCP Packer bucket iteration. + [GH-11872](https://github.com/hashicorp/packer/pull/11872) +* data/hcp_packer_image: Add support for `channel` as input argument to + retrieve an image from the associated iteration. If + using several images from a single iteration, you may prefer sourcing an + iteration first, and referencing it for subsequent uses, as every + `hcp_packer_image` with the channel set will generate a potentially + billable HCP Packer request but if several `hcp_packer_image`s use a + shared `hcp_packer_iteration` that will only generate one potentially + billable request. + [GH-11865](https://github.com/hashicorp/packer/pull/11865) + + + +### BUG FIXES + +* core/hcl2: Fix crash when parsing malformed provisioner override blocks. + [GH-11881](https://github.com/hashicorp/packer/pull/11881) +* core/hcl2: Fix crash when running `packer validate` on templates containing + one or more HCP Packer data sources. + [GH-11883](https://github.com/hashicorp/packer/pull/11883) + + +## 1.8.2 (June 21, 2022) + +### NOTES: +* The Packer plugin SDK includes the latest version of the go-getter library, + which has been updated to address the vulnerabilities listed in + [HCSEC-2022-13](https://discuss.hashicorp.com/t/hcsec-2022-13-multiple-vulnerabilities-in-go-getter-library/39930). + The updated SDK contains changes that can + be breaking for some plugins as the updated go-getter settings in the SDK + prevent reading/writing through symlinks and to sub-directories that + require upward path traversal (e.g /tmp/.../etc/hosts). The updates also + includes a 30 minute maximum timeout for file downloading, which can be an + issue for very large or slow downloads if they exceed more than 30 minutes + to complete. + +### SECURITY: +* Bump packer-plugin-sdk to v0.3.0 to address reported vulnerabilities within + the go-getter library. + [GH-11843](https://github.com/hashicorp/packer/pull/11843) +* Bump plugins relying on go-getter for downloading remote files to address + reported vulnerabilities within the go-getter library. See [HCSEC-2022-13](https://discuss.hashicorp.com/t/hcsec-2022-13-multiple-vulnerabilities-in-go-getter-library/39930) for details. + [GH-11844](https://github.com/hashicorp/packer/pull/11844) + +### FEATURES: +* Future Scaffolding: This release contains changes that allow Packer core to + validate that a newly built image is a direct child of a HCP Packer + registry source image. This feature is only available for HCP Packer + enabled builds using the `hcp_packer_image` data source for setting a + builder's source image. + [GH-11832](https://github.com/hashicorp/packer/pull/11832) + +### PLUGINS: + +External plugins have been pinned to the following versions. Please see their + respective changelogs for details on plugin specific bug fixes and + improvements. + +* azure@v1.1.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-azure/releases/tag/v1.1.0) +* hyperv@v1.0.4 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-hyperv/releases/tag/v1.0.4) +* parallels@v1.0.3 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-parallels/releases/tag/v1.0.3) +* proxmox@v1.0.8 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-proxmox/releases/tag/v1.0.8) +* qemu@v1.0.5 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-qemu/releases/tag/v1.0.5) +* vagrant@v1.0.3 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-vagrant/releases/tag/v1.0.3) +* virtualbox@v1.0.4 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-virtualbox/releases/tag/v1.0.4) +* vmware@v1.0.7 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-vmware/releases/tag/v1.0.7) +* vsphere@v1.0.5 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-vsphere/releases/tag/v1.0.5) + +### IMPROVEMENTS: +* Add `pause_after` configuration argument to Powershell provisioner. + [GH-11792](https://github.com/hashicorp/packer/pull/11792) +* HCP Packer data sources will now fail for revoked iterations to prevent building non-compliant images. + [GH-11854](https://github.com/hashicorp/packer/pull/11854) + +### BUG FIXES: + +* Add missing support for the `env` configuration argument in remote shell + provisioners. [GH-11819](https://github.com/hashicorp/packer/pull/11819) +* The preinst and postrm user scripts, including the service configuration + directives, have been removed from the Packer rpm installations packages, + as Packer does not require a service user in order to run. + [GH-11831](https://github.com/hashicorp/packer/pull/11831) + +## 1.8.1 (May 27, 2022) + +### NOTES: +* All bundled plugins have been updated to their latest release to address any + open issues pertaining to the legacy SSH key algorithm fix made to the + Packer plugin SDK. + [GH-11761](https://github.com/hashicorp/packer/pull/11761) + [GH-11802](https://github.com/hashicorp/packer/pull/11802) +* This release contains the latest golang.org/x/crypto/ssh module which + implements client authentication support for signature algorithms based on + SHA-2 for use with existing RSA keys. Previously, a client would fail to + authenticate with RSA keys to servers that reject signature algorithms + based on SHA-1. + +### FEATURES: +* **New Datasource** HTTP data source retrieves information from a HTTP + endpoint to be used during Packer builds. + [GH-11658](https://github.com/hashicorp/packer/pull/11658) + +### PLUGINS: + +External plugins have been pinned to the following versions. Please see their + respective changelogs for details on plugin specific bug fixes and + improvements. + +* alicloud@v1.0.2 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-alicloud/releases/tag/v1.0.2) +* amazon@v1.1.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-amazon/releases/tag/v1.1.0) +* ansible@v1.0.2 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-ansible/releases/tag/v1.0.2) +* azure@v1.0.8 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-azure/releases/tag/v1.0.8) +* chef@v1.0.2 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-chef/releases/tag/v1.0.2) +* cloudstack@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-cloudstack/releases/tag/v1.0.1) +* converge@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-converge/releases/tag/v1.0.1) +* digitalocean@v1.0.3 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-digitalocean/releases/tag/v1.0.3) +* docker@v1.0.5 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-docker/releases/tag/v1.0.5) +* googlecompute@v1.0.13 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-googlecompute/releases/tag/v1.0.13) +* hcloud@v1.0.4 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-hcloud/releases/tag/v1.0.4) +* hyperone@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-hyperone/releases/tag/v1.0.1) +* hyperv@v1.0.3 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-hyperv/releases/tag/v1.0.3) +* inspec@v1.0.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-inspec/releases/tag/v1.0.0) +* jdcloud@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-jdcloud/releases/tag/v1.0.1) +* linode@v1.0.3 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-linode/releases/tag/v1.0.3) +* lxc@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-lxc/releases/tag/v1.0.1) +* lxd@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-lxd/releases/tag/v1.0.1) +* ncloud@v1.0.3 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-ncloud/releases/tag/v1.0.3) +* oneandone@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-oneandone/releases/tag/v1.0.1) +* openstack@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-openstack/releases/tag/v1.0.1) +* oracle@v1.0.2 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-oracle/releases/tag/v1.0.2) +* parallels@v1.0.2 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-parallels/releases/tag/v1.0.2) +* profitbricks@v1.0.2 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-profitbricks/releases/tag/v1.0.2) +* proxmox@v1.0.7 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-proxmox/releases/tag/v1.0.7) +* puppet@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-puppet/releases/tag/v1.0.1) +* qemu@v1.0.4 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-qemu/releases/tag/v1.0.4) +* salt@v1.0.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-salt/releases/tag/v1.0.0) +* tencentcloud@v1.0.5 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-tencentcloud/releases/tag/v1.0.5) +* triton@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-triton/releases/tag/v1.0.1) +* ucloud@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-ucloud/releases/tag/v1.0.1) +* vagrant@v1.0.2 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-vagrant/releases/tag/v1.0.2) +* virtualbox@v1.0.3 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-virtualbox/releases/tag/v1.0.3) +* vmware@v1.0.5 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-vmware/releases/tag/v1.0.5) +* vsphere@v1.0.4 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-vsphere/releases/tag/v1.0.4) +* yandex@v1.1.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-yandex/releases/tag/v1.1.1) + +### IMPROVEMENTS: +* Bump hcp-sdk-go to latest version. + [GH-11763](https://github.com/hashicorp/packer/pull/11763) +* Plugins installed via the `packer plugins install` command will now + automatically load for builds using HCL2 templates without a + required_plugins block. + [GH-11712](https://github.com/hashicorp/packer/pull/11712) +* Plugins installed via the `packer plugins install` command will now + automatically load for builds using JSON templates. + [GH-11712](https://github.com/hashicorp/packer/pull/11712) + +### BUG FIXES: +* Bump github.com/hashicorp/go-getter to fix the untarring of OVA files + regression. [GH-11675](https://github.com/hashicorp/packer/pull/11675) +* Bump github.com/hashicorp/packer-plugin-sdk to address legacy SSH key + algorithms in SSH communicator. + [GH-11774](https://github.com/hashicorp/packer/pull/11774) + [GH-11754](https://github.com/hashicorp/packer/pull/11754) + +## 1.8.0 (March 4, 2022) + +### NOTES: +* **Breaking Change**: Support for the following architectures `ppc64le, mips, + mips64, mipsle, mipsle64, s390x` have been removed from the Packer releases + page. Packer, along with the HashiCorp maintained Packer plugins, have been + updated to release binaries for the HashiCorp supported architectures arm, + arm64, 386, and amd64. A full list of supported platforms can be found + on the [Packer Downloads](https://www.packer.io/downloads) page. + [GH-11564](https://github.com/hashicorp/packer/pull/11564) + [GH-11601](https://github.com/hashicorp/packer/pull/11601) + [GH-11603](https://github.com/hashicorp/packer/pull/11603) + +* **Breaking Change**: The `packer-plugin-check`(github.com/hashicorp/packer/cmd/packer-plugins-check) has been + replaced by the [`packer-sdc plugin-check` command](https://github.com/hashicorp/packer- plugin sdk/tree/main/cmd/packer-sdc#packer-sdc). + Plugin maintainers who may be using the packer-plugin-check as part of their + release pipeline are encouraged to move to the packer-sdc command. As an + alternative, maintainers can continue to use the packer-plugin-check by + pinning the command to Packer 1.7.10. + [GH-11317](https://github.com/hashicorp/packer/pull/11317) + +### FEATURES +* **New Command** `packer plugins` command and subcommands to manage external + plugins. [GH-11553](https://github.com/hashicorp/packer/pull/11553) + [GH-11625](https://github.com/hashicorp/packer/pull/11625) + +### IMPROVEMENTS +* core: Add a `env` argument to provisioner blocks that allow for setting a + map of key/value pairs to inject prior to the execute_command. The env argument + is an alternative to using environment_vars for setting environment variables, + which has the added ability to read from Packer datasources. [GH-11569](https://github.com/hashicorp/packer/pull/11569) +* core: Bump version of go-getter to allow for downloading ISOs with PGP signed + checksums. [GH-11495](https://github.com/hashicorp/packer/pull/11495) +* core: Docker images are now available for all supported architectures that + the linux binaries are built for including arm, arm64, 386, and amd64 + [GH-11564](https://github.com/hashicorp/packer/pull/11564) + [GH-11601](https://github.com/hashicorp/packer/pull/11601) + [GH-11603](https://github.com/hashicorp/packer/pull/11603) +* core: Packer's Linux package service configs and pre/post install scripts are + now available under .release/linux. + [GH-11601](https://github.com/hashicorp/packer/pull/11601) + [GH-11603](https://github.com/hashicorp/packer/pull/11603) +* core: Packer's Linux packages are now available for all supported Linux + architectures including arm, arm64, 386, and amd64 + [GH-11564](https://github.com/hashicorp/packer/pull/11564) + [GH-11601](https://github.com/hashicorp/packer/pull/11601) + [GH-11603](https://github.com/hashicorp/packer/pull/11603) +* core: The dockerfile that is used to build the packer docker image available + at hashicorp/packer now lives in the root of this repo. The README that + describes how to build the docker targets defined in the Dockerfile are + available under ./release/docker/README.md. + [GH-11564](https://github.com/hashicorp/packer/pull/11564) + [GH-11601](https://github.com/hashicorp/packer/pull/11601) + [GH-11603](https://github.com/hashicorp/packer/pull/11603) +* core: The packer-plugin-check command has been removed. Plugin maintainers + should update their release configuration to use the `packer-sdc plugin- + check` command. [GH-11317](https://github.com/hashicorp/packer/pull/11317) + + +### BUG FIXES +* core/hcl2: Fixes an issue preventing builds from pausing between provisioners when + the `--debug` argument has been passed. + [GH-11537](https://github.com/hashicorp/packer/pull/11537) +* core/hcl2: Fixes a data loss issue when merging an empty-object map to a non- + empty map variable. + [GH-11566](https://github.com/hashicorp/packer/pull/11566) +* core/hcl2: Fixes a regression where references to locals via the lookup + function were failing to find defined keys. + [GH-11566](https://github.com/hashicorp/packer/pull/11566) +* core/hcl2: Fixes an issue where HCP Packer build labels from the first + completed build image were being copied to all images within the same + build. [GH-11574](https://github.com/hashicorp/packer/pull/11574) + [GH-11584](https://github.com/hashicorp/packer/pull/11584) +* core: HCP Packer datasources will no longer fail for iterations with + scheduled revocations. + [GH-11619](https://github.com/hashicorp/packer/pull/11619) +* core: Packer darwin builds now use macOS system DNS resolver for resolving + hostnames. [GH-9710](https://github.com/hashicorp/packer/issues/9710) + [GH-11564](https://github.com/hashicorp/packer/pull/11564) + +## 1.7.10 (February 02, 2022) + +### NOTES: + +This patch release fixes a crash in m1 caused by the psutils dep, + https://github.com/hashicorp/packer/pull/11546, originally reported by the + HomeBrew team here https://github.com/hashicorp/packer/issues/11512 + +### IMPROVEMENTS +* Print all locals errors when there is a circular error. + [GH-11527](https://github.com/hashicorp/packer/pull/11527 + +### BUG FIXES +* Prevent duplicate local block creation. + [GH-11534](https://github.com/hashicorp/packer/pull/11534) +* Update psutils dep to avoid crash on Arm64 Macs. + [GH-11546](https://github.com/hashicorp/packer/pull/11546) + +## 1.7.9 (January 19, 2022) +### NOTES: + +* HCP Packer compatible plugins will contain a "HCP Packer Ready" badge at the + top of on their documentation page. + [GH-11456](https://github.com/hashicorp/packer/pull/11456) +* Packer core [documentation](https://packer.io/docs) now supports versioning + on Packer.io. Users can select documentation for the version of Packer they + are using. [GH-11434](https://github.com/hashicorp/packer/pull/11434) +* The `hcp_packer_registry.labels` argument has been deprecated in favor of the + new `bucket_labels` argument to support custom user generated bucket + labels. [GH-11401](https://github.com/hashicorp/packer/pull/11401) +* The code of the Outscale builder was handed over to the outscale org + [GH-11428](https://github.com/hashicorp/packer/pull/11428) +* Version information for external plugins has been added as a badge to the + plugin documentation. + [GH-11457](https://github.com/hashicorp/packer/pull/11457) +* The following unmaintained external provisioner plugins have been archived: chef, + converge, inspec, salt-masterless, puppet. More + information on what this deprecation means for users can be found on the + blog post - [Plans to Archive Unmaintained Provisioner + Plugins](https://www.hashicorp.com/blog/plans-to-archive-unmaintained-packer-provisioner-plugins) + + +### PLUGINS: + +External plugins have been pinned to the following versions. Please see + their respective changelogs for details on plugin specific bug fixes and + improvements. + +* alicloud@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-alicloud/releases/tag/v1.0.1) +* amazon@v1.0.6 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-amazon/releases/tag/v1.0.6) +* ansible@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-ansible/releases/tag/v1.0.1) +* azure@v1.0.4 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-azure/releases/tag/v1.0.4) +* chef@v1.0.2 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-chef/releases/tag/v1.0.2) +* cloudstack@v1.0.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-cloudstack/releases/tag/v1.0.0) +* converge@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-converge/releases/tag/v1.0.1) +* digitalocean@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-digitalocean/releases/tag/v1.0.1) +* docker@v1.0.3 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-docker/releases/tag/v1.0.3) +* googlecompute@v1.0.9 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-googlecompute/releases/tag/v1.0.9) +* hcloud@v1.0.2 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-hcloud/releases/tag/v1.0.2) +* hyperone@v1.0.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-hyperone/releases/tag/v1.0.0) +* hyperv@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-hyperv/releases/tag/v1.0.1) +* inspec@v1.0.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-inspec/releases/tag/v1.0.0) +* jdcloud@v1.0.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-jdcloud/releases/tag/v1.0.0) +* linode@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-linode/releases/tag/v1.0.1) +* lxc@v1.0.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-lxc/releases/tag/v1.0.0) +* lxd@v1.0.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-lxd/releases/tag/v1.0.0) +* ncloud@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-ncloud/releases/tag/v1.0.1) +* oneandone@v1.0.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-oneandone/releases/tag/v1.0.0) +* openstack@v1.0.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-openstack/releases/tag/v1.0.0) +* oracle@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-oracle/releases/tag/v1.0.1) +* outscale@v1.0.2 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-outscale/releases/tag/v1.0.2) +* parallels@v1.0.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-parallels/releases/tag/v1.0.0) +* profitbricks@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-profitbricks/releases/tag/v1.0.1) +* proxmox@v1.0.4 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-proxmox/releases/tag/v1.0.4) +* puppet@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-puppet/releases/tag/v1.0.1) +* qemu@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-qemu/releases/tag/v1.0.1) +* salt@v1.0.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-salt/releases/tag/v1.0.0) +* tencentcloud@v1.0.3 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-tencentcloud/releases/tag/v1.0.3) +* triton@v1.0.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-triton/releases/tag/v1.0.0) +* ucloud@v1.0.0 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-ucloud/releases/tag/v1.0.0) +* vagrant@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-vagrant/releases/tag/v1.0.1) +* virtualbox@v1.0.1 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-virtualbox/releases/tag/v1.0.1) +* vmware@v1.0.5 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-vmware/releases/tag/v1.0.5) +* vsphere@v1.0.2 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-vsphere/releases/tag/v1.0.2) +* yandex@v1.0.3 - [CHANGELOG](https://github.com/hashicorp/packer-plugin-yandex/releases/tag/v1.0.3) +* scaleway@v1.0.4 - [CHANGELOG](https://github.com/scaleway/packer-plugin-scaleway/releases/tag/v1.0.4) + +### IMPROVEMENTS: + +* core/hcl2: Add `bucket_labels` argument to the `hcp_packer_registry` block to + support custom user generated bucket labels. + [GH-11401](https://github.com/hashicorp/packer/pull/11401) +* core/hcl2: Add `build_labels` argument to the `hcp_packer_registry` block to + support custom user generated build labels. + [GH-11401](https://github.com/hashicorp/packer/pull/11401) +* core/hcl2: Allow for the use of variables and locals within a `build` block. + [GH-11421](https://github.com/hashicorp/packer/pull/11421) +* core/hcl2: Allow for the use of variables and locals within a + `hcp_packer_registry` block. + [GH-11421](https://github.com/hashicorp/packer/pull/11421) +* core/website: Add HCP Packer Ready badge to supported plugins. + [GH-11456](https://github.com/hashicorp/packer/pull/11456) +* core/website: Add Packer version selection toggle to Packer core + documentation. [GH-11434](https://github.com/hashicorp/packer/pull/11434) +* core/website: Add version information to external plugin documentation. + [GH-11456](https://github.com/hashicorp/packer/pull/11456) +* core/website: Extract external plugins documentation from `packer.io/docs/` + into `packer.io/plugins`. + [GH-11464](https://github.com/hashicorp/packer/pull/11464) +* core: Add Packer user agent information to HCP Packer client requests. + [GH-11455](https://github.com/hashicorp/packer/pull/11455) +* core: Bump github.com/hashicorp/packer-plugin-sdk from 0.2.9 to 0.2.11 to + prevent HCP Packer builds from failing when no SourceImageID is + provided. [GH-11459](https://github.com/hashicorp/packer/pull/11459) +* core: Bump to latest preview version of hashicorp/hcp-sdk-go to prevent HCP + Packer builds from trying to update a revoked iteration. + [GH-11492](https://github.com/hashicorp/packer/pull/11492) +* provisioner/powwershell: Tiny tweaks and fixes for the PowerShell + provisioner. [GH-11410](https://github.com/hashicorp/packer/pull/11410) + +### BUG FIXES: + +* core/hcl2: Allow the use of `build.name` for naming provisioners and + post-processors. [GH-11432](https://github.com/hashicorp/packer/pull/11432) +* core/hcl2: Fix crash when a provisioner `timeout` argument is improperly + formatted. [GH-11382](https://github.com/hashicorp/packer/pull/11382) +* core/hcl2: HCP Packer builds containing metadata not expected by Packer core + will no longer fail the build. + [GH-11458](https://github.com/hashicorp/packer/pull/11458) +* provisioner/file: File provisioner will now perform a noop when no source + file content is specified; previously missing content resulted in a hard + fail. [GH-11349](https://github.com/hashicorp/packer/pull/11349) + + +## 1.7.8 (October 27, 2021) + +### BUG FIXES +* builder/amazon: Bump plugin to latest version to address a variable + interpolation issue for builder `run_tags`. [GH-11360](https://github.com/hashicorp/packer/pull/11360) + +## 1.7.7 (October 19, 2021) + +### NOTES: + +* The code of the Scaleway builder was handed over to the scaleway org + [GH-11298](https://github.com/hashicorp/packer/pull/11298) + [GH-11296](https://github.com/hashicorp/packer/pull/11296) + +### IMPROVEMENTS: +* Improved support and user experience for HCP Packer registry. + [GH-11304](https://github.com/hashicorp/packer/pull/11304) + [GH-11315](https://github.com/hashicorp/packer/pull/11315) + [GH-11320](https://github.com/hashicorp/packer/pull/11320) + [GH-11319](https://github.com/hashicorp/packer/pull/11319) + +* core: Allow to use build variables in a post processor. + [GH-11323](https://github.com/hashicorp/packer/pull/11323) +* core: Allow use in top level source variables in a build.source block. + [GH-11318](https://github.com/hashicorp/packer/pull/11318) +* core: Show successful message upon successful packer validate. + [GH-11337](https://github.com/hashicorp/packer/pull/11337) +* packer init: better error handling. + [GH-11330](https://github.com/hashicorp/packer/pull/11330) + +## 1.7.6 (September 28, 2021) + +### BUG FIXES: +* core: Update Go module dependencies to fix an issue preventing Go 1.16 users + from installing the packer-plugins-check command via go install. [GH-11282] + +## 1.7.5 (September 14, 2021) + +### NOTES: +The Exoscale builder and post-processor are no longer vendored with Packer + core, users of the Exoscale plugin should use `packer init` to install the + latest version of the plugin. See the [Exoscale Plugin + Documentation](https://github.com/exoscale/packer-plugin-exoscale#exoscale-packer-plugin) for more information. [GH-11237] + +### FEATURES: +* **Future Scaffolding** This release contains no-op refactors in preparation + for connecting Packer to the HCP Packer Registry. + +### IMPROVEMENTS: +* core: Upgrade to Go 1.17. [GH-11237] +* hcl2_upgrade: support strftime function. [GH-11220] +* provisioner/file: add option to set content + tests. [GH-11209] +* provisioner/inspec: Remove inspec provisioner from Packer core. [GH-11230] +* provisioner/salt-masterless: Remove salt-masterless provisioner from Packer + core to github.com/hashicorp/packer-plugin-salt/provisioner/salt- + masterless. [GH-11229] + +### BUG FIXES: +* builder/azure: Bump plugin to latest version to address a vulnerable + dependency in azure-sdk-for-go. [GH-11162] +* builder/googlecompute: Bump plugin to latest version to fix an issue when + building an instance via an IAP tunnel. [GH-11235] +* builder/ncloud: Bump ncloud dependency to fix dependency that got deleted + from github [GH-11224] +* core: Fix `{{packer_version}}` interpolation regression for HCL and JSON + templates. [GH-11200] +* hcl2_upgrade: Fix panic when file does not exist. [GH-11206] +* hcl2_upgrade: special case: vsphere fix. [GH-11216] + +## 1.7.4 (July 20, 2021) + +### BUG FIXES: + +* builder/outscale: Update Outscale multi-component plugin to fix `go build` + failures due to missing Go module dependencies. [GH-11147] + +## 1.7.3 (June 15, 2021) + +### IMPROVEMENTS: + +We've extracted a majority of HashiCorp-maintained and community plugins from the Packer Core repository. They now live in their own multi-component plugin repositories. This is not a breaking change as we are enabling backwards compatibility in this release by vendoring components back into Packer. +However, we encourage users to begin using `packer init` to download and install plugins to get the latest updates to each plugin, and to prepare for Packer v2.0 when we will stop vendoring the above plugins into the main Packer binary. + +The following repositories have been created, and their components have been deleted from the "github.com/hashicorp/packer" repository. + +* "github.com/hashicorp/packer-plugin-alicloud" [GH-10932] +* "github.com/hashicorp/packer-plugin-amazon" [GH-10800] +* "github.com/hashicorp/packer-plugin-ansible" [GH-10912] +* "github.com/hashicorp/packer-plugin-azure" [GH-10979] +* "github.com/hashicorp/packer-plugin-chef" [GH-10921] +* "github.com/hashicorp/packer-plugin-cloudstack" [GH-10934] +* "github.com/hashicorp/packer-plugin-converge" [GH-10956] +* "github.com/hashicorp/packer-plugin-digitalocean" [GH-10961] +* "github.com/hashicorp/packer-plugin-docker" [GH-10695] +* "github.com/hashicorp/packer-plugin-googlecompute" [GH-10890] +* "github.com/hashicorp/packer-plugin-hcloud" [GH-10966] +* "github.com/hashicorp/packer-plugin-hyperone" [GH-10949] +* "github.com/hashicorp/packer-plugin-hyperv" [GH-10949] +* "github.com/hashicorp/packer-plugin-inspec" +* "github.com/hashicorp/packer-plugin-ionos-cloud" +* "github.com/hashicorp/packer-plugin-jdcloud" [GH-10946] +* "github.com/hashicorp/packer-plugin-linode" [GH-10947] +* "github.com/hashicorp/packer-plugin-lxc" [GH-10965] +* "github.com/hashicorp/packer-plugin-lxd" [GH-10965] +* "github.com/hashicorp/packer-plugin-ncloud" [GH-10937] +* "github.com/hashicorp/packer-plugin-openstack" [GH-10933] +* "github.com/hashicorp/packer-plugin-oracle" [GH-10962] +* "github.com/hashicorp/packer-plugin-outscale" [GH-10941] +* "github.com/hashicorp/packer-plugin-parallels" [GH-10936] +* "github.com/hashicorp/packer-plugin-profitbricks" [GH-11084] +* "github.com/hashicorp/packer-plugin-proxmox" [GH-10930] +* "github.com/hashicorp/packer-plugin-puppet" [GH-10943] +* "github.com/hashicorp/packer-plugin-qemu" [GH-10929] +* "github.com/hashicorp/packer-plugin-salt" +* "github.com/hashicorp/packer-plugin-scaleway" [GH-10939] +* "github.com/hashicorp/packer-plugin-tencentcloud" [GH-10967] +* "github.com/hashicorp/packer-plugin-triton" [GH-10963] +* "github.com/hashicorp/packer-plugin-ucloud" [GH-10953] +* "github.com/hashicorp/packer-plugin-vagrant" [GH-10960] +* "github.com/hashicorp/packer-plugin-virtualbox" [GH-10910] +* "github.com/hashicorp/packer-plugin-vmware" [GH-10920] +* "github.com/hashicorp/packer-plugin-vsphere" [GH-10896] +* "github.com/hashicorp/packer-plugin-yandex" [GH-10970] + +The following components will not be removed from the main packer binary: + +* `null` builder +* `file` builder +* `breakpoint` provisioner + +* `file` provisioner +* `powershell` provisioner +* `shell` provisioner +* `shell-local` provisioner +* `sleep` provisioner +* `windows-restart` provisioner +* `windows-shell` provisioner + +* `artifice` post-processor +* `checksum` post-processor +* `compress` post-processor +* `manifest` post-processor +* `shell-local` post-processor + +### Bug Fixes: +* builder/azure: Add `keep_os_disk` parameter to control OS disk deletion + [GH-10045] +* builder/azure: Stop SIG timout from being overridden by PollingDuration + [GH-10816] +* builder/azure: Support shared image gallery storage account type [GH-10863] +* builder/proxmox: Proxmox builder use ipv4 address instead of always ipv6. + [GH-10858] +* core/hcl2_upgrade: Allow hcl2_upgrade continue with unknown builders. + [GH-11049] +* core/hcl2_upgrade: Improve regex to fix escaping on split function. + [GH-11083] +* core/hcl: Fix Invalid provisioner pause_before panic [GH-10978] +* core: HCL "index" function now actually returns the index of the element + [GH-11008] +* core: Implemented DEFAULT_NAME handling for datasource plugins [GH-11026] + +### Enhancements: + +* builder/azure: Added custom nicname and osdiskname [GH-10938] +* builder/azure: Add support for shared image gallery storage account type + [GH-10863] +* builder/digitalocean: support ecdsa, ed25519, dsa temporary key types. + [GH-10856] +* builder/ncloud: Support ncloud vpc version [GH-10870] +* core/fmt: When reading from stdin `packer fmt` will output the contents of + the formatted file even if the input was already formatted. [GH-11047] +* core/hcl: HCL variables are now supported within the `name`, `only`, + `except`, and `keep_input_artifact` fields for post-processor blocks. + [GH-11094] +* core/hcl: Running `packer build` with an `-only` or `-exclude` flag will now + inform the user if no match was found. [GH-11050] +* post-processor/compress: Add bzip2 support to post-processor [GH-10867] +* post-processor/googlecompute-import: Add Image Storage Locations field + [GH-10864] +* Removed the golang "vendor" directory in favor of go modules. This should not + affect end users. [GH-10916] + +## 1.7.2 (April 05, 2021) + +### IMPROVEMENTS: + +* builder/alicloud: Add `ramrole` configuration to ECS instance. [GH-10845] + +### BUG FIXES: + +* builder/proxmox: Update Proxmox Go API to ensure only the first non-loopback + IPv4 address gets returned. [GH-10858] +* builder/vsphere: Fix primary disk resize on clone. [GH-10848] +* core: Fix bug where call to "packer version" sent output to stderr instead of + stdout. [GH-10850] + +## 1.7.1 (March 31, 2021) + +### NOTES: + +* builder/amazon: Has been vendored in this release and will no longer be + updated with Packer core. In Packer v1.8.0 the plugin will be removed + entirely. The `amazon` components will continue to work as expected until + then, but for the latest offerings of the Amazon plugin, users are + encourage to use the `packer init` command to install the latest release + version. For more details see [Installing Packer + Plugins](https://www.packer.io/docs/plugins#installing-plugins) +* builder/docker: Has been vendored in this release and will no longer be + updated with Packer core. In Packer v1.8.0 the plugin will be removed + entirely. The `docker` builder will continue to work as expected until + then, but for the latest offerings of the Docker plugin, users are + encourage to use the `packer init` command to install the latest release + version. For more details see [Installing Packer + Plugins](https://www.packer.io/docs/plugins#installing-plugins) +* darwin/arm64: Packer now includes the darwin/arm64 binary to its releases to + supports the new OSX M1. [GH-10804] +* post-processor/docker-\*: Have been vendored in this release and will no + longer be updated with Packer core. In Packer v1.8.0 the plugin will be + removed entirely. The `docker` builder will continue to work as expected + until then, but for the latest offerings of the Docker plugin, users are + encourage to use the `packer init` command to install the latest release + version. For more details see [Installing Packer + Plugins](https://www.packer.io/docs/plugins#installing-plugins) +* post-processor/exoscale-import: Has been vendored in this release and will no + longer be updated with Packer core. In Packer v1.8.0 the plugin will be + removed entirely. The `exoscale-import` post-processor will continue to + work as expected until then, but for the latest offerings of the Exoscale + plugin, users are encourage to use the `packer init` command to install the + latest release version. For more details see [Exoscale Plugin + Repostiroy](https://github.com/exoscale/packer-plugin-exoscale). [GH-10709] + +### IMPROVEMENTS +* builder/amazon: allow creation of ebs snapshots without volumes. [GH-9591] +* builder/amazon: Fix issue for multi-region AMI build that fail when + encrypting with KMS and sharing across accounts. [GH-10754] +* builder/azure: Add client_cert_token_timeout option. [GH-10528] +* builder/google: Make Windows password timeout configurable. [GH-10727] +* builder/google: Update public GCP image project as gce-uefi-images are + deprecated. [GH-10724] +* builder/oracle-oci: Update Oracle Go SDK to add support for OCI flexible + shapes. [GH-10833] +* builder/proxmox: Allow using API tokens for Proxmox authentication. + [GH-10797] +* builder/qemu: Added firmware option. [GH-10683] +* builder/scaleway: add support for timeout in shutdown step. [GH-10503] +* builder/vagrant: Fix logging to be clearer when Vagrant builder overrides + values retrieved from vagrant's ssh_config call. [GH-10743] +* builder/virtualbox: Added ISO builder option to create additional disks. + [GH-10674] +* builder/virtualbox: Add options for nested virtualisation and RTC time base. + [GH-10736] +* builder/virtualbox: Add template options for chipset, firmware, nic, graphics + controller, and audio controller. [GH-10671] +* builder/virtualbox: Support for "virtio" storage and ISO drive. [GH-10632] +* builder/vmware: Added "attach_snapshot" parameter to vmware vmx builder. + [GH-10651] +* command/fmt: Adding recursive flag to formatter to format subdirectories. + [GH-10457] +* core/hcl2: Add legacy_isotime function. [GH-10780] +* core/hcl2: Add support for generating `dynamic` blocks within a `build` + block. [GH-10825] +* core/hcl2: Add templatefile function. [GH-10776] +* core/hcl2_upgrade: hcl2_upgrade command can now upgrade json var-files. + [GH-10676] +* core/init: Add implicit required_plugin blocks feature. [GH-10732] +* core: Add http_content option to serve variables from HTTP at preseed. + [GH-10801] +* core: Change template parsing error to include warning about file extensions. + [GH-10652] +* core: Update to gopsutil v3.21.1 to allow builds to work for darwin arm64. + [GH-10697] +* provisioner/inspec: Allow non-zero exit codes for inspec provisioner. + [GH-10723] + +### BUG FIXES +* buider/azure: Update builder to ensure a proper clean up Azure temporary + managed Os disks. [GH-10713] +* builder/amazon: Update amazon SDK to fix an SSO login issue. [GH-10668] +* builder/azure: Don't overwrite subscription id if unset. [GH-10659] +* builder/azure: Set default for the parameter client_cert_token_timeout + [GH-10783] +* builder/google: Add new configuration field `windows_password_timeout` to + allow user to set configurable timeouts. [GH-10727] +* builder/hyperv: Make Packer respect winrm_host flag in winrm connect func. + [GH-10748] +* builder/openstack: Make Packer respect winrm_host flag in winrm connect func. + [GH-10748] +* builder/oracle-oci: Update Oracle Go SDK to fix issue with reading key file. + [GH-10560] [GH-10774] +* builder/outscale: Fix omi_description that was ignored in Osc builder + [GH-10792] +* builder/parallels: Make Packer respect winrm_host flag in winrm connect func. + [GH-10748] +* builder/proxmox: Fixes issue when using `additional_iso_files` in HCL enabled + templates. [GH-10772] +* builder/qemu: Make Packer respect winrm_host flag in winrm connect func. + [GH-10748] +* builder/virtualbox: Make Packer respect winrm_host flag in winrm connect + func. [GH-10748] +* builder/vmware: Added a fallback file check when trying to determine the + network-mapping configuration. [GH-10543] +* builder/vsphere: Fix invalid device configuration issue when creating a + vm with multiple disk on the same controller. [GH-10844] +* builder/vsphere: Fix issue where boot command would fail the build do to a + key typing error. This change will now retry to type the key on error + before giving up. [GH-10541] +* core/hcl2_upgrade: Check for nil config map when provisioner/post-processor + doesn't have config. [GH-10730] +* core/hcl2_upgrade: Fix escaped quotes in template functions [GH-10794] +* core/hcl2_upgrade: Make hcl2_upgrade command correctly translate + pause_before. [GH-10654] +* core/hcl2_upgrade: Make json variables using template engines get stored as + locals so they can be properly interpolated. [GH-10685] +* core/init: Fixes issue where `packer init` was failing to install valid + plugins containing a 'v' within its name. [GH-10760] +* core: Packer will now show a proper error message when failing to load the + contents of PACKER_CONFIG. [GH-10766] +* core: Pin Packer to Golang 1.16 to fix code generation issues. [GH-10702] +* core: Templates previously could not interpolate the environment variable + PACKER_LOG_PATH. [GH-10660] +* post-processor/vagrant-cloud: Override direct upload based on box size + [GH-10820] +* provisioner/chef-solo: HCL2 templates can support the json_string option. + [GH-10655] +* provisioner/inspec: Add new configuration field `valid_exit_codes` to allow + for non-zero exit codes. [GH-10723] +* provisioner/salt-masterless: Update urls for the bootstrap scripts used by + salt-masterless provide. [GH-10755] + +## 1.7.0 (February 17, 2021) + +### FEATURES +* **New Command** (HCL only) `packer init` command will download plugins defined + in a new `required_plugins` block [GH-10304] [GH-10633]. +* **New Plugin Type** Data sources can be implemented (blog post forthcoming). + [GH-10440] +* **New Plugin** Aws Secrets Manager data source [GH-10505] [GH-10467] + +### BACKWARDS INCOMPATIBILITIES +* core: The API that the Packer core uses to communicate with community plugins + has changed; maintainers of community plugins will need to upgrade their + plugins in order to make them compatible with v1.7.0. An upgrade guide will + be available on our guides page https://www.packer.io/guides. + +### IMPROVEMENTS +* builder/amazon: Add `skip_create_ami` option for testing and situations where + artifact is not the ami. [GH-10531] +* builder/amazon: Add IMDSv2 support for AWS EBS builder. [GH-10546] +* builder/amazon: Add resource tags in the launch template used to request spot + instances. [GH-10456] +* builder/openstack: Add `skip_create_image` option for testing and situations + where artifact is not the image. [GH-10496] +* builder/oracle-oci: Add retry strategies to oci calls [GH-10591] +* core/fmt: The `packer fmt` can now read from stdin. [GH-10500] +* core/hcl: Add regex and regexall hcl2 template functions. [GH-10601] +* core/hcl: Templates now support "sensitive" locals. [GH-10509] +* core/hcl: Templates now support error-cleanup-provisioner. [GH-10604] +* hcl2_upgrade: Command now comes with a flag so you can control whether output + templates are annotated with helpful comments. [GH-10619] +* hcl2_upgrade: Command now gracefully handles options with template engine + interpolations. [GH-10625] +* hcl2_upgrade: Command will convert amazon filters to use the ami data source. + [GH-10491] + +### BUG FIXES +* amazon/ebssurrogate: Apply snapshot tags at same time as when taking + snapshot. [GH-10150] +* builder/amazon: Fix bug where validation fails if optional iops value is + unset. [GH-10518] +* builder/amazon: Wrap API call to get filtered image in a retry. [GH-10610] +* builder/bsusurrogate: override bsu when omi root device is set. [GH-10490] +* builder/google: Fix bug where Packer would fail when run by users who do not + have permission to access the metadata, even though the metadata is not + necessary to the run. [GH-10458] +* builder/profitbricks: Profitbricks builder could not connect using SSH + communicator. [GH-10549] +* builder/proxmox: Ensure ISOs in additional_iso_files are mounted during VM + creation. [GH-10586] +* builder/proxmox: Improve cloud init error logging for proxmox builder. + [GH-10499] +* builder/qemu: Fix bug where vnc_min_port set to value greater then 5900 could + prevent Packer from connecting to QEMU. [GH-10450] [GH-10451] +* builder/qemu: Fix regression with cd indexing when disk_interface is `ide`. + [GH-10519] +* builder/vmware-esx: Skip credential validation, which requires ovftool to be + installed, if we are not exporting an image. [GH-10520] +* builder/yandex: Fix cloud-init config for ubuntu 20.04. [GH-10522] +* builder/yandex: Fix incorrect access to `instance_id`. [GH-10522] +* core/hcl: Fix bug where []uint8 types could not be passed to plugins. +* core/hcl: fix bug where HCL core could not handle passing []uint8 to plugins. + [GH-10516] +* core/hcl: Fix force flag for hcl2 provisioners and post-processors. + [GH-10571] +* post-processor/vsphere: Fix regression where Packer would not check the exit + status after streaming UI from the ovftool command. [GH-10468] +* post-processor/yandex-export: Changed dhclient command and supported + configuring disk for exportupdate-dump-method. Also added support for + `file` builder. [GH-10488] + +## 1.6.6 (December 16, 2020) + +### FEATURES +* **New command** `fmt` allows users to format existing HCL2 configuration + files into a canonical style. Please see [fmt command + docs](https://packer.io/docs/commands/fmt) for more details. [GH-10225] + [GH-10377] +* **New function** `env` allows users to set the default value of a variable to + the value of an environment variable. Please see [env function + docs](https://www.packer.io/docs/templates/hcl_templates/functions/contextual/env) for + more details. [GH-10240] +* **Future Scaffolding** This release contains a large number of no-op + refactoring changes. The Packer team at HashiCorp is preparing to split the + plugins and core to make it easier for our third party maintainers and + community members to release and maintain plugins, just like HashiCorp did + with the Terraform Core-Provider split. The Packer team is committed to + making sure that this split is seamless for our users and for our community + maintainers -- if you are a community maintainer, you may want to follow + along with some of the work by looking at the + [core-plugin-split github tag.](https://github.com/hashicorp/packer/pulls?q=is%3Apr+label%3Acore-plugin-split) + No one needs to do anything, yet, but we felt it was worth calling out all + the work that isn't making it into the changelog. We will be following up + with lots of documentation and communication in early 2021 with more + information. + +### IMPROVEMENTS +* builder/amazon-ebs: Add tags to launch templates. [GH-10203] +* builder/amazon: Add support for Amazon EBS gp3 volumes. [Gh-10338] +* builder/amazon: Increase default max_retries to lessen throttling issues. + [GH-10290] +* builder/amazon: Support AWS gp3 volumes [GH-10338] +* builder/amazon: Support root volume encryption for amazon-chroot. [GH-10243] +* builder/amazon: Validate IOPS ratio. [GH-10199] +* builder/azure-arm: Add Azure CLI authentication support to builder. + [GH-10157] +* builder/azure-arm: Create keyvaults with SoftDelete enabled. [GH-10210] +* builder/digitalocean: New option to provision with private ip. [GH-10093] +* builder/google: Add `wait_to_add_ssh_keys` option to delay the addition of + SSH configuration that may be disrupted during an instance boot sequence. + [GH-10320] +* builder/google: Add support for creating shielded VMs. [GH-10172] +* builder/googlecompute-export: Add logging.write to service account scopes. + [GH-10316] +* builder/oracle-oci: Support image launch mode. [GH-10212] +* builder/outscale: Add outscale.hk endpoint support [GH-10207] +* builder/outscale: Add x509 certificate support. [GH-10161] +* builder/proxmox: New config option for boot-order. [GH-10260] +* builder/scaleway: Use the SDK functions to load profile from file and env. + [GH-10181] +* builder/virtualbox: Allow attaching guest additions with "none" communicator. + [GH-10306] +* builder/vmware: Make compatible with MacOS BigSur by using Apple DHCP leases + instead of VMWare leases [GH-10384] +* builder/vsphere: New option to add additional storage to a cloned vm. + [GH-10287] +* builder/yandex: More resilient image mounting and initialization. [GH-10335] +* builder/yandex: Update user-data to not use cloud-config fields to prevent + possible user data collisions. [GH-10385] +* core/hcl: Update to `hcl2_upgrade` command to support complex variable values + and packer version blocks. [GH-10221] +* hcl2upgrade: Update command to fix `env` call upgrade. [GH-10244] +* post-processor/vagrant-cloud: Add support for uploading directly to storage + on Vagrant Cloud. [GH-10193] +* post-processor/yandex-export: Add retries and wait after disk attach + operation. [GH-10303] +* post-processor/yandex-export: Show progress on export. [GH-10368] +* post-processor/yandex-export: Use ssh communicator in export. [GH-10352] +* post-processor/yandex-export: Verify the access to a specific bucket. + [GH-10188] +* provisioner/salt-masterless: Call winrepo.update_git_repos and + pkg.refresh_db. [GH-10201] + +### BUG FIXES +* builder/amazon: Fix retry logic in AWS spot instance tagging. [GH-10394] +* builder/amazon: Fix single `tag` interpolation to allow for templating engine + usage. [GH-10224] +* builder/google: Fix crash when using the `-on-error` build flag. [GH-10247] +* builder/google: Fix issue with service account detection when running Packer + on a compute instance with `use_os_login` enabled. [GH-10360] +* builder/qemu: Fix duplication of main disk when setting "disk_image: true". + [GH-10337] +* builder/qemu: Fix nil pointer dereference when loading values from state. + [GH-10249] +* builder/qemu: Fix panic when disk_image=true and source image has no file + extension. [GH-10226] +* builder/vagrant: Return error if ssh-config command fails. [GH-10213] +* builder/vsphere: WaitForIP should not return an error if an IP is not found + [GH-10321] +* builder/yandex: Change disk creation method to manual. [GH-10250] +* builder/yandex: Fix issue with UserAgent string. [GH-10361] +* builder/yandex: Fixed using cloud config when using IPv6. [GH-10297] +* core/hcl: Ensure the `reverse` function does not break when given a value of + type list. [GH-10380] +* post-processor/yandex-export: Check service account id. [GH-10305] + +## 1.6.5 (October 30, 2020) + +### FEATURES: +* New Builder(s): Proxmox builder has been split into two new builders + `proxmox-iso` and `promox-clone`. See [Proxmox + Builder](https://packer.io/docs/builders/proxmox) for more information on + the builder. For users of the previous `proxmox` builder please use `packer + fix` to migrate your templates to the new `promox-iso` builder. [GH-9262] + +### BUG FIXES: + +* builder/amazon: SSM connection now recovers from reboots. [GH-10003] +* builder/azure-arm: Fix build failures due to the deletion of additional + managed disks defined in "disk_additional_size". [GH-10163] +* builder/azure-chroot: Fix typo in option `exlude_from_latest` to + `exclude_from_latest`. Old name will still be respected. [GH-10034] +* builder/googlecompute: Fix HCL image_encryption_key fields and use the same + casing in JSON and HCL2 [GH-10173] +* builder/openstack: Fix source image validation regression when using filters. + [GH-10065] +* builder/proxmox: Fix unhandled buildvar type for HCL2 enabled build + templates. [GH-10154] +* builder/qemu: Fix a regression where Packer would not wait properly in + step_shutdown when a null communicator was used. [GH-10178] +* builder/qemu: Fix crash in step_run of qemu when loading commhostport form + the statebag in a situation where the communicator is none. [GH-10145] +* builder/vsphere-clone: Packer was not respecting the "destroy" flag set in + the content library config, and always keeping the source vm. This has been + fixed. [GH-10165] +* builder/vsphere: Ensure builds are able to continue when no communicator has + been specified `"communicator": "none"`. [GH-9964] +* builder/vsphere: Fix CD uploads so that Packer does not try to delete a CD + that was not successfully uploaded. [GH-10155] +* core/hcl: Hide sensitive variables from output. [GH-10031] +* core/hcl: Packer HCL's "Coalesce" function now behaves same way as + Terraform's. [GH-10016] +* core: Fix artifact handling so that input artifacts are properly preserved in + postprocessors that don't modify artifacts. [GH-9996] +* core: Fix pathing in cd_files to copy proper directory tree when user + provided absolute paths. [GH-10022] +* provisioner/ansible: Ansible galaxy no longer forces use of collections in v1 + files. [GH-10010] + +### IMPROVEMENTS: + +* builder/amazon-ebssurrogate: Apply snapshot tags at snapshot creation time. + [GH-10150] +* builder/amazon: Add `io2` as a supported volume type. [GH-10102] +* builder/amazon: Add support for source instance tenancy [GH-10085] +* builder/google: Add service account impersonation. [GH-9968] [GH-10054] +* builder/googlecompute: Add `skip_create_image` option. [GH-10115] +* builder/googlecompute: Allow users to select the algorithm to use when + generating temporary SSH keypair [GH-10111] +* builder/linode: Add `state_timeout` attribute to Linode builder. [GH-10128] +* builder/oracle-oci: New option to specify image compartment separate from + build compartment. [GH-10040] +* builder/oracle-oci: New option to specify boot volume size. [GH-10017] +* builder/oracle: Add `base_image_filter` option as alternative to + `base_image_ocid` [GH-10116] +* builder/outscale: Migrate to new Outscale SDK. [GH-10056] +* builder/proxmox: split Proxmox into proxmox-iso and proxmox-clone. [GH-9626] + [GH-10166] +* builder/scaleway: Allow the user to use an image label (eg ubuntu_focal) + instead of a hardcoded UUID on the Scaleway builder. [GH-10061] +* builder/vsphere: Skip iso download if hashed file is already present on + remote datastore. [GH-10143] +* builder/yandex: Add support for IAM credentials in the token field and + YC_TOKEN environment variable. [GH-10158] +* core/hcl: Add ability to set version restrictions [GH-10149] +* core/hcl: Add build.name variable so users can access build name in addition + to source name. [GH-10114] +* core/hcl: Add consul_key function to HCL templates. [GH-10119] +* core/hcl: Add HCL2 aws_secretsmanager function [GH-10124] +* core/hcl: Add packer.version variable to hcl configs so users can access the + Packer release version. [GH-10117] +* core: Let user provide type of generated ssh key instead of always doing ssh- + rsa [GH-10101] + +## 1.6.4 (September 30, 2020) + +### BUG FIXES: +* builder/amazon: Fix authentication issue when using instance profiles or + assumed roles for loading session-derived credentials. [GH-10007] +* builder/azure: Fix crash when using `azure_tag` or `azure_tags` configuration + options. [GH-10014] +* builder/qemu: Ensure `qemu_img_args` are honored during the disk convert + step. [GH-10001] + + ## 1.6.3 (September 25, 2020) + +### IMPROVEMENTS: +* builder/amazon: Add `pause_before_ssm` option to pause for some time before + establishing a Session Manager session; defaults to 10s. [GH-9988] +* builder/amazon: Implement assume_role option that matches Terraform behavior. + [GH-9981] +* builder/azure: Support publishing to a Shared Image Gallery with a different + subscription id [GH-9875] +* builder/openstack: Add `external_source_image_url` and + `external_source_image_format` to support building images from external + source URLs. [GH-9992] +* builder/openstack: Include API requests and responses as part of the debug + log output. [GH-9972] +* builder/oracle-oci: Add `create_vnic_details` option for launch details. + [GH-9856] +* builder/oracle-oci: Allow freeform and defined tags to be added to an instance. + [GH-9802] +* builder/proxmox: Add `io_thread` option for supporting io threads when using + a `virtio-scsi-single` controller with a `scsi` or `virtio` disk type. + [GH-9969] +* builder/proxmox: Add ability to specify interfaces for http_directory and VM. + [GH-9874] +* builder/proxmox: Allow the mounting of multiple ISOs via the `cd_drive` + option. [GH-9653] +* builder/proxmox: Fix boot command special keys. [GH-9885] +* builder/qemu: Add `qemu_img_args` option to set special cli flags for calls + to qemu-img [GH-9956] +* builder/qemu: Add `skip_resize_disk` option to skip the resizing of QCOW2 + images. [GH-9896] [GH-9860] +* builder/qemu: Skip qemu-img convert on MacOS to prevent the creation of + corrupt images [QEMU + #1776920](https://bugs.launchpad.net/qemu/+bug/1776920) [GH-9949] +* builder/scaleway: Change default boottype to local. [GH-9853] +* builder/scaleway: Update scaleway to use non-deprecated sdk. [GH-9902] +* builder/vmware: Add `vnc_over_websocket` to allow the sending of a + `boot_command` to hosts running ESXi 6.7 and above. [GH-9938] +* builder/vmware: Allow user to set vmware tools source path. [GH-9983] +* builder/vsphere-clone: Add ability to set `mac_address` [GH-9930] +* builder/vsphere-clone: Add floppy_files, cd_files, and iso_paths options. + [GH-9963] +* builder/vsphere-iso: Add NVMe controller support. [GH-9880] +* builder/vsphere: Look for a default resource pool when root resource pool is + not found. [GH-9809] +* core: Add support for running cygwin/msys2 based cd/iso creation tool + [GH-9954] +* core: New `cd_files` option to mount iso for modern OSes which don't support + floppies. [GH-9796] [GH-9919] [GH-9928] [GH-9932] [GH-9941] +* HCL2: When the type of a variable is not known evaluate setting as a literal + string instead of a variable name. [GH-9863] +* post-processor/vagrant: Support the use of template variables within + Vagrantfile templates. [GH-9923] +* post-processor/yandex-import: Allow custom API endpoint. [GH-9850] +* provisioner/ansible: Add support for Ansible Galaxy Collections. [GH-9903] + +### BUG FIXES: +* builder/amazon-ebs: Fix issue where retrying on invalid IAM instance profile + error was creating multiple spot instances. [GH-9946] +* builder/amazon-ebssurrogate: Fix issue where builder defaults to AWS managed + key even when custom `kms_key_id` is set. [GH-9959] +* builder/amazon: Update ssm_driver log polling logic to prevent infinite loops + when SSM driver is terminated outside of Packer. [GH-9991] +* builder/azure: Fix crash when using HCL2 configs. [GH-9984] [GH-9985] +* builder/qemu: Fix hardcoded lowerbound causing negative ports [GH-9905] +* builder/qemu: Skip compaction when backing file is used. [GH-9918] +* builder/scaleway: Add pre validate step to prevent the creation of multiple + images with the same name. [GH-9840] +* builder/vmware-iso: Prevent the use of reserved SCSI ID 0:7 when attaching + multiple disks. [GH-9940] +* builder/vsphere: Fix overly strict iso_path validation regex. [GH-9855] +* command/console: Prevent failure when there are unknown vars. [GH-9864] +* command/inspect: Allow unset variables in HCL2 and JSON. [GH-9832] +* core: Prevent the UI progressbar from hanging and crashing when there is no + TTY available. [GH-9974] +* core: Use $APPDATA over $HOME on Windows hosts when determining homedir. + [GH-9830] +* post-processor/digitalocean-import: Fix crash caused by empty artifact.Files + slice. [GH-9857] +* post-processor/yandex-export: Check for error after runner completes. + [GH-9925] +* post-processor/yandex-export: Set metadata key to expected value on error. + [GH-9849] +* post-processor/yandex-import: Fix S3 URL construct process. [GH-9931] + +## 1.6.2 (August 28, 2020) + +### FEATURES: +* **New command** `hcl2_upgrade` is a JSON to HCL2 transpiler that allows users + to transform an existing JSON configuration template into its HCL2 template + equivalent. Please see [hcl2_upgrade command + docs](https://packer.io/docs/commands/hcl2_upgrade) for more details. + [GH-9659] + +### IMPROVEMENTS: +* builder/amazon: Add all of the custom AWS template engines to `build` + template function for use by provisioners. [GH-9751] +* builder/amazon: Add aws_polling config option to override env variables. + [GH-9777] +* builder/azure: Add FreeBSD support to azure/chroot builder. [GH-9697] +* builder/vmware-esx: Add `network_name` option to vmware so that users can set + a network without using vmx data. [GH-9718] +* builder/vmware-vmx: Add additional disk configuration option. Previously + only implemented for vmware-iso builder [GH-9815] +* builder/vmware: Add a `remote_output_directory option` so users can tell + Packer where on a datastore to create a vm. [GH-9784] +* builder/vmware: Add option to export to ovf or ova from a local vmware build + [GH-9825] +* builder/vmware: Add progress tracker to vmware-esx5 iso upload. [GH-9779] +* builder/vsphere-iso: Add support for building on a single ESXi host + [GH-9793] +* builder/vsphere: Add new `directory_permission` config export option. + [GH-9704] +* builder/vsphere: Add option to import OVF templates to the Content Library + [GH-9755] +* builder/vsphere: Add step and options to customize cloned VMs. [GH-9665] +* builder/vsphere: Update `iso_paths` to support reading ISOs from Content + Library paths [GH-9801] +* core/hcl: Add provisioner "override" option to HCL2 templates. [GH-9764] +* core/hcl: Add vault integration as an HCL2 function function. [GH-9746] +* core: Add colored prefix to progress bar so it's clearer what build each + progress bar belongs to. [GH-9780] +* core: Ui now pretty prints build durations. [GH-9749] +* core: When a build is cancelled, Packer will skip postprocessors gracefully + rather than failing them. [GH-9720] +* integrations/secretsmanager: Add support for plaintext non-key/pair secrets. + [GH-9773] +* post-processor/vsphere: Improve UI to catch bad credentials and print errors. + [GH-9649] +* provisioner/ansible-remote: Add `ansible_ssh_extra_args` so users can specify + extra arguments to pass to ssh [GH-9821] +* provisioner/file: Clean up, bugfix, and document previously-hidden `sources` + option. [GH-9725] [GH-9735] +* provisioner/salt-masterless: Add option to option to download community + SaltStack formulas. [GH-9726] + +### BUG FIXES: +* build: Fix bug in code generator that caused generation to fail in nested + packer/packer dirs [GH-9728] +* build: Fix Makefile so that default target doesn't crash and creates dev + binaries. [GH-9706] +* builder/amazon-ebssurrogate: Make skip_save_build_region option work in the + ebssurrogate builder, not just the ebs builder. [GH-9666] +* builder/amazon: Add retry logic to the spot instance creation step to handle + "Invalid IAM Instance Profile name" errors [GH-9810] +* builder/amazon: Update the `aws_secretsmanager` function to read from the AWS + credentials file for obtaining default region information; fixes the + 'MissingRegion' error when AWS_REGION is not set [GH-9781] +* builder/file: Make sure that UploadDir receives the interpolated destination. + [GH-9698] +* builder/googlecompute: Fix bug where startup script hang would cause export + to hang. [GH-9708] +* builder/hyperv: Send boot command in small chunks to make it more stable. + [GH-9765] +* builder/scaleway: Fix config issue that made scaleway builder fail when used + with HCL2 config. [GH-9677] +* builder/vmware: Fully destroy vm if it was cancelled or errored. This will + make orphaned vms easier to destroy through vCenter. [GH-9782] +* builder/vsphere: Fix `alt`, `ctrl`, and `shift` keypresses in the + boot_command. [GH-9702] [GH-9739] +* builder/vsphere: Fix bug where Packer timed out if two interfaces were + defined but only one had an available IP. [GH-9748] +* builder/vsphere: Fix the configuration_parameters option so that it is always + applied, not just when the tool sync policy is set. [GH-9713] +* communicator: Fix `pause_before_connect` option to force a reconnect after + the pause. [GH-9772] +* core: Make `max_retries` provisioner option a string to allow variable + interpolation. [GH-9673] +* post-processor/vsphere-template: Fix ReregisterVM to default to true instead + of false. [GH-9736] +* post-processor/yandex-export: Fix issue when validating region_name [GH-9814] +* provisioner/inspec: Fix the 'Unsupported argument; An argument named + "command"' error when using the inspec provisioner in an HCL2 configuration + [GH-9800] + +## 1.6.1 (July 30, 2020) + +### BACKWARDS INCOMPATABILITIES: + +* HCL: builder/vsphere: Add option to add a xhci/usb3 controller; changes + controller value to an array of strings. [GH-9574] +* HCL: New HCL-only `post-processors` block to run chained post-processors + after a build [GH-9638]. Before this, defining multiple `post-processor` + blocks after provisioning steps would run them sequentially, now doing this + makes them start from the build artifact. To queue post-processors you now + have to define them in a `post-processors` block. [GH-9638] +* post-processor/vSphere: We have fixed a bug in the ovftool URL encoding. This + may mean that you have performed an encoding workaround that is no longer + necessary. [GH-9589] + +### FEATURES: +* **New post-processor** Yandex Import [GH-9553] + +### IMPROVEMENTS: +* builder/amazon-ebs: Wrap CreateImage call in a retry to account for eventual + consistency [GH-9579] +* builder/azure: Disable SSH password authentication unless password is + explicitly specified. [GH-9603] +* builder/docker: Add options for --cap-add, --cap-drop, --tmpfs, --device + [GH-9565] +* builder/file: Create parent directories of target file, if they don't exist. + [GH-9452] +* builder/googlecompute: Add `wrap_startup_script` configuration option to + allow the disabling of Packer's startup script wrapper [GH-9505] +* builder/googlecompute: Add support for oslogin via the `use_os_login` + configuration option [GH-9339] +* builder/googlecompute: Make IAP tunnel timeout configurable. [GH-9545] +* builder/googlecompute: Support using WinRM over an IAP tunnel [GH-9610] +* builder/hyper-v: Include secure boot template in box.xml [GH-9552] +* builder/hyperone: Add support for custom username in vm creation. [GH-9497] +* builder/hyperone: Skip chroot device discovery. [GH-9489] +* builder/openstack: Bump gophercloud to latest version [GH-9573] +* builder/proxmox: Add option to disable KVM hardware virtualization in Proxmox + builder [GH-9587] +* builder/proxmox: Add support for multiple NIC packet queues [GH-9597] +* builder/proxmox: Enable Proxmox builder to toggle the firewall parameter for + network interfaces. [GH-9487] +* builder/proxmox: Update Proxmox storagePoolTypes [GH-9418] +* builder/qemu: Add 'cdrom_interface' option to QEMU builder [GH-9483] +* builder/tencentcloud: Add `source_image_name` to support getting source image + by name [GH-9619] +* builder/tencentcloud: Update cvm root disk type to `CLOUD_PREMIUM` [GH-9663] +* builder/ucloud: New access config options and run config options. [GH-9466] +* builder/vsphere-clone: Add `boot_command` support to vsphere-clone builder, + including support for starting an HTTP server +* builder/vsphere-clone: Add `vapp` configuration option [GH-9507] +* builder/vsphere: Add ability to define multiple disk controllers [GH-9519] +* builder/vsphere: Add `boot_command` support to vsphere-clone builder. [GH-9397] +* builder/vsphere: Add `content_library_destination` to import VM template to a + Content Library [GH-9551] +* builder/vsphere: Add `force_bios_setup` configuration option [GH-9563] +* builder/vsphere: Add option to add a xhci/usb3 controller [GH-9574] +* builder/vsphere: Create vm output folders if they don't exist [GH-9402] +* builder/vsphere: Fix file size descriptor when exporting OFV [GH-9568] +* builder/vsphere: Look at all available IPs in the waiting for IP loop. + [GH-9450] +* builder/vsphere: Match network to host when multiple networks are found + [GH-9556] +* builder/vsphere: Update vsphere `boot_command` to bring it in line with other + builders' boot_command functionality. [GH-9406] +* builder/vsphere: Use datacenter inventory path for find folder [GH-9390] +* builder/vsphere: Use value from "ip_wait_address" option to determine the + default for the http server IP [GH-9441] +* builder/yandex: Allow set `min_disk_size` for an image. +* builder/yandex: Support authentication by Service Account on instance + [GH-9383] +* builder/yandex: yandex: Add new property `min_disk_size` of built image + [GH-9594] +* communicator/ssh: Add support for OpenSSH certificate signing [GH-9521] +* communicator/ssh: Allow users to provide a list of ciphers that they want + Packer to support. [GH-9453] +* core/hcl2: Add possibility to name singular build.source blocks to + differentiate their output and to filter on them [GH-9490] +* core/hcl2: Add the "inspect" command for hcl2 configs. [GH-9468] +* core/hcl2: HCL configs now respect only/except using build names instead of + types. [GH-9454] +* core/hcl: Allow use of `keep_input_artifact` in post processors. [GH-9477] +* core/hcl: Share build info with Provisioner and Post-Processor via HCL2 + variables [GH-9444] [GH-9534] [GH-9622] +* core: Add on-error flag option to run error-cleanup-provisioner [GH-9429] +* core: communicator/ssh: Add new `ssh_key_exchange_algorithms` option to + supply custom key exchange algorithms in SSH client [GH-9634] +* core: refactor initialization out from Packer core to allow `validate + --syntax-only` to no error when a variable is not set [GH-9627] +* hcl2: Handle uint64 buildvars [GH-9585] +* post-processor/yandex-export: Allow users to utilize generated variables in + templating. [GH-9555] +* post-processor/yandex-export: Support Authentication by Service Account Key + file [GH-9379] +* post-processor/yandex-import: Support creating an Image based on another one + [GH-9614] +* post-processor/yandex-import: Support using URL from yandex-export [GH-9601] +* provisioner/ansible: Add template option for templating the inventory file + lines [GH-9438] + +### BUG FIXES: +* builder/amazon: Change "Resource" field in + `temporary_iam_instance_profile_policy_document` to be an array of strings, + not just a single string. [GH-9509] +* builder/amazon: HCL2: Add singular `run_volume_tag` block to ebs & ebssurrogate + builders. [GH-9457] +* builder/amazon: Retry fetching block device mappings if empty. [GH-9480] +* builder/azure: Fix data disks URI. [GH-9467] +* builder/googlecompute: Fix issue with `use_iap` globally changing a user's + gcloud project configuration, by temporarily setting "project" via project + flag and not via `gcloud config` [GH-9662] +* builder/googlecompute: Fix the "secure boot" validation for uefi_compatible + images. [GH-9371] +* builder/qemu: Only set up localhost port forwarding if skipnatmapping is + false. [GH-9479] +* builder/vagrant: Fix box file validation for remote box files specified using + `source_path` [GH-9660] +* builder/vagrant: Improve validation and error handling around synced_folder. + Make sure that synced folder can be defined relative to Packer run + directory, not the Vagrant output directory. [GH-9577] +* builder/virtualbox-vm: Fix regression where builder would fail if the vm had + no snapshots. [GH-9435] +* builder/vmware-iso: Try checksum remote file instead of local file. [GH-9584] +* builder/vmware: Fix a race that occurred when parsing the network config. + [GH-9387] +* builder/vmware: Update vendor library, enabling retries on 502 errors + [GH-9391] +* builder/vsphere-clone: Fix SSH public key injection for cloned templates + [GH-9507] +* builder/vsphere: Clean up folder path to remove leading slashes. [GH-9542] +* builder/vsphere: Deduplicate where Firmware is set in vsphere-iso builder + [GH-9557] +* builder/vsphere: Fix vsphere ToolsSyncTime and ToolsUpgradePolicy [GH-9515] +* builder/vsphere: vSphere driver context is no longer cancelled when Packer + context is cancelled. [GH-9576] +* communicator/winrm: Add the "no_proxy" environment variable option to have + winrm bypass the proxy set by the http_proxy or https_proxy environment + vars, when connecting to the remote instance. [GH-9267] +* core: Do not print download progress bar if a machine-readable UI is chosen. + [GH-9448] +* post-processor/amazon-import: Add support for retrying RequestLimitExceeded + errors when importing an image [GH-9537] +* post-processor/docker-import: Fix crash when using docker-import with HCL. + [GH-9670] +* post-processor/vsphere: Fix password encoding in vsphere post-processor + ovftool call [GH-9589] +* post-processor/yandex-export: Fix error handling and docs. [GH-9554] +* provisioner/ansible-local: Fix agent auth in SSH communicator [GH-9639] +* provisioner/ansible: Correct check for whether PackerHttpAddr is implemented + or not [GH-9498] +* provisioner/ansible: Quote extra-var packer_build_name to handle names with + spaces [GH-9590] + +## 1.6.0 (June 09, 2020) + +### BACKWARDS INCOMPATIBILITIES: +* builder/vsphere-iso: The deprecated fields `network`, `network_card`, + `disk_size`, `disk_thin_provisioned`, and `disk_eagerly_scrub` have been + removed. Run `packer fix template.json` to migrate an existing template + [GH-9149] + +* For all iso-based builders, the `iso_checksum_type` and `iso_checksum_url` + fields have been removed in favor of simply setting the `iso_checksum` + field. Use the [packer fix](https://www.packer.io/docs/commands/fix/) + command to update a config file. See the [iso_checksum field + docs](https://www.packer.io/docs/builders/virtualbox/iso/#iso_checksum) to + read more about this. [GH-8437] + +### FEATURES: +* **New post-processor** Yandex Export [GH-9124] +* builder/amazon: Add SSM Session Manager as a SSH interface connection + [GH-9082] +* builder/google: Implement iap proxy for googlecompute [GH-9105] +* HCL2/core: Add command/console support for HCL2 config files [GH-9359] +* HCL2/core: Add command/validate support for HCL2 config files [GH-9346] +* HCL2/core: it is now possible to set source fields from the `build` block + [GH-9291] + +### IMPROVEMENTS: +* builder/amazon: Added new `build` template function field: + SourceAMICreationDate [GH-9277] +* builder/azure-arm: Support User Assigned Managed Identity [GH-9293] +* builder/azure-chroot: Copy data disks between shared image galleries + [GH-9323] +* builder/googlecompute: Changed default disk size. [GH-9071] +* builder/googlecompute: New option to specify storage location for GCP images + [GH-9326] +* builder/qemu: Add support for using a network bridge [GH-9159] +* builder/qemu: Added `skip_nat_mapping` option to skip the communicator (SSH + or WinRM) automatic port forward and use the guest port directly. [GH-9307] +* builder/qemu: Replace deprecated `ssh_host_port_min` and `ssh_host_port_max` + by `host_port_min` and `host_port_max`. [GH-9307] +* builder/virtualbox: Add `output_filename` config option to allow user to set + a custom filename instead of forcing it to be the same as vm_name. + [GH-9174] +* builder/vsphere: Add `floppy_label` option. [GH-9187] +* builder/vsphere: Add `ip_wait_address` to allow packer to wait for an ip + address that fits within the range (cidr address). [GH-9358] +* builder/vsphere: Add `tools_sync_time` and `tools_upgrade_policy` options to + set time synchronization and automatic VMware Tools upgrade. [GH-9366] +* core/hcl: Enable "force", "debug", and "on-error" command line flags for hcl2 + builds. [GH-9234] +* core/hcl: Enable only/except for provisioners and post-processors [GH-9357] +* core/hcl: Support named builds in HCL2 templates [GH-9245] +* core/interpolation: Add support for specifying a particular key to fetch from + the AWS Secrets Manager [GH-9202] [GH-9286] +* core: HCL logs now display source type and source name (`type.name`) in logs + to differentiate more easily who says what. [GH-9257] +* core: update consul and vault dependencies [GH-9205] +* core: Update vendored "go-getter" library with checksum fixes. +* core: Users can now configure the http bind address of the server launched to + serve files from http_directory [GH-9313] +* core: When a template contains a deprecated option, Packer will now encourage + the user to call `packer fix`. [GH-9325] +* post-processor/docker-push: Support pushing multiple tags [GH-9182] +* post-processor/docker-tag: Change field name of docker tag to "tags" instead + of "tag" since it's a list. Keep "tag" for backwards compatibility. + [GH-9183] +* provisioner/windows-restart: Display full hostname, instead of just the + NetBIOS name [GH-9335] + +### BUG FIXES: +* buidler/vsphere-iso: Fix bug validating iso_urls [GH-9321] +* builder/azure-arm: Update runtime constants with a valid DataDiskName + [GH-9251] +* builder/digitalocean: Use correct image type for Droplet creates. [GH-9212] +* builder/openstack: Don't error if metadata can't be set. Old versions of + openstack don't support that API call. [GH-9198] +* builder/vagrant: Use absolute path for package_include files to prevent them + from having to be relative to the output vagrant directory. [GH-9260] +* builder/virtualbox: Fix bug using checksum files. [GH-9101] +* builder/vmware: Fix ssh connection when the guest platform's dhcpcd switches + the IP address in-between a build. [GH-9322] +* builder/vsphere-iso: Use the Datacenter's VmFolder call instead of manually + generating folder path [GH-9342] +* builder/vsphere: Add option not to set host during datastore upload. [GH-9100 +* builder/vsphere: Fix crash in the driver for an interface conversion of + types.AnyType nil to types.ManagedObjectReference. [GH-9354] +* builder/vsphere: Fix iso config prepare being called incorrectly, which + caused `iso_url` field to fail. [GH-9197] +* builder/yandex: Do not require 'gpu-standard-v1' platform_id for any GPU- + based config. [GH-9356] +* core: Ensure HTTP server information `PackerHTTPIP`, `PackerHTTPPort`, and + `PackerHTTPAddr` are available via the `build` template engine for all + supported builders [GH-9238] +* core: Fix `iso_url` to accept SMB shared files UNC and windows network paths. + [GH-8954] +* core: Fix regression that broke use of pwd when retrieving the checksum from + a file [GH-9129]. +* post-processor/vagrant: Add "provider_override" template option to allow + artifacts from the Artifice post-processor [GH-9239] +* post-processor/vsphere-template: Add VSphere builder's artifact to vsphere- + template's supported types [GH-9146] +* postprocessor/artifice: Update various core post-processors to accept + artifacts from the Artifice post-processor [GH-9239] +* provisioner/ansible-remote: Fix the arg order to make sure that the playbook + file is the last item in the call [GH-9279] +* provisioner/ansible: Ansible provisioner doesn't force ssh key checking if + user wants to use password to connect instead. [GH-9350] +* provisioner/inspec: Fix build variables interpolation [GH-9262] +* provisioner/powershell: fix interpolation of execute_command in cleanup + script call. [GH-9275] +* provisioner/powershell: Fix long-wait retry loop caused by cleanup logic + [GH-9226] +* provisioner/salt-masterless: Ignore the CmdArgs field in hcl2 [GH-9290] + +## 1.5.6 (May 1, 2020) + +### BACKWARDS INCOMPATIBILITIES: +* core/hcl2: HCL2 singular blocks: use key/value or name/value depending on how + the service names things [GH-9078] +* core/hcl2: Maps are now treated as settable arguments as opposed to blocks. + For example `tags = {}` instead of `tags {}` [GH-9035] +* `packer build` command: removed option to set `parallel=false`, use + `-parallel-builds=1` for this. + +### FEATURES: +* **New Builder** azure-dtl allows creation of devtestlabs images in Azure + [GH-8987] + +* **New Core Feature** provisioners now support a `max_retries` option that can + be used for retrying a provisioner on error [GH-9061] + +* **New Post-Processor**: `yandex-export` Upload built image in Yandex Object Storage. + +### IMPROVEMENTS: +* builder/azure-arm: Add `boot_diag_storage_account` option for enabling boot + diagnostics on a virtual machine [GH-9053] +* builder/azure-arm: Add support for setting custom resource names [GH-9028] +* builder/azure-arm: Data disk names are now randomly generated [GH-8986] +* builder/azure: Add shared image destination for azure-chroot [GH-9021] +* builder/azure: Add shared image source for chroot builder [GH-9070] +* builder/google: Add support for custom shielded images [GH-8970] +* builder/google: Allow `source_image_project_id` to be a list of several + projects to search. [GH-8679] +* builder/hyperv: Add `boot_order` option to override the default boot order + [GH-9046] +* builder/oracle-oci: Allow Instance Principal Auth for Oracle OCI builder + [GH-8893] +* builder/oracle-oci: Update Oracle SDK. [GH-9104] +* builder/proxmox: Add ability to add a cloud-init drive [GH-9045] +* builder/vsphere: Add `disable_shutdown` option to support manual vm shutdown + over the default automatic shutdown step [GH-9095] +* builder/vsphere: Add `vgpu_profile` option for specifying vGPU profiles + [GH-8946] +* builder/vsphere: Add support for EFI Secure Boot [GH-9018] +* builder/vsphere: Add support for specifying vGPU profiles [GH-8946] +* builder/yandex: Add `target_image_folder_id ` option for changing the folder + where a built image will be saved to [GH-9080] +* core/hcl2: HCL mode builds now honor -only and -except options [GH-8947] +* core/hcl2: Set `packer_build_name` and `packer_builder_type` variables for + builder provisioners and post-processors [GH-8956] +* core/HCL: HCL mode now honors -only and -except options. [GH-8947] +* core: New template function: aws_secretsmanager [GH-9099] +* provisioner/ansible: Add `use_proxy` option to not use localhost proxy adapter. Removes + need for ansible connection_plugin when using WinRM. [GH-8625] +* provisioner/powershell: Add `debug_mode` option to help with debugging + generated scripts [GH-8996] +* provisioner/powershell: Add cleanup step to remove temporarily created + scripts; cleanup can be skipped by setting the `skip_clean` option + [GH-8908] + +### BUG FIXES: +* builder/amazon: Fix bug with `launch_block_device_mappings` in spot + instances. [GH-8945] +* builder/azure-arm: Fix issue where managed image builds were using a + different `location` then what was specified in the build configuration + [GH-9068] +* builder/azure: Allow Managed Data Disks to be used with Azure Shared Image + Gallery [GH-8912] +* builder/azure: Fix SSH connection for temporary admin users specified in + `ssh_username` [GH-9103] +* builder/osc: Make compliant with oAPI spec for Outscale osc-bsu builder + [GH-9093] +* builder/qemu: Remove `net_device` pre-validation [GH-8979] +* builder/tencentcloud: Update builder to handle the + InstanceOperationInProgress error [GH-9069] +* builder/vsphere-iso: disk_size is no longer required if storage is defined + [GH-8975] +* builder/vsphere: Add exported files to VSphere artifact [GH-9020] +* builder/vsphere: Fix issue where -force is not working with vsphere builders + [GH-9039] +* core: Fix crash in wrapperreadline helper when calling `os.NewFile` on + unknown file descriptor [GH-9037] +* core: Make sure CLI variables supersede variables from var files [GH-8964] +* provisioner/powershell: Fix integer decoding issue in the execution policy + parser [GH-8997] + +## 1.5.5 (March 25,2020) + +### BACKWARDS INCOMPATIBILITIES: +* core: Interpolation of undefined variables will now error as expected, in + previous versions variables were allowed to be set as a command line + argument but that was because the validation was being ignored + +### IMPROVEMENTS: +* builder/azure: Add support for configurable KeyVault SKU [GH-8879] +* builder/hyperv: Add `first_boot_device` setting to allow the selection of the + initial device or device class used for booting the VM. [GH-8714] +* builder/hyperv: Fix Hyper-V compacted disk size comparison [GH-8811] +* builder/openstack: Add new `image_auto_accept_members` option [GH-8931] +* builder/proxmox: Add ability to specify vga adapter [GH-8892] +* builder/proxmox: Add onboot directive support [GH-8935] +* builder/tencentcloud: Show tencentcloud image id after copy to destination + region. [GH-8763] +* builder/vmware-iso: Add `cleanup_remote_cache` config option to [GH-8917] +* builder/vmware-iso: Do not perform dial test of NIC when ssh bastion is + required [GH-8877] +* builder/vsphere-clone: Add ability to export VM to OVF file [GH-8764] +* builder/vsphere-iso: Add ability to define multiple disks. [GH-8787] +* builder/vsphere-iso: Add ability to export VM to OVF file [GH-8764] +* builder/vsphere-iso: Add support for eagerly zeroed / scrubbed disks. + [GH-8756] +* builder/vsphere-iso: Add the remote iso first so that it is first in boot + order, and clarify boot behavior. [GH-8732] +* communicator/ssh: Add flag to enable support for keyboard-interactive auth to + connect bastion [GH-8847] +* core/hcl2: Add support for singular blocks [GH-8889] +* core/hcl2: Add support in HCL2 configs for dynamic blocks, document for loops + and splat expressions [GH-8720] +* core/hcl2: Fix HCL2 local variables decoding to allow local usage within + another local in the same locals block [GH-8755] +* core/hcl2: Import new replace and regex_replace funcs from go-cty + + documentation [GH-8863] +* core: Enable hcl files as var files in HCL mode [GH-8882] +* core: Make "build" engine template variables SSHPublicKey and SSHPrivateKey + strings [GH-8829] + +### Bug Fixes: +* builder/azure: Fix `winrm_password` attribution and allow users to set + `winrm_username` [GH-8928] +* builder/azure: Fix azure key vault cleanup failure [GH-8905] +* builder/azure: Fix HCL2 bug that prevented Azure and other builders from + loading properly. [GH-8785] +* builder/googlecompute: Fix WinRMPassword template engine. [GH-8890] +* builder/googlecompute: Replace deprecated "sshKeys" metadata with "ssh-keys" + to fix SSH authentication issue [GH-8942] +* builder/proxmox: Add new validation to catch that template_name cannot + contain spaces. [GH-8799] +* builder/proxmox: Bump proxmox-api-go to fix upstream bug where users hit open + file limit. [GH-8800] +* builder/vagrant: Fix path validation in ssh config step. [GH-8826] +* builder/virtualbox-vm: Fix crash when VM has no snapshots. [GH-8906] +* builder/virtualbox: Remove all floppy controllers before adding a new one. + [GH-8828] +* builder/vsphere-clone: Fix issue preventing the cloning of VMs with the same + name in different folders [GH-8938] +* builder/vsphere-iso: Fix issue preventing the creation of VMs with the same + name in different folders [GH-8938] +* builder/vsphere: Fix network object interface panic. [GH-8753] +* core/hcl2: Fix crash when an unset variable is used [GH-8837] +* core/hcl2: Fix logic for parsing literal value variables [GH-8834] +* core/hcl2: Make sure locals are evaluated only after variables are. [GH-8918] +* core: Fix "build" template engine interpolation for certain fields in certain + provisioners. [GH-8771] +* core: Fix bug where user var recursion could fail intermittently when used + with env vars [GH-8875] +* core: Interpolation of undefined variables will now error as expected +* plugins: Make plugin discovery stricter with respect to periods so that users + can disable plugins by renaming the extension [GH-8735] +* provisioner/salt: Fix `no_exit_on_failure` config to work correctly as + expected. [GH-9119] +* provisioner/shell: "inline" config option is now a template engine. [GH-8883] + +## 1.5.4 (February 14, 2020) +no-change release to fix code-signing on OSX binaries. Since checksums for these +binaries has changed, we are releasing a second time to prevent confusion. + +## 1.5.3 (February 14, 2020) + +### IMPROVEMENTS: +* builder/vsphere: Add ability to define multiple NICs for vsphere-iso + [GH-8739] +* builder/vsphere: Add option to remove CD-ROM drives. [GH-8690] +* core: Add validation to catch when users accidentally add duplicate fields to + template [GH-8725] + +### Bug Fixes: +* core/hcl2: Fix template prepare/validation for HCL2 templates [GH-8742] +* core: Fix `build` template function interpolation [GH-8727] + +## 1.5.2 (February 12, 2020) +**New Builder** The vsphere-iso builder, previously maintained by JetBrains, +has been merged with the Packer core. It will be officially supported by the +Packer team at HashiCorp moving forward. [GH-8480] + +**HCL2 variables & functions** HCL2 configurations can now use `variable`, +`variables`, `locals`, and functions [GH-8588]. + +### IMPROVEMENTS: +* builder/alicloud: Add AlicloudProfile option. [GH-8560] +* builder/amazon: Add max_retries option to aws builders [GH-8709] +* builder/amazon: Add source AMI owner ID/name to template engines [GH-8550] +* builder/amazon: Update instance waiters to use global waiter settings set by + `AWS_POLL_DELAY_SECONDS` and `AWS_TIMEOUT_SECONDS` [GH-8699] +* builder/azure: Allow users to use custom key vault for storing Windows + certificates [GH-8704] +* builder/azure: Set expiry for image versions in SIG [GH-8561] +* builder/proxmox: Add option to upload the boot ISO rather than pointing out a + previously manually uploaded one. [GH-8624] +* builder/vagrant: Fix a crash in the Vagrant driver [GH-8607] +* builder/yandex: Add service account ID to config [GH-8717] +* communicator/winrm: Users can now override winrm_host with a static IP even + when using cloud builders. [GH-8675] +* core/hcl2: Fix bug preventing reading slices within other slices [GH-8669] +* core: Interpolation within post-processors can now access build-specific + values like Host IP, communicator password, and more [GH-8632] +* core: Add `PACKER_PLUGIN_PATH` to list of supported paths for plugin + discovery [GH-8616] +* core: clean up messy log line in plugin execution. [GH-8542] +* core: Ensure `PACKER_HTTP_ADDR` is always set for any builder that provides a + HTTP server for file transfer [GH-8654] +* core: Fix loading external plugins defined in PACKER_CONFIG [GH-8582] +* core: Log name of postprocessor running to disambiguate long chains of post- + processors. [GH-8613] +* core: Packer can use isos in-place on Windows again, instead of copying them + into its cache. [GH-7627] +* core: step_download: return without error if Urls is empty [GH-8579] +* post-processor/vsphere-template] Simplify method to use vm.MarkAsTemplate + (optionally) [GH-8511] +* scripts: Fix some issues with mapstructure-to-hcl2 code generator. [GH-8574] +* scripts: Update Vagrant bootstrapping scripts [GH-8604] + +### Bug Fixes: +* builder/alicloud: Fix "security group doesn't exist" error when there are >10 + security groups. [GH-8535] +* builder/amazon: Allow AWS builder pre-validation to pass when subnet filters + are present [GH-8622] +* builder/azure: Fix bug where deployments were not being cleaned up: [GH-8496] +* builder/azure: Fix issue where WinRMPassword was being left unset [GH-8670] +* builder/lxd: Fix file uploading issue when using the file provisioner + [GH-8636] +* builder/null: Fix crash when configuring builder using HCL2. [GH-8612] +* builder/osc: Fix ssh host detection in Public Cloud and Nets [GH-8414] +* builder/vagrant: Fix bug with reading key from a path with spaces [GH-8605] +* builder/virtualbox-ovf: Remove config dependency from StepImport [GH-8509] +* builder/virtualbox-vm: use config as a non pointer to avoid a panic [GH-8576] +* core: Fix crash when build.sources is set to an invalid name [GH-8569] +* core: Fix error loading .packerconfig [GH-8623] +* core: Fix loading local ISO files when using `iso_target_path` [GH-8689] +* core: Fix loading of external plugins. GH-8543] +* post-processor/docker-tag: Fix regression if no tags were specified. + [GH-8593] +* post-processor/vagrant-cloud: Update error handling for Vagrant Cloud API + [GH-8594] +* post-processor/vagrant: correctly handle the diskSize property as a qemu size + string [GH-8567] +* provisioner/ansible: Fix password sanitization to account for empty string + values. [GH-8570] +* provisioner/shell: Fix bug with shell provisioner failing to clean up the + environment var file when env_var_file is true. [GH-8639] + +## 1.5.1 (December 20, 2019) +This was a fast-follow release to fix a number of panics that we introduced when +making changes for HCL2. + +### IMPROVEMENTS: +* builder/alicloud: Add show_expired option for describing images [GH-8425] + +### Bug Fixes: +* builder/cloudstack: Fix panics associated with loading config [GH-8513] +* builder/hyperv/iso: Fix panics associated with loading config [GH-8513] +* builder/hyperv/vmcx: Fix panics associated with loading config [GH-8513] +* builder/jdcloud: Update jdcloud statebag to use pointers for config [GH-8518] +* builder/linode: Fix panics associated with loading config [GH-8513] +* builder/lxc: Fix panics associated with loading config [GH-8513] +* builder/lxd: Fix panics associated with loading config [GH-8513] +* builder/oneandone: Fix panics associated with loading config [GH-8513] +* builder/oracle/classic: Fix panics associated with loading config [GH-8513] +* builder/oracle/oci: Fix panics associated with loading config [GH-8513] +* builder/osc/bsuvolume: Fix panics associated with loading config [GH-8513] +* builder/parallels/pvm: Fix panics associated with loading config [GH-8513] +* builder/profitbricks: Fix panics associated with loading config [GH-8513] +* builder/scaleway: Fix panics associated with loading config [GH-8513] +* builder/vagrant: Fix panics associated with loading config [GH-8513] +* builder/virtualbox/ovf: Fix panics associated with loading config [GH-8513] +* builder/virtualbox: Configure NAT interface before forwarded port mapping + #8514 +* post-processor/vagrant-cloud: Configure NAT interface before forwarded port + mapping [GH-8514] + +## 1.5.0 (December 18, 2019) + +### IMPROVEMENTS: +* builder/amazon: Add no_ephemeral template option to remove ephemeral drives + from launch mappings. [GH-8393] +* builder/amazon: Add validation for "subnet_id" when specifying "vpc_id" + [GH-8360] [GH-8387] [GH-8391] +* builder/amazon: allow enabling ena/sr-iov on ebssurrogate spot instances + [GH-8397] +* builder/amazon: Retry runinstances aws api call to mitigate throttling + [GH-8342] +* builder/hyperone: Update builder schema and tags [GH-8444] +* builder/qemu: Add display template option for qemu. [GH-7676] +* builder/qemu: Disk Size is now read as a string to support units. [GH-8320] + [GH-7546] +* builder/qemu: Add fixer to convert disk size from int to string [GH-8390] +* builder/qemu: Disk Size is now read as a string to support units. [GH-8320] + [GH-7546] +* builder/qemu: When a user adds a new drive in qemuargs, process it to make + sure that necessary settings are applied to that drive. [GH-8380] +* builder/vmware: Fix error message when ovftool is missing [GH-8371] +* core: Cleanup logging for external plugins [GH-8471] +* core: HCL2 template support is now in beta. [GH-8423] +* core: Interpolation within provisioners can now access build-specific values + like Host IP, communicator password, and more. [GH-7866] +* core: Various fixes to error handling. [GH-8343] [GH-8333] [GH-8316] + [GH-8354] [GH-8361] [GH-8363] [GH-8370] +* post-processor/docker-tag: Add support for multiple tags. [GH-8392] +* post-processor/shell-local: Add "valid_exit_codes" option to shell-local. + [GH-8401] +* provisioner/chef-client: Add version selection option. [GH-8468] +* provisioner/shell-local: Add "valid_exit_codes" option to shell-local. + [GH-8401] +* provisioner/shell: Add support for the "env_var_format" parameter [GH-8319] + +### BUG FIXES: +* builder/amazon: Fix request retry mechanism to launch aws instance [GH-8430] +* builder/azure: Fix PollDuration option which was overridden in some clients. + [GH-8490] +* builder/hyperv: Fix bug in checking VM name that could cause flakiness if + many VMs are defined. [GH-8357] +* builder/vagrant: Use absolute path for Vagrantfile [GH-8321] +* builder/virtualbox: Fix panic in snapshot builder. [GH-8336] [GH-8329] +* communicator/winrm: Resolve ntlm nil pointer bug by bumping go-ntlmssp + dependency [GH-8369] +* communicator: Fix proxy connection settings to use "SSHProxyUsername" and + "SSHProxyPassword" where relevant instead of bastion username and password. + [GH-8375] +* core: Fix bug where Packer froze if asked to log an extremely long line + [GH-8356] +* core: Fix iso_target_path option; don't cache when target path is non-nil + [GH-8394] +* core: Return exit code 1 when builder type is not found [GH-8474] +* core: Return exit code 1 when builder type is not found [GH-8475] +* core: Update to newest version of go-tty to re-enable CTRL-S and CTRL-Q usage + [GH-8364] + +### BACKWARDS INCOMPATIBILITIES: +* builder/amazon: Complete deprecation of clean_ami_name template func + [GH-8320] [GH-8193] +* core: Changes have been made to both the Prepare() method signature on the + builder interface and on the Provision() method signature on the + provisioner interface. [GH-7866] +* provisioner/ansible-local: The "galaxycommand" option has been renamed to + "galaxy_command". A fixer has been written for this, which can be invoked + with `packer fix`. [GH-8411] + +## 1.4.5 (November 4, 2019) + +### IMPROVEMENTS: +* added ucloud-import post-processsor to import custom image for UCloud UHost + instance [GH-8261] +* builder/amazon: New option to specify IAM policy for a temporary instance + profile [GH-8247] +* builder/amazon: improved validation around encrypt_boot and kms_key_id for a + better experience [GH-8288] +* builder/azure-arm: Allow specification of polling duration [GH-8226] +* builder/azure-chroot: Add Azure chroot builder [GH-8185] & refactored some + common code together after it [GH-8269] +* builder/azure: Deploy NSG if list of IP addresses is provided in config + [GH-8203] +* builder/azure: Set correct user agent for Azure client set [GH-8259] +* builder/cloudstack: Add instance_display_name for cloudstack builder + [GH-8280] +* builder/hyperv: Add the additional_disk_size option tho the hyperv vmcx + builder. [GH-8246] +* builder/openstack: Add option to discover provisioning network [GH-8279] +* builder/oracle-oci: Support defined tags for oci builder [GH-8172] +* builder/proxmox: Add ability to select CPU type [GH-8201] +* builder/proxmox: Add support for SCSI controller selection [GH-8199] +* builder/proxmoz: Bump Proxmox dependency: [GH-8241] +* builder/tencent: Add retry on remote api call [GH-8250] +* builder/vagrant: Pass through logs from vagrant in real time rather than + buffering until command is complete [GH-8274] +* builder/vagrant: add insert_key option for toggling whether to add Vagrant's + insecure key [GH-8274] +* builder/virtualbox: enabled pcie disks usage, but this feature is in beta and + won't work out of the box yet [GH-8305] +* communicator/winrm: Prevent busy loop while waiting for WinRM connection + [GH-8213] +* core: Add strftime function in templates [GH-8208] +* core: Improve error message when comment is bad [GH-8267] +* post-processor/amazon-import: delete intermediary snapshots [GH-8307] +* Fix various dropped errors an removed unused code: [GH-8230] [GH-8265] + [GH-8276] [GH-8281] [GH-8309] [GH-8311] [GH-8304] [GH-8303] [GH-8293] + +### BUG FIXES: +* builder/amazon: Fix region copy for non-ebs amazon builders [GH-8212] +* builder/amazon: Fix spot instance bug where builder would fail if one + availability zone could not support the requested spot instance type, even + if another AZ could do so. [GH-8184] +* builder/azure: Fix build failure after a retry config generation error. + [GH-8209] +* builder/docker: Use a unique temp dir for each build to prevent concurrent + builds from stomping on each other [GH-8192] +* builder/hyperv: Improve filter for determining which files to compact + [GH-8248] +* builder/hyperv: Use first adapter, rather than failing, when multiple + adapters are attached to host OS's VM switch [GH-8234] +* builder/openstack: Fix setting openstack metadata for use_blockstorage_volume + [GH-8186] +* builder/openstack: Warn instead of failing on terminate if instance is + already shut down [GH-8176] +* post-processor/digitalocean-import: Fix panic when 'image_regions' not set + [GH-8179] +* provisioner/powershell: Fix powershell syntax error causing failed builds + [GH-8195] + +## 1.4.4 (October 1, 2019) + +### IMPROVEMENTS: +**New Core Feature** Error cleanup provisioner [GH-8155] +* builder/amazon: Add ability to set `run_volume_tags` [GH-8051] +* builder/amazon: Add AWS API call reties on AMI prevalidation [GH-8034] +* builder/azure: Refactor client config [GH-8121] +* builder/cloudstack: New step to detach iso. [GH-8106] +* builder/googlecompute: Fail fast when image name is invalid. [GH-8112] +* builder/googlecompute: Users can now query Vault for an Oauth token rather + than setting an account file [GH-8143] +* builder/hcloud: Allow selecting image based on filters [GH-7945] +* builder/hyper-v: Decrease the delay between Hyper-V VM startup and hyper-v + builder's ability to send keystrokes to the target VM. [GH-7970] +* builder/openstack: Store WinRM password for provisioners to use [GH-7940] +* builder/proxmox: Shorten default boot_key_interval to 5ms from 100ms + [GH-8088] +* builder/proxmox: Allow running the template VM in a Proxmox resource pool + [GH-7862] +* builder/ucloud: Make ucloud builder's base url configurable [GH-8095] +* builder/virtualbox-vm: Make target snapshot optional [GH-8011] [GH-8004] +* builder/vmware: Allow user to attach floppy files to remote vmx builds + [GH-8132] +* builder/yandex: Add ability to retry API requests [GH-8142] +* builder/yandex: Support GPU instances and set source image by name [GH-8091] +* communicator/ssh: Support for SSH port tunneling [GH-7918] +* core: Add a new `floppy_label` option [GH-8099] +* core: Added version compatibility to console command [GH-8080] +* post-processor/vagrant-cloud: Allow blank access_token for private vagrant + box hosting [GH-8097] +* post-processor/vagrant-cloud: Allow use of the Artifice post-processor with + the Vagrant Cloud post-processor [GH-8018] [GH-8027] +* post-processor/vsphere: Removed redundant whitelist check for builders, + allowing users to use post-processor without the VMWare builder [GH-8064] + +### BUG FIXES: +* builder/amazon: Fix FleetID crash. [GH-8013] +* builder/amazon: Gracefully handle rate limiting when retrieving winrm + password. [GH-8087] +* builder/amazon: Fix race condition in spot instance launching [GH-8165] +* builder/amazon: Amazon builders now respect ssh_host option [GH-8162] +* builder/amazon: Update the AWS sdk to resolve some credential handling issues + [GH-8131] +* builder/azure: Avoid a panic in getObjectIdFromToken [GH-8047] +* builder/googlecompute: Fix crash caused by nil account file. [GH-8102] +* builder/hyper-v: Fix when management interface is not part of virtual switch + [GH-8017] +* builder/openstack: Fix dropped error when creating image client. [GH-8110] +* builder/openstack: Fix race condition created when adding metadata [GH-8016] +* builder/outscale: Get SSH Host from VM.Nics instead of VM Root [GH-8077] +* builder/proxmox: Bump proxmox api dep, fixing bug with checking http status + during boot command [GH-8083] +* builder/proxmox: Check that disk format is set when pool type requires it + [GH-8084] +* builder/proxmox: Fix panic caused by cancelling build [GH-8067] [GH-8072] +* builder/qemu: Fix dropped error when retrieving version [GH-8050] +* builder/vagrant: Fix dropped errors in code and tests. [GH-8118] +* builder/vagrant: Fix provisioning boxes, define source and output boxes + [GH-7957] +* builder/vagrant: Fix ssh and package steps to use source syntax. [GH-8125] +* builder/vagrant: Use GlobalID when provided [GH-8092] +* builder/virtualbox: Fix windows pathing problem for guest additions checksum + download. [GH-7996] +* builder/virtualbox: LoadSnapshots succeeds even if machine has no snapshots + [GH-8096] +* builder/vmware: fix dropped test errors [GH-8170] +* core: Fix bug where sensitive variables containing commas were not being + properly sanitized in UI calls. [GH-7997] +* core: Fix handling of booleans where "unset" is a value distinct from + "false". [GH-8021] +* core: Fix tests that swallowed errors in goroutines [GH-8094] +* core: Fix bug where Packer could no longer run as background process [GH-8101] +* core: Fix zsh auto-completion [GH-8160] +* communicator/ssh: Friendlier message warning user that their creds may be + wrong [GH-8167] +* post-processor/amazon-import: Fix non-default encryption. [GH-8113] +* post-processor/vagrant-cloud: Fix dropped errors [GH-8156] +* provisioner/ansible: Fix provisioner dropped errors [GH-8045] + +### BACKWARDS INCOMPATIBILITIES: +* core: "sed" template function has been deprecated in favor of "replace" and + "replace_all" functions [GH-8119] + +## 1.4.3 (August 14, 2019) + +### IMPROVEMENTS: +* **new builder** UCloud builder [GH-7775] +* **new builder** Outscale [GH-7459] +* **new builder** VirtualBox Snapshot [GH-7780] +* **new builder** JDCloud [GH-7962] +* **new post-processor** Exoscale Import post-processor [GH-7822] [GH-7946] +* build: Change Makefile to behave differently inside and outside the gopath + when generating code. [GH-7827] +* builder/amazon: Don't calculate spot bids; Amazon has changed spot pricing to + no longer require this. [GH-7813] +* builder/google: Add suse-byos-cloud to list of public GCP cloud image + projects [GH-7935] +* builder/openstack: New `image_min_disk` option [GH-7290] +* builder/openstack: New option `use_blockstorage_volume` to set openstack + image metadata [GH-7792] +* builder/openstack: Select instance network on which to assign floating ip + [GH-7884] +* builder/qemu: Implement VNC password functionality [GH-7836] +* builder/scaleway: Allow removing volume after image creation for Scaleway + builder [GH-7887] +* builder/tencent: Add `run_tags` to option to tag instance. [GH-7810] +* builder/tencent: Remove unnecessary image name validation check. [GH-7786] +* builder/tencent: Support data disks for tencentcloud builder [GH-7815] +* builder/vmware: Fix intense CPU usage because of poorly handled errors. + [GH-7877] +* communicator: Use context for timeouts, interruption in ssh and winrm + communicators [GH-7868] +* core: Change how on-error=abort is handled to prevent EOF errors that mask + real issues [GH-7913] +* core: Clean up logging vs ui call in step download [GH-7936] +* core: New environment var option to allow user to set location of config + directory [GH-7912] +* core: Remove obsolete Cancel functions from builtin provisioners [GH-7917] +* post-processor/vagrant: Add option to allow box Vagrantfiles to be generated + during the build [GH-7951] +* provisioner/ansible: Add support for installing roles with ansible-galaxy + [GH-7916 +* provisioner/salt-masterless: Modify file upload to handle non-root case. + [GH-7833] + +### BUG FIXES: +* builder/amazon: Add error to warn users of spot_tags regression. [GH-7989] +* builder/amazon: Allow EC2 Spot Fleet packer instances to run in parallel + [GH-7818] +* builder/amazon: Fix failures and duplication in Amazon region copy and + encryption step. [GH-7870] [GH-7923] +* builder/amazon: No longer store names of volumes which get deleted on + termination inside ebssurrogate artifact. [GH-7829] +* builder/amazon: Update aws-sdk-go to v1.22.2, resolving some AssumeRole + issues [GH-7967] +* builder/azure: Create configurable polling duration and set higher default + for image copies to prevent timeouts on successful copies [GH-7920] +* builder/digitalocean: increase timeout for Digital Ocean snapshot creation. + [GH-7841] +* builder/docker: Check container os, not host os, when creating container dir + default [GH-7939] +* builder/docker: Fix bug where PACKER_TMP_DIR was created with root perms on + linux [GH-7905] +* builder/docker: Fix file download hang caused by blocking ReadAll call + [GH-7814] +* builder/google: Fix outdated oauth URL. [GH-7835] [GH-7927] +* builder/hyperv: Improve code for detecting IP address [GH-7880] +* builder/ucloud: Update the api about stop instance to fix the read-only image + build by ucloud-uhost [GH-7914] +* builder/vagrant: Fix bug where source_path was being used instead of box_name + when generating the Vagrantfile. [GH-7859] +* builder/virtualbox: Honor value of 'Comment' field in ssh keypair generation. + [GH-7922] +* builder/vmware: Fix validation regression that occurred when user provided a + checksum file [GH-7804] +* buildere/azure: Fix crash with managed images not published to shared image + gallery. [GH-7837] +* communicator/ssh: Move ssh_interface back into individual builders from ssh + communicator to prevent validation issues where it isn't implemented. + [GH-7831] +* console: Fix console help text [GH-7960] +* core: Fix bug in template parsing where function errors were getting + swallowed. [GH-7854] +* core: Fix regression where a local filepath containing `//` was no longer + properly resolving to `/`. [GH-7888] +* core: Fix regression where we could no longer access isos on SMB shares. + [GH-7800] +* core: Make ssh_host template option always override all builders' IP + discovery. [GH-7832] +* core: Regenerate boot_command PEG code [GH-7977] +* fix: clean up help text and fixer order to make sure all fixers are called + [GH-7903] +* provisioner/inspec: Use --input-file instead of --attrs to avoid deprecation + warning [GH-7893] +* provisioner/salt-masterless: Make salt-masterless provisioner respect + disable_sudo directive for all commands [GH-7774] + +## 1.4.2 (June 26, 2019) + +### IMPROVEMENTS: +* **new feature:** Packer console [GH-7726] +* builder/alicloud: cleanup image and snapshot if target image is still not + available after timeout [GH-7744] +* builder/alicloud: let product API determine the default value of io_optimized + [GH-7747] +* builder/amazon: Add new `skip_save_build_region` option to fix naming + conflicts when building in a region you don't want the final image saved + in. [GH-7759] +* builder/amazon: Add retry for temp key-pair generation in amazon-ebs + [GH-7731] +* builder/amazon: Enable encrypted AMI sharing across accounts [GH-7707] +* builder/amazon: New SpotInstanceTypes feature for spot instance users. + [GH-7682] +* builder/azure: Allow users to publish Managed Images to Azure Shared Image + Gallery (same Subscription) [GH-7778] +* builder/azure: Update Azure SDK for Go to v30.0.0 [GH-7706] +* builder/cloudstack: Add tags to instance upon creation [GH-7526] +* builder/docker: Better windows defaults [GH-7678] +* builder/google: Add feature to import user-data from a file [GH-7720] +* builder/hyperv: Abort build if there's a name collision [GH-7746] +* builder/hyperv: Clarify pathing requirements for hyperv-vmcx [GH-7790] +* builder/hyperv: Increase MaxRamSize to match modern Windows [GH-7785] +* builder/openstack: Add image filtering on properties. [GH-7597] +* builder/qemu: Add additional disk support [GH-7791] +* builder/vagrant: Allow user to override vagrant ssh-config details [GH-7782] +* builder/yandex: Gracefully shutdown instance, allow metadata from file, and + create preemptible instance type [GH-7734] +* core: scrub out sensitive variables in scrub out sensitive variables logs + [GH-7743] + +### BUG FIXES: +* builder/alicloud: Fix describing snapshots issue when image_ignore_data_disks + is provided [GH-7736] +* builder/amazon: Fix bug in region copy which produced badly-named AMIs in the + build region. [GH-7691] +* builder/amazon: Fix failure that happened when spot_tags was set but ami_tags + wasn't [GH-7712] +* builder/cloudstack: Update go-cloudstack sdk, fixing compatibility with + CloudStack v 4.12 [GH-7694] +* builder/proxmox: Update proxmox-api-go dependency, fixing issue calculating + VMIDs. [GH-7755] +* builder/tencent: Correctly remove tencentcloud temporary keypair. [GH-7787] +* core: Allow timestamped AND colorless ui messages [GH-7769] +* core: Apply logSecretFilter to output from ui.Say [GH-7739] +* core: Fix "make bin" command to use reasonable defaults. [GH-7752] +* core: Fix user var interpolation for variables set via -var-file and from + command line [GH-7733] +* core: machine-readable UI now writes UI calls to logs. [GH-7745] +* core: Switch makefile to use "GO111MODULE=auto" to allow for modern gomodule + usage. [GH-7753] +* provisioner/ansible: prevent nil pointer dereference after a language change + [GH-7738] +* provisioner/chef: Accept chef license by default to prevent hangs in latest + Chef [GH-7653] +* provisioner/powershell: Fix crash caused by error in retry logic check in + powershell provisioner [GH-7657] +* provisioner/powershell: Fix null file descriptor error that occurred when + remote_path provided is a directory and not a file. [GH-7705] + +## 1.4.1 (May 15, 2019) + +### IMPROVEMENTS: +* **new builder:** new proxmox builder implemented [GH-7490] +* **new builder:** new yandex cloud builder implemented [GH-7484] +* **new builder:** new linode builder implemented [GH-7508] +* build: Circle CI now generates test binaries for all pull requests [GH-7624] + [GH-7625] [GH-7630] +* builder/alicloud: Support encryption with default service key [GH-7574] +* builder/amazon: Users of chroot and ebssurrogate builders may now choose + between "x86_64" and "arm64" architectures when registering their AMIs. + [GH-7620] +* builder/amazon: Users of the ebssurrogage builder may now choose to omit + certain launch_block_devices from the final AMI mapping by using the + omit_from_artifact feature. [GH-7612] +* builder/azure: Update Azure SDK [GH-7563] +* builder/docker: Better error messaging with container downloads. [GH-7513] +* builder/google: add image encryption support [GH-7551] +* builder/hyperv: Add keep_registered option to hyperv [GH-7498] +* builder/qemu: Replace dot-based parsing with hashicorp/go-version [GH-7614] +* builder/vmware: Add 30 minute timeout for destroying a VM [GH-7553] +* core: Cleanup cache of used port after closing [GH-7613] +* core: New option to set number of builds running in parallel & test + BuildCommand more [GH-7501] +* packer compiles on s390x [GH-7567] +* provisioner/file: Added warnings about writeable locations [GH-7494] + + +### BUG FIXES: +* builder/amazon: Fix bug that always encrypted build region with default key. + [GH-7507] +* builder/amazon: Fix bug that wasn't deleting unencrypted temporary snapshots + [GH-7521] +* builder/amazon: Fix EBSsurrogate copy, encryption, and deletion of temporary + unencrypted amis. [GH-7598] +* builder/hyperv: Fixes IP detection error if more than one VMNetworkAdapter is + found [GH-7480] +* builder/qemu: Fix mistake switching ssh port mix/max for vnc port min/max + [GH-7615] +* builder/vagrant: Fix bug with builder and vagrant-libvirt plugin [GH-7633] +* builder/virtualbox: Don't fail download when checksum is not set. [GH-7512] +* builder/virtualbox: Fix ovf download failures by using local ovf files in + place instead of symlinking [GH-7497] +* builder/vmware: Fix panic configuring VNC for remote builds [GH-7509] +* core/build: Allow building Packer on solaris by removing progress bar and tty + imports on solaris [GH-7618] +* core: Fix race condition causing hang [GH-7579] +* core: Fix tty related panics [GH-7517] +* core: Step download: Always copy local files on windows rather than + symlinking them [GH-7575] +* packer compiles on Solaris again [GH-7589] [GH-7618] +* post-processor/vagrant: Fix bug in retry logic that caused failed upload to + report success. [GH-7554] + +## 1.4.0 (April 11, 2019) + +### IMPROVEMENTS: +* builder/alicloud: Improve error message for conflicting images name [GH-7415] +* builder/amazon-chroot: Allow users to specify custom block device mapping + [GH-7370] +* builder/ansible: Documentation fix explaining how to use ansible 2.7 + winrm + [GH-7461] +* builder/azure-arm: specify zone resilient image from config [GH-7211] +* builder/docker: Add support for windows containers [GH-7444] +* builder/openstack: Allow both ports and networks in openstack builder + [GH-7451] +* builder/openstack: Expose force_delete for openstack builder [GH-7395] +* builder/OpenStack: Support Application Credential Authentication [GH-7300] +* builder/virtualbox: Add validation for 'none' communicator. [GH-7419] +* builder/virtualbox: create ephemeral SSH key pair for build process [GH-7287] +* core: Add functionality to marshal a Template to valid Packer JSON [GH-7339] +* core: Allow user variables to be interpreted within the variables section + [GH-7390] +* core: Incorporate the go-getter to handle downloads [GH-6999] +* core: Lock Packer VNC ports using a lock file to prevent collisions [GH-7422] +* core: Print VerifyChecksum log for the download as ui.Message output + [GH-7387] +* core: Users can now set provisioner timeouts [GH-7466] +* core: Switch to using go mod for managing dependencies [GH-7270] +* core: Select a new VNC port if initial port is busy [GH-7423] +* post-processor/googlecompute-export: Set network project id to builder + [GH-7359] +* post-processor/vagrant-cloud: support for the vagrant builder [GH-7397] +* post-processor/Vagrant: Option to ignore SSL verification when using on- + premise vagrant cloud [GH-7377] +* postprocessor/amazon-import: Support S3 and AMI encryption. [GH-7396] +* provisioner/shell provisioner/windows-shell: allow to specify valid exit + codes [GH-7385] +* core: Filter sensitive variables out of the ui as well as the logs + [GH-7462] + +### BUG FIXES: +* builder/alibaba: Update to latest Alibaba Cloud official image to fix + acceptance tests [GH-7375] +* builder/amazon-chroot: Fix building PV images and where mount_partition is + set [GH-7337] +* builder/amazon: Fix http_proxy env var regression [GH-7361] +* builder/azure: Fix: Power off before taking snapshot (windows) [GH-7464] +* builder/hcloud: Fix usage of freebsd64 rescue image [GH-7381] +* builder/vagrant: windows : fix docs and usage [GH-7416] [GH-7417] +* builder/vmware-esxi: properly copy .vmxf files in remote vmx builds [GH-7357] +* core: fix bug where Packer didn't pause in debug on certain linux platforms. + [GH-7352] +* builder/amazon: Fix bug copying encrypted images between regions [GH-7342] + +### BACKWARDS INCOMPATIBILITIES: +* builder/amazon: Change `temporary_security_group_source_cidr` to + `temporary_security_group_source_cidrs` and allow it to accept a list of + strings. [GH-7450] +* builder/amazon: If users do not pass any encrypt setting, retain any initial + encryption setting of the AMI. [GH-6787] +* builder/docker: Update docker's default config to use /bin/sh instead of + /bin/bash [GH-7106] +* builder/hyperv: Change option names cpu->cpus and ram_size->memory to bring + naming in line with vmware and virtualbox builders [GH-7447] +* builder/oracle-classic: Remove default ssh_username from oracle classic + builder, but add note to docs with oracle's default user. [GH-7446] +* builder/scaleway: Renamed attribute api_access_key to organization_id. + [GH-6983] +* Change clean_image name and clean_ami_name to a more general clean_resource + name for Googlecompute, Azure, and AWS builders. [GH-7456] +* core/post-processors: Change interface for post-processors to allow an + overridable default for keeping input artifacts. [GH-7463] + +## 1.3.5 (February 28, 2019) + +### IMPROVEMENTS: +* builder/alicloud: Update aliyun sdk to support eu-west-1 region [GH-7338] +* builder/amazon: AWS users can now use the Vault AWS engine to generate + temporary credentials. [GH-7282] +* builder/azure: IMDS to get subscription for Azure MSI [GH-7332] +* builder/openstack: Replaced deprecated compute/ api with imageservice/ + [GH-7038] +* builder/virtualbox: New "guest_additions_interface" option to enable + attaching via a SATA interface. [GH-7298] +* builder/vmware: Add `cores` option for specifying the number of cores per + socket. [GH-7191] +* bulder/openstac: Deprecated compute/v2/images API [GH-7268] +* core: Add validation check to help folks who swap their iso_path and + checksum_path [GH-7311] +* fixer/amazon: Make the amazon-private-ip fixer errors more visible [GH-7336] +* post-processor/googlecompute-export: Extend auth for the GCE-post-processors + to act like the GCE builder. [GH-7222] +* post-processor/googlecompute-import: Extend auth for the GCE-post-processors + to act like the GCE builder. [GH-7222] +* post-processor/manifest: Add "custom_data" key to packer manifest post- + processor [GH-7248] + +### BUG FIXES: +* builder/amazon: Fix support for aws-us-gov [GH-7347] +* builder/amazon: Move snapshot deletion to cleanup phase. [GH-7343] +* builder/azure: Fixed Azure interactive authentication [GH-7276] +* builder/cloudstack: Updated sdk version; can now use ostype name in + template_os option. [GH-7264] +* builder/google: Change metadata url to use a FQDN fixing bug stemming from + differing DNS/search domains. [GH-7260] +* builder/hyper-v: Fix integer overflows in 32-bit builds [GH-7251] +* builder/hyper-v: Fix regression where we improperly handled spaces in switch + names [GH-7266] +* builder/openstack: Pass context So we know to cancel during WaitForImage + [GH-7341] +* builder/vmware-esxi: Strip \r\n whitespace from end of names of + files stored on esxi. [GH-7310] +* builder/vmware: Add "--noSSLVerify" to args in ovftool Validation [GH-7314] +* core: clean up Makefile [GH-7254][GH-7265] +* core: Fixes mismatches in checksums for dependencies for Go 1.11.4+ [GH-7261] +* core: make sure 'only' option is completely ignored by post-processors + [GH-7262] +* core: name a post-processor to its type when it is not named [GH-7330] +* provisioner/salt: Force powershell to overwrite duplicate files [GH-7281] + +### Features: +* **new builder** `vagrant` allows users to call vagrant to provision starting + from vagrant boxes and save them as new vagrant boxes. [GH-7221] +* **new builder:** `hyperone` for building new images on HyperOne Platform on + top of existing image or from the scratch with the use of chroot. [GH-7294] +* **new post-processor** `digitalocean-import`Add digitalocean-import post- + processor. [GH-7060] +* **new provisioner**`inspec` Added inspec.io provisioner [GH-7180] +* communicator: Add configurable pause after communicator can connect but + before it performs provisioning tasks [GH-7317] [GH-7351] + +## 1.3.4 (January 30, 2019) +### IMPROVEMENTS: +* builder/alicloud: delete copied image and snapshots if corresponding options + are specified [GH-7050] +* builder/amazon: allow to interpolate more variables [GH-7059] +* builder/amazon: Check that the KMS key ID is valid [GH-7090] +* builder/amazon: Clean up logging for aws waiters so that it only runs once + per builder [GH-7080] +* builder/amazon: don't Cleanup Temp Keys when there is no communicator to + avoid a panic [GH-7100] [GH-7095] +* builder/amazon: Don't try to guess region from metadata if not set & update + aws-sdk-go [GH-7230] +* builder/amazon: Import error messages should now contain reason for failure + [GH-7207] +* builder/azure: add certificate authentication [GH-7189] +* builder/azure: allow to configure disk caching [GH-7061] +* builder/azure: use deallocate instead of just power-off [GH-7203] +* builder/hyperv: Add support for legacy network adapters on Hyper-V. [GH-7128] +* builder/hyperv: Allow user to set `version` option in the New-VM command. + [GH-7136] +* builder/openstack: Add `volume_size` option [GH-7130] +* builder/openstack: Don't require network v2 [GH-6933] +* builder/openstack: Support for tagging new images [GH-7037] +* builder/qemu: Add configuration options to specify cpu count and memory size + [GH-7156] +* builder/qemu: Add support for whpx accelerator to qemu builder [GH-7151] +* builder/vmware: Escape query as suggested in issue #7200 [GH-7223] +* core/shell: Add env vars "PACKER_HTTP_IP" and "PACKER_HTTP_PORT" to shell + provisioners [GH-7075] +* core: allow to use `-except` on post-processors [GH-7183] +* core: Clean up internal handling and creation of temporary directories + [GH-7102] +* core: Deprecate mitchellh/go-homedir package in favor of os/user [GH-7062] +* core: Download checksum match failures will now log the received checksum. + [GH-7210] +* core: Explicitly set ProxyFromEnvironment in httpclients when creating an aws + session [GH-7226] +* core: make packer inspect not print sensitive variables [GH-7084] +* post-processor/google: Add new `guest-os-features` option. [GH-7218] +* postprocessor/docker-import: Added `change` support [GH-7127] +* provisioner/ansible-remote: add `-o IdentitiesOnly=yes` as a default flag + [GH-7115] +* provisioner/chef-client: Elevated support for chef-client provisioner + [GH-7078] +* provisioner/puppet: Elevated support for puppet-* provisioner [GH-7078] +* provisioner/windows-restart: wait for already-scheduled reboot [GH-7056] and + ignore reboot specific errors [GH-7071] + + +### BUG FIXES: +* builder/azure: Ensure the Windows Guest Agent is fully functional before + Sysprep is executed. [GH-7176] +* builder/azure: Fix snapshot regression [GH-7111] +* builder/docker: Ensure that entrypoint and arguments get passed to docker, + not the image. [GH-7091] +* builder/hcloud: fix go mod dependency [GH-7099] +* builder/hcloud: prevent panic when ssh key was not passed [GH-7118] +* builder/hyperv: Fix the Hyper-V gen 1 guest boot order. [GH-7147] +* builder/hyperv: hyper-v builder no longer ignores `ssh_host` option. + [GH-7154] +* builder/oracle-oci: Fix crash that occurs when image is nil [GH-7126] +* builder/parallels: Fix attaching prl tools [GH-7158] +* builder/virtualbox: Fix handling of portcount argument for version 6 beta + [GH-7174] [GH-7094] +* builder/vmware: Fix bug caused by 'nil' dir field in artifact struct when + building locally [GH-7116] +* communicator/docker: Fix docker file provisioner on Windows [GH-7163] +* core: prioritize AppData over default user directory ( UserProfile ) + [GH-7166] +* core: removed a flaky race condition in tests [GH-7119] +* postprocessor/vsphere: Stop setting HDDOrder, since it was breaking uploads + [GH-7108] + + +## 1.3.3 (December 5, 2018) +### IMPROVEMENTS: +* builder/alicloud: Add options for system disk properties [GH-6939] +* builder/alicloud: Apply tags to relevant snapshots [GH-7040] +* builder/alicloud: Support creating image without data disks [GH-7022] +* builder/amazon: Add option for skipping TLS verification [GH-6842] +* builder/azure: Add options for Managed Image OS Disk and Data Disk snapshots + [GH-6980] +* builder/hcloud: Add `snapshot_labels` option to hcloud builder [GH-7046] +* builder/hcloud: Add ssh_keys config to hcloud builder [GH-7028] +* builder/hcloud: Update hcloud-go version and support builds using rescue mode + [GH-7034] +* builder/oracle: Parameterized volume size support for Oracle classic builder + [GH-6918] +* builder/parallels: Add configuration options to parallels builder to specify + cpu count and memory size [GH-7018] +* builder/virtualbox: Add configuration options to virtualbox builder to + specify cpu count and memory size [GH-7017] +* builder/virtualbox: expose the VBoxManage export --iso option [GH-5950] +* builder/vmware: Add configuration options to vmware builder to specify cpu + count and memory size [GH-7019] +* builder/vmware: Add new display_name template option [GH-6984] +* builder/vmware: Extend vmware-vmx builder to allow esxi builds. [GH-4591] + [GH-6927] +* builder/vmware: Validate username/password for ovftool during prepare. + [GH-6977] +* builder/vmware: Warn users if their vmx_data overrides data that Packer uses + the template engine to set in its default vmx template. [GH-6987] +* communicator/ssh: Expand user path for SSH private key [GH-6946] +* core: Add a sed template engine [GH-6580] +* core: More explicit error message in rpc/ui.go [GH-6981] +* core: Replaced unsafe method of determining homedir with os/user + implementation [GH-7036] +* core: Update vagrantfile's go version. [GH-6841] +* post-processor/amazon-import: Support ova, raw, vmdk, and vhdx formats in the + amazon-import post-processor. [GH-6938] +* post-processor/vsphere-template: Add option to snapshot vm before marking as + template [GH-6969] +* provisioner/breakpoint: Add a new breakpoint provisioner. [GH-7058] +* provisioner/powershell: Allow Powershell provisioner to use service accounts + [GH-6972] +* provisioner/shell: Add PauseAfter option to shell provisioner [GH-6913] + +### BUG FIXES: +* builder/amazon: Better error handling of region/credential guessing from + metadata [GH-6931] +* builder/amazon: move region validation to run so that we don't break + validation when no credentials are set [GH-7032] +* builder/hyperv: Remove -Copy:$false when calling Hyper-V\Compare-VM + compatability report [GH-7030] +* builder/qemu: Do not set detect-zeroes option when we want it "off" [GH-7064] +* builder/vmware-esxi: Create export directories for vmx and ovf file types + [GH-6985] +* builder/vmware: Correctly parse version for VMware Fusion Tech Preview + [GH-7016] +* builder/vmware: Escape vSphere username when putting it into the export call + [GH-6962] +* post-processor/vagrant: Add "hvf" as a libvirt driver [GH-6955] +* provisioner/ansible: inventory is no longer set to inventory_directory + [GH-7065] + +## 1.3.2 (October 29, 2018) +### IMPROVEMENTS: +* builder/alicloud: Add new `disable_stop_instance` option. [GH-6764] +* builder/alicloud: Support adding tags to image. [GH-6719] +* builder/alicloud: Support ssh with private ip address. [GH-6688] +* builder/amazon: Add support to explicitly control ENA support [GH-6872] +* builder/amazon: Add suppport for `vpc_filter`, `subnet_filter`, and + `security_group_filter`. [GH-6374] +* builder/amazon: Add validation for required `device_name` parameter in + `block_device_mappings`. [GH-6845] +* builder/amazon: Clean up security group wait code. [GH-6843] +* builder/amazon: Update aws-sdk-go to v1.15.54, adding support for + `credential_source`. [GH-6849] +* builder/amazon: Use DescribeRegions for aws region validation. [GH-6512], + [GH-6904] +* builder/azure: Add new `shared_image_gallery` option. [GH-6798] +* builder/googlecompute: Return an error if `startup_script_file` is specified, + but file does not exist. [GH-6848] +* builder/hcloud: Add Hetzner Cloud builder. [GH-6871] +* builder/openstack: Add new `disk_format` option. [GH-6702] +* builder/openstack: Fix bug where `source_image_name` wasn't being used to + properly find a UUID. [GH-6751] +* builder/openstack: Wait for volume availability when cleaning up [GH-6703] +* builder/qemu: Add `disk_detect_zeroes` option. [GH-6827] +* builder/scaleway: Add `boottype` parameter to config. [GH-6772] +* builder/scaleway: Update scaleway-cli vendor. [GH-6771] +* core: New option to add timestamps to UI output. [GH-6784] +* post-processor/vagrant-cloud: Validate vagrant cloud auth token doing an auth + request [GH-6914] +* provisioner/file: Improve error messaging when file destination is a + directory with no trailing slash. [GH-6756] +* provisioner/powershell: Provide better error when Packer can't find + Powershell executable. [GH-6817] +* provisioner/shell-local: Add ability to specify OSs where shell-local can run + [GH-6878] + +### BUG FIXES: +* builder/alicloud: Fix ssh configuration pointer issues that could cause a bug + [GH-6720] +* builder/alicloud: Fix type error in step_create_tags [GH-6763] +* builder/amazon: Error validating credentials is no longer obscured by a + region validation error. and some region validation refactors and + improvements [GH-6865] +* builder/amazon: Fix error calculating defaults in AWS waiters. [GH-6727] +* builder/amazon: Increase default wait for image import to one hour. [GH-6818] +* builder/amazon: Waiter now fails rather than hanging for extra time when an + image import fails. [GH-6747] +* builder/azure: Updated Azure/go-ntlmssp dependency to resolve an issue with + the winrm communicator not connecting to Windows machines requiring NTLMv2 + session security +* builder/digitalocean: Fix ssh configuration pointer issues that could cause a + panic [GH-6729] +* builder/hyperv/vmcx: Allow to set generation from buildfile [GH-6909] +* builder/scaleway: Fix issues with ssh keys. [GH-6768] +* core: Fix error where logging was always enabled when Packer was run from + inside Terraform. [GH-6758] +* core: Fix issue with with names containing spaces in ESX5Driver and in ssh + communicator [GH-6891], [GH-6823] +* core: Fix logger so it doesn't accidentally try to format unescaped strings. + [GH-6824] +* core: Fix race conditions in progress bar code [GH-6858], [GH-6788], + [GH-6851] +* core: Fix various places in multiple builders where config was not being + passed as a pointer. [GH-6739] +* post-processor/manifest: No longer provides an empty ID string for Azure's + managed image artifact [GH-6822] +* provisioner/powershell: Fix a bug in the way we set the ProgressPreference + variable in the default `execute_command` [GH-6838] +* provisioner/windows-restart: Fix extraneous break which forced early exit + from our wait loop. [GH-6792] + +## 1.3.1 (September 13, 2018) + +### IMPROVEMENTS: +* builder/amazon: automatically decode encoded authorization messages if + possible [GH-5415] +* builder:amazon: Optional cleanup of the authorized keys file [GH-6713] +* builder/qemu: Fixed bug where a -device in the qemuargs would override the default network settings, resulting in no network [GH-6807] + +### BUG FIXES: +* builder/amazon: fix bugs relating to spot instances provisioning [GH-6697] + [GH-6693] +* builder/openstack: fix ssh keypair not attached [GH-6701] +* core: progressbar: fix deadlock locking builds after first display [GH-6698] + +## 1.3.0 (September 11, 2018) + +### IMPROVEMENTS: +* azure/arm: Retry cleanup of individual resources on error [GH-6644] +* builder/alicloud: Support source image coming from marketplace [GH-6588] +* builder/amazon-chroot: Add new `root_volume_type` option. [GH-6669] +* builder/amazon-chroot: If you have a PV source AMI, with the Amazon Chroot + builder, and the destination AMI is type HVM, you can now enable + ena_support, example: [GH-6670] +* builder/amazon-chroot: New feature `root_volume_tags` to tag the created + volumes. [GH-6504] +* builder/amazon: Create a random interim AMI name when encrypt_boot is true so + that ami name is not searchable. [GH-6657] +* builder/azure: Implement clean_image_name template engine. [GH-6558] +* builder/cloudstack: Add option to use a fixed port via public_port. [GH-6532] +* builder/digitalocean: Add support for tagging to instances [GH-6546] +* builder/googlecompute: Add new `min_cpu_platform` feature [GH-6607] +* builder/googlecompute: Update the list of public image projects that we + search, based on GCE documentation. [GH-6648] +* builder/lxc: Allow unplivileged LXC containers. [GH-6279] +* builder/oci: Add `metadata` feature to Packer config. [GH-6498] +* builder/openstack: Add support for getting config from clouds-public.yaml. + [GH-6595] +* builder/openstack: Add support for ports. [GH-6570] +* builder/openstack: Add support for source_image_filter. [GH-6490] +* builder/openstack: Migrate floating IP usage to Network v2 API from Compute + API. [GH-6373] +* builder/openstack: Support Block Storage volumes as boot volume. [GH-6596] +* builder/oracle-oci: Add support for freeform tagging of OCI images [GH-6338] +* builder/qemu: add ssh agent support. [GH-6541] +* builder/qemu: New `use_backing_file` feature [GH-6249] +* builder/vmware-iso: Add support for disk compaction [GH-6411] +* builder/vmware-iso: Try to use ISO files uploaded to the datastore when + building remotely instead of uploading them freshly every time [GH-5165] +* command/validate: Warn users if config needs fixing. [GH-6423] +* core: Add a 'split' function to parse template variables. [GH-6357] +* core: Add a template function allowing users to read keys from consul + [GH-6577] +* core: Add a template function allowing users to read keys from vault + [GH-6533] +* core: Add progress-bar to download step. [GH-5851] +* core: Create a new root-level Packer template option, "sensitive-variables" + which allows users to list which variables they would like to have scrubbed + from the Packer logs. [GH-6610] +* core: Create new config options, "boot_keygroup_interval" and + "boot_key_interval" that can be set at the builder-level to supercede + PACKER_KEY_INTERVAL for the bootcommand. [GH-6616] +* core: Deduplicate ui and log lines that stream to terminal [GH-6611] +* core: Refactor and deduplicate ssh code across builders. This should be a no- + op but is a big win for maintainability. [GH-6621] [GH-6613] +* post-processor/compress: Add support for xz compression [GH-6534] +* post-processor/vagrant: Support for Docker images. [GH-6494] +* post-processor/vsphere: Add new `esxi_host` option. [GH-5366] +* postprocessor/vagrant: Add support for Azure. [GH-6576] +* provisioner/ansible: Add new "extra var", packer_http_addr. [GH-6501] +* provisioner/ansible: Enable {{.WinRMPassword}} template engine. [GH-6450] +* provisioner/shell-local: Create PACKER_HTTP_ADDR environment variable + [GH-6503] + + +### BUG FIXES: +* builder/amazon-ebssurrogate: Clean up volumes at end of build. [GH-6514] +* builder/amazon: Increase default waiter timeout for AWS + WaitUntilImageAvailable command [GH-6601] +* builder/amazon: Increase the MaxRetries in the Amazon client from the default + to 20, to work around users who regularly reach their requestlimit and are + being throttled. [GH-6641] +* builder/amazon: Properly apply environment overrides to our custom-written + waiters. [GH-6649] +* builder/azure: Generated password satisfies Azure password requirements + [GH-6480] +* builder/hyper-v: Builder no longer errors if skip_compaction isn't true when + skip_export is true, and compaction efficiency is improved [GH-6393] +* builder/lxc: Correctly pass "config" option to "lxc launch". [GH-6563] +* builder/lxc: Determine lxc root according to the running user [GH-6543] +* builder/lxc: Fix file copying for unprivileged LXC containers [GH-6544] +* builder/oracle-oci: Update OCI sdk, fixing validation bug that occurred when + RSA key was encrypted. [GH-6492] +* builder/vmware-iso: Fix crash caused by invalid datacenter url. [GH-6529] +* builder/vmware: Maintain original boot order during CreateVMX step for + vmware-iso builder [GH-6204] +* communicator/chroot: Fix quote escaping so that ansible provisioner works + properly. [GH-6635] +* core: Better error handling in downloader when connection error occurs. + [GH-6557] +* core: Fix broken pathing checks in checksum files. [GH-6525] +* provisioner/shell Create new template option allowing users to choose to + source env vars from a file rather than declaring them inline. This + resolves a bug that occurred when users had complex quoting in their + `execute_command`s [GH-6636] +* provisioner/shell-local: Windows inline scripts now default to being appended + with ".cmd", fixing a backwards incompatibility in v1.2.5 [GH-6626] +* provisioner/windows-restart: Provisioner now works when used in conjunction + with SSH communicator [GH-6606] + +### BACKWARDS INCOMPATIBILITIES: +* builder/amazon: "owners" field on source_ami_filter is now required for + secuirty reasons. [GH-6585] +* builder/vmware-iso: validation will fail for templates using esxi that have the "disk_type_id" set to something other than "thin" or "" and that do not have "skip_compaction": true also set. Use `packer fix` to fix this. [GH-6411] + +## 1.2.5 (July 16, 2018) + +### BUG FIXES: +* builder/alickoud: Fix issue where internet_max_bandwidth_out template option + was not being passed to builder. [GH-6416] +* builder/alicloud: Fix an issue with VPC cleanup. [GH-6418] +* builder/amazon-chroot: Fix communicator bug that broke chroot builds. + [GH-6363] +* builder/amazon: Replace packer's waiters with those from the AWS sdk, solving + several timeout bugs. [GH-6332] +* builder/azure: update azure-sdk-for-go, fixing 32-bit build errors. [GH-6479] +* builder/azure: update the max length of managed_image_resource_group to match + new increased length of 90 characters. [GH-6477] +* builder/hyper-v: Fix secure boot template feature so that it properly passes + the temolate for MicrosoftUEFICertificateAuthority. [GH-6415] +* builder/hyperv: Fix bug in HyperV IP lookups that was causing breakages in + FreeBSD/OpenBSD builds. [GH-6416] +* builder/qemu: Fix error race condition in qemu builder that caused convert to + fail on ubuntu 18.x [GH-6437] +* builder/qemu: vnc_bind_address was not being passed to qemu. [GH-6467] +* builder/virtualbox: Allow iso_url to be a symlink. [GH-6370] +* builder/vmware: Don't fail on DHCP lease files that cannot be read, fixing + bug where builder failed on NAT networks that don't serve DHCP. [GH-6415] +* builder/vmware: Fix bug where we couldn't discover IP if vm_name differed + from the vmx displayName. [GH-6448] +* builder/vmware: Fix validation to prevent hang when remopte_password is not + sent but vmware is building on esxi. [GH-6424] +* builder/vmware:Correctly default the vm export format to ovf; this is what + the docs claimed we already did, but we didn't. [GH-4538] +* communicator/winrm: Revert an attempt to determine whether remote upload + destinations were files or directories, as this broke uploads on systems + without Powershell installed. [GH-6481] +* core: Fix bug in parsing of iso checksum files that arose when setting + iso_url to a relative filepath. [GH-6488] +* core: Fix Packer crash caused by improper error handling in the downloader. + [GH-6381] +* fix: Fix bug where fixer for ssh_private_ip that failed when boolean values + are passed as strings. [GH-6458] +* provisioner/powershell: Make upload of powershell variables retryable, in + case of system restarts. [GH-6388] + +### IMPROVEMENTS: +* builder/amazon: Add the ap-northeast-3 region. [GH-6385] +* builder/amazon: Spot requests may now have tags applied using the `spot_tags` + option [GH-5452] +* builder/cloudstack: Add support for Projectid and new config option + prevent_firewall_changes. [GH-6487] +* builder/openstack: Add support for token authorization and cloud.yaml. + [GH-6362] +* builder/oracle-oci: Add new "instance_name" template option. [GH-6408] +* builder/scaleway: Add new "bootscript" parameter, allowing the user to not + use the default local bootscript [GH-6439] +* builder/vmware: Add support for linked clones to vmware-vmx. [GH-6394] +* debug: The -debug flag will now cause Packer to pause between provisioner + scripts in addition to Packer steps. [GH-4663] +* post-processor/googlecompute-import: Added new googlecompute-import post- + processor [GH-6451] +* provisioner/ansible: Add new "playbook_files" option to execute multiple + playbooks within one provisioner call. [GH-5086] + +## 1.2.4 (May 29, 2018) + +### BUG FIXES: + +* builder/amazon: Can now force the chroot builder to mount an entire block + device instead of a partition [GH-6194] +* builder/azure: windows-sql-cloud is now in the default list of projects to + check for provided images. [GH-6210] +* builder/chroot: A new template option, `nvme_device_path` has been added to + provide a workaround for users who need the amazon-chroot builder to mount + a NVMe volume on their instances. [GH-6295] +* builder/hyper-v: Fix command for mounting multiple disks [GH-6267] +* builder/hyperv: Enable IP retrieval for Server 2008 R2 hosts. [GH-6219] +* builder/hyperv: Fix bug in MAC address specification on Hyper-V. [GH-6187] +* builder/parallels-pvm: Add missing disk compaction step. [GH-6202] +* builder/vmware-esxi: Remove floppy files from the remote server on cleanup. + [GH-6206] +* communicator/winrm: Updated dependencies to fix a race condition [GH-6261] +* core: When using `-on-error=[abort|ask]`, output the error to the user. + [GH-6252] +* provisioner/puppet: Extra-Arguments are no longer prematurely + interpolated. [GH-6215] +* provisioner/shell: Remove file stat that was causing problems uploading files + [GH-6239] + +### IMPROVEMENTS: + +* builder/amazon: Amazon builders other than `chroot` now support T2 unlimited + instances [GH-6265] +* builder/azure: Allow device login for US government cloud. [GH-6105] +* builder/azure: Devicelogin Support for Windows [GH-6285] +* builder/azure: Enable simultaneous builds within one resource group. + [GH-6231] +* builder/azure: Faster deletion of Azure Resource Groups. [GH-6269] +* builder/azure: Updated Azure SDK to v15.0.0 [GH-6224] +* builder/hyper-v: Hyper-V builds now connect to vnc display by default when + building [GH-6243] +* builder/hyper-v: New `use_fixed_vhd_format` allows vm export in an Azure- + compatible format [GH-6101] +* builder/hyperv: New config option for specifying what secure boot template to + use, allowing secure boot of linux vms. [GH-5883] +* builder/qemu: Add support for hvf accelerator. [GH-6193] +* builder/scaleway: Fix SSH communicator connection issue. [GH-6238] +* core: Add opt-in Packer top-level command autocomplete [GH-5454] +* post-processor/shell-local: New options have been added to create feature + parity with the shell-local provisioner. This feature now works on Windows + hosts. [GH-5956] +* provisioner/chef: New config option allows user to skip cleanup of chef + client staging directory. [GH-4300] +* provisioner/shell-local: Can now access automatically-generated WinRM + password as variable [GH-6251] +* provisoner/shell-local: New options have been added to create feature parity + with the shell-local post-processor. This feature now works on Windows + hosts. [GH-5956] +* builder/virtualbox: Use HTTPS to download guest editions, now that it's + available. [GH-6406] + +## 1.2.3 (April 25, 2018) + +### BUG FIXES: + +* builder/azure: Azure CLI may now be logged into several accounts. [GH-6087] +* builder/ebssurrogate: Snapshot all launch devices. [GH-6056] +* builder/hyper-v: Fix CopyExportedVirtualMachine script so it works with + links. [GH-6082] +* builder/oracle-classic: Fix panics when cleaning up resources that haven't + been created. [GH-6095] +* builder/parallels: Allow user to cancel build while the OS is starting up. + [GH-6166] +* builder/qemu: Avoid warning when using raw format. [GH-6080] +* builder/scaleway: Fix compilation issues on solaris/amd64. [GH-6069] +* builder/virtualbox: Fix broken scancodes in boot_command. [GH-6067] +* builder/vmware-iso: Fail in validation if user gives wrong remote_type value. + [GH-4563] +* builder/vmware: Fixed a case-sensitivity issue when determining the network + type during the cloning step in the vmware-vmx builder. [GH-6057] +* builder/vmware: Fixes the DHCP lease and configuration pathfinders for VMware + Player. [GH-6096] +* builder/vmware: Multi-disk VM's can be properly handled by the compacting + stage. [GH-6074] +* common/bootcommand: Fix numerous bugs in the boot command code, and make + supported features consistent across builders. [GH-6129] +* communicator/ssh: Stop trying to discover whether destination is a directory + from uploader. [GH-6124] +* post-processor/vagrant: Large VMDKs should no longer show a 0-byte size on OS + X. [GH-6084] +* post-processor/vsphere: Fix encoding of spaces in passwords for upload. + [GH-6110] +* provisioner/ansible: Pass the inventory_directory configuration option to + ansible -i when it is set. [GH-6065] +* provisioner/powershell: fix bug with SSH communicator + cygwin. [GH-6160] +* provisioner/powershell: The {{.WinRMPassword}} template variable now works + with parallel builders. [GH-6144] + +### IMPROVEMENTS: + +* builder/alicloud: Update aliyungo common package. [GH-6157] +* builder/amazon: Expose more source ami data as template variables. [GH-6088] +* builder/amazon: Setting `force_delete` will only delete AMIs owned by the + user. This should prevent failures where we try to delete an AMI with a + matching name, but owned by someone else. [GH-6111] +* builder/azure: Users of Powershell provisioner may access the randomly- + generated winrm password using the template variable {{.WinRMPassword}}. + [GH-6113] +* builder/google: Users of Powershell provisioner may access the randomly- + generated winrm password using the template variable {{.WinRMPassword}}. + [GH-6141] +* builder/hyper-v: User can now configure hyper-v disk block size. [GH-5941] +* builder/openstack: Add configuration option for `instance_name`. [GH-6041] +* builder/oracle-classic: Better validation of destination image name. + [GH-6089] +* builder/oracle-oci: New config options for user data and user data file. + [GH-6079] +* builder/oracle-oci: use the official OCI sdk instead of handcrafted client. + [GH-6142] +* builder/triton: Add support to Skip TLS Verification of Triton Certificate. + [GH-6039] +* provisioner/ansible: Ansible users may provide a custom inventory file. + [GH-6107] +* provisioner/file: New `generated` tag allows users to upload files created + during Packer run. [GH-3891] + +## 1.2.2 (March 26, 2018) + +### BUG FIXES: + +* builder/amazon: Fix AWS credential defaulting [GH-6019] +* builder/LXC: make sleep timeout easily configurable [GH-6038] +* builder/virtualbox: Correctly send multi-byte scancodes when typing boot + command. [GH-5987] +* builder/virtualbox: Special boot-commands no longer overwrite previous + commands [GH-6002] +* builder/vmware: Default to disabling XHCI bus for USB on the vmware-iso + builder. [GH-5975] +* builder/vmware: Handle multiple devices per VMware network type [GH-5985] +* communicator/ssh: Handle errors uploading files more gracefully [GH-6033] +* provisioner/powershell: Fix environment variable file escaping. [GH-5973] + + +### IMPROVEMENTS: + +* builder/amazon: Added new region `cn-northwest-1`. [GH-5960] +* builder/amazon: Users may now access the amazon-generated administrator + password [GH-5998] +* builder/azure: Add support concurrent deployments in the same resource group. + [GH-6005] +* builder/azure: Add support for building with additional disks. [GH-5944] +* builder/azure: Add support for marketplace plan information. [GH-5970] +* builder/azure: Make all command output human readable. [GH-5967] +* builder/azure: Respect `-force` for managed image deletion. [GH-6003] +* builder/google: Add option to specify a service account, or to run without + one. [GH-5991] [GH-5928] +* builder/oracle-oci: Add new "use_private_ip" option. [GH-5893] +* post-processor/vagrant: Add LXC support. [GH-5980] +* provisioner/salt-masterless: Added Windows support. [GH-5702] +* provisioner/salt: Add windows support to salt provisioner [GH-6012] [GH-6012] + + +## 1.2.1 (February 23, 2018) + +### BUG FIXES: + +* builder/amazon: Fix authorization using assume role. [GH-5914] +* builder/hyper-v: Fix command collisions with VMWare PowerCLI. [GH-5861] +* builder/vmware-iso: Fix panic when building on esx5 remotes. [GH-5931] +* builder/vmware: Fix issue detecting host IP. [GH-5898] [GH-5900] +* provisioner/ansible-local: Fix conflicting escaping schemes for vars provided + via `--extra-vars`. [GH-5888] + +### IMPROVEMENTS: + +* builder/oracle-classic: Add `snapshot_timeout` option to control how long we + wait for the snapshot to be created. [GH-5932] +* builder/oracle-classic: Add support for WinRM connections. [GH-5929] + + +## 1.2.0 (February 9, 2018) + +### BACKWARDS INCOMPATIBILITIES: + +* 3rd party plugins: We have moved internal dependencies, meaning your 3rd + party plugins will no longer compile (however existing builds will still + work fine); the work to fix them is minimal and documented in GH-5810. + [GH-5810] +* builder/amazon: The `ssh_private_ip` option has been removed. Instead, please + use `"ssh_interface": "private"`. A fixer has been written for this, which + can be invoked with `packer fix`. [GH-5876] +* builder/openstack: Extension support has been removed. To use OpenStack + builder with the OpenStack Newton (Oct 2016) or earlier, we recommend you + use Packer v1.1.2 or earlier version. +* core: Affects Windows guests: User variables containing Powershell special + characters no longer need to be escaped. [GH-5376] +* provisioner/file: We've made destination semantics more consistent across the + various communicators. In general, if the destination is a directory, files + will be uploaded into the directory instead of failing. This mirrors the + behavior of `rsync`. There's a chance some users might be depending on the + previous buggy behavior, so it's worth ensuring your configuration is + correct. [GH-5426] +* provisioner/powershell: Regression from v1.1.1 forcing extra escaping of + environment variables in the non-elevated provisioner has been fixed. + [GH-5515] [GH-5872] + +### IMPROVEMENTS: + +* **New builder:** `ncloud` for building server images using the NAVER Cloud + Platform. [GH-5791] +* **New builder:** `oci-classic` for building new custom images for use with + Oracle Cloud Infrastructure Classic Compute. [GH-5819] +* **New builder:** `scaleway` - The Scaleway Packer builder is able to create + new images for use with Scaleway BareMetal and Virtual cloud server. + [GH-4770] +* builder/amazon: Add `kms_key_id` option to block device mappings. [GH-5774] +* builder/amazon: Add `skip_metadata_api_check` option to skip consulting the + amazon metadata service. [GH-5764] +* builder/amazon: Add Paris region (eu-west-3) [GH-5718] +* builder/amazon: Give better error messages if we have trouble during + authentication. [GH-5764] +* builder/amazon: Remove Session Token (STS) from being shown in the log. + [GH-5665] +* builder/amazon: Replace `InstanceStatusOK` check with `InstanceReady`. This + reduces build times universally while still working for all instance types. + [GH-5678] +* builder/amazon: Report which authentication provider we're using. [GH-5764] +* builder/amazon: Timeout early if metadata service can't be reached. [GH-5764] +* builder/amazon: Warn during prepare if we didn't get both an access key and a + secret key when we were expecting one. [GH-5762] +* builder/azure: Add validation for incorrect VHD URLs [GH-5695] +* builder/docker: Remove credentials from being shown in the log. [GH-5666] +* builder/google: Support specifying licenses for images. [GH-5842] +* builder/hyper-v: Allow MAC address specification. [GH-5709] +* builder/hyper-v: New option to use differential disks and Inline disk + creation to improve build time and reduce disk usage [GH-5631] +* builder/qemu: Add Intel HAXM support to QEMU builder [GH-5738] +* builder/triton: Triton RBAC is now supported. [GH-5741] +* builder/triton: Updated triton-go dependencies, allowing better error + handling. [GH-5795] +* builder/vmware-iso: Add support for cdrom and disk adapter types. [GH-3417] +* builder/vmware-iso: Add support for setting network type and network adapter + type. [GH-3417] +* builder/vmware-iso: Add support for usb/serial/parallel ports. [GH-3417] +* builder/vmware-iso: Add support for virtual soundcards. [GH-3417] +* builder/vmware-iso: More reliably retrieve the guest networking + configuration. [GH-3417] +* builder/vmware: Add support for "super" key in `boot_command`. [GH-5681] +* communicator/ssh: Add session-level keep-alives [GH-5830] +* communicator/ssh: Detect dead connections. [GH-4709] +* core: Gracefully clean up resources on SIGTERM. [GH-5318] +* core: Improved error logging in floppy file handling. [GH-5802] +* core: Improved support for downloading and validating a uri containing a + Windows UNC path or a relative file:// scheme. [GH-2906] +* post-processor/amazon-import: Allow user to specify role name in amazon- + import [GH-5817] +* post-processor/docker: Remove credentials from being shown in the log. + [GH-5666] +* post-processor/google-export: Synchronize credential semantics with the + Google builder. [GH-4148] +* post-processor/vagrant: Add vagrant post-processor support for Google + [GH-5732] +* post-processor/vsphere-template: Now accepts artifacts from the vSphere post- + processor. [GH-5380] +* provisioner/amazon: Use Amazon SDK's InstanceRunning waiter instead of + InstanceStatusOK waiter [GH-5773] +* provisioner/ansible: Improve user retrieval. [GH-5758] +* provisioner/chef: Add support for 'trusted_certs_dir' chef-client + configuration option [GH-5790] +* provisioner/chef: Added Policyfile support to chef-client provisioner. + [GH-5831] + +### BUG FIXES: + +* builder/alicloud-ecs: Attach keypair before starting instance in alicloud + builder [GH-5739] +* builder/amazon: Fix tagging support when building in us-gov/china. [GH-5841] +* builder/amazon: NewSession now inherits MaxRetries and other settings. + [GH-5719] +* builder/virtualbox: Fix interpolation ordering so that edge cases around + guest_additions_url are handled correctly [GH-5757] +* builder/virtualbox: Fix regression affecting users running Packer on a + Windows host that kept Packer from finding Virtualbox guest additions if + Packer ran on a different drive from the one where the guest additions were + stored. [GH-5761] +* builder/vmware: Fix case where artifacts might not be cleaned up correctly. + [GH-5835] +* builder/vmware: Fixed file handle leak that may have caused race conditions + in vmware builder [GH-5767] +* communicator/ssh: Add deadline to SSH connection to prevent Packer hangs + after script provisioner reboots vm [GH-4684] +* communicator/winrm: Fix issue copying empty directories. [GH-5763] +* provisioner/ansible-local: Fix support for `--extra-vars` in + `extra_arguments`. [GH-5703] +* provisioner/ansible-remote: Fixes an error where Packer's private key can be + overridden by inherited `ansible_ssh_private_key` options. [GH-5869] +* provisioner/ansible: The "default extra variables" feature added in Packer + v1.0.1 caused the ansible-local provisioner to fail when an --extra-vars + argument was specified in the extra_arguments configuration option; this + has been fixed. [GH-5335] +* provisioner/powershell: Regression from v1.1.1 forcing extra escaping of + environment variables in the non-elevated provisioner has been fixed. + [GH-5515] [GH-5872] + + +## 1.1.3 (December 8, 2017) + +### IMPROVEMENTS: + +* builder/alicloud-ecs: Add security token support and set TLS handshake + timeout through environment variable. [GH-5641] +* builder/amazon: Add a new parameter `ssh_interface`. Valid values include + `public_ip`, `private_ip`, `public_dns` or `private_dns`. [GH-5630] +* builder/azure: Add sanity checks for resource group names [GH-5599] +* builder/azure: Allow users to specify an existing resource group to use, + instead of creating a new one for every run. [GH-5548] +* builder/hyper-v: Add support for differencing disk. [GH-5458] +* builder/vmware-iso: Improve logging of network errors. [GH-5456] +* core: Add new `packer_version` template engine. [GH-5619] +* core: Improve logic checking for downloaded ISOs in case where user has + provided more than one URL in `iso_urls` [GH-5632] +* provisioner/ansible-local: Add ability to clean staging directory. [GH-5618] + +### BUG FIXES: + +* builder/amazon: Allow `region` to appear in `ami_regions`. [GH-5660] +* builder/amazon: `C5` instance types now build more reliably. [GH-5678] +* builder/amazon: Correctly set AWS region if given in template along with a + profile. [GH-5676] +* builder/amazon: Prevent `sriov_support` and `ena_support` from being used + with spot instances, which would cause a build failure. [GH-5679] +* builder/hyper-v: Fix interpolation context for user variables in + `boot_command` [GH-5547] +* builder/qemu: Set default disk size to 40960 MB to prevent boot failures. + [GH-5588] +* builder/vmware: Correctly detect Windows boot on vmware workstation. + [GH-5672] +* core: Fix windows path regression when downloading ISOs. [GH-5591] +* provisioner/chef: Fix chef installs on Windows. [GH-5649] + +## 1.1.2 (November 15, 2017) + +### IMPROVEMENTS: + +* builder/amazon: Correctly deregister AMIs when `force_deregister` is set. + [GH-5525] +* builder/digitalocean: Add `ipv6` option to enable on droplet. [GH-5534] +* builder/docker: Add `aws_profile` option to control the aws profile for ECR. + [GH-5470] +* builder/google: Add `clean_image_name` template engine. [GH-5463] +* builder/google: Allow selecting container optimized images. [GH-5576] +* builder/google: Interpolate network and subnetwork values, rather than + relying on an API call that packer may not have permission for. [GH-5343] +* builder/hyper-v: Add `disk_additional_size` option to allow for up to 64 + additional disks. [GH-5491] +* builder/hyper-v: Also disable automatic checkpoints for gen 2 VMs. [GH-5517] +* builder/lxc: Add new `publish_properties` field to set image properties. + [GH-5475] +* builder/lxc: Add three new configuration option categories to LXC builder: + `create_options`, `start_options`, and `attach_options`. [GH-5530] +* builder/triton: Add `source_machine_image_filter` option to select an image + ID based on a variety of parameters. [GH-5538] +* builder/virtualbox-ovf: Error during prepare if source path doesn't exist. + [GH-5573] +* builder/virtualbox-ovf: Retry while removing VM to solve for transient + errors. [GH-5512] +* communicator/ssh: Add socks 5 proxy support. [GH-5439] +* core/iso_config: Support relative paths in checksum file. [GH-5578] +* core: Rewrite vagrantfile code to make cross-platform development easier. + [GH-5539] +* post-processor/docker-push: Add `aws_profile` option to control the aws + profile for ECR. [GH-5470] +* post-processor/vsphere: Properly capture `ovftool` output. [GH-5499] + +### BUG FIXES: + +* builder/amazon: Add a delay option to security group waiter. [GH-5536] +* builder/amazon: Fix regressions relating to spot instances and EBS volumes. + [GH-5495] +* builder/amazon: Set region from profile, if profile is set, rather than being + overridden by metadata. [GH-5562] +* builder/docker: Remove `login_email`, which no longer exists in the docker + client. [GH-5511] +* builder/hyperv: Fix admin check that was causing powershell failures. + [GH-5510] +* builder/oracle: Defaulting of OCI builder region will first check the packer + template and the OCI config file. [GH-5407] +* builder/triton: Fix a bug where partially created images can be reported as + complete. [GH-5566] +* post-processor/vsphere: Use the vm disk path information to re-create the vmx + datastore path. [GH-5567] +* provisioner/windows-restart: Wait for restart no longer endlessly loops if + user specifies a custom restart check command. [GH-5563] + +## 1.1.1 (October 13, 2017) + +### IMPROVEMENTS: + +* **New builder:** `hyperv-vmcx` for building images from existing VMs. + [GH-4944] [GH-5444] +* builder/amazon-instance: Add `.Token` as a variable in the + `BundleUploadCommand` template. [GH-5288] +* builder/amazon: Add `temporary_security_group_source_cidr` option to control + ingress to source instances. [GH-5384] +* builder/amazon: Output AMI Name during prevalidation. [GH-5389] +* builder/amazon: Support template functions in tag keys. [GH-5381] +* builder/amazon: Tag volumes on creation instead of as a separate step. + [GH-5417] +* builder/docker: Add option to set `--user` flag when running `exec`. + [GH-5406] +* builder/docker: Set file owner to container user when uploading. Can be + disabled by setting `fix_upload_owner` to `false`. [GH-5422] +* builder/googlecompute: Support setting labels on the resulting image. + [GH-5356] +* builder/hyper-v: Add `vhd_temp_path` option to control where the VHD resides + while it's being provisioned. [GH-5206] +* builder/hyper-v: Allow vhd or vhdx source images instead of just ISO. + [GH-4944] [GH-5444] +* builder/hyper-v: Disable automatic checkpoints. [GH-5374] +* builder/virtualbox-ovf: Add `keep_registered` option. [GH-5336] +* builder/vmware: Add `disable_vnc` option to prevent VNC connections from + being made. [GH-5436] +* core: Releases will now be built for ppc64le. +* post-processor/vagrant: When building from a builder/hyper-v artifact, link + instead of copy when available. [GH-5207] + + +### BUG FIXES: + +* builder/cloudstack: Fix panic if build is aborted. [GH-5388] +* builder/hyper-v: Respect `enable_dynamic_memory` flag. [GH-5363] +* builder/puppet-masterless: Make sure directories created with sudo are + writable by the packer user. [GH-5351] +* provisioner/chef-solo: Fix issue installing chef-solo on Windows. [GH-5357] +* provisioner/powershell: Fix issue setting environment variables by writing + them to a file, instead of the command line. [GH-5345] +* provisioner/powershell: Fix issue where powershell scripts could hang. + [GH-5082] +* provisioner/powershell: Fix Powershell progress stream leak to stderr for + normal and elevated commands. [GH-5365] +* provisioner/puppet-masterless: Fix bug where `puppet_bin_dir` wasn't being + respected. [GH-5340] +* provisioner/puppet: Fix setting facter vars on Windows. [GH-5341] + + +## 1.1.0 (September 12, 2017) + +### IMPROVEMENTS: + +* builder/alicloud: Update alicloud go sdk and enable multi sites support for + alicloud [GH-5219] +* builder/amazon: Upgrade aws-sdk-go to 1.10.14, add tags at instance run time. + [GH-5196] +* builder/azure: Add object_id to windows_custom_image.json. [GH-5285] +* builder/azure: Add support for storage account for managed images. [GH-5244] +* builder/azure: Update pkcs12 package. [GH-5301] +* builder/cloudstack: Add support for Security Groups. [GH-5175] +* builder/docker: Uploading files and directories now use docker cp. [GH-5273] + [GH-5333] +* builder/googlecompute: Add `labels` option for labeling launched instances. + [GH-5308] +* builder/googlecompute: Add support for accelerator api. [GH-5137] +* builder/profitbricks: added support for Cloud API v4. [GH-5233] +* builder/vmware-esxi: Remote builds now respect `output_directory` [GH-4592] +* builder/vmware: Set artifact ID to `VMName`. [GH-5187] +* core: Build solaris binary by default. [GH-5268] [GH-5248] +* core: Remove LGPL dependencies. [GH-5262] +* provisioner/puppet: Add `guest_os_type` option to add support for Windows. + [GH-5252] +* provisioner/salt-masterless: Also use sudo to clean up if we used sudo to + install. [GH-5240] + +### BACKWARDS INCOMPATIBILITIES: + +* builder/amazon: Changes way that AMI artifacts are printed out after build, + aligning them to builder. Could affect output parsing. [GH-5281] +* builder/amazon: Split `enhanced_networking` into `sriov_support` and + `ena_support` to support finer grained control. Use `packer fix + ` to automatically update your template to use `ena_support` + where previously there was only `enhanced_networking`. Make sure to also + add `sriov_support` if you need that feature, and to ensure `ena_support` + is what you intended to be in your template. [GH-5284] +* builder/cloudstack: Setup temporary SSH keypair; backwards incompatible in + the uncommon case that the source image allowed SSH auth with password but + not with keypair. [GH-5174] +* communicator/ssh: Renamed `ssh_disable_agent` to + `ssh_disable_agent_forwarding`. Need to run fixer on packer configs that + use `ssh_disable_agent`. [GH-5024] +* communicator: Preserve left-sided white-space in remote command output. Make + sure any scripts that parse this output can handle the new whitespace + before upgrading. [GH-5167] +* provisioner/shell: Set default for `ExpectDisconnect` to `false`. If your + script causes the connection to be reset, you should set this to `true` to + prevent errors. [GH-5283] + +### BUG FIXES: + +* builder/amazon: `force_deregister` works in all regions, not just original + region. [GH-5250] +* builder/docker: Directory uploads now use the same semantics as the rest of + the communicators. [GH-5333] +* builder/vmware: Fix timestamp in default VMName. [GH-5274] +* builder/winrm: WinRM now waits to make sure commands can run successfully + before considering itself connected. [GH-5300] +* core: Fix issue where some builders wouldn't respect `-on-error` behavior. + [GH-5297] +* provisioner/windows-restart: The first powershell provisioner after a restart + now works. [GH-5272] + +### FEATURES: + +* **New builder**: Oracle Cloud Infrastructure (OCI) builder for creating + custom images. [GH-4554] +* **New builder:** `lxc` for building lxc images. [GH-3523] +* **New builder:** `lxd` for building lxd images. [GH-3625] +* **New post-processor**: vSphere Template post-processor to be used with + vmware-iso builder enabling user to mark a VM as a template. [GH-5114] + +## 1.0.4 (August 11, 2017) + +### IMPROVEMENTS: + +* builder/alicloud: Increase polling timeout. [GH-5148] +* builder/azure: Add `private_virtual_network_with_public_ip` option to + optionally obtain a public IP. [GH-5222] +* builder/googlecompute: use a more portable method of obtaining zone. + [GH-5192] +* builder/hyperv: Properly interpolate user variables in template. [GH-5184] +* builder/parallels: Remove soon to be removed --vmtype flag in createvm. + [GH-5172] +* contrib: add json files to zsh completion. [GH-5195] + +### BUG FIXES: + +* builder/amazon: Don't delete snapshots we didn't create. [GH-5211] +* builder/amazon: fix builds when using the null communicator. [GH-5217] +* builder/docker: Correctly handle case when uploading an empty directory. + [GH-5234] +* command/push: Don't push variables if they are unspecified. Reverts to + behavior in 1.0.1. [GH-5235] +* command/push: fix handling of symlinks. [GH-5226] +* core: Strip query parameters from ISO URLs when checking against a checksum + file. [GH-5181] +* provisioner/ansible-remote: Fix issue where packer could hang communicating + with ansible-remote. [GH-5146] + +## 1.0.3 (July 17, 2017) + +### IMPROVEMENTS: +* builder/azure: Update to latest Azure SDK, enabling support for managed + disks. [GH-4511] +* builder/cloudstack: Add default cidr_list [ 0.0.0.0/0 ]. [GH-5125] +* builder/cloudstack: Add support for ssh_agent_auth. [GH-5130] +* builder/cloudstack: Add support for using a HTTP server. [GH-5017] +* builder/cloudstack: Allow reading api_url, api_key, and secret_key from env + vars. [GH-5124] +* builder/cloudstack: Make expunge optional and improve logging output. + [GH-5099] +* builder/googlecompute: Allow using URL's for network and subnetwork. + [GH-5035] +* builder/hyperv: Add support for floppy_dirs with hyperv-iso builder. +* builder/hyperv: Add support for override of system %temp% path. +* core: Experimental Android ARM support. [GH-5111] +* post-processor/atlas: Disallow packer push of vagrant.box artifacts to atlas. + [GH-4780] +* postprocessor/atlas: Disallow pushing vagrant.box artifacts now that Vagrant + cloud is live. [GH-4780] + +### BUG FIXES: +* builder/amazon: Fix panic that happens if ami_block_device_mappings is empty. + [GH-5059] +* builder/azure: Write private SSH to file in debug mode. [GH-5070] [GH-5074] +* builder/cloudstack: Properly report back errors. [GH-5103] [GH-5123] +* builder/docker: Fix windows filepath in docker-toolbox call [GH-4887] +* builder/docker: Fix windows filepath in docker-toolbox call. [GH-4887] +* builder/hyperv: Use SID to verify membership in Admin group, fixing for non- + english users. [GH-5022] +* builder/hyperv: Verify membership in the group Hyper-V Administrators by SID + not name. [GH-5022] +* builder/openstack: Update gophercloud version, fixing builds > 1 hr long. + [GH-5046] +* builder/parallels: Skip missing paths when looking for unnecessary files. + [GH-5058] +* builder/vmware-esxi: Fix VNC port discovery default timeout. [GH-5051] +* communicator/ssh: Add ProvisionerTypes to communicator tests, resolving panic + [GH-5116] +* communicator/ssh: Resolve race condition that sometimes truncates ssh + provisioner stdout [GH-4719] +* post-processor/checksum: Fix interpolation of "output". [GH-5112] +* push: Push vars in packer config, not just those set from command line and in + var-file. [GH-5101] + +## 1.0.2 (June 21, 2017) + +### BUG FIXES: +* communicator/ssh: Fix truncated stdout from remote ssh provisioner. [GH-5050] +* builder/amazon: Fix bugs related to stop instance command. [GH-4719] +* communicator/ssh: Fix ssh connection errors. [GH-5038] +* core: Remove logging that shouldn't be there when running commands. [GH-5042] +* provisioner/shell: Fix bug where scripts were being run under `sh`. [GH-5043] + +### IMPROVEMENTS: + +* provisioner/windows-restart: make it clear that timeouts come from the + provisioner, not winrm. [GH-5040] + +## 1.0.1 (June 19, 2017) + +### IMPROVEMENTS: + +* builder/amazon: Allow amis to be copied to other regions, encrypted with + custom KMS keys. [GH-4948] +* builder/amazon: Allow configuration of api endpoint to support api-compatible + cloud providers. [GH-4896] +* builder/amazon: Fix regex used for ami name validation [GH-4902] +* builder/amazon: Look up vpc from subnet id if no vpc was specified. [GH-4879] +* builder/amazon: Print temporary security group name to the UI. [GH-4997] +* builder/amazon: Support Assume Role with MFA and ECS Task Roles. Also updates + to a newer version of aws-sdk-go. [GH-4996] +* builder/amazon: Use retry logic when creating instance tags. [GH-4876] +* builder/amazon: Validate ami name. [GH-4762] +* builder/azure: Add build output to artifact. [GH-4953] +* builder/azure: Use disk URI as artifact ID. [GH-4981] +* builder/digitalocean: Added support for monitoring. [GH-4782] +* builder/digitalocean: Support for copying snapshot to other regions. + [GH-4893] +* builder/hyper-v: Remove the check for administrator rights when sending key + strokes to Hyper-V. [GH-4687] # builder/openstack: Fix private key error + message to match documentation [GH-4898] +* builder/null: Support SSH agent auth [GH-4956] +* builder/openstack: Add ssh agent support. [GH-4655] +* builder/openstack: Support client x509 certificates. [GH-4921] +* builder/parallels-iso: Configuration of disk type, plain or expanding. + [GH-4621] +* builder/triton: An SSH agent can be used to authenticate requests, making + `triton_key_material` optional. [GH-4838] +* builder/triton: If no source machine networks are specified, instances are + started on the default public and internal networks. [GH-4838] +* builder/virtualbox: Add sata port count configuration option. [GH-4699] +* builder/virtualbox: Don't add port forwarding when using "none" communicator. + [GH-4960] +* builder/vmware: Add option to remove interfaces from the vmx. [GH-4927] +* builder/vmware: Properly remove mounted CDs on OS X. [GH-4810] +* builder/vmware: VNC probe timeout is configurable. [GH-4919] +* command/push: add `-sensitive` flag to mark pushed vars are sensitive. + [GH-4970] +* command/push: Vagrant support in Terraform Enterprise is deprecated. + [GH-4950] +* communicator/ssh: Add ssh agent support for bastion connections. [GH-4940] +* communicator/winrm: Add NTLM authentication support. [GH-4979] +* communicator/winrm: Add support for file downloads. [GH-4748] +* core: add telemetry for better product support. [GH-5015] +* core: Build binaries for arm64 [GH-4892] +* post-processor/amazon-import: Add support for `license_type`. [GH-4634] +* post-processor/vagrant-cloud: Get vagrant cloud token from environment. + [GH-4982] +* provisioner/ansible-local: Add extra-vars `packer_build_name`, + `packer_builder_type`, and `packer_http_addr`. [GH-4821] +* provisioner/ansible: Add `inventory_directory` option to control where to + place the generated inventory file. [GH-4760] +* provisioner/ansible: Add `skip_version_check` flag for when ansible will be + installed from a prior provisioner. [GH-4983] +* provisioner/ansible: Add extra-vars `packer_build_name` and + `packer_builder_type`. [GH-4821] +* provisioner/chef-solo: Add option to select Chef version. [GH-4791] +* provisioner/salt: Add salt bin directory configuration. [GH-5009] +* provisioner/salt: Add support for grains. [GH-4961] +* provisioner/shell: Use `env` to set environment variables to support freebsd + out of the box. [GH-4909] +* website/docs: Clarify language, improve formatting. [GH-4866] +* website/docs: Update docker metadata fields that can be changed. [GH-4867] + + +### BUG FIXES: + +* builder/amazon-ebssurrogate: Use ami device settings when creating the AMI. + [GH-4972] +* builder/amazon: don't try to delete extra volumes during clean up. [GH-4930] +* builder/amazon: fix `force_delete_snapshot` when the launch instance has + extra volumes. [GH-4931] +* builder/amazon: Only delete temporary key if we created one. [GH-4850] +* builder/azure: Replace calls to panic with error returns. [GH-4846] +* communicator/winrm: Use KeepAlive to keep long-running connections open. + [GH-4952] +* core: Correctly reject config files which have junk after valid json. + [GH-4906] +* post-processor/checksum: fix crash when invalid checksum is used. [GH-4812] +* post-processor/vagrant-cloud: don't read files to upload in to memory first. + [GH-5005] +* post-processor/vagrant-cloud: only upload once under normal conditions. + [GH-5008] +* provisioner/ansible-local: Correctly set the default staging directory under + Windows. [GH-4792] + +### FEATURES: + +* **New builder:** `alicloud-ecs` for building Alicloud ECS images. [GH-4619] + + +## 1.0.0 (April 4, 2017) + +### BUG FIXES: + +* builder/amazon: Fix b/c issue by reporting again the tags we create. + [GH-4704] +* builder/amazon: Fix crash in `step_region_copy`. [GH-4642] +* builder/googlecompute: Correct values for `on_host_maintenance`. [GH-4643] +* builder/googlecompute: Use "default" service account. [GH-4749] +* builder/hyper-v: Don't wait for shutdown_command to return. [GH-4691] +* builder/virtualbox: fix `none` communicator by allowing skipping upload of + version file. [GH-4678] +* builder/virtualbox: retry removing floppy controller. [GH-4705] +* communicator/ssh: don't return error if we can't close connection. [GH-4741] +* communicator/ssh: fix nil pointer error. [GH-4690] +* core: fix version number +* core: Invoking packer `--help` or `--version` now exits with status 0. + [GH-4723] +* core: show correct step name when debugging. [GH-4672] +* communicator/winrm: Directory uploads behave more like scp. [GH-4438] + +### IMPROVEMENTS: + +* builder/amazon-chroot: Ability to give an empty list in `copy_files` to + prevent the default `/etc/resolv.conf` file from being copied. If + `copy_files` isn't given at all, the default behavior remains. [GH-4708] +* builder/amazon: set force_deregister to true on -force. [GH-4649] +* builder/amazon: validate ssh key name/file. [GH-4665] +* builder/ansible: Clearer error message when we have problems getting the + ansible version. [GH-4694] +* builder/hyper-v: validate output dir in step, not in config. [GH-4645] +* More diligently try to complete azure-setup.sh. [GH-4752] +* website: fix display on ios devices. [GH-4618] + +## 0.12.3 (March 1, 2017) + +### BACKWARDS INCOMPATIBILITIES: + +* provisioner/ansible: by default, the staging dir will be randomized. [GH-4472] + +### FEATURES: + +* **New builder:** `ebs-surrogate` for building AMIs from EBS volumes. [GH-4351] + +### IMPROVEMENTS: + +* builder/amazon-chroot: support encrypted boot volume. [GH-4584] +* builder/amazon: Add BuildRegion and SourceAMI template variables. [GH-4399] +* builder/amazon: Change EC2 Windows password timeout to 20 minutes. [GH-4590] +* builder/amazon: enable ena when `enhanced_networking` is set. [GH-4578] +* builder/azure:: add two new config variables for temp_compute_name and + temp_resource_group_name. [GH-4468] +* builder/docker: create export dir if needed. [GH-4439] +* builder/googlecompute: Add `on_host_maintenance` option. [GH-4544] +* builder/openstack: add reuse_ips option to try to re-use existing IPs. + [GH-4564] +* builder/vmware-esxi: try for longer to connect to vnc port. [GH-4480] + [GH-4610] +* builder/vmware: allow extra options for ovftool. [GH-4536] +* builder/vmware: don't cache ip address so we know if it changes. [GH-4532] +* communicator/docker: preserve file mode. [GH-4443] +* communicator/ssh: Use SSH agent when enabled for bastion step. [GH-4598] +* communicator/winrm: support ProxyFromEnvironment. [GH-4463] +* core: don't show ui color if we're not colorized. [GH-4525] +* core: make VNC links clickable in terminal. [GH-4497] [GH-4498] +* docs: add community page. [GH-4550] +* post-processor/amazon-import: support AMI attributes on import [GH-4216] +* post-processor/docker-import: print stderr on docker import failure. + [GH-4529] + +### BUG FIXES: + +* builder/amazon-ebsvolume: Fix interpolation of block_device. [GH-4464] +* builder/amazon: Fix ssh agent authentication. [GH-4597] +* builder/docker: Don't force tag if using a docker version that doesn't + support it. [GH-4560] +* builder/googlecompute: fix bug when creating image from custom image_family. + [GH-4518] +* builder/virtualbox: remove guest additions before saving image. [GH-4496] +* core: always check for an error first when walking a path. [GH-4467] +* core: update crypto/ssh lib to fix large file uploads. [GH-4546] +* provisioner/chef-client: only upload knife config if we're cleaning. + [GH-4534] + +## 0.12.2 (January 20, 2017) + +### FEATURES: + +* **New builder:** `triton` for building images for Joyent Triton. [GH-4325] +* **New provisioner:** `converge` for provisioning with converge.sh. [GH-4326] + +### IMPROVEMENTS: + +* builder/hyperv-iso: add `iso_target_extension` option. [GH-4294] +* builder/openstack: Add support for instance metadata. [GH-4361] +* builder/openstack: Attempt to use existing floating IPs before allocating a + new one. [GH-4357] +* builder/parallels-iso: add `iso_target_extension` option. [GH-4294] +* builder/qemu: add `iso_target_extension` option. [GH-4294] +* builder/qemu: add `use_default_display` option for osx compatibility. + [GH-4293] +* builder/qemu: Detect input disk image format during copy/convert. [GH-4343] +* builder/virtualbox-iso: add `iso_target_extension` option. [GH-4294] +* builder/virtualbox: add `skip_export` option to skip exporting the VM after + build completes. [GH-4339] +* builder/vmware & builder/qemu: Allow configurable delay between keystrokes + when typing boot command. [GH-4403] +* builder/vmware-iso: add `iso_target_extension` option. [GH-4294] +* builder/vmware-iso: add `skip_export` option to skip exporting the VM after + build completes. [GH-4378] +* builder/vmware: Try to use `ip address` to find host IP. [GH-4411] +* common/step_http\_server: set `PACKER_HTTP_ADDR` env var for accessing http + server from inside builder. [GH-4409] +* provisioner/powershell: Allow equals sign in value of environment variables. + [GH-4328] +* provisioner/puppet-server: Add default facts. [GH-4286] + +### BUG FIXES: + +* builder/amazon-chroot: Panic in AMI region copy step. [GH-4341] +* builder/amazon: Crashes when new EBS vols are used. [GH-4308] +* builder/amazon: Fix crash in amazon-instance. [GH-4372] +* builder/amazon: fix run volume tagging [GH-4420] +* builder/amazon: fix when using non-existent security\_group\_id. [GH-4425] +* builder/amazon: Properly error if we don't have the + ec2:DescribeSecurityGroups permission. [GH-4304] +* builder/amazon: Properly wait for security group to exist. [GH-4369] +* builder/docker: Fix crash when performing log in to ECR with an invalid URL. + [GH-4385] +* builder/openstack: fix for finding resource by ID. [GH-4301] +* builder/qemu: Explicitly set WinRMPort for StepConnect. [GH-4321] +* builder/virtualbox: Explicitly set WinRMPort for StepConnect. [GH-4321] +* builder/virtualbox: Pause between each boot command element in -debug. + [GH-4346] +* builder/vmware builder/parallels: Fix hang when shutting down windows in + certain cases. [GH-4436] +* command/push: Don't interpolate variables when pushing. [GH-4389] +* common/step_http_server: make port range inclusive. [GH-4398] +* communicator/winrm: update winrm client, resolving `MaxMemoryPerShellMB` + errors and properly error logging instead of panicking. [GH-4412] [GH-4424] +* provider/windows-shell: Allows equals sign in env var value. [GH-4423] + +## 0.12.1 (December 15, 2016) + +### BACKWARDS INCOMPATIBILITIES: + +* `ssh_username` is now required if using communicator ssh. [GH-4172] +* builder/amazon: Change `shutdown_behaviour` to `shutdown_behavior`. Run + "packer fix template.json" to migrate a template. [GH-4285] +* builder/openstack: No long supports the `api_key` option for rackspace. + [GH-4283] +* post-processor/manifest: Changed `filename` field to be `output`, to be more + consistent with other post-processors. `packer fix` will fix this for you. + [GH-4192] +* post-processor/shell-local: Now runs per-builder instead of per-file. The + filename is no longer passed in as an argument to the script, but instead + needs to be gleaned from the manifest post-processor. [GH-4189] + +### FEATURES: + +* **New builder:** "Hyper-V" Added new builder for Hyper-V on Windows. + [GH-2576] +* **New builder:** "1&1" Added new builder for [1&1](https://www.1and1.com/). + [GH-4163] + +### IMPROVEMENTS: + +* builder/amazon-ebs: Support specifying KMS key for encryption. [GH-4023] +* builder/amazon-ebsvolume: Add artifact output. [GH-4141] +* builder/amazon: Add `snapshot_tag` overrides. [GH-4015] +* builder/amazon: Added new region London - eu-west-2. [GH-4284] +* builder/amazon: Added ca-central-1 to list of known aws regions. [GH-4274] +* builder/amazon: Adds `force_delete_snapshot` flag to also cleanup snapshots + if we're removing a preexisting image, as with `force_deregister_image`. + [GH-4223] +* builder/amazon: Support `snapshot_users` and `snapshot_groups` for sharing + ebs snapshots. [GH-4243] +* builder/cloudstack: Support reusing an already associated public IP. + [GH-4149] +* builder/docker: Introduce docker commit changes, author, and message. + [GH-4202] +* builder/googlecompute: Support `source_image_family`. [GH-4162] +* builder/googlecompute: enable support for Google Compute XPN. [GH-4288] +* builder/openstack: Added `image_members` to add new members to image after + it's created. [GH-4283] +* builder/openstack: Added `image_visibility` field to specify visibility of + created image. [GH-4283] +* builder/openstack: Automatically reauth as needed. [GH-4262] +* builder/virtualbox-ovf: Can now give a URL to an ova file. [GH-3982] +* communicator/ssh: adds ability to download download directories and + wildcards, fix destination file mode (not hardcoded anymore). [GH-4210] +* post-processor/shell-local: support spaces in script path. [GH-4144] +* provisioner/ansible: Allow `winrm` communicator. [GH-4209] +* provisioner/salt: Bootstrap fallback on wget if curl failed. [GH-4244] + +### BUG FIXES: + +* builder/amazon: Correctly assign key from `ssh_keypair_name` to source + instance. [GH-4222] +* builder/amazon: Fix `source_ami_filter` ignores `owners`. [GH-4235] +* builder/amazon: Fix launching spot instances in EC2 Classic [GH-4204] +* builder/qemu: Fix issue where multiple commands on a single line + in boot_command wouldn't be parsed correctly. [GH-4269] +* core: Unbreak glob patterns in `floppy_files`. [GH-3890] +* post-processor/checksum: cleanup, and fix output to specified file with + more than one artifacts. [GH-4210] +* post-processor/checksum: reset hash after each artifact file. [GH-4210] +* provisioner/file: fix for directory download. [GH-4210] +* provisioner/file: fix issue uploading multiple files to a directory, + mentioned in [GH-4049]. [GH-4210] +* provisioner/shell: Treat disconnects as retryable when running cleanup. If + you have a reboot in your script, we'll now wait until the host is + available before attempting to cleanup the script. [GH-4197] + +## 0.12.0 (November 15, 2016) + +### FEATURES: + +* **New builder:** "cloudstack" Can create new templates for use with + CloudStack taking either an ISO or existing template as input. [GH-3909] +* **New builder:** "profitbricks" Builder for creating images in the + ProfitBricks cloud. [GH-3660] +* **New builder:** "amazon-ebsvolume" Can create Amazon EBS volumes which are + preinitialized with a filesystem and data. [GH-4088] + + +### IMPROVEMENTS: + +* builder/amazon: Allow polling delay override with `AWS_POLL_DELAY_SECONDS`. + [GH-4083] +* builder/amazon: Allow use of local SSH Agent. [GH-4050] +* builder/amazon: Dynamic source AMI [GH-3817] +* builder/amazon: Show AMI ID found when using `source_ami_filter`. [GH-4096] +* builder/googlecompute: Support `ssh_private_key_file` in communicator. + [GH-4101] +* builder/googlecompute: Support custom scopes. [GH-4043] +* command/push: Fix variable pushes to Atlas. Still needs Atlas server to be + updated before the issue will be fixed completely. [GH-4089] +* communicator/ssh: Improved SSH upload performance. [GH-3940] +* contrib/azure-setup.sh: Support for azure-cli 0.10.7. [GH-4133] +* docs: Fix command line variable docs. [GH-4143] +* post-processor/vagrant: Fixed inconsistency between vagrant-libvirt driver + and packer QEMU accelerator. [GH-4104] +* provisioner/ansible: Move info messages to log [GH-4123] +* provisioner/puppet: Add `puppet_bin_dir` option. [GH-4014] +* provisioner/salt: Add `salt_call_args` option. [GH-4158] + +### BUG FIXES: + +* builder/amazon: Fixed an error where we wouldn't fail the build even if we + timed out waiting for the temporary security group to become available. + [GH-4099] +* builder/amazon: Properly cleanup temporary key pairs. [GH-4080] +* builder/google: Fix issue where we'd hang waiting for a startup script + which doesn't exist. [GH-4102] +* builder/qemu: Fix keycodes for ctrl, shift and alt keys. [GH-4115] +* builder/vmware: Fix keycodes for ctrl, shift and alt keys. [GH-4115] +* builder/vmware: Fixed build error when shutting down. [GH-4041] +* common/step_create_floppy: Fixed support for 1.44MB floppies on Windows. + [GH-4135] +* post-processor/googlecompute-export: Fixes scopes. [GH-4147] +* provisioner/powershell: Reverted [GH-3371] fixes quoting issue. [GH-4069] +* scripts: Fix build under Windows for go 1.5. [GH-4142] + +## 0.11.0 (October 21, 2016) + +### BACKWARDS INCOMPATIBILITIES: + +* VNC and VRDP-like features in VirtualBox, VMware, and QEMU now configurable + but bind to 127.0.0.1 by default to improve security. See the relevant + builder docs for more info. +* Docker builder requires Docker > 1.3 +* provisioner/chef-solo: default staging directory renamed to + `packer-chef-solo`. [GH-3971] + +### FEATURES: + +* **New Checksum post-processor**: Create a checksum file from your build + artifacts as part of your build. [GH-3492] [GH-3790] +* **New build flag** `-on-error` to allow inspection and keeping artifacts on + builder errors. [GH-3885] +* **New Google Compute Export post-processor**: exports an image from a Packer + googlecompute builder run and uploads it to Google Cloud Storage. + [GH-3760] +* **New Manifest post-processor**: writes metadata about packer's output + artifacts data to a JSON file. [GH-3651] + + +### IMPROVEMENTS: + +* builder/amazon: Added `disable_stop_instance` option to prevent automatic + shutdown when the build is complete. [GH-3352] +* builder/amazon: Added `shutdown_behavior` option to support `stop` or + `terminate` at the end of the build. [GH-3556] +* builder/amazon: Added `skip_region_validation` option to allow newer or + custom AWS regions. [GH-3598] +* builder/amazon: Added `us-east-2` and `ap-south-1` regions. [GH-4021] + [GH-3663] +* builder/amazon: Support building from scratch with amazon-chroot builder. + [GH-3855] [GH-3895] +* builder/amazon: Support create an AMI with an `encrypt_boot` volume. + [GH-3382] +* builder/azure: Add `os_disk_size_gb`. [GH-3995] +* builder/azure: Add location to setup script. [GH-3803] +* builder/azure: Allow user to set custom data. [GH-3996] +* builder/azure: Made `tenant_id` optional. [GH-3643] +* builder/azure: Now pre-validates `capture_container_name` and + `capture_name_prefix` [GH-3537] +* builder/azure: Removed superfluous polling code for deployments. [GH-3638] +* builder/azure: Support for a user defined VNET. [GH-3683] +* builder/azure: Support for custom images. [GH-3575] +* builder/azure: tag all resources. [GH-3764] +* builder/digitalocean: Added `user_data_file` support. [GH-3933] +* builder/digitalocean: Fixes timeout waiting for snapshot. [GH-3868] +* builder/digitalocean: Use `state_timeout` for unlock and off transitions. + [GH-3444] +* builder/docker: Improved support for Docker pull from Amazon ECR. [GH-3856] +* builder/google: Add `-force` option to delete old image before creating new + one. [GH-3918] +* builder/google: Add image license metadata. [GH-3873] +* builder/google: Added support for `image_family` [GH-3531] +* builder/google: Added support for startup scripts. [GH-3639] +* builder/google: Create passwords for Windows instances. [GH-3932] +* builder/google: Enable to select NVMe images. [GH-3338] +* builder/google: Signal that startup script fished via metadata. [GH-3873] +* builder/google: Use gcloud application default credentials. [GH-3655] +* builder/google: provision VM without external IP address. [GH-3774] +* builder/null: Can now be used with WinRM. [GH-2525] +* builder/openstack: Added support for `ssh_password` instead of generating + ssh keys. [GH-3976] +* builder/parallels: Add support for ctrl, shift and alt keys in + `boot_command`. [GH-3767] +* builder/parallels: Copy directories recursively with `floppy_dirs`. + [GH-2919] +* builder/parallels: Now pauses between `boot_command` entries when running + with `-debug` [GH-3547] +* builder/parallels: Support future versions of Parallels by using the latest + driver. [GH-3673] +* builder/qemu: Add support for ctrl, shift and alt keys in `boot_command`. + [GH-3767] +* builder/qemu: Added `vnc_bind_address` option. [GH-3574] +* builder/qemu: Copy directories recursively with `floppy_dirs`. [GH-2919] +* builder/qemu: Now pauses between `boot_command` entries when running with + `-debug` [GH-3547] +* builder/qemu: Specify disk format when starting qemu. [GH-3888] +* builder/virtualbox-iso: Added `hard_drive_nonrotational` and + `hard_drive_discard` options to enable trim/discard. [GH-4013] +* builder/virtualbox-iso: Added `keep_registered` option to skip cleaning up + the image. [GH-3954] +* builder/virtualbox: Add support for ctrl, shift and alt keys in + `boot_command`. [GH-3767] +* builder/virtualbox: Added `post_shutdown_delay` option to wait after + shutting down to prevent issues removing floppy drive. [GH-3952] +* builder/virtualbox: Added `vrdp_bind_address` option. [GH-3566] +* builder/virtualbox: Copy directories recursively with `floppy_dirs`. + [GH-2919] +* builder/virtualbox: Now pauses between `boot_command` entries when running + with `-debug` [GH-3542] +* builder/vmware-vmx: Added `tools_upload_flavor` and `tools_upload_path` to + docs. +* builder/vmware: Add support for ctrl, shift and alt keys in `boot_command`. + [GH-3767] +* builder/vmware: Added `vnc_bind_address` option. [GH-3565] +* builder/vmware: Adds passwords for VNC. [GH-2325] +* builder/vmware: Copy directories recursively with `floppy_dirs`. [GH-2919] +* builder/vmware: Handle connection to VM with more than one NIC on ESXi + [GH-3347] +* builder/vmware: Now paused between `boot_command` entries when running with + `-debug` [GH-3542] +* core: Supress plugin discovery from plugins. [GH-4002] +* core: Test floppy disk files actually exist. [GH-3756] +* core: setting `PACKER_LOG=0` now disables logging. [GH-3964] +* post-processor/amazon-import: Support `ami_name` for naming imported AMI. + [GH-3941] +* post-processor/compress: Added support for bgzf compression. [GH-3501] +* post-processor/docker: Improved support for Docker push to Amazon ECR. + [GH-3856] +* post-processor/docker: Preserve tags when running docker push. [GH-3631] +* post-processor/vagrant: Added vsphere-esx hosts to supported machine types. + [GH-3967] +* provisioner/ansible-local: Support for ansible-galaxy. [GH-3350] [GH-3836] +* provisioner/ansible: Improved logging and error handling. [GH-3477] +* provisioner/ansible: Support scp. [GH-3861] +* provisioner/chef: Added `knife_command` option and added a correct default + value for Windows. [GH-3622] +* provisioner/chef: Installs 64bit chef on Windows if available. [GH-3848] +* provisioner/file: Now makes destination directory. [GH-3692] +* provisioner/puppet: Added `execute_command` option. [GH-3614] +* provisioner/salt: Added `custom_state` to specify state to run instead of + `highstate`. [GH-3776] +* provisioner/shell: Added `expect_disconnect` flag to fail if remote + unexpectedly disconnects. [GH-4034] +* scripts: Added `help` target to Makefile. [GH-3290] +* vendor: Moving from Godep to govendor. See `CONTRIBUTING.md` for details. + [GH-3956] +* website: code examples now use inconsolata. Improve code font rendering on + linux. + +### BUG FIXES: + +* builder/amazon: Add 0.5 cents to discovered spot price. [GH-3662] +* builder/amazon: Allow using `ssh_private_key_file` and `ssh_password`. + [GH-3953] +* builder/amazon: Fix packer crash when waiting for SSH. [GH-3865] +* builder/amazon: Honor ssh_private_ip flag in EC2-Classic. [GH-3752] +* builder/amazon: Properly clean up EBS volumes on failure. [GH-3789] +* builder/amazon: Use `temporary_key_pair_name` when specified. [GH-3739] +* builder/amazon: retry creating tags on images since the images might take + some time to become available. [GH-3938] +* builder/azure: Fix authorization setup script failing to creating service + principal. [GH-3812] +* builder/azure: check for empty resource group. [GH-3606] +* builder/azure: fix token validity test. [GH-3609] +* builder/docker: Fix file provisioner dotfile matching. [GH-3800] +* builder/docker: fix docker builder with ansible provisioner. [GH-3476] +* builder/qemu: Don't fail on communicator set to `none`. [GH-3681] +* builder/qemu: Make `ssh_host_port_max` an inclusive bound. [GH-2784] +* builder/virtualbox: Make `ssh_host_port_max` an inclusive bound. [GH-2784] +* builder/virtualbox: Respect `ssh_host` [GH-3617] +* builder/vmware: Do not add remotedisplay.vnc.ip to VMX data on ESXi + [GH-3740] +* builder/vmware: Don't check for poweron errors on ESXi. [GH-3195] +* builder/vmware: Re-introduce case sensitive VMX keys. [GH-2707] +* builder/vmware: Respect `ssh_host`/`winrm_host` on ESXi. [GH-3738] +* command/push: Allows dot (`.`) in image names. [GH-3937] +* common/iso_config: fix potential panic when iso checksum url was given but + not the iso url. [GH-4004] +* communicator/ssh: fixed possible panic when reconnecting fails. [GH-4008] +* communicator/ssh: handle error case where server closes the connection but + doesn't give us an error code. [GH-3966] +* post-processor/shell-local: Do not set execute bit on artifact file. + [GH-3505] +* post-processor/vsphere: Fix upload failures with vsphere. [GH-3321] +* provisioner/ansible: Properly set host key checking even when a custom ENV + is specified. [GH-3568] +* provisioner/file: Fix directory download. [GH-3899] +* provisioner/powershell: fixed issue with setting environment variables. + [GH-2785] +* website: improved rendering on iPad. [GH-3780] + +## 0.10.2 (September 20, 2016) + +### BUG FIXES: + +* Rebuilding with OS X Sierra and go 1.7.1 to fix bug in Sierra + +## 0.10.1 (May 7, 2016) + +### FEATURES: + +* `azure-arm` builder: Can now build Windows images, and supports additional + configuration. Please refer to the documentation for details. + +### IMPROVEMENTS: + +* core: Added support for `ATLAS_CAFILE` and `ATLAS_CAPATH` [GH-3494] +* builder/azure: Improved build cancellation and cleanup of partially- + provisioned resources. [GH-3461] +* builder/azure: Improved logging. [GH-3461] +* builder/azure: Added support for US Government and China clouds. [GH-3461] +* builder/azure: Users may now specify an image version. [GH-3461] +* builder/azure: Added device login. [GH-3461] +* builder/docker: Added `privileged` build option. [GH-3475] +* builder/google: Packer now identifies its version to the service. [GH-3465] +* provisioner/shell: Added `remote_folder` and `remote_file` options + [GH-3462] +* post-processor/compress: Added support for `bgzf` format and added + `format` option. [GH-3501] + +### BUG FIXES: + +* core: Fix hang after pressing enter key in `-debug` mode. [GH-3346] +* provisioner/chef: Use custom values for remote validation key path + [GH-3468] + +## 0.10.0 (March 14, 2016) + +### BACKWARDS INCOMPATIBILITIES: + +* Building Packer now requires go >= 1.5 (>= 1.6 is recommended). If you want + to continue building with go 1.4 you can remove the `azurearmbuilder` line + from `command/plugin.go`. + +### FEATURES: + +* **New `azure-arm` builder**: Build virtual machines in Azure Resource + Manager + +### IMPROVEMENTS: + +* builder/google: Added support for `disk_type` [GH-2830] +* builder/openstack: Added support for retrieving the Administrator password + when using WinRM if no `winrm_password` is set. [GH-3209] +* provisioner/ansible: Added the `empty_groups` parameter. [GH-3232] +* provisioner/ansible: Added the `user` parameter. [GH-3276] +* provisioner/ansible: Don't use deprecated ssh option with Ansible 2.0 + [GH-3291] +* provisioner/puppet-masterless: Add `ignore_exit_codes` parameter. [GH-3349] + +### BUG FIXES: + +* builders/parallels: Handle `output_directory` containing `.` and `..` + [GH-3239] +* provisioner/ansible: os.Environ() should always be passed to the ansible + command. [GH-3274] + +## 0.9.0 (February 19, 2016) + +### BACKWARDS INCOMPATIBILITIES: + +* Packer now ships as a single binary, including plugins. If you install packer + 0.9.0 over a previous packer installation, **you must delete all of the + packer-* plugin files** or packer will load out-of-date plugins from disk. +* Release binaries are now provided via . +* Packer 0.9.0 is now built with Go 1.6. +* core: Plugins that implement the Communicator interface must now implement + a DownloadDir method. [GH-2618] +* builder/amazon: Inline `user_data` for EC2 is now base64 encoded + automatically. [GH-2539] +* builder/parallels: `parallels_tools_host_path` and `guest_os_distribution` + have been replaced by `guest_os_type`; use `packer fix` to update your + templates. [GH-2751] + +### FEATURES: + +* **Chef on Windows**: The chef provisioner now has native support for + Windows using Powershell and WinRM. [GH-1215] +* **New `vmware-esxi` feature**: Packer can now export images from vCloud or + vSphere during the build. [GH-1921] +* **New Ansible Provisioner**: `ansible` provisioner supports remote + provisioning to keep your build image cleaner. [GH-1969] +* **New Amazon Import post-processor**: `amazon-import` allows you to upload an + OVA-based VM to Amazon EC2. [GH-2962] +* **Shell Local post-processor**: `shell-local` allows you to run shell + commands on the host after a build has completed for custom packaging or + publishing of your artifacts. [GH-2706] +* **Artifice post-processor**: Override packer artifacts during post- + processing. This allows you to extract artifacts from a packer builder and + use them with other post-processors like compress, docker, and Atlas. + +### IMPROVEMENTS: + +* core: Packer plugins are now compiled into the main binary, reducing file + size and build times, and making packer easier to install. The overall + plugin architecture has not changed and third-party plugins can still be + loaded from disk. Please make sure your plugins are up-to-date! [GH-2854] +* core: Packer now indicates line numbers for template parse errors. [GH-2742] +* core: Scripts are executed via `/usr/bin/env bash` instead of `/bin/bash` + for broader compatibility. [GH-2913] +* core: `target_path` for builder downloads can now be specified. [GH-2600] +* core: WinRM communicator now supports HTTPS protocol. [GH-3061] +* core: Template syntax errors now show line, column, offset. [GH-3180] +* core: SSH communicator now supports downloading directories. [GH-2618] +* builder/amazon: Add support for `ebs_optimized` [GH-2806] +* builder/amazon: You can now specify `0` for `spot_price` to switch to on + demand instances. [GH-2845] +* builder/amazon: Added `ap-northeast-2` (Seoul) [GH-3056] +* builder/amazon: packer will try to derive the AZ if only a subnet is + specified. [GH-3037] +* builder/digitalocean: doubled instance wait timeouts to power off or + shutdown (now 4 minutes) and to complete a snapshot (now 20 minutes) + [GH-2939] +* builder/google: `account_file` can now be provided as a JSON string + [GH-2811] +* builder/google: added support for `preemptible` instances. [GH-2982] +* builder/google: added support for static external IPs via `address` option + [GH-3030] +* builder/openstack: added retry on WaitForImage 404. [GH-3009] +* builder/openstack: Can specify `source_image_name` instead of the ID + [GH-2577] +* builder/openstack: added support for SSH over IPv6. [GH-3197] +* builder/parallels: Improve support for Parallels 11. [GH-2662] +* builder/parallels: Parallels disks are now compacted by default. [GH-2731] +* builder/parallels: Packer will look for Parallels in + `/Applications/Parallels Desktop.app` if it is not detected automatically + [GH-2839] +* builder/qemu: qcow2 images are now compacted by default. [GH-2748] +* builder/qemu: qcow2 images can now be compressed. [GH-2748] +* builder/qemu: Now specifies `virtio-scsi` by default. [GH-2422] +* builder/qemu: Now checks for version-specific options. [GH-2376] +* builder/qemu: Can now bypass disk cache using `iso_skip_cache` [GH-3105] +* builder/qemu: `` in `boot_command` now accepts an arbitrary duration + like [GH-3129] +* builder/qemu: Expose `{{ .SSHHostPort }}` in templates. [GH-2884] +* builder/virtualbox: Added VRDP for debugging. [GH-3188] +* builder/vmware-esxi: Added private key auth for remote builds via + `remote_private_key_file` [GH-2912] +* post-processor/atlas: Added support for compile ID. [GH-2775] +* post-processor/docker-import: Can now import Artifice artifacts. [GH-2718] +* provisioner/chef: Added `encrypted_data_bag_secret_path` option. [GH-2653] +* provisioner/puppet: Added the `extra_arguments` parameter. [GH-2635] +* provisioner/salt: Added `no_exit_on_failure`, `log_level`, and improvements + to salt command invocation. [GH-2660] + +### BUG FIXES: + +* core: Random number generator is now seeded. [GH-2640] +* core: Packer should now have a lot less race conditions. [GH-2824] +* builder/amazon: The `no_device` option for block device mappings is now handled correctly. [GH-2398] +* builder/amazon: AMI name validation now matches Amazon's spec. [GH-2774] +* builder/amazon: Use snapshot size when volume size is unspecified. [GH-2480] +* builder/amazon: Pass AccessKey and SecretKey when uploading bundles for + instance-backed AMIs. [GH-2596] +* builder/parallels: Added interpolation in `prlctl_post` [GH-2828] +* builder/vmware: `format` option is now read correctly. [GH-2892] +* builder/vmware-esxi: Correct endless loop in destroy validation logic + [GH-2911] +* provisioner/shell: No longer leaves temp scripts behind. [GH-1536] +* provisioner/winrm: Now waits for reboot to complete before continuing with provisioning. [GH-2568] +* post-processor/artifice: Fix truncation of files downloaded from Docker. [GH-2793] + + +## 0.8.6 (Aug 22, 2015) + +### IMPROVEMENTS: + +* builder/docker: Now supports Download so it can be used with the file + provisioner to download a file from a container. [GH-2585] +* builder/docker: Now verifies that the artifact will be used before the build + starts, unless the `discard` option is specified. This prevent failures + after the build completes. [GH-2626] +* post-processor/artifice: Now supports glob-like syntax for filenames. [GH-2619] +* post-processor/vagrant: Like the compress post-processor, vagrant now uses a + parallel gzip algorithm to compress vagrant boxes. [GH-2590] + +### BUG FIXES: + +* core: When `iso_url` is a local file and the checksum is invalid, the local + file will no longer be deleted. [GH-2603] +* builder/parallels: Fix interpolation in `parallels_tools_guest_path` [GH-2543] + +## 0.8.5 (Aug 10, 2015) + +### FEATURES: + +* **[Beta]** Artifice post-processor: Override packer artifacts during post- + processing. This allows you to extract artifacts from a packer builder + and use them with other post-processors like compress, docker, and Atlas. + +### IMPROVEMENTS: + +* Many docs have been updated and corrected; big thanks to our contributors! +* builder/openstack: Add debug logging for IP addresses used for SSH. [GH-2513] +* builder/openstack: Add option to use existing SSH keypair. [GH-2512] +* builder/openstack: Add support for Glance metadata. [GH-2434] +* builder/qemu and builder/vmware: Packer's VNC connection no longer asks for + an exclusive connection. [GH-2522] +* provisioner/salt-masterless: Can now customize salt remote directories. [GH-2519] + +### BUG FIXES: + +* builder/amazon: Improve instance cleanup by storing id sooner. [GH-2404] +* builder/amazon: Only fetch windows password when using WinRM communicator. [GH-2538] +* builder/openstack: Support IPv6 SSH address. [GH-2450] +* builder/openstack: Track new IP address discovered during RackConnect. [GH-2514] +* builder/qemu: Add 100ms delay between VNC key events. [GH-2415] +* post-processor/atlas: atlas_url configuration option works now. [GH-2478] +* post-processor/compress: Now supports interpolation in output config. [GH-2414] +* provisioner/powershell: Elevated runs now receive environment variables. [GH-2378] +* provisioner/salt-masterless: Clarify error messages when we can't create or + write to the temp directory. [GH-2518] +* provisioner/salt-masterless: Copy state even if /srv/salt exists already. [GH-1699] +* provisioner/salt-masterless: Make sure /etc/salt exists before writing to it. [GH-2520] +* provisioner/winrm: Connect to the correct port when using NAT with + VirtualBox / VMware. [GH-2399] + +## Note: 0.8.3 was pulled and 0.8.4 was skipped. + +## 0.8.2 (July 17, 2015) + +### IMPROVEMENTS: + +* builder/docker: Add option to use a Pty. [GH-2425] + +### BUG FIXES: + +* core: Fix crash when `min_packer_version` is specified in a template. [GH-2385] +* builder/amazon: Fix EC2 devices being included in EBS mappings. [GH-2459] +* builder/googlecompute: Fix default name for GCE images. [GH-2400] +* builder/null: Fix error message with missing ssh_host. [GH-2407] +* builder/virtualbox: Use --portcount on VirtualBox 5.x. [GH-2438] +* provisioner/puppet: Packer now correctly handles a directory for manifest_file. [GH-2463] +* provisioner/winrm: Fix potential crash with WinRM. [GH-2416] + +## 0.8.1 (July 2, 2015) + +### IMPROVEMENTS: + +* builder/amazon: When debug mode is enabled, the Windows administrator + password for Windows instances will be shown. [GH-2351] + +### BUG FIXES: + +* core: `min_packer_version` field in configs work. [GH-2356] +* core: The `build_name` and `build_type` functions work in provisioners. [GH-2367] +* core: Handle timeout in SSH handshake. [GH-2333] +* command/build: Fix reading configuration from stdin. [GH-2366] +* builder/amazon: Fix issue with sharing AMIs when using `ami_users` [GH-2308] +* builder/amazon: Fix issue when using multiple Security Groups. [GH-2381] +* builder/amazon: Fix for tag creation when creating new ec2 instance. [GH-2317] +* builder/amazon: Fix issue with creating AMIs with multiple device mappings. [GH-2320] +* builder/amazon: Fix failing AMI snapshot tagging when copying to other + regions. [GH-2316] +* builder/amazon: Fix setting AMI launch permissions. [GH-2348] +* builder/amazon: Fix spot instance cleanup to remove the correct request. [GH-2327] +* builder/amazon: Fix `bundle_prefix` not interpolating `timestamp` [GH-2352] +* builder/amazon-instance: Fix issue with creating AMIs without specifying a + virtualization type. [GH-2330] +* builder/digitalocean: Fix builder using private IP instead of public IP. [GH-2339] +* builder/google: Set default communicator settings properly. [GH-2353] +* builder/vmware-iso: Setting `checksum_type` to `none` for ESX builds + now works. [GH-2323] +* provisioner/chef: Use knife config file vs command-line params to + clean up nodes so full set of features can be used. [GH-2306] +* post-processor/compress: Fixed crash in compress post-processor plugin. [GH-2311] + +## 0.8.0 (June 23, 2015) + +### BACKWARDS INCOMPATIBILITIES: + +* core: SSH connection will no longer request a PTY by default. This + can be enabled per builder. +* builder/digitalocean: no longer supports the v1 API which has been + deprecated for some time. Most configurations should continue to + work as long as you use the `api_token` field for auth. +* builder/digitalocean: `image`, `region`, and `size` are now required. +* builder/openstack: auth parameters have been changed to better + reflect OS terminology. Existing environment variables still work. + +### FEATURES: + +* **WinRM:** You can now connect via WinRM with almost every builder. + See the docs for more info. [GH-2239] +* **Windows AWS Support:** Windows AMIs can now be built without any + external plugins: Packer will start a Windows instance, get the + admin password, and can use WinRM (above) to connect through. [GH-2240] +* **Disable SSH:** Set `communicator` to "none" in any builder to disable SSH + connections. Note that provisioners won't work if this is done. [GH-1591] +* **SSH Agent Forwarding:** SSH Agent Forwarding will now be enabled + to allow access to remote servers such as private git repos. [GH-1066] +* **SSH Bastion Hosts:** You can now specify a bastion host for + SSH access (works with all builders). [GH-387] +* **OpenStack v3 Identity:** The OpenStack builder now supports the + v3 identity API. +* **Docker builder supports SSH**: The Docker builder now supports containers + with SSH, just set `communicator` to "ssh" [GH-2244] +* **File provisioner can download**: The file provisioner can now download + files out of the build process. [GH-1909] +* **New config function: `build_name`**: The name of the currently running + build. [GH-2232] +* **New config function: `build_type`**: The type of the currently running + builder. This is useful for provisioners. [GH-2232] +* **New config function: `template_dir`**: The directory to the template + being built. This should be used for template-relative paths. [GH-54] +* **New provisioner: shell-local**: Runs a local shell script. [GH-770] +* **New provisioner: powershell**: Provision Windows machines + with PowerShell scripts. [GH-2243] +* **New provisioner: windows-shell**: Provision Windows machines with + batch files. [GH-2243] +* **New provisioner: windows-restart**: Restart a Windows machines and + wait for it to come back online. [GH-2243] +* **Compress post-processor supports multiple algorithms:** The compress + post-processor now supports lz4 compression and compresses gzip in + parallel for much faster throughput. + +### IMPROVEMENTS: + +* core: Interrupt handling for SIGTERM signal as well. [GH-1858] +* core: HTTP downloads support resuming. [GH-2106] + * builder/*: Add `ssh_handshake_attempts` to configure the number of + handshake attempts done before failure. [GH-2237] +* builder/amazon: Add `force_deregister` option for automatic AMI + deregistration. [GH-2221] +* builder/amazon: Now applies tags to EBS snapshots. [GH-2212] +* builder/amazon: Clean up orphaned volumes from Source AMIs. [GH-1783] +* builder/amazon: Support custom keypairs. [GH-1837] +* builder/amazon-chroot: Can now resize the root volume of the resulting + AMI with the `root_volume_size` option. [GH-2289] +* builder/amazon-chroot: Add `mount_options` configuration option for providing + options to the `mount` command. [GH-2296] +* builder/digitalocean: Save SSH key to pwd if debug mode is on. [GH-1829] +* builder/digitalocean: User data support. [GH-2113] +* builder/googlecompute: Option to use internal IP for connections. [GH-2152] +* builder/parallels: Support Parallels Desktop 11. [GH-2199] +* builder/openstack: Add `rackconnect_wait` for Rackspace customers to wait for + RackConnect data to appear +* builder/openstack: Add `ssh_interface` option for rackconnect for users that + have prohibitive firewalls +* builder/openstack: Flavor names can be used as well as refs +* builder/openstack: Add `availability_zone` [GH-2016] +* builder/openstack: Machine will be stopped prior to imaging if the + cluster supports the `startstop` extension. [GH-2223] +* builder/openstack: Support for user data. [GH-2224] +* builder/qemu: Default accelerator to "tcg" on Windows. [GH-2291] +* builder/virtualbox: Added option: `ssh_skip_nat_mapping` to skip the + automatic port forward for SSH and to use the guest port directly. [GH-1078] +* builder/virtualbox: Added SCSI support +* builder/vmware: Support for additional disks. [GH-1382] +* builder/vmware: Can now customize the template used for adding disks. [GH-2254] +* command/fix: After fixing, the template is validated. [GH-2228] +* command/push: Add `-name` flag for specifying name from CLI. [GH-2042] +* command/push: Push configuration in templates supports variables. [GH-1861] +* post-processor/docker-save: Can be chained. [GH-2179] +* post-processor/docker-tag: Support `force` option. [GH-2055] +* post-processor/docker-tag: Can be chained. [GH-2179] +* post-processor/vsphere: Make more fields optional, support empty + resource pools. [GH-1868] +* provisioner/puppet-masterless: `working_directory` option. [GH-1831] +* provisioner/puppet-masterless: `packer_build_name` and + `packer_build_type` are default facts. [GH-1878] +* provisioner/puppet-server: `ignore_exit_codes` option added. [GH-2280] + +### BUG FIXES: + +* core: Fix potential panic for post-processor plugin exits. [GH-2098] +* core: `PACKER_CONFIG` may point to a non-existent file. [GH-2226] +* builder/amazon: Allow spaces in AMI names when using `clean_ami_name` [GH-2182] +* builder/amazon: Remove deprecated ec2-upload-bundle parameter. [GH-1931] +* builder/amazon: Use IAM Profile to upload bundle if provided. [GH-1985] +* builder/amazon: Use correct exit code after SSH authentication failed. [GH-2004] +* builder/amazon: Retry finding created instance for eventual + consistency. [GH-2129] +* builder/amazon: If no AZ is specified, use AZ chosen automatically by + AWS for spot instance. [GH-2017] +* builder/amazon: Private key file (only available in debug mode) + is deleted on cleanup. [GH-1801] +* builder/amazon: AMI copy won't copy to the source region. [GH-2123] +* builder/amazon: Validate AMI doesn't exist with name prior to build. [GH-1774] +* builder/amazon: Improved retry logic around waiting for instances. [GH-1764] +* builder/amazon: Fix issues with creating Block Devices. [GH-2195] +* builder/amazon/chroot: Retry waiting for disk attachments. [GH-2046] +* builder/amazon/chroot: Only unmount path if it is mounted. [GH-2054] +* builder/amazon/instance: Use `-i` in sudo commands so PATH is inherited. [GH-1930] +* builder/amazon/instance: Use `--region` flag for bundle upload command. [GH-1931] +* builder/digitalocean: Wait for droplet to unlock before changing state, + should lower the "pending event" errors. +* builder/digitalocean: Ignore invalid fields from the ever-changing v2 API +* builder/digitalocean: Private images can be used as a source. [GH-1792] +* builder/docker: Fixed hang on prompt while copying script +* builder/docker: Use `docker exec` for newer versions of Docker for + running scripts. [GH-1993] +* builder/docker: Fix crash that could occur at certain timed ctrl-c. [GH-1838] +* builder/docker: validate that `export_path` is not a directory. [GH-2105] +* builder/google: `ssh_timeout` is respected. [GH-1781] +* builder/openstack: `ssh_interface` can be used to specify the interface + to retrieve the SSH IP from. [GH-2220] +* builder/qemu: Add `disk_discard` option. [GH-2120] +* builder/qemu: Use proper SSH port, not hardcoded to 22. [GH-2236] +* builder/qemu: Find unused SSH port if SSH port is taken. [GH-2032] +* builder/virtualbox: Bind HTTP server to IPv4, which is more compatible with + OS installers. [GH-1709] +* builder/virtualbox: Remove the floppy controller in addition to the + floppy disk. [GH-1879] +* builder/virtualbox: Fixed regression where downloading ISO without a + ".iso" extension didn't work. [GH-1839] +* builder/virtualbox: Output dir is verified at runtime, not template + validation time. [GH-2233] +* builder/virtualbox: Find unused SSH port if SSH port is taken. [GH-2032] +* builder/vmware: Add 100ms delay between keystrokes to avoid subtle + timing issues in most cases. [GH-1663] +* builder/vmware: Bind HTTP server to IPv4, which is more compatible with + OS installers. [GH-1709] +* builder/vmware: Case-insensitive match of MAC address to find IP. [GH-1989] +* builder/vmware: More robust IP parsing from ifconfig output. [GH-1999] +* builder/vmware: Nested output directories for ESXi work. [GH-2174] +* builder/vmware: Output dir is verified at runtime, not template + validation time. [GH-2233] +* command/fix: For the `virtualbox` to `virtualbox-iso` builder rename, + provisioner overrides are now also fixed. [GH-2231] +* command/validate: don't crash for invalid builds. [GH-2139] +* post-processor/atlas: Find common archive prefix for Windows. [GH-1874] +* post-processor/atlas: Fix index out of range panic. [GH-1959] +* post-processor/vagrant-cloud: Fixed failing on response +* post-processor/vagrant-cloud: Don't delete version on error. [GH-2014] +* post-processor/vagrant-cloud: Retry failed uploads a few times +* provisioner/chef-client: Fix permissions issues on default dir. [GH-2255] +* provisioner/chef-client: Node cleanup works now. [GH-2257] +* provisioner/puppet-masterless: Allow manifest_file to be a directory +* provisioner/salt-masterless: Add `--retcode-passthrough` to salt-call +* provisioner/shell: chmod executable script to 0755, not 0777. [GH-1708] +* provisioner/shell: inline commands failing will fail the provisioner. [GH-2069] +* provisioner/shell: single quotes in env vars are escaped. [GH-2229] +* provisioner/shell: Temporary file is deleted after run. [GH-2259] +* provisioner/shell: Randomize default script name to avoid strange + race issues from Windows. [GH-2270] + +## 0.7.5 (December 9, 2014) + +### FEATURES: + +* **New command: `packer push`**: Push template and files to HashiCorp's + Atlas for building your templates automatically. +* **New post-processor: `atlas`**: Send artifact to HashiCorp's Atlas for + versioning and storing artifacts. These artifacts can then be queried + using the API, Terraform, etc. + +### IMPROVEMENTS: + +* builder/googlecompute: Support for ubuntu-os-cloud project +* builder/googlecompute: Support for OAuth2 to avoid client secrets file +* builder/googlecompute: GCE image from persistent disk instead of tarball +* builder/qemu: Checksum type "none" can be used +* provisioner/chef: Generate a node name if none available +* provisioner/chef: Added ssl_verify_mode configuration + +### BUG FIXES: + +* builder/parallels: Fixed attachment of ISO to cdrom device +* builder/parallels: Fixed boot load ordering +* builder/digitalocean: Fixed decoding of size +* builder/digitalocean: Fixed missing content-type header in request +* builder/digitalocean: Fixed use of private IP +* builder/digitalocean: Fixed the artifact ID generation +* builder/vsphere: Fixed credential escaping +* builder/qemu: Fixed use of CDROM with disk_image +* builder/aws: Fixed IP address for SSH in VPC +* builder/aws: Fixed issue with multiple block devices +* builder/vmware: Upload VMX to ESX5 after editing +* communicator/docker: Fix handling of symlinks during upload +* provisioner/chef: Fixed use of sudo in some cases +* core: Fixed build name interpolation +* postprocessor/vagrant: Fixed check for Vagrantfile template + +## 0.7.2 (October 28, 2014) + +### FEATURES: + +* builder/digitalocean: API V2 support. [GH-1463] +* builder/parallels: Don't depend on _prl-utils_. [GH-1499] + +### IMPROVEMENTS: + +* builder/amazon/all: Support new AWS Frankfurt region. +* builder/docker: Allow remote `DOCKER_HOST`, which works as long as + volumes work. [GH-1594] +* builder/qemu: Can set cache mode for main disk. [GH-1558] +* builder/qemu: Can build from pre-existing disk. [GH-1342] +* builder/vmware: Can specify path to Fusion installation with environmental + variable `FUSION_APP_PATH`. [GH-1552] +* builder/vmware: Can specify the HW version for the VMX. [GH-1530] +* builder/vmware/esxi: Will now cache ISOs/floppies remotely. [GH-1479] +* builder/vmware/vmx: Source VMX can have a disk connected via SATA. [GH-1604] +* post-processors/vagrant: Support Qemu (libvirt) boxes. [GH-1330] +* post-processors/vagrantcloud: Support self-hosted box URLs. + +### BUG FIXES: + +* core: Fix loading plugins from pwd. [GH-1521] +* builder/amazon: Prefer token in config if given. [GH-1544] +* builder/amazon/all: Extended timeout for waiting for AMI. [GH-1533] +* builder/virtualbox: Can read VirtualBox version on FreeBSD. [GH-1570] +* builder/virtualbox: More robust reading of guest additions URL. [GH-1509] +* builder/vmware: Always remove floppies/drives. [GH-1504] +* builder/vmware: Wait some time so that post-VMX update aren't + overwritten. [GH-1504] +* builder/vmware/esxi: Retry power on if it fails. [GH-1334] +* builder/vmware-vmx: Fix issue with order of boot command support. [GH-1492] +* builder/amazon: Extend timeout and allow user override. [GH-1533] +* builder/parallels: Ignore 'The fdd0 device does not exist' [GH-1501] +* builder/parallels: Rely on Cleanup functions to detach devices. [GH-1502] +* builder/parallels: Create VM without hdd and then add it later. [GH-1548] +* builder/parallels: Disconnect cdrom0. [GH-1605] +* builder/qemu: Don't use `-redir` flag anymore, replace with + `hostfwd` options. [GH-1561] +* builder/qemu: Use `pc` as default machine type instead of `pc-1.0`. +* providers/aws: Ignore transient network errors. [GH-1579] +* provisioner/ansible: Don't buffer output so output streams in. [GH-1585] +* provisioner/ansible: Use inventory file always to avoid potentially + deprecated feature. [GH-1562] +* provisioner/shell: Quote environmental variables. [GH-1568] +* provisioner/salt: Bootstrap over SSL. [GH-1608] +* post-processors/docker-push: Work with docker-tag artifacts. [GH-1526] +* post-processors/vsphere: Append "/" to object address. [GH-1615] + +## 0.7.1 (September 10, 2014) + +### FEATURES: + +* builder/vmware: VMware Fusion Pro 7 is now supported. [GH-1478] + +### BUG FIXES: + +* core: SSH will connect slightly faster if it is ready immediately. +* provisioner/file: directory uploads no longer hang. [GH-1484] +* provisioner/file: fixed crash on large files. [GH-1473] +* scripts: Windows executable renamed to packer.exe. [GH-1483] + +## 0.7.0 (September 8, 2014) + +### BACKWARDS INCOMPATIBILITIES: + +* The authentication configuration for Google Compute Engine has changed. + The new method is much simpler, but is not backwards compatible. + `packer fix` will _not_ fix this. Please read the updated GCE docs. + +### FEATURES: + +* **New Post-Processor: `compress`** - Gzip compresses artifacts with files. +* **New Post-Processor: `docker-save`** - Save an image. This is similar to + export, but preserves the image hierarchy. +* **New Post-Processor: `docker-tag`** - Tag a created image. +* **New Template Functions: `upper`, `lower`** - See documentation for + more details. +* core: Plugins are automatically discovered if they're named properly. + Packer will look in the PWD and the directory with `packer` for + binaries named `packer-TYPE-NAME`. +* core: Plugins placed in `~/.packer.d/plugins` are now automatically + discovered. +* builder/amazon: Spot instances can now be used to build EBS backed and + instance store images. [GH-1139] +* builder/docker: Images can now be committed instead of exported. [GH-1198] +* builder/virtualbox-ovf: New `import_flags` setting can be used to add + new command line flags to `VBoxManage import` to allow things such + as EULAs to be accepted. [GH-1383] +* builder/virtualbox-ovf: Boot commands and the HTTP server are supported. + [GH-1169] +* builder/vmware: VMware Player 6 is now supported. [GH-1168] +* builder/vmware-vmx: Boot commands and the HTTP server are supported. + [GH-1169] + +### IMPROVEMENTS: + +* core: `isotime` function can take a format. [GH-1126] +* builder/amazon/all: `AWS_SECURITY_TOKEN` is read and can also be + set with the `token` configuration. [GH-1236] +* builder/amazon/all: Can force SSH on the private IP address with + `ssh_private_ip`. [GH-1229] +* builder/amazon/all: String fields in device mappings can use variables. [GH-1090] +* builder/amazon-instance: EBS AMIs can be used as a source. [GH-1453] +* builder/digitalocean: Can set API URL endpoint. [GH-1448] +* builder/digitalocean: Region supports variables. [GH-1452] +* builder/docker: Can now specify login credentials to pull images. +* builder/docker: Support mounting additional volumes. [GH-1430] +* builder/parallels/all: Path to tools ISO is calculated automatically. [GH-1455] +* builder/parallels-pvm: `reassign_mac` option to choose whether or not + to generate a new MAC address. [GH-1461] +* builder/qemu: Can specify "none" acceleration type. [GH-1395] +* builder/qemu: Can specify "tcg" acceleration type. [GH-1395] +* builder/virtualbox/all: `iso_interface` option to mount ISO with SATA. [GH-1200] +* builder/vmware-vmx: Proper `floppy_files` support. [GH-1057] +* command/build: Add `-color=false` flag to disable color. [GH-1433] +* post-processor/docker-push: Can now specify login credentials. [GH-1243] +* provisioner/chef-client: Support `chef_environment`. [GH-1190] + +### BUG FIXES: + +* core: nicer error message if an encrypted private key is used for + SSH. [GH-1445] +* core: Fix crash that could happen with a well timed double Ctrl-C. + [GH-1328] [GH-1314] +* core: SSH TCP keepalive period is now 5 seconds (shorter). [GH-1232] +* builder/amazon-chroot: Can properly build HVM images now. [GH-1360] +* builder/amazon-chroot: Fix crash in root device check. [GH-1360] +* builder/amazon-chroot: Add description that Packer made the snapshot + with a time. [GH-1388] +* builder/amazon-ebs: AMI is deregistered if an error. [GH-1186] +* builder/amazon-instance: Fix deprecation warning for `ec2-bundle-vol` + [GH-1424] +* builder/amazon-instance: Add `--no-filter` to the `ec2-bundle-vol` + command by default to avoid corrupting data by removing package + manager certs. [GH-1137] +* builder/amazon/all: `delete_on_termination` set to false will work. +* builder/amazon/all: Fix race condition on setting tags. [GH-1367] +* builder/amazon/all: More descriptive error messages if Amazon only + sends an error code. [GH-1189] +* builder/docker: Error if `DOCKER_HOST` is set. +* builder/docker: Remove the container during cleanup. [GH-1206] +* builder/docker: Fix case where not all output would show up from + provisioners. +* builder/googlecompute: add `disk_size` option. [GH-1397] +* builder/googlecompute: Auth works with latest formats on Google Cloud + Console. [GH-1344] +* builder/openstack: Region is not required. [GH-1418] +* builder/parallels-iso: ISO not removed from VM after install. [GH-1338] +* builder/parallels/all: Add support for Parallels Desktop 10. [GH-1438] +* builder/parallels/all: Added some navigation keys. [GH-1442] +* builder/qemu: If headless, sdl display won't be used. [GH-1395] +* builder/qemu: Use `512M` as `-m` default. [GH-1444] +* builder/virtualbox/all: Search `VBOX_MSI_INSTALL_PATH` for path to + `VBoxManage` on Windows. [GH-1337] +* builder/virtualbox/all: Seed RNG to avoid same ports. [GH-1386] +* builder/virtualbox/all: Better error if guest additions URL couldn't be + detected. [GH-1439] +* builder/virtualbox/all: Detect errors even when `VBoxManage` exits + with a zero exit code. [GH-1119] +* builder/virtualbox/iso: Append timestamp to default name for parallel + builds. [GH-1365] +* builder/vmware/all: No more error when Packer stops an already-stopped + VM. [GH-1300] +* builder/vmware/all: `ssh_host` accepts templates. [GH-1396] +* builder/vmware/all: Don't remount floppy in VMX post step. [GH-1239] +* builder/vmware/vmx: Do not re-add floppy disk files to VMX. [GH-1361] +* builder/vmware-iso: Fix crash when `vnc_port_min` and max were the + same value. [GH-1288] +* builder/vmware-iso: Finding an available VNC port on Windows works. [GH-1372] +* builder/vmware-vmx: Nice error if Clone is not supported (not VMware + Fusion Pro). [GH-787] +* post-processor/vagrant: Can supply your own metadata.json. [GH-1143] +* provisioner/ansible-local: Use proper path on Windows. [GH-1375] +* provisioner/file: Mode will now be preserved. [GH-1064] + +## 0.6.1 (July 20, 2014) + +### FEATURES: + +* **New post processor:** `vagrant-cloud` - Push box files generated by + vagrant post processor to Vagrant Cloud. [GH-1289] +* Vagrant post-processor can now packer Hyper-V boxes. + +### IMPROVEMENTS: + +* builder/amazon: Support for enhanced networking on HVM images. [GH-1228] +* builder/amazon-ebs: Support encrypted EBS volumes. [GH-1194] +* builder/ansible: Add `playbook_dir` option. [GH-1000] +* builder/openstack: Add ability to configure networks. [GH-1261] +* builder/openstack: Skip certificate verification. [GH-1121] +* builder/parallels/all: Add ability to select interface to connect to. +* builder/parallels/pvm: Support `boot_command`. [GH-1082] +* builder/virtualbox/all: Attempt to use local guest additions ISO + before downloading from internet. [GH-1123] +* builder/virtualbox/ovf: Supports `guest_additions_mode` [GH-1035] +* builder/vmware/all: Increase cleanup timeout to 120 seconds. [GH-1167] +* builder/vmware/all: Add `vmx_data_post` for modifying VMX data + after shutdown. [GH-1149] +* builder/vmware/vmx: Supports tools uploading. [GH-1154] + +### BUG FIXES: + +* core: `isotime` is the same time during the entire build. [GH-1153] +* builder/amazon-common: Sort AMI strings before outputting. [GH-1305] +* builder/amazon: User data can use templates/variables. [GH-1343] +* builder/amazon: Can now build AMIs in GovCloud. +* builder/null: SSH info can use templates/variables. [GH-1343] +* builder/openstack: Workaround for gophercloud.ServerById crashing. [GH-1257] +* builder/openstack: Force IPv4 addresses from address pools. [GH-1258] +* builder/parallels: Do not delete entire CDROM device. [GH-1115] +* builder/parallels: Errors while creating floppy disk. [GH-1225] +* builder/parallels: Errors while removing floppy drive. [GH-1226] +* builder/virtualbox-ovf: Supports guest additions options. [GH-1120] +* builder/vmware-iso: Fix esx5 path separator in windows. [GH-1316] +* builder/vmware: Remote ESXi builder now uploads floppy. [GH-1106] +* builder/vmware: Remote ESXi builder no longer re-uploads ISO every + time. [GH-1244] +* post-processor/vsphere: Accept DOMAIN\account usernames. [GH-1178] + * provisioner/chef-*: Fix remotePaths for Windows. [GH-394] + +## 0.6.0 (May 2, 2014) + +### FEATURES: + +* **New builder:** `null` - The null builder does not produce any + artifacts, but is useful for debugging provisioning scripts. [GH-970] +* **New builder:** `parallels-iso` and `parallels-pvm` - These can be + used to build Parallels virtual machines. [GH-1101] +* **New provisioner:** `chef-client` - Provision using a the `chef-client` + command, which talks to a Chef Server. [GH-855] +* **New provisioner:** `puppet-server` - Provision using Puppet by + communicating to a Puppet master. [GH-796] +* `min_packer_version` can be specified in a Packer template to force + a minimum version. [GH-487] + +### IMPROVEMENTS: + +* core: RPC transport between plugins switched to MessagePack +* core: Templates array values can now be comma separated strings. + Most importantly, this allows for user variables to fill + array configurations. [GH-950] +* builder/amazon: Added `ssh_private_key_file` option. [GH-971] +* builder/amazon: Added `ami_virtualization_type` option. [GH-1021] +* builder/digitalocean: Regions, image names, and sizes can be + names that are looked up for their valid ID. [GH-960] +* builder/googlecompute: Configurable instance name. [GH-1065] +* builder/openstack: Support for conventional OpenStack environmental + variables such as `OS_USERNAME`, `OS_PASSWORD`, etc. [GH-768] +* builder/openstack: Support `openstack_provider` option to automatically + fill defaults for different OpenStack variants. [GH-912] +* builder/openstack: Support security groups. [GH-848] +* builder/qemu: User variable expansion in `ssh_key_path` [GH-918] +* builder/qemu: Floppy disk files list can also include globs + and directories. [GH-1086] +* builder/virtualbox: Support an `export_opts` option which allows + specifying arbitrary arguments when exporting the VM. [GH-945] +* builder/virtualbox: Added `vboxmanage_post` option to run vboxmanage + commands just before exporting. [GH-664] +* builder/virtualbox: Floppy disk files list can also include globs + and directories. [GH-1086] +* builder/vmware: Workstation 10 support for Linux. [GH-900] +* builder/vmware: add cloning support on Windows. [GH-824] +* builder/vmware: Floppy disk files list can also include globs + and directories. [GH-1086] +* command/build: Added `-parallel` flag so you can disable parallelization + with `-no-parallel`. [GH-924] +* post-processors/vsphere: `disk_mode` option. [GH-778] +* provisioner/ansible: Add `inventory_file` option. [GH-1006] +* provisioner/chef-client: Add `validation_client_name` option. [GH-1056] + +### BUG FIXES: + +* core: Errors are properly shown when adding bad floppy files. [GH-1043] +* core: Fix some URL parsing issues on Windows. +* core: Create Cache directory only when it is needed. [GH-367] +* builder/amazon-instance: Use S3Endpoint for ec2-upload-bundle arg, + which works for every region. [GH-904] +* builder/digitalocean: updated default image_id. [GH-1032] +* builder/googlecompute: Create persistent disk as boot disk via + API v1. [GH-1001] +* builder/openstack: Return proper error on invalid instance states. [GH-1018] +* builder/virtualbox-iso: Retry unregister a few times to deal with + VBoxManage randomness. [GH-915] +* provisioner/ansible: Fix paths when provisioning Linux from + Windows. [GH-963] +* provisioner/ansible: set cwd to staging directory. [GH-1016] +* provisioners/chef-client: Don't chown directory with Ubuntu. [GH-939] +* provisioners/chef-solo: Deeply nested JSON works properly. [GH-1076] +* provisioners/shell: Env var values can have equal signs. [GH-1045] +* provisioners/shell: chmod the uploaded script file to 0777. [GH-994] +* post-processor/docker-push: Allow repositories with ports. [GH-923] +* post-processor/vagrant: Create parent directories for `output` path. [GH-1059] +* post-processor/vsphere: datastore, network, and folder are no longer + required. [GH-1091] + +## 0.5.2 (02/21/2014) + +### FEATURES: + +* **New post-processor:** `docker-import` - Import a Docker image and give it + a specific repository/tag. +* **New post-processor:** `docker-push` - Push an imported image to + a registry. + +### IMPROVEMENTS: + +* core: Most downloads made by Packer now use a custom user agent. [GH-803] +* builder/googlecompute: SSH private key will be saved to disk if `-debug` is + specified. [GH-867] +* builder/qemu: Can specify the name of the qemu binary. [GH-854] +* builder/virtualbox-ovf: Can specify import options such as "keepallmacs". + [GH-883] + +### BUG FIXES: + +* core: Fix crash case if blank parameters are given to Packer. [GH-832] +* core: Fix crash if big file uploads are done. [GH-897] +* core: Fix crash if machine-readable output is going to a closed + pipe. [GH-875] +* builder/docker: user variables work properly. [GH-777] +* builder/qemu: reboots are now possible in provisioners. [GH-864] +* builder/virtualbox,vmware: iso\_checksum is not required if the + checksum type is "none" +* builder/virtualbox,vmware/qemu: Support for additional scancodes for + `boot_command` such as ``, ``, ``, etc. [GH-808] +* communicator/ssh: Send TCP keep-alives on connections. [GH-872] +* post-processor/vagrant: AWS/DigitalOcean keep input artifacts by + default. [GH-55] +* provisioners/ansible-local: Properly upload custom playbooks. [GH-829] +* provisioners/ansible-local: Better error if ansible isn't installed. + [GH-836] + +## 0.5.1 (01/02/2014) + +### BUG FIXES: + +* core: If a stream ID loops around, don't let it use stream ID 0. [GH-767] +* core: Fix issue where large writes to plugins would result in stream + corruption. [GH-727] +* builders/virtualbox-ovf: `shutdown_timeout` config works. [GH-772] +* builders/vmware-iso: Remote driver works properly again. [GH-773] + +## 0.5.0 (12/30/2013) + +### BACKWARDS INCOMPATIBILITIES: + +* "virtualbox" builder has been renamed to "virtualbox-iso". Running your + template through `packer fix` will resolve this. +* "vmware" builder has been renamed to "vmware-iso". Running your template + through `packer fix` will resolve this. +* post-processor/vagrant: Syntax for overriding by provider has changed. + See the documentation for more information. Running your template + through `packer fix` should resolve this. +* post-processor/vsphere: Some available configuration options were + changed. Running your template through `packer fix` should resolve + this. +* provisioner/puppet-masterless: The `execute_command` no longer has + the `Has*` variables, since the templating language now supports + comparison operations. See the Go documentation for more info: + http://golang.org/pkg/text/template/ + +### FEATURES: + +* **New builder:** Google Compute Engine. You can now build images for + use in Google Compute Engine. See the documentation for more information. + [GH-715] +* **New builder:** "virtualbox-ovf" can build VirtualBox images from + an existing OVF or OVA. [GH-201] +* **New builder:** "vmware-vmx" can build VMware images from an existing + VMX. [GH-201] +* Environmental variables can now be accessed as default values for + user variables using the "env" function. See the documentation for more + information. +* "description" field in templates: write a human-readable description + of what a template does. This will be shown in `packer inspect`. +* Vagrant post-processor now accepts a list of files to include in the + box. +* All provisioners can now have a "pause\_before" parameter to wait + some period of time before running that provisioner. This is useful + for reboots. [GH-737] + +### IMPROVEMENTS: + +* core: Plugins communicate over a single TCP connection per plugin now, + instead of sometimes dozens. Performance around plugin communication + dramatically increased. +* core: Build names are now template processed so you can use things + like user variables in them. [GH-744] +* core: New "pwd" function available globally that returns the working + directory. [GH-762] +* builder/amazon/all: Launched EC2 instances now have a name of + "Packer Builder" so that they are easily recognizable. [GH-642] +* builder/amazon/all: Copying AMIs to multiple regions now happens + in parallel. [GH-495] +* builder/amazon/all: Ability to specify "run\_tags" to tag the instance + while running. [GH-722] +* builder/digitalocean: Private networking support. [GH-698] +* builder/docker: A "run\_command" can be specified, configuring how + the container is started. [GH-648] +* builder/openstack: In debug mode, the generated SSH keypair is saved + so you can SSH into the machine. [GH-746] +* builder/qemu: Floppy files are supported. [GH-686] +* builder/qemu: Next `run_once` option tells Qemu to run only once, + which is useful for Windows installs that handle reboots for you. + [GH-687] +* builder/virtualbox: Nice errors if Packer can't write to + the output directory. +* builder/virtualbox: ISO is ejected prior to export. +* builder/virtualbox: Checksum type can be "none" [GH-471] +* builder/vmware: Can now specify path to the Fusion application. [GH-677] +* builder/vmware: Checksum type can be "none" [GH-471] +* provisioner/puppet-masterless: Can now specify a `manifest_dir` to + upload manifests to the remote machine for imports. [GH-655] + +### BUG FIXES: + +* core: No colored output in machine-readable output. [GH-684] +* core: User variables can now be used for non-string fields. [GH-598] +* core: Fix bad download paths if the download URL contained a "." + before a "/" [GH-716] +* core: "{{timestamp}}" values will always be the same for the entire + duration of a build. [GH-744] +* builder/amazon: Handle cases where security group isn't instantly + available. [GH-494] +* builder/virtualbox: don't download guest additions if disabled. [GH-731] +* post-processor/vsphere: Uploads VM properly. [GH-694] +* post-processor/vsphere: Process user variables. +* provisioner/ansible-local: all configurations are processed as templates + [GH-749] +* provisioner/ansible-local: playbook paths are properly validated + as directories, not files. [GH-710] +* provisioner/chef-solo: Environments are recognized. [GH-726] + +## 0.4.1 (December 7, 2013) + +### IMPROVEMENTS: + +* builder/amazon/ebs: New option allows associating a public IP with + non-default VPC instances. [GH-660] +* builder/openstack: A "proxy\_url" setting was added to define an HTTP + proxy to use when building with this builder. [GH-637] + +### BUG FIXES: + +* core: Don't change background color on CLI anymore, making things look + a tad nicer in some terminals. +* core: multiple ISO URLs works properly in all builders. [GH-683] +* builder/amazon/chroot: Block when obtaining file lock to allow + parallel builds. [GH-689] +* builder/amazon/instance: Add location flag to upload bundle command + so that building AMIs works out of us-east-1. [GH-679] +* builder/qemu: Qemu arguments are templated. [GH-688] +* builder/vmware: Cleanup of VMX keys works properly so cd-rom won't + get stuck with ISO. [GH-685] +* builder/vmware: File cleanup is more resilient to file delete races + with the operating system. [GH-675] +* provisioner/puppet-masterless: Check for hiera config path existence + properly. [GH-656] + +## 0.4.0 (November 19, 2013) + +### FEATURES: + +* Docker builder: build and export Docker containers, easily provisioned + with any of the Packer built-in provisioners. +* QEMU builder: builds a new VM compatible with KVM or Xen using QEMU. +* Remote ESXi builder: builds a VMware VM using ESXi remotely using only + SSH to an ESXi machine directly. +* vSphere post-processor: Can upload VMware artifacts to vSphere +* Vagrant post-processor can now make DigitalOcean provider boxes. [GH-504] + +### IMPROVEMENTS: + +* builder/amazon/all: Can now specify a list of multiple security group + IDs to apply. [GH-499] +* builder/amazon/all: AWS API requests are now retried when a temporary + network error occurs as well as 500 errors. [GH-559] +* builder/virtualbox: Use VBOX\_INSTALL\_PATH env var on Windows to find + VBoxManage. [GH-628] +* post-processor/vagrant: skips gzip compression when compression_level=0 +* provisioner/chef-solo: Encrypted data bag support. [GH-625] + +### BUG FIXES: + +* builder/amazon/chroot: Copying empty directories works. [GH-588] +* builder/amazon/chroot: Chroot commands work with shell provisioners. [GH-581] +* builder/amazon/chroot: Don't choose a mount point that is a partition of + an already mounted device. [GH-635] +* builder/virtualbox: Ctrl-C interrupts during waiting for boot. [GH-618] +* builder/vmware: VMX modifications are now case-insensitive. [GH-608] +* builder/vmware: VMware Fusion won't ask for VM upgrade. +* builder/vmware: Ctrl-C interrupts during waiting for boot. [GH-618] +* provisioner/chef-solo: Output is slightly prettier and more informative. + +## 0.3.11 (November 4, 2013) + +### FEATURES: + +* builder/amazon/ebs: Ability to specify which availability zone to create + instance in. [GH-536] + +### IMPROVEMENTS: + +* core: builders can now give warnings during validation. warnings won't + fail the build but may hint at potential future problems. +* builder/digitalocean: Can now specify a droplet name +* builder/virtualbox: Can now disable guest addition download entirely + by setting "guest_additions_mode" to "disable" [GH-580] +* builder/virtualbox,vmware: ISO urls can now be https. [GH-587] +* builder/virtualbox,vmware: Warning if shutdown command is not specified, + since it is a common case of data loss. + +### BUG FIXES: + +* core: Won't panic when writing to a bad pipe. [GH-560] +* builder/amazon/all: Properly scrub access key and secret key from logs. + [GH-554] +* builder/openstack: Properly scrub password from logs. [GH-554] +* builder/virtualbox: No panic if SSH host port min/max is the same. [GH-594] +* builder/vmware: checks if `ifconfig` is in `/sbin` [GH-591] +* builder/vmware: Host IP lookup works for non-C locales. [GH-592] +* common/uuid: Use cryptographically secure PRNG when generating + UUIDs. [GH-552] +* communicator/ssh: File uploads that exceed the size of memory no longer + cause crashes. [GH-561] + +## 0.3.10 (October 20, 2013) + +### FEATURES: + +* Ansible provisioner + +### IMPROVEMENTS: + +* post-processor/vagrant: support instance-store AMIs built by Packer. [GH-502] +* post-processor/vagrant: can now specify compression level to use + when creating the box. [GH-506] + +### BUG FIXES: + +* builder/all: timeout waiting for SSH connection is a failure. [GH-491] +* builder/amazon: Scrub sensitive data from the logs. [GH-521] +* builder/amazon: Handle the situation where an EC2 instance might not + be immediately available. [GH-522] +* builder/amazon/chroot: Files copied into the chroot remove destination + before copy, fixing issues with dangling symlinks. [GH-500] +* builder/digitalocean: don't panic if erroneous API response doesn't + contain error message. [GH-492] +* builder/digitalocean: scrub API keys from config debug output. [GH-516] +* builder/virtualbox: error if VirtualBox version cant be detected. [GH-488] +* builder/virtualbox: detect if vboxdrv isn't properly setup. [GH-488] +* builder/virtualbox: sleep a bit before export to ensure the session + is unlocked. [GH-512] +* builder/virtualbox: create SATA drives properly on VirtualBox 4.3. [GH-547] +* builder/virtualbox: support user templates in SSH key path. [GH-539] +* builder/vmware: support user templates in SSH key path. [GH-539] +* communicator/ssh: Fix issue where a panic could arise from a nil + dereference. [GH-525] +* post-processor/vagrant: Fix issue with VirtualBox OVA. [GH-548] +* provisioner/salt: Move salt states to correct remote directory. [GH-513] +* provisioner/shell: Won't block on certain scripts on Windows anymore. + [GH-507] + +## 0.3.9 (October 2, 2013) + +### FEATURES: + +* The Amazon chroot builder is now able to run without any `sudo` privileges + by using the "command_wrapper" configuration. [GH-430] +* Chef provisioner supports environments. [GH-483] + +### BUG FIXES: + +* core: default user variable values don't need to be strings. [GH-456] +* builder/amazon-chroot: Fix errors with waiting for state change. [GH-459] +* builder/digitalocean: Use proper error message JSON key (DO API change). +* communicator/ssh: SCP uploads now work properly when directories + contain symlinks. [GH-449] +* provisioner/chef-solo: Data bags and roles path are now properly + populated when set. [GH-470] +* provisioner/shell: Windows line endings are actually properly changed + to Unix line endings. [GH-477] + +## 0.3.8 (September 22, 2013) + +### FEATURES: + +* core: You can now specify `only` and `except` configurations on any + provisioner or post-processor to specify a list of builds that they + are valid for. [GH-438] +* builders/virtualbox: Guest additions can be attached rather than uploaded, + easier to handle for Windows guests. [GH-405] +* provisioner/chef-solo: Ability to specify a custom Chef configuration + template. +* provisioner/chef-solo: Roles and data bags support. [GH-348] + +### IMPROVEMENTS: + +* core: User variables can now be used for integer, boolean, etc. + values. [GH-418] +* core: Plugins made with incompatible versions will no longer load. +* builder/amazon/all: Interrupts work while waiting for AMI to be ready. +* provisioner/shell: Script line-endings are automatically converted to + Unix-style line-endings. Can be disabled by setting "binary" to "true". + [GH-277] + +### BUG FIXES: + +* core: Set TCP KeepAlives on internally created RPC connections so that + they don't die. [GH-416] +* builder/amazon/all: While waiting for AMI, will detect "failed" state. +* builder/amazon/all: Waiting for state will detect if the resource (AMI, + instance, etc.) disappears from under it. +* builder/amazon/instance: Exclude only contents of /tmp, not /tmp + itself. [GH-437] +* builder/amazon/instance: Make AccessKey/SecretKey available to bundle + command even when they come from the environment. [GH-434] +* builder/virtualbox: F1-F12 and delete scancodes now work. [GH-425] +* post-processor/vagrant: Override configurations properly work. [GH-426] +* provisioner/puppet-masterless: Fix failure case when both facter vars + are used and prevent_sudo. [GH-415] +* provisioner/puppet-masterless: User variables now work properly in + manifest file and hiera path. [GH-448] + +## 0.3.7 (September 9, 2013) + +### BACKWARDS INCOMPATIBILITIES: + +* The "event_delay" option for the DigitalOcean builder is now gone. + The builder automatically waits for events to go away. Run your templates + through `packer fix` to get rid of these. + +### FEATURES: + +* **NEW PROVISIONER:** `puppet-masterless`. You can now provision with + a masterless Puppet setup. [GH-234] +* New globally available template function: `uuid`. Generates a new random + UUID. +* New globally available template function: `isotime`. Generates the + current time in ISO standard format. +* New Amazon template function: `clean_ami_name`. Substitutes '-' for + characters that are illegal to use in an AMI name. + +### IMPROVEMENTS: + +* builder/amazon/all: Ability to specify the format of the temporary + keypair created. [GH-389] +* builder/amazon/all: Support the NoDevice flag for block mappings. [GH-396] +* builder/digitalocean: Retry on any pending event errors. +* builder/openstack: Can now specify a project. [GH-382] +* builder/virtualbox: Can now attach hard drive over SATA. [GH-391] +* provisioner/file: Can now upload directories. [GH-251] + +### BUG FIXES: + +* core: Detect if SCP is not enabled on the other side. [GH-386] +* builder/amazon/all: When copying AMI to multiple regions, copy + the metadata (tags and attributes) as well. [GH-388] +* builder/amazon/all: Fix panic case where eventually consistent + instance state caused an index out of bounds. +* builder/virtualbox: The `vm_name` setting now properly sets the OVF + name of the output. [GH-401] +* builder/vmware: Autoanswer VMware dialogs. [GH-393] +* command/inspect: Fix weird output for default values for optional vars. + +## 0.3.6 (September 2, 2013) + +### FEATURES: + +* User variables can now be specified as "required", meaning the user + MUST specify a value. Just set the default value to "null". [GH-374] + +### IMPROVEMENTS: + +* core: Much improved interrupt handling. For example, interrupts now + cancel much more quickly within provisioners. +* builder/amazon: In `-debug` mode, the keypair used will be saved to + the current directory so you can access the machine. [GH-373] +* builder/amazon: In `-debug` mode, the DNS is outputted. +* builder/openstack: IPv6 addresses supported for SSH. [GH-379] +* communicator/ssh: Support for private keys encrypted using PKCS8. [GH-376] +* provisioner/chef-solo: You can now use user variables in the `json` + configuration for Chef. [GH-362] + +### BUG FIXES: + +* core: Concurrent map access is completely gone, fixing rare issues + with runtime memory corruption. [GH-307] +* core: Fix possible panic when ctrl-C during provisioner run. +* builder/digitalocean: Retry destroy a few times because DO sometimes + gives false errors. +* builder/openstack: Properly handle the case no image is made. [GH-375] +* builder/openstack: Specifying a region is now required in a template. +* provisioners/salt-masterless: Use filepath join to properly join paths. + +## 0.3.5 (August 28, 2013) + +### FEATURES: + +* **NEW BUILDER:** `openstack`. You can now build on OpenStack. [GH-155] +* **NEW PROVISIONER:** `chef-solo`. You can now provision with Chef + using `chef-solo` from local cookbooks. +* builder/amazon: Copy AMI to multiple regions with `ami_regions`. [GH-322] +* builder/virtualbox,vmware: Can now use SSH keys as an auth mechanism for + SSH using `ssh_key_path`. [GH-70] +* builder/virtualbox,vmware: Support SHA512 as a checksum type. [GH-356] +* builder/vmware: The root hard drive type can now be specified with + "disk_type_id" for advanced users. [GH-328] +* provisioner/salt-masterless: Ability to specify a minion config. [GH-264] +* provisioner/salt-masterless: Ability to upload pillars. [GH-353] + +### IMPROVEMENTS: + +* core: Output message when Ctrl-C received that we're cleaning up. [GH-338] +* builder/amazon: Tagging now works with all amazon builder types. +* builder/vmware: Option `ssh_skip_request_pty` for not requesting a PTY + for the SSH connection. [GH-270] +* builder/vmware: Specify a `vmx_template_path` in order to customize + the generated VMX. [GH-270] +* command/build: Machine-readable output now contains build errors, if any. +* command/build: An "end" sentinel is outputted in machine-readable output + for artifact listing so it is easier to know when it is over. + +### BUG FIXES: + +* core: Fixed a couple cases where a double ctrl-C could panic. +* core: Template validation fails if an override is specified for a + non-existent builder. [GH-336] +* core: The SSH connection is heartbeated so that drops can be + detected. [GH-200] +* builder/amazon/instance: Remove check for ec2-ami-tools because it + didn't allow absolute paths to work properly. [GH-330] +* builder/digitalocean: Send a soft shutdown request so that files + are properly synced before shutdown. [GH-332] +* command/build,command/validate: If a non-existent build is specified to + '-only' or '-except', it is now an error. [GH-326] +* post-processor/vagrant: Setting OutputPath with a timestamp now + always works properly. [GH-324] +* post-processor/vagrant: VirtualBox OVA formats now turn into + Vagrant boxes properly. [GH-331] +* provisioner/shell: Retry upload if start command fails, making reboot + handling much more robust. + +## 0.3.4 (August 21, 2013) + +### IMPROVEMENTS: + +* post-processor/vagrant: the file being compressed will be shown + in the UI. [GH-314] + +### BUG FIXES: + +* core: Avoid panics when double-interrupting Packer. +* provisioner/shell: Retry shell script uploads, making reboots more + robust if they happen to fail in this stage. [GH-282] + +## 0.3.3 (August 19, 2013) + +### FEATURES: + +* builder/virtualbox: support exporting in OVA format. [GH-309] + +### IMPROVEMENTS: + +* core: All HTTP downloads across Packer now support the standard + proxy environmental variables (`HTTP_PROXY`, `NO_PROXY`, etc.) [GH-252] +* builder/amazon: API requests will use HTTP proxy if specified by + environmental variables. +* builder/digitalocean: API requests will use HTTP proxy if specified + by environmental variables. + +### BUG FIXES: + +* core: TCP connection between plugin processes will keep-alive. [GH-312] +* core: No more "unused key keep_input_artifact" for post processors. [GH-310] +* post-processor/vagrant: `output_path` templates now work again. + +## 0.3.2 (August 18, 2013) + +### FEATURES: + +* New command: `packer inspect`. This command tells you the components of + a template. It respects the `-machine-readable` flag as well so you can + parse out components of a template. +* Packer will detect its own crashes (always a bug) and save a "crash.log" + file. +* builder/virtualbox: You may now specify multiple URLs for an ISO + using "iso_url" in a template. The URLs will be tried in order. +* builder/vmware: You may now specify multiple URLs for an ISO + using "iso_url" in a template. The URLs will be tried in order. + +### IMPROVEMENTS: + +* core: built with Go 1.1.2 +* core: packer help output now loads much faster. +* builder/virtualbox: guest_additions_url can now use the `Version` + variable to get the VirtualBox version. [GH-272] +* builder/virtualbox: Do not check for VirtualBox as part of template + validation; only check at execution. +* builder/vmware: Do not check for VMware as part of template validation; + only check at execution. +* command/build: A path of "-" will read the template from stdin. +* builder/amazon: add block device mappings. [GH-90] + +### BUG FIXES: + +* windows: file URLs are easier to get right as Packer + has better parsing and error handling for Windows file paths. [GH-284] +* builder/amazon/all: Modifying more than one AMI attribute type no longer + crashes. +* builder/amazon-instance: send IAM instance profile data. [GH-294] +* builder/digitalocean: API request parameters are properly URL + encoded. [GH-281] +* builder/virtualbox: download progress won't be shown until download + actually starts. [GH-288] +* builder/virtualbox: floppy files names of 13 characters are now properly + written to the FAT12 filesystem. [GH-285] +* builder/vmware: download progress won't be shown until download + actually starts. [GH-288] +* builder/vmware: interrupt works while typing commands over VNC. +* builder/virtualbox: floppy files names of 13 characters are now properly + written to the FAT12 filesystem. [GH-285] +* post-processor/vagrant: Process user variables. [GH-295] + +## 0.3.1 (August 12, 2013) + +### IMPROVEMENTS: + +* provisioner/shell: New setting `start_retry_timeout` which is the timeout + for the provisioner to attempt to _start_ the remote process. This allows + the shell provisioner to work properly with reboots. [GH-260] + +### BUG FIXES: + +* core: Remote command output containing '\r' now looks much better + within the Packer output. +* builder/vmware: Fix issue with finding driver files. [GH-279] +* provisioner/salt-masterless: Uploads work properly from Windows. [GH-276] + +## 0.3.0 (August 12, 2013) + +### BACKWARDS INCOMPATIBILITIES: + +* All `{{.CreateTime}}` variables within templates (such as for AMI names) + are now replaced with `{{timestamp}}`. Run `packer fix` to fix your + templates. + +### FEATURES: + +* **User Variables** allow you to specify variables within your templates + that can be replaced using the command-line, files, or environmental + variables. This dramatically improves the portability of packer templates. + See the documentation for more information. +* **Machine-readable output** can be enabled by passing the + `-machine-readable` flag to _any_ Packer command. +* All strings in a template are now processed for variables/functions, + so things like `{{timestamp}}` can be used everywhere. More features will + be added in the future. +* The `amazon` builders (all of them) can now have attributes of their + resulting AMIs modified, such as access permissions and product codes. + +### IMPROVEMENTS: + +* builder/amazon/all: User data can be passed to start the instances. [GH-253] +* provisioner/salt-masterless: `local_state_tree` is no longer required, + allowing you to use shell provisioner (or others) to bring this down. + [GH-269] + +### BUG FIXES: + +* builder/amazon/ebs,instance: Retry deleing security group a few times. + [GH-278] +* builder/vmware: Workstation works on Windows XP now. [GH-238] +* builder/vmware: Look for files on Windows in multiple locations + using multiple environmental variables. [GH-263] +* provisioner/salt-masterless: states aren't deleted after the run + anymore. [GH-265] +* provisioner/salt-masterless: error if any commands exit with a non-zero + exit status. [GH-266] + +## 0.2.3 (August 7, 2013) + +### IMPROVEMENTS: + +* builder/amazon/all: Added Amazon AMI tag support. [GH-233] + +### BUG FIXES: + +* core: Absolute/relative filepaths on Windows now work for iso_url + and other settings. [GH-240] +* builder/amazon/all: instance info is refreshed while waiting for SSH, + allowing Packer to see updated IP/DNS info. [GH-243] + +## 0.2.2 (August 1, 2013) + +### FEATURES: + +* New builder: `amazon-chroot` can create EBS-backed AMIs without launching + a new EC2 instance. This can shave minutes off of the AMI creation process. + See the docs for more info. +* New provisioner: `salt-masterless` will provision the node using Salt + without a master. +* The `vmware` builder now works with Workstation 9 on Windows. [GH-222] +* The `vmware` builder now works with Player 5 on Linux. [GH-190] + +### IMPROVEMENTS: + +* core: Colors won't be outputted on Windows unless in Cygwin. +* builder/amazon/all: Added `iam_instance_profile` to launch the source + image with a given IAM profile. [GH-226] + +### BUG FIXES: + +* builder/virtualbox,vmware: relative paths work properly as URL + configurations. [GH-215] +* builder/virtualbox,vmware: fix race condition in deleting the output + directory on Windows by retrying. + +## 0.2.1 (July 26, 2013) + +### FEATURES: + +* New builder: `amazon-instance` can create instance-storage backed + AMIs. +* VMware builder now works with Workstation 9 on Linux. + +### IMPROVEMENTS: + +* builder/amazon/all: Ctrl-C while waiting for state change works +* builder/amazon/ebs: Can now launch instances into a VPC for added protection. [GH-210] +* builder/virtualbox,vmware: Add backspace, delete, and F1-F12 keys to the boot + command. +* builder/virtualbox: massive performance improvements with big ISO files because + an expensive copy is avoided. [GH-202] +* builder/vmware: CD is removed prior to exporting final machine. [GH-198] + +### BUG FIXES: + +* builder/amazon/all: Gracefully handle when AMI appears to not exist + while AWS state is propagating. [GH-207] +* builder/virtualbox: Trim carriage returns for Windows to properly + detect VM state on Windows. [GH-218] +* core: build names no longer cause invalid config errors. [GH-197] +* command/build: If any builds fail, exit with non-zero exit status. +* communicator/ssh: SCP exit codes are tested and errors are reported. [GH-195] +* communicator/ssh: Properly change slash direction for Windows hosts. [GH-218] + +## 0.2.0 (July 16, 2013) + +### BACKWARDS INCOMPATIBILITIES: + +* "iso_md5" in the virtualbox and vmware builders is replaced with + "iso_checksum" and "iso_checksum_type" (with the latter set to "md5"). + See the announce below on `packer fix` to automatically fix your templates. + +### FEATURES: + +* **NEW COMMAND:** `packer fix` will attempt to fix templates from older + versions of Packer that are now broken due to backwards incompatibilities. + This command will fix the backwards incompatibilities introduced in this + version. +* Amazon EBS builder can now optionally use a pre-made security group + instead of randomly generating one. +* DigitalOcean API key and client IDs can now be passed in as + environmental variables. See the documentation for more details. +* VirtualBox and VMware can now have `floppy_files` specified to attach + floppy disks when booting. This allows for unattended Windows installs. +* `packer build` has a new `-force` flag that forces the removal of + existing artifacts if they exist. [GH-173] +* You can now log to a file (instead of just stderr) by setting the + `PACKER_LOG_FILE` environmental variable. [GH-168] +* Checksums other than MD5 can now be used. SHA1 and SHA256 can also + be used. See the documentation on `iso_checksum_type` for more info. [GH-175] + +### IMPROVEMENTS: + +* core: invalid keys in configuration are now considered validation + errors. [GH-104] +* core: all builders now share a common SSH connection core, improving + SSH reliability over all the builders. +* amazon-ebs: Credentials will come from IAM role if available. [GH-160] +* amazon-ebs: Verify the source AMI is EBS-backed before launching. [GH-169] +* shell provisioner: the build name and builder type are available in + the `PACKER_BUILD_NAME` and `PACKER_BUILDER_TYPE` env vars by default, + respectively. [GH-154] +* vmware: error if shutdown command has non-zero exit status. + +### BUG FIXES: + +* core: UI messages are now properly prefixed with spaces again. +* core: If SSH connection ends, re-connection attempts will take + place. [GH-152] +* virtualbox: "paused" doesn't mean the VM is stopped, improving + shutdown detection. +* vmware: error if guest IP could not be detected. [GH-189] + +## 0.1.5 (July 7, 2013) + +### FEATURES: + +* "file" uploader will upload files from the machine running Packer to the + remote machine. +* VirtualBox guest additions URL and checksum can now be specified, allowing + the VirtualBox builder to have the ability to be used completely offline. + +### IMPROVEMENTS: + +* core: If SCP is not available, a more descriptive error message + is shown telling the user. [GH-127] +* shell: Scripts are now executed by default according to their shebang, + not with `/bin/sh`. [GH-105] +* shell: You can specify what interpreter you want inline scripts to + run with `inline_shebang`. +* virtualbox: Delete the packer-made SSH port forwarding prior to + exporting the VM. + +### BUG FIXES: + +* core: Non-200 response codes on downloads now show proper errors. + [GH-141] +* amazon-ebs: SSH handshake is retried. [GH-130] +* vagrant: The `BuildName` template property works properly in + the output path. +* vagrant: Properly configure the provider-specific post-processors so + things like `vagrantfile_template` work. [GH-129] +* vagrant: Close filehandles when copying files so Windows can + rename files. [GH-100] + +## 0.1.4 (July 2, 2013) + +### FEATURES: + +* virtualbox: Can now be built headless with the "Headless" option. [GH-99] +* virtualbox: and codes for waiting 5 and 10 seconds + during the boot sequence, respectively. [GH-97] +* vmware: Can now be built headless with the "Headless" option. [GH-99] +* vmware: and codes for waiting 5 and 10 seconds + during the boot sequence, respectively. [GH-97] +* vmware: Disks are defragmented and compacted at the end of the build. + This can be disabled using "skip_compaction" + +### IMPROVEMENTS: + +* core: Template syntax errors now show line and character number. [GH-56] +* amazon-ebs: Access key and secret access key default to + environmental variables. [GH-40] +* virtualbox: Send password for keyboard-interactive auth. [GH-121] +* vmware: Send password for keyboard-interactive auth. [GH-121] + +### BUG FIXES: + +* vmware: Wait until shut down cleans up properly to avoid corrupt + disk files. [GH-111] + +## 0.1.3 (July 1, 2013) + +### FEATURES: + +* The VMware builder can now upload the VMware tools for you into + the VM. This is opt-in, you must specify the `tools_upload_flavor` + option. See the website for more documentation. + +### IMPROVEMENTS: + +* digitalocean: Errors contain human-friendly error messages. [GH-85] + +### BUG FIXES: + +* core: More plugin server fixes that avoid hangs on OS X 10.7. [GH-87] +* vagrant: AWS boxes will keep the AMI artifact around. [GH-55] +* virtualbox: More robust version parsing for uploading guest additions. [GH-69] +* virtualbox: Output dir and VM name defaults depend on build name, + avoiding collisions. [GH-91] +* vmware: Output dir and VM name defaults depend on build name, + avoiding collisions. [GH-91] + +## 0.1.2 (June 29, 2013) + +### IMPROVEMENTS: + +* core: Template doesn't validate if there are no builders. +* vmware: Delete any VMware files in the VM that aren't necessary for + it to function. + +### BUG FIXES: + +* core: Plugin servers consider a port in use if there is any + error listening to it. This fixes I18n issues and Windows. [GH-58] +* amazon-ebs: Sleep between checking instance state to avoid + RequestLimitExceeded. [GH-50] +* vagrant: Rename VirtualBox ovf to "box.ovf" [GH-64] +* vagrant: VMware boxes have the correct provider type. +* vmware: Properly populate files in artifact so that the Vagrant + post-processor works. [GH-63] + +## 0.1.1 (June 28, 2013) + +### BUG FIXES: + +* core: plugins listen explicitly on 127.0.0.1, fixing odd hangs. [GH-37] +* core: fix race condition on verifying checksum of large ISOs which + could cause panics. [GH-52] +* virtualbox: `boot_wait` defaults to "10s" rather than 0. [GH-44] +* virtualbox: if `http_port_min` and max are the same, it will no longer + panic. [GH-53] +* vmware: `boot_wait` defaults to "10s" rather than 0. [GH-44] +* vmware: if `http_port_min` and max are the same, it will no longer + panic. [GH-53] + +## 0.1.0 (June 28, 2013) + +* Initial release + diff --git a/content/packer/LICENSE b/content/packer/LICENSE new file mode 100644 index 0000000000..91ed1781f8 --- /dev/null +++ b/content/packer/LICENSE @@ -0,0 +1,92 @@ +License text copyright (c) 2020 MariaDB Corporation Ab, All Rights Reserved. +"Business Source License" is a trademark of MariaDB Corporation Ab. + +Parameters + +Licensor: HashiCorp, Inc. +Licensed Work: Packer Version 1.10.0 or later. The Licensed Work is (c) 2024 + HashiCorp, Inc. +Additional Use Grant: You may make production use of the Licensed Work, provided + Your use does not include offering the Licensed Work to third + parties on a hosted or embedded basis in order to compete with + HashiCorp's paid version(s) of the Licensed Work. For purposes + of this license: + + A "competitive offering" is a Product that is offered to third + parties on a paid basis, including through paid support + arrangements, that significantly overlaps with the capabilities + of HashiCorp's paid version(s) of the Licensed Work. If Your + Product is not a competitive offering when You first make it + generally available, it will not become a competitive offering + later due to HashiCorp releasing a new version of the Licensed + Work with additional capabilities. In addition, Products that + are not provided on a paid basis are not competitive. + + "Product" means software that is offered to end users to manage + in their own environments or offered as a service on a hosted + basis. + + "Embedded" means including the source code or executable code + from the Licensed Work in a competitive offering. "Embedded" + also means packaging the competitive offering in such a way + that the Licensed Work must be accessed or downloaded for the + competitive offering to operate. + + Hosting or using the Licensed Work(s) for internal purposes + within an organization is not considered a competitive + offering. HashiCorp considers your organization to include all + of your affiliates under common control. + + For binding interpretive guidance on using HashiCorp products + under the Business Source License, please visit our FAQ. + (https://www.hashicorp.com/license-faq) +Change Date: Four years from the date the Licensed Work is published. +Change License: MPL 2.0 + +For information about alternative licensing arrangements for the Licensed Work, +please contact licensing@hashicorp.com. + +Notice + +Business Source License 1.1 + +Terms + +The Licensor hereby grants you the right to copy, modify, create derivative +works, redistribute, and make non-production use of the Licensed Work. The +Licensor may make an Additional Use Grant, above, permitting limited production use. + +Effective on the Change Date, or the fourth anniversary of the first publicly +available distribution of a specific version of the Licensed Work under this +License, whichever comes first, the Licensor hereby grants you rights under +the terms of the Change License, and the rights granted in the paragraph +above terminate. + +If your use of the Licensed Work does not comply with the requirements +currently in effect as described in this License, you must purchase a +commercial license from the Licensor, its affiliated entities, or authorized +resellers, or you must refrain from using the Licensed Work. + +All copies of the original and modified Licensed Work, and derivative works +of the Licensed Work, are subject to this License. This License applies +separately for each version of the Licensed Work and the Change Date may vary +for each version of the Licensed Work released by Licensor. + +You must conspicuously display this License on each original or modified copy +of the Licensed Work. If you receive the Licensed Work in original or +modified form from a third party, the terms and conditions set forth in this +License apply to your use of that work. + +Any use of the Licensed Work in violation of this License will automatically +terminate your rights under this License for the current and all other +versions of the Licensed Work. + +This License does not grant you any right in any trademark or logo of +Licensor or its affiliates (provided that you may use a trademark or logo of +Licensor as expressly required by this License). + +TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED ON +AN "AS IS" BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS, +EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND +TITLE. diff --git a/content/packer/README.md b/content/packer/README.md new file mode 100644 index 0000000000..05a280b2b8 --- /dev/null +++ b/content/packer/README.md @@ -0,0 +1,65 @@ +# Packer +[![License: BUSL-1.1](https://img.shields.io/badge/License-BUSL--1.1-yellow.svg)](LICENSE) +[![Build Status](https://github.com/hashicorp/packer/actions/workflows/build.yml/badge.svg)](https://github.com/hashicorp/packer/actions/workflows/build.yml) +[![Discuss](https://img.shields.io/badge/discuss-packer-3d89ff?style=flat)](https://discuss.hashicorp.com/c/packer) +=== + +

+ + HashiCorp Packer logo + +

+ +Packer is a tool for building identical machine images for multiple platforms +from a single source configuration. + +Packer is lightweight, runs on every major operating system, and is highly +performant, creating machine images for multiple platforms in parallel. Packer +supports various platforms through external plugin integrations, the full list of which can +be found at https://developer.hashicorp.com/packer/integrations. + +The images that Packer creates can easily be turned into [Vagrant](http://www.vagrantup.com) boxes. + +## Quick Start + +### Packer + +There is a great [introduction and getting started guide](https://learn.hashicorp.com/tutorials/packer/get-started-install-cli) +for building a Docker image on your local machine without using any paid cloud resources. + +Alternatively, you can refer to [getting started with AWS](https://developer.hashicorp.com/packer/tutorials/aws-get-started) to +learn how to build a machine image for an external cloud provider. + +### HCP Packer + +HCP Packer registry stores Packer image metadata, enabling you to track your image lifecycle. + +To get started with building an AWS machine image to HCP Packer for referencing in Terraform refer +to the collection of [HCP Packer Tutorials](https://developer.hashicorp.com/packer/tutorials/hcp-get-started). + +## Documentation + +Comprehensive documentation is viewable on the Packer website at https://developer.hashicorp.com/packer/docs. + +## Contributing to Packer + +See +[CONTRIBUTING.md](https://github.com/hashicorp/packer/blob/master/.github/CONTRIBUTING.md) +for best practices and instructions on setting up your development environment +to work on Packer. + +## Unmaintained Plugins +As contributors' circumstances change, development on a community maintained +plugin can slow. When this happens, HashiCorp may use GitHub's option to archive the +plugin’s repository, to clearly signal the plugin's status to users. + +What does **unmaintained** mean? + +1. The code repository and all commit history will still be available. +1. Documentation will remain on the Packer website. +1. Issues and pull requests are monitored as a best effort. +1. No active development will be performed by HashiCorp. + +If you are interested in maintaining an unmaintained or archived plugin, please reach out to us at packer@hashicorp.com. + + diff --git a/content/packer/content/docs/builders/community-supported.mdx b/content/packer/content/docs/builders/community-supported.mdx new file mode 100644 index 0000000000..bbf9e79d2b --- /dev/null +++ b/content/packer/content/docs/builders/community-supported.mdx @@ -0,0 +1,22 @@ +--- +description: | + Community-supported builders are developed and maintained by third-parties and not HashiCorp. Use them with Packer to extend Packer functionality. +page_title: Community-supported builders +--- + +# Community-supported builders + +The following builders are developed and maintained by various members of the +Packer community, not by HashiCorp. For more information on how to use community +builders, refer to our docs on [extending Packer](/packer/docs/plugins/creation). + +- ARM builders + + - [packer-plugin-arm-image](https://github.com/solo-io/packer-plugin-arm-image): Lets you extend onto existing system images. + - [packer-builder-arm](https://github.com/mkaczanowski/packer-builder-arm): Lets you extend or build new images with a variety of options, such as custom partition tables. + +- [Exoscale builder](https://github.com/exoscale/packer-plugin-exoscale) - Creates Exoscale custom templates based on a compute instance snapshot. + +- [Citrix XenServer/Citrix Hypervisor](https://github.com/xenserver/packer-builder-xenserver) - Plugin for creating [Citrix XenServer/Citrix Hypervisor](https://xenserver.org/) images from an ISO image or from an existing template. + +- [XCP-NG/Citrix XenServer/Citrix Hypervisor/Updated Fork](https://github.com/ddelnano/packer-plugin-xenserver) - Plugin for creating [XCP-NG/Citrix XenServer/Citrix Hypervisor](https://xcp-ng.org/) images from an ISO image or from an existing template. This is a fork of the orginal and reccomended by the developers of XCP-NG. \ No newline at end of file diff --git a/content/packer/content/docs/builders/file.mdx b/content/packer/content/docs/builders/file.mdx new file mode 100644 index 0000000000..904cbd4523 --- /dev/null +++ b/content/packer/content/docs/builders/file.mdx @@ -0,0 +1,71 @@ +--- +description: | + The `file` builder creates an artifact from a file. Use the `file` builder to debug post-processors without incurring long wait times. +page_title: file builder reference +--- + + + + + +# `file` builder + +The `file` builder creates an artifact from a file. You can use it to debug post-processors without incurring long wait times. + +Artifact `BuilderId`: `packer.file` + +## Basic Example + +Below is a fully functioning example. It create a file at `target` with the +specified `content`. + + + + +```hcl +source "file" "basic-example" { + content = "Lorem ipsum dolor sit amet" + target = "dummy_artifact" +} + +build { + sources = ["sources.file.basic-example"] +} +``` + + + + +```json +{ + "type": "file", + "content": "Lorem ipsum dolor sit amet", + "target": "dummy_artifact" +} +``` + + + + +## Configuration Reference + +Configuration options are organized below into two categories: required and +optional. Within each category, the available options are alphabetized and +described. + +Any [communicator](/packer/docs/templates/legacy_json_templates/communicator) defined is ignored. + +### Required + +- `target` (string) - The path for the artifact file that will be created. If + the path contains directories that don't exist, Packer will create them, too. + +### Optional + +You can only define one of `source` or `content`. If none of them is defined +the artifact will be empty. + +- `source` (string) - The path for a file which will be copied as the + artifact. + +- `content` (string) - The content that will be put into the artifact. diff --git a/content/packer/content/docs/builders/index.mdx b/content/packer/content/docs/builders/index.mdx new file mode 100644 index 0000000000..08b5dd48ee --- /dev/null +++ b/content/packer/content/docs/builders/index.mdx @@ -0,0 +1,21 @@ +--- +description: | + Builders create machines and generate images from them for various platforms. Learn about the types of builders you can use in your Packer templates. +page_title: Builders overview +--- + +# Builders overview + +Builders create machines and generate images from those machines for various platforms. Some builders in Packer perform helper tasks, such as running provisioners. + +Packer has the following types of builders: + +- [Plugins](/packer/plugins): Plugins that you install have their own associated set of builders. For example, EC2, VMware, and VirtualBox use their own separate sets of builders. +- [`file`](/packer/docs/builders/file): The `file` builder creates an artifact from a file. +- [`null`](/packer/docs/builders/null): The `null` builder sets up an SSH connection and runs the provisioners. +- [Custom](/packer/docs/plugins/creation/custom-builders): You can write new builders for new or existing platforms. +- [Community-supported](/packer/docs/builders/community-supported): The Packer community develops and maintains builders for several additional platforms. + +Refer to the [`source`](/packer/docs/templates/hcl_templates/blocks/source) block documentation to learn more about configuring builders in the Packer templating language. + + diff --git a/content/packer/content/docs/builders/null.mdx b/content/packer/content/docs/builders/null.mdx new file mode 100644 index 0000000000..207c375e01 --- /dev/null +++ b/content/packer/content/docs/builders/null.mdx @@ -0,0 +1,53 @@ +--- +description: | + The `null` builder creates an SSH connection and runs provisioners. Use the `null` builder to debug provisioners without incurring long wait times. +page_title: null builder reference +--- + + + + + +# `null` builder + +The `null` builder sets up an SSH connection and runs provisioners. You can use it to debug provisioners without incurring long wait times. It does not create a images or artifacts. + +## Basic Example + +Below is a fully functioning example. It doesn't do anything useful, since no +provisioners are defined, but it will connect to the specified host via ssh. + + + + +```hcl +source "null" "basic-example" { + ssh_host = "127.0.0.1" + ssh_username = "foo" + ssh_password = "bar" +} + +build { + sources = ["sources.null.basic-example"] +} +``` + + + + +```json +{ + "type": "null", + "ssh_host": "127.0.0.1", + "ssh_username": "foo", + "ssh_password": "bar" +} +``` + + + + +## Configuration Reference + +The null builder has no configuration parameters other than the +[communicator](/packer/docs/templates/legacy_json_templates/communicator) settings. diff --git a/content/packer/content/docs/commands/build.mdx b/content/packer/content/docs/commands/build.mdx new file mode 100644 index 0000000000..208e9abf64 --- /dev/null +++ b/content/packer/content/docs/commands/build.mdx @@ -0,0 +1,71 @@ +--- +description: | + The `packer build` command builds all of the artifacts defined in a template. Builds can run in parallel or sequentially. +page_title: packer build - Commands +--- + +# `packer build` command reference + + + +Starting August 1st, 2025, the source for many official HashiCorp-maintained Packer plugins is moving from GitHub releases to the official HashiCorp release site, [releases.hashicorp.com](https://releases.hashicorp.com). Refer to [Install HashiCorp-maintained plugins](/packer/docs/plugins/install#install-hashicorp-maintained-plugins) for more information. + + + +The `packer build` command takes a template and runs all the builds within it +in order to generate a set of artifacts. The various builds specified within a +template are executed in parallel, unless otherwise specified. And the +artifacts that are created will be outputted at the end of the build. + +## Options + +- `-color=false` - Disables colorized output. Enabled by default. + +- `-debug` - Disables parallelization and enables debug mode. Debug mode + flags the builders that they should output debugging information. The exact + behavior of debug mode is left to the builder. In general, builders usually + will stop between each step, waiting for keyboard input before continuing. + This will allow the user to inspect state and so on. + +`@include 'commands/except.mdx'` + +- `-force` - Forces a builder to run when artifacts from a previous build + prevent a build from running. The exact behavior of a forced build is left + to the builder. In general, a builder supporting the forced build will + remove the artifacts from the previous build. This will allow the user to + repeat a build without having to manually clean these artifacts beforehand. + +- `-on-error=cleanup` (default), `-on-error=abort`, `-on-error=ask`, `-on-error=run-cleanup-provisioner` - + Selects what to do when the build fails during provisioning. Please note that + this only affects the build during the provisioner run, not during the + post-processor run, because it is related to whether or not to keep the + instance running and related artifacts like generated SSH keys on the system + when a provisioner fails. + + - `cleanup` cleans up after the previous steps, deleting temporary files and virtual machines. + - `abort` exits without any cleanup, which might require the next build to use `-force`. + - `ask` presents a prompt and waits for you to decide to clean up, abort, or retry + the failed step. + - `run-cleanup-provisioner` aborts and exits without any cleanup besides + the [error-cleanup-provisioner](/packer/docs/templates/legacy_json_templates/provisioners#on-error-provisioner) if one is defined. + +`@include 'commands/only.mdx'` + +- `-parallel-builds=N` - Limit the number of builds to run in parallel, 0 + means no limit (defaults to 0). + +- `-timestamp-ui` - Enable prefixing of each ui output with an RFC3339 + timestamp. + +- `-var` - Set a variable in your Packer template. This option can be used + multiple times. This is useful for setting version numbers for your build. + +- `-var-file` - Set template variables from a file. + +- `-warn-on-undeclared-var` - Setting this flag will yield a warning for each assignment within + a variable definitions file (*.pkrvars.hcl | *.pkrvars.json) that does not have an accompanying + variable block. This can occur when using a var-file that contains a large amount of unused variables + for a given HCL2 template. For HCL2 template builds defining a value for a variable in a var-file is + not enough on its own for Packer to function, as there also needs to be a variable block definition in + the template files `pkr.hcl` for the variable. By default `packer build` will not warn when a var-file + contains one or more undeclared variables. diff --git a/content/packer/content/docs/commands/console.mdx b/content/packer/content/docs/commands/console.mdx new file mode 100644 index 0000000000..b418c93cde --- /dev/null +++ b/content/packer/content/docs/commands/console.mdx @@ -0,0 +1,155 @@ +--- +description: | + The `packer console` command starts an interactive console, letting you experiment with Packer variable interpolations. +page_title: packer console command reference +--- + +# `packer console` command reference + +The `packer console` command allows you to experiment with Packer variable +interpolations. You may access variables in the Packer config you called the +console with, or provide variables when you call console using the -var or +-var-file command line options. + +~> **Note:** `console` is available from version 1.4.2 and above. + +~> **Note:** For HCL2 `console` is available from version 1.6.0 and above, use +`packer console --config-type=hcl2` to try it without a config file. Go +templating ( or `{{..}}` calls ) will not work in HCL2 mode. + +Type in the interpolation to test and hit `` to see the result. + +To exit the console, type "exit" and hit ``, or use Control-C. + +```shell-session +$ packer console my_template.json +``` + +The full list of options that the console command will accept is visible in the +help output, which can be seen via `packer console -h`. + +## Options + +- `-var` - Set a variable in your Packer template. This option can be used + multiple times. This is useful for setting version numbers for your build. + example: `-var "myvar=asdf"` + +- `-var-file` - Set template variables from a file. + example: `-var-file myvars.json` + +## REPL commands + +- `help` - displays help text for Packer console. + +- `exit` - exits the console + +- `variables` - prints a list of all variables read into the console from the + `-var` option, `-var-files` option, and template. + +## Usage Examples - repl session ( JSON ) + +Let's say you launch a console using a Packer template `example_template.json`: + +```shell-session +$ packer console example_template.json +``` + +You'll be dropped into a prompt that allows you to enter template functions and +see how they're evaluated; for example, if the variable `myvar` is defined in +your example_template's variable section: + +```json +"variables":{ + "myvar": "asdfasdf" +}, +... +``` + +and you enter `` {{user `myvar`}} `` in the Packer console, you'll see the value of +myvar: + +```shell-session +> {{user `myvar`}} +asdfasdf +``` + +From there you can test more complicated interpolations: + +```shell-session +> {{user `myvar`}}-{{timestamp}} +asdfasdf-1559854396 +``` + +And when you're done using the console, just type "exit" or CTRL-C + +```shell-session +> exit +$ +``` + +If you'd like to provide a variable or variable files, you'd do this: + +```shell-session +$ packer console -var "myvar=fdsafdsa" -var-file myvars.json example_template.json +``` + +If you don't have specific variables or var files you want to test, and just +want to experiment with a particular template engine, you can do so by simply +calling `packer console` without a template file. + +## Usage Examples - piped commands ( JSON ) + +If you'd like to just see a specific single interpolation without launching +the REPL, you can do so by echoing and piping the string into the console +command: + +```shell-session +$ echo {{timestamp}} | packer console +1559855090 +``` + +## Usage Examples - repl session ( HCL2 ) + +~> **Note:** For HCL2 `console` is available from version 1.6.0 and above, use +`packer console --config-type=hcl2` to try it without a config file. Go +templating ( or `{{..}}` calls ) will not work in HCL2 mode. + +Without a config file, `packer console` can be used to experiment with the +expression syntax and [built-in functions](/packer/docs/templates/hcl_templates/functions). + +### Starting + +To start a session on a folder containing HCL2 config files, run: + +```shell-session +packer console folder/ +``` + +Because `folder/` is a folder Packer will start in HCL2 mode, you can also +directly pass an HCL2 formatted config file: + +```shell-session +packer console file.pkr.hcl +``` + +Because the file is suffixed with `.pkr.hcl` Packer will start in HCL2 mode. + +When you just want to play around without a config file you can set the +`--config-type=hcl2` option and Packer will start in HCL2 mode: + +```shell-session +packer console --config-type=hcl2 +``` + +### Scripting + +The `packer console` command can be used in non-interactive scripts by piping +newline-separated commands to it. Only the output from the final command is +printed unless an error occurs earlier. + +For example: + +```shell-session +$ echo "1 + 5" | packer console +6 +``` diff --git a/content/packer/content/docs/commands/fix.mdx b/content/packer/content/docs/commands/fix.mdx new file mode 100644 index 0000000000..61c095e78c --- /dev/null +++ b/content/packer/content/docs/commands/fix.mdx @@ -0,0 +1,40 @@ +--- +description: | + The `packer fix` command updates backward incompatible templates for the running version of Packer. +page_title: packer fix command reference +--- + +# `packer fix` command reference + + +The `packer fix` command takes a template and finds backward incompatible +parts of it and brings it up to date so it can be used with the latest version +of Packer. After you update to a new Packer release, you should run the fix +command to make sure your templates work with the new release. + +-> **JSON template-only command**: You cannot use the `packer fix` command to update HCL2 templates. + +The fix command will output the changed template to standard out, so you should +redirect standard out using standard OS-specific techniques if you want to save it +to a file. For example, on Linux systems, you may want to do this: + +```shell-session +$ packer fix old.json > new.json +``` + +If fixing fails for any reason, the fix command will exit with a non-zero exit +status. Error messages appear on standard error, so if you're redirecting +output, you'll still see error messages. + +-> **Even when Packer fix doesn't do anything** to the template, the +template will be outputted to standard out. Things such as configuration key +ordering and indentation may be changed. The output format however, is +pretty-printed for human readability. + +The full list of fixes that the fix command performs is visible in the help +output, which can be seen via `packer fix -h`. + +## Options + +- `-validate=false` - Disables validation of the fixed template. True by + default. diff --git a/content/packer/content/docs/commands/fmt.mdx b/content/packer/content/docs/commands/fmt.mdx new file mode 100644 index 0000000000..23e7909ffd --- /dev/null +++ b/content/packer/content/docs/commands/fmt.mdx @@ -0,0 +1,66 @@ +--- +description: | + The `packer fmt` Packer command formats HCL2 configuration files to a canonical format and style to help you prevent coding errors. +page_title: packer fmt command reference +--- + +# `packer fmt` command reference + +The `packer fmt` Packer command is used to format HCL2 configuration files to +a canonical format and style. JSON files (.json) are not modified. This command +applies a subset of HCL language style conventions, along with other minor +adjustments for readability. + +`packer fmt` will display the name of the configuration file(s) that need formatting, +and write any formatted changes back to the original configuration file(s). + +Example usage: + +Check if configuration file(s) need to be formatted, but don't write the changes. + +```shell-session +$ packer fmt -check . +my-template.pkr.hcl + +``` + +Format a configuration file, writing the changes back to the original file. + +```shell-session +$ packer fmt my-template.pkr.hcl +my-template.pkr.hcl + +``` + +Format multiple configuration files, writing the changes back to respective original files. + +```shell-session +$ packer fmt my-template.pkr.hcl my-varfile.pkrvars.hcl +my-template.pkr.hcl +my-varfile.pkrvars.hcl + +``` + +Format a configuration file, reading from stdin and writing to stdout. + +```shell-session +$ packer fmt - + +// You can use pipes to combine this feature with other command line options +$ cat my-template.pkr.hcl | packer fmt - +``` + +## Options + +- `-check` - Checks if the input is formatted. Exit status will be 0 if all + input is properly formatted and non-zero otherwise. + +- `-diff` - Display diffs of any formatting change + +- `-write=false` - Don't write formatting changes to source files + (always disabled if using -check) + +- `-` - read formatting changes from stdin and write them to stdout. + +- `-recursive` Also process files in subdirectories. By default, only the + given directory (or current directory) is processed. diff --git a/content/packer/content/docs/commands/hcl2_upgrade.mdx b/content/packer/content/docs/commands/hcl2_upgrade.mdx new file mode 100644 index 0000000000..005d0281fd --- /dev/null +++ b/content/packer/content/docs/commands/hcl2_upgrade.mdx @@ -0,0 +1,136 @@ +--- +description: | + The `packer hcl2_upgrade` Packer command transpiles a JSON + configuration template into HCL2 so you can transition to HCL templates. +page_title: packer hcl2_upgrade command reference +--- + +# `packer hcl2_upgrade` command reference + +The `packer hcl2_upgrade` Packer command transpiles a JSON +configuration template to it's formatted HCL2 counterpart. The command +returns a zero exit status on success and a non-zero exit status on failure. + +-> **This command is beta**. We do not recommend using beta functionality in production environments. To report an issue and provide feedback, [open a GitHub +issue](https://github.com/hashicorp/packer/issues/new/choose). + +## Usage + +```shell-session +$ packer hcl2_upgrade my-template.json + +Successfully created my-template.json.pkr.hcl +``` + +## Upgrading variables file + +From **v1.7.1**, the `hcl2_upgrade` command can upgrade a variables file. + + + + +```json +{ + "variables": { + "aws_region": null, + "aws_secondary_region": "{{ env `AWS_DEFAULT_REGION` }}", + "aws_secret_key": "", + "aws_access_key": "" + }, + "sensitive-variables": ["aws_secret_key", "aws_access_key"] +} +``` + + + + +```hcl +variable "aws_access_key" { + type = string + default = "" + sensitive = true +} + +variable "aws_region" { + type = string +} + +variable "aws_secondary_region" { + type = string + default = "${env("AWS_DEFAULT_REGION")}" +} + +variable "aws_secret_key" { + type = string + default = "" + sensitive = true +} +``` + + + + +## Go template functions + +`hcl2_upgrade` will do its best to transform your Go _template calls_ to HCL2, +here is the list of calls that should get transformed: + +- `` {{ user `my_var` }} `` becomes `${var.my_var}`. +- `` {{ env `my_var` }} `` becomes `${var.my_var}`. Packer HCL2 supports + environment variables through input variables. See + [docs](/packer/docs/templates/hcl_templates/variables#environment-variables) + for more info. +- `{{ timestamp }}` becomes `${local.timestamp}`, the local variable + will be created for all generated files. +- `` {{ build `ID` }} `` becomes `${build.ID}`. + +The rest of the calls should remain Go template calls for now, this will be +improved over time. + +-> **Note**: The `hcl2_upgrade` command does its best to transform template +calls to their JSON counterpart, but it might fail. In that case the +`hcl2_upgrade` command will simply output the local HCL2 block without +transformation and with the error message in a comment. We are currently +working on improving this part of the transformer. + +## Options + +- `-output-file` - Filename of the hcl2 generated template. Defaults to + JSON_TEMPLATE.pkr.hcl; for example, if the file is called + "packerparty.json", the default output-file is "packerparty.json.pkr.hcl". +- `-with-annotations` - Adds helpful comments to the HCL template with + information about the generated HCL2 blocks. + +## User variables using other user variables + +Packer JSON recently started allowing using user variables from variables. In +HCL2, input variables cannot use functions nor other variables and are +virtually static, local variables must be used instead to craft more dynamic +variables. + +For v1.7.0 and lower, `hcl2_upgrade` doesn't upgrade variables to local variables, +and it is up to you to upgrade them manually. Upgrade to **v1.7.1** to let the command do it +automatically for you. + +Here is an example of a local variable using a string input variables: + +```hcl +variable "foo" { + default = "Hello," +} + +variable "bar" { + default = "World!" +} + +locals { + baz = "${var.foo} ${var.bar}" +} +``` + +## Upgrading templates that use third-party community plugins + +If your template references a plugin that is not bundled with the main Packer +binary, you need to make sure that the [plugin is installed](/packer/docs/plugins#installing-plugins) +or you will get an `unknown builder type` error. Packer needs to load the plugin +to transpose the template. diff --git a/content/packer/content/docs/commands/index.mdx b/content/packer/content/docs/commands/index.mdx new file mode 100644 index 0000000000..5f9d6bc561 --- /dev/null +++ b/content/packer/content/docs/commands/index.mdx @@ -0,0 +1,162 @@ +--- +description: | + The Packer command-line interface lets you perform Packer operations. Use the `packer` CLI command with subcommands, flags, and options to build and manage artifacts and install and manage plugins. +page_title: Packer commands overview +--- + +# Packer Commands Overview + +Packer is controlled using a command-line interface. All interaction with +Packer is done via the `packer` tool. Like many other command-line tools, the +`packer` tool takes a subcommand to execute, and that subcommand may have +additional options as well. Subcommands are executed with `packer SUBCOMMAND`, +where "SUBCOMMAND" is the actual command you wish to execute. + +If you run `packer` by itself, help will be displayed showing all available +subcommands and a brief synopsis of what they do. In addition to this, you can +run any `packer` command with the `-h` flag to output more detailed help for a +specific subcommand. + +The documentation contains information about each subcommand. + +## Machine-Readable Output + +By default, the output of Packer is very human-readable. It uses nice +formatting, spacing, and colors in order to make Packer a pleasure to use. +However, Packer was built with automation in mind. To that end, Packer supports +a fully machine-readable output setting, allowing you to use Packer in +automated environments. + +Because the machine-readable output format was made with Unix tools in mind, it +is `awk`/`sed`/`grep`/etc. friendly and provides a familiar interface without +requiring you to learn a new format. + +### Enabling Machine-Readable Output + +The machine-readable output format can be enabled by passing the +`-machine-readable` flag to any Packer command. This immediately enables all +output to become machine-readable on stdout. Logging, if enabled, continues to +appear on stderr. An example of the output is shown below: + +```shell-session +$ packer -machine-readable version +1498365963,,version,1.0.2 +1498365963,,version-prelease, +1498365963,,version-commit,3ead2750b+CHANGES +1498365963,,ui,say,Packer v1.0.2 +``` + +The format will be covered in more detail later. But as you can see, the output +immediately becomes machine-friendly. Try some other commands with the +`-machine-readable` flag to see! + +~>; The `-machine-readable` flag is designed for automated environments and +is mutually-exclusive with the `-debug` flag, which is designed for interactive +environments. + +### Format for Machine-Readable Output + +The machine readable format is a line-oriented, comma-delimited text format. +This makes it more convenient to parse using standard Unix tools such as `awk` +or `grep` in addition to full programming languages like Ruby or Python. + +The format is: + +```text +timestamp,target,type,data... +``` + +Each component is explained below: + +- `timestamp` is a Unix timestamp in UTC of when the message was printed. + +- `target` When you call `packer build` this can be either empty or + individual build names, e.g. `amazon-ebs`. It is normally empty when builds + are in progress, and the build name when artifacts of particular builds are + being referred to. + +- `type` is the type of machine-readable message being outputted. The two + most common `type`s are `ui` and `artifact` + +- `data` is zero or more comma-separated values associated with the prior + type. The exact amount and meaning of this data is type-dependent, so you + must read the documentation associated with the type to understand fully. + +Within the format, if data contains a comma, it is replaced with +`%!(PACKER_COMMA)`. This was preferred over an escape character such as `\'` +because it is more friendly to tools like `awk`. + +Newlines within the format are replaced with their respective standard escape +sequence. Newlines become a literal `\n` within the output. Carriage returns +become a literal `\r`. + +### Machine-Readable Message Types + +Here's an incomplete list of types you may see in the machine-readable output: + +You'll see these data types when you run `packer build`: + +- `ui`: this means that the information being provided is a human-readable + string that would be sent to stdout even if we aren't in machine-readable + mode. There are three "data" subtypes associated with this type: + + - `say`: in a non-machine-readable format, this would be bolded. Normally + it is used for announcements about beginning new steps in the build + process + + - `message`: the most commonly used message type, used for basic updates + during the build process. + + - `error`: reserved for errors + +- `artifact-count`: This data type tells you how many artifacts a particular + build produced. + +- `artifact`: This data type tells you information about what Packer created + during its build. An example of output follows the pattern + `timestamp, buildname, artifact, artifact_number, key, value` where `key` + and `value` contain information about the artifact. + + For example: + + ```text + 1539967803,,ui,say,\n==> Builds finished. The artifacts of successful builds are: + 1539967803,amazon-ebs,artifact-count,2 + 1539967803,amazon-ebs,artifact,0,builder-id,mitchellh.amazonebs + 1539967803,amazon-ebs,artifact,0,id,eu-west-1:ami-04d23aca8bdd36e30 + 1539967803,amazon-ebs,artifact,0,string,AMIs were created:\neu-west-1: ami-04d23aca8bdd36e30\n + 1539967803,amazon-ebs,artifact,0,files-count,0 + 1539967803,amazon-ebs,artifact,0,end + 1539967803,,ui,say,--> amazon-ebs: AMIs were created:\neu-west-1: ami-04d23aca8bdd36e30\n + 1539967803,amazon-ebs,artifact,1,builder-id, + 1539967803,amazon-ebs,artifact,1,id, + 1539967803,amazon-ebs,artifact,1,string, + 1539967803,amazon-ebs,artifact,1,files-count,0 + 2018/10/19 09:50:03 waiting for all plugin processes to complete... + 1539967803,amazon-ebs,artifact,1,end + ``` + +You'll see these data types when you run `packer version`: + +- `version`: what version of Packer is running + +- `version-prerelease`: Data will contain `dev` if version is prerelease, and + otherwise will be blank. + +- `version-commit`: The git hash for the commit that the branch of Packer is + currently on; most useful for Packer developers. + +## Autocompletion + +The `packer` command features opt-in subcommand autocompletion that you can +enable for your shell with `packer -autocomplete-install`. After doing so, you +can invoke a new shell and use the feature. + +For example, assume a tab is typed at the end of each prompt line: + +```shell-session +$ packer p +plugin build +$ packer build - +-color -debug -except -force -machine-readable -on-error -only -parallel -timestamp -var -var-file +``` diff --git a/content/packer/content/docs/commands/init.mdx b/content/packer/content/docs/commands/init.mdx new file mode 100644 index 0000000000..d99afdf2c2 --- /dev/null +++ b/content/packer/content/docs/commands/init.mdx @@ -0,0 +1,81 @@ +--- +description: | + The `packer init` command downloads and installs the plugins specified in a Packer template. +page_title: packer init command reference +--- + +# `packer init` command reference + + + +Starting August 1st, 2025, the source for many official HashiCorp-maintained Packer plugins is moving from GitHub releases to the official HashiCorp release site, [releases.hashicorp.com](https://releases.hashicorp.com). Refer to [Install HashiCorp-maintained plugins](/packer/docs/plugins/install#install-hashicorp-maintained-plugins) for more information. + + + + +The `packer init` command initializes Packer according to an HCL template configuration. Refer to [Installing Plugins](/packer/docs/plugins/install) for additional information about installing plugins. + +## Description + +Use the `packer init` command to download and install plugins according to the `required_plugins` block in Packer templates written in HCL. Refer to [Specifying plugin requirements](/packer/docs/templates/hcl_templates/blocks/packer#specifying-plugin-requirements) in the template configuration reference for additional information about configuring the `required_plugins` block. + +Legacy JSON templates are not supported. You can convert your JSON template files to HCL using the [hcl2_upgrade](/packer/docs/commands/hcl2_upgrade) command. + +We recommend running the `packer init` command as the first step when working with a new or existing template. You can run the command multiple times. Subsequent runs may produce errors, but the command never deletes already-installed plugins. + +### Third-party plugin verification + +We recommend that you vet and verify any third-party plugins you want to install. + +### Installation location + +By default, Packer installs plugins into the plugins directory at `$HOME/.config/packer/plugins` on Unix and `%APPDATA%\packer.d\plugins` on Windows, but you can specify a different directory using the `PACKER_PLUGIN_PATH` environment variable. Refer to the [Packer configuration reference](/packer/docs/configure) for additional information. + +## Usage + +Use the following syntax to run the `packer init` command: + +```shell-session +$ packer init +``` +The command will process any template file that ends with `pkr.hcl`. + +The template must contain all dependencies when running the command on a single template file. The command fails if the template is intended to be built as a bundle of partials. + +For variable definitions, it is recommended to use the extensions `.pkrvars.hcl` or `.auto.pkrvars.hcl`. When you run `packer init` in the directory, these variable definition files will be automatically excluded from processing. + +## Examples + +The following example installs the plugins specified in a template from the current directory: + +```shell-session +$ packer init . +``` + +The following example installs the plugins specified in a template named `template.pkr.hcl` from the current directory: +```shell-session +$ packer init template.pkr.hcl +``` + +The following example installs the plugins specified in the `builds/foo/` directory: + +```shell-session +$ packer init builds/foo/. +``` + +The following example installs the plugins specified in a template from the `builds/foo/template.pkr.hcl` path: + +```shell-session +$ packer init builds/foo/template.pkr.hcl +``` + +## Arguments + +You can pass the following arguments: + +- Packer template: Specify the path to either an HCL2 template or a directory containing at least one valid HCL2 template and related dependencies. + +## Options + +- `-upgrade`: Use this option to upgrade plugins that are already installed to the latest available version. Packer upgrades to the latest version in accordance with the version constraints specified in the template. +- `-force`: Use this option to force Packer to reinstall plugins. diff --git a/content/packer/content/docs/commands/inspect.mdx b/content/packer/content/docs/commands/inspect.mdx new file mode 100644 index 0000000000..7378a169ad --- /dev/null +++ b/content/packer/content/docs/commands/inspect.mdx @@ -0,0 +1,53 @@ +--- +description: > + The `packer inspect` command outputs the variables, builders, and provisioners a template uses so you can review the template without reading the HCL. +page_title: packer inspect command reference +--- + +# `packer inspect` command reference + +The `packer inspect` command takes a template and outputs the various +components a template defines. This can help you quickly learn about a template +without having to dive into the HCL itself. The command will tell you things +like what variables a template accepts, the builders it defines, the +provisioners it defines and the order they'll run, and more. + +This command is extra useful when used with [machine-readable +output](/packer/docs/commands) enabled. The command outputs the components +in a way that is parseable by machines. + +The command doesn't validate the actual configuration of the various components +(that is what the `validate` command is for), but it will validate the syntax +of your template by necessity. + +## Example + +Given a basic template, here is an example of what the output might look like: + +```shell-session +$ packer inspect template.pkr.hcl +> input-variables: + +var.aws_access_key: "" +var.aws_secret_key: "" + +> local-variables: + +> builds: + + > : + + sources: + + amazon-ebs.foo + amazon-instance.bar + virtualbox-iso.basic + + provisioners: + + shell + + post-processors: + + +``` diff --git a/content/packer/content/docs/commands/plugins/index.mdx b/content/packer/content/docs/commands/plugins/index.mdx new file mode 100644 index 0000000000..71452fa2e0 --- /dev/null +++ b/content/packer/content/docs/commands/plugins/index.mdx @@ -0,0 +1,29 @@ +--- +description: | + The `packer plugins` command group contains subcommands for managing Packer plugins. +page_title: packer plugins command reference +--- + +# `packer plugins` command reference + +The `plugins` command group contains subcommands for installing, uninstalling, and viewing Packer plugins. + +```shell-session +$ packer plugins -h +Usage: packer plugins [options] [args] + This command groups subcommands for interacting with Packer plugins. + +Related but not under the "plugins" command : + +- "packer init " will install all plugins required by a config. + +Subcommands: + install Install latest Packer plugin [matching version constraint] + installed List all installed Packer plugin binaries + remove Remove Packer plugins [matching a version] + required List plugins required by a config +``` + +## Related + +- [`packer init`](/packer/docs/commands/init) will install all required plugins. diff --git a/content/packer/content/docs/commands/plugins/install.mdx b/content/packer/content/docs/commands/plugins/install.mdx new file mode 100644 index 0000000000..79079b3de7 --- /dev/null +++ b/content/packer/content/docs/commands/plugins/install.mdx @@ -0,0 +1,62 @@ +--- +description: | + The `packer plugins install` command manually downloads and installs Packer plugins without having to update and initialize a build template. +page_title: packer plugins install command reference +--- + +# `packer plugins install` command reference + + + +Starting August 1st, 2025, the source for many official HashiCorp-maintained Packer plugins is moving from GitHub releases to the official HashiCorp release site, [releases.hashicorp.com](https://releases.hashicorp.com). Refer to [Install HashiCorp-maintained plugins](/packer/docs/plugins/install#install-hashicorp-maintained-plugins) for more information. + + + +The `packer plugins install` command downloads and installs the most recent version of a plugin binary. Refer to [Installing Plugins](/packer/docs/plugins/install) for additional information about installing plugins. + +## Description + +Use the `packer plugins install` command to manually install plugins instead of specifying plugins in a template and initializing Packer. Refer to the [`packer init` command documentation](/packer/docs/commands/init) for information about installing plugins specified in a Packer template during initialization. + +### Third-party plugin verification + +HashiCorp only verifies plugins from GitHub under the`hashicorp/*` namespace. We recommend that you vet and verify any third-party plugins you want to install. + +### Installation directory + +By default, Packer installs plugins into the plugins directory at `$HOME/.config/packer/plugins` on Unix and `%APPDATA%\packer.d\plugins` on Windows, but you can specify a different directory using the `PACKER_PLUGIN_PATH environment variable. +Plugin installation requires access to temporary files under `TMPDIR`. If the system's temp directory is non-writable or non-executable, use TMPDIR to override the location of the temporary file store used by Packer. +Refer to the [Packer configuration reference](/packer/docs/configure) for additional information. + +## Usage + +Use the following syntax to run the `packer plugins install` command: + +## Examples + +Specify a version number to install a specific version of the plugin. The following example installs the version `1.0.1` of `my-plugin` from a remote source: + +```shell-session +$ packer plugins install github.com/hashicorp/my-plugin 1.0.1 +``` + +Use the `--path` option to specify a local plugin binary. The following example installs `my-plugin` from a local binary: + +```shell-session +$ packer plugins install --path my-plugin github.com/hashicorp/my-plugin +``` + +## Arguments + +You can pass the following arguments: + +- ``: A remote source where the plugin binary is available. To remotely download from a source, the plugin must be hosted on GitHub. For locally-sourced binaries, you must also include the `--path` option. Refer to [Installing Plugins](/packer/docs/plugins/install) for additional information about installing plugins. +- ``: A specific version of the plugin to download and install. Packer accepts semantic version numbers that follow the `..` format. If omitted, the command installs the latest plugin version. + +## Options + +You can use the following options: + +`--path`: Use this option to specify a local plugin binary. You must use this option to install plugins from a local source. +`--force`: Use this option to force Packer to reinstall the plugin. + diff --git a/content/packer/content/docs/commands/plugins/installed.mdx b/content/packer/content/docs/commands/plugins/installed.mdx new file mode 100644 index 0000000000..7a8b09e81c --- /dev/null +++ b/content/packer/content/docs/commands/plugins/installed.mdx @@ -0,0 +1,21 @@ +--- +description: | + The `packer plugins installed` command lists the installed Packer plugins. +page_title: packer plugins installed command reference +--- + +# `packer plugins installed` command reference + +The `plugins installed` subcommand lists installed Packer plugins. + +```shell-session +$ packer plugins installed -h +Usage: packer plugins installed + + This command lists all installed plugin binaries that match with the current + OS and architecture. Packer's API version will be ignored. +``` + +## Related + +- [`packer init`](/packer/docs/commands/init) will install all required plugins. diff --git a/content/packer/content/docs/commands/plugins/remove.mdx b/content/packer/content/docs/commands/plugins/remove.mdx new file mode 100644 index 0000000000..0c7bcf3a81 --- /dev/null +++ b/content/packer/content/docs/commands/plugins/remove.mdx @@ -0,0 +1,45 @@ +--- +description: | + The `packer plugins remove` command removes one or more versions of an installed Packer plugin. +page_title: packer plugin remove command reference +--- + +# `packer plugins remove` command reference + +The `packer plugins remove` subcommand removes one or more versions of an installed Packer plugin. + +The command is flexible enough to remove all versions at once or just a single version at a time. +- If a plugin's source address is specified, without a version constraint, all the versions of that plugin will be removed. +- If a version constraint is specified, only the specified version will be removed. +- If a direct path to a plugin is specified, only the specified version will be removed. + +Refer to the examples below for details on usage. +```shell-session +To remove a plugin matching a version constraint for the current OS and architecture. + + packer plugins remove github.com/hashicorp/happycloud v1.2.3 + +To remove all versions of a plugin for the current OS and architecture omit the version constraint. + + packer plugins remove github.com/hashicorp/happycloud + +To remove a single plugin binary from the Packer plugin directory specify the absolute path to an installed binary. This syntax does not allow for version matching. + + packer plugins remove ~/.config/plugins/github.com/hashicorp/happycloud/packer-plugin-happycloud_v1.0.0_x5.0_linux_amd64 +``` + +## Remove all installed plugins +The `plugins remove` command can be used in conjunction with the `plugins installed` command to remove all +Packer plugins by piping the results of `plugins installed` to `plugins removed`. + +On a Unix based OS with xargs installed you can remove all plugin versions using the following command: +```shell-session +~> packer plugins installed | xargs -n1 packer plugins remove +~/.config/packer/plugins/github.com/hashicorp/amazon/packer-plugin-amazon_v1.3.2_x5.0_darwin_arm64 +~/.config/packer/plugins/github.com/hashicorp/azure/packer-plugin-azure_v2.1.3_x5.0_darwin_arm64 +``` + +## Related + +- [`packer init`](/packer/docs/commands/init) will install all required plugins. +- [`packer plugins install`](/packer/docs/commands/plugins/install) will install a single plugin. diff --git a/content/packer/content/docs/commands/plugins/required.mdx b/content/packer/content/docs/commands/plugins/required.mdx new file mode 100644 index 0000000000..0842336246 --- /dev/null +++ b/content/packer/content/docs/commands/plugins/required.mdx @@ -0,0 +1,30 @@ +--- +description: | + The `packer plugins required` command lists all of the plugins required by a Packer configuration and any installed binaries that satisfy the requirements. +page_title: packer plugins required command reference +--- + +# `packer plugins required` command reference + +The `plugins required` command lists all plugins required by a Packer configuration and +all the installed binaries that match the constraint. The first binary +is the most up-to-date installed version and will be the one picked by Packer in a build. + +```shell-session +$ packer plugins required -h +Usage: packer plugins required + + This command will list every Packer plugin required by a Packer config, in + packer.required_plugins blocks. All binaries matching the required version + constrain and the current OS and Architecture will be listed. The most recent + version (and the first of the list) will be the one picked by Packer during a + build. + + + Ex: packer plugins required require.pkr.hcl + Ex: packer plugins required path/to/folder/ +``` + +## Related + +- [`packer init`](/packer/docs/commands/init) will install all required plugins. \ No newline at end of file diff --git a/content/packer/content/docs/commands/validate.mdx b/content/packer/content/docs/commands/validate.mdx new file mode 100644 index 0000000000..1eb1f057c2 --- /dev/null +++ b/content/packer/content/docs/commands/validate.mdx @@ -0,0 +1,71 @@ +--- +description: | + The `packer validate` command validates the syntax and + configuration in a Packer template, helping you prevent errors. +page_title: packer validate command reference +--- + +# `packer validate` command reference + +The `packer validate` Packer command is used to validate the syntax and +configuration of a [template](/packer/docs/templates). The command will +return a zero exit status on success, and a non-zero exit status on failure. +Additionally, if a template doesn't validate, any error messages will be +outputted. + +Example usage: + +```shell-session +$ packer validate my-template.pkr.hcl +Template validation failed. Errors are shown below. + +Errors validating build 'vmware'. 1 error(s) occurred: + +* Either a path or inline script must be specified. +``` + +## Options + +- `-syntax-only` - Only the syntax of the template is checked. The + configuration is not validated. + +- `-evaluate-datasources` - Evaluate all data sources when validating a template. + This is only valid on HCL2 templates, since JSON templates do not feature + datasources, this option will be ignored. + + ~> **Warning:** Data sources may rely on external services for fetching data, + which can incur some costs at validation if the services being contacted are + billing per operation. + +- `-except=foo,bar,baz` - Validates all the builds except those with the + comma-separated names. In legacy JSON templates, build names default to the + types of their builders (e.g. `docker` or + `amazon-ebs` or `virtualbox-iso`), unless a specific `name` attribute is + specified within the configuration. In HCL2 templates, the "name" is the + source block's "name" label, unless an in-build source definition adds the + "name" configuration option. + +- `-no-warn-undeclared-var` - Silence warnings when the variable definition + file contains variable assignments for undeclared variables. This can occur + when using a var-file that contains a large amount of unused variables for a + given HCL2 template. For HCL2 template defining a value for a variable in a + var-file is not enough on its own for Packer to function, as there also needs + to be a variable block definition in the template files `pkr.hcl` for the + variable. By default `packer validate` will warn when a var-file contains one + or more undeclared variables. + +- `-only=foo,bar,baz` - Only validate the builds with the given comma-separated + names. In legacy JSON templates, build names default to the + types of their builders (e.g. `docker` or + `amazon-ebs` or `virtualbox-iso`), unless a specific `name` attribute is + specified within the configuration. In HCL2 templates, the "name" is the + source block's "name" label, unless an in-build source definition adds the + "name" configuration option. + +- `-machine-readable` Sets all output to become machine-readable on stdout. + Logging, if enabled, continues to appear on stderr. + +- `-var` - Set a variable in your Packer template. This option can be used + multiple times. This is useful for setting version numbers for your build. + +- `-var-file` - Set template variables from a file. diff --git a/content/packer/content/docs/communicators/index.mdx b/content/packer/content/docs/communicators/index.mdx new file mode 100644 index 0000000000..600c35039e --- /dev/null +++ b/content/packer/content/docs/communicators/index.mdx @@ -0,0 +1,25 @@ +--- +description: | + Communicators are remote-access programs Packer uses to perform actions, such as upload files and execute scripts, on the machines Packer creates. +page_title: Communicators overview +--- + +# Communicators overview + +Communicators are the mechanism Packer uses to upload files, execute scripts, +etc. with the machine being created. + +Communicators are configured within the +[builder](/packer/docs/templates/legacy_json_templates/builders) section. Packer currently supports +three kinds of communicators: + +- `none`: Packer does not use a communicator and cannot use most provisioners. +- [`ssh`](/packer/docs/communicators/ssh): Packer establishes an SSH connection to the machine. This is usually the default. +- [`winrm`](/packer/docs/communicators/winrm): Packer establishes a WinRM connection. + +In addition to the above, some builders have custom communicators they can use. +For example, the Docker builder has a "docker" communicator that uses +`docker exec` and `docker cp` to execute scripts and copy files. + +For more details on how to use each communicator, click the links above to be +taken to each communicator's page. diff --git a/content/packer/content/docs/communicators/ssh.mdx b/content/packer/content/docs/communicators/ssh.mdx new file mode 100644 index 0000000000..c4e003bdd1 --- /dev/null +++ b/content/packer/content/docs/communicators/ssh.mdx @@ -0,0 +1,93 @@ +--- +description: | + The `ssh` communicator establishes an SSH connection so that Packer can perform actions, such as upload files and execute scripts, on the machine it creates. +page_title: Establish an SSH connection +--- + +# Establish an SSH Connection + +This topic describes how to use the `ssh` communicator to establish an SSH connection to the machine it creates so that Packer can perform actions, such as upload files and execute scripts. + +## Introduction + +Communicators are the mechanism Packer uses to upload files, execute scripts, +etc. on the machine being created, and are configured within the +[builder](/packer/docs/templates/legacy_json_templates/builders) section. + +The `ssh` communicator does this by using the SSH protocol. It is the default +communicator for a majority of builders. + +If you have an SSH agent configured on the host running Packer, and SSH agent +authentication is enabled in the communicator config, Packer will automatically +forward the SSH agent to the remote host. + +## Getting Ready to Use the `ssh` Communicator + +The `ssh` communicator is the default communicator for a majority of builders, but +depending on your builder it may not work "out of the box". + +If you are building from a cloud image (for example, building on Amazon), there +is a good chance that your cloud provider has already preconfigured SSH on the +image for you, meaning that all you have to do is configure the communicator in +the Packer template. + +However, if you are building from a brand-new and unconfigured operating system +image, you will almost always have to perform some extra work to configure SSH +on the guest machine. For most operating system distributions, this work will +be performed by a [boot command](/packer/plugins/builders/vmware/iso#boot-configuration) +that references a file which provides answers to the normally-interactive +questions you get asked when installing an operating system. The name of this +file varies by operating system; some common examples are the "preseed" file +required by Debian, the "kickstart" file required by CentOS or the +"answer file", also known as the Autounattend.xml file, required by Windows. +For simplicity's sake, we'll refer to this file as the "preseed" file in the +rest of the documentation. + +If you are unfamiliar with how to use a preseed file for automatic +bootstrapping of an image, please either take a look at our +[quick guides](/packer/guides/automatic-operating-system-installs) to +image bootstrapping, or research automatic configuration for your specific +guest operating system. Knowing how to automatically initalize your operating +system is critical for being able to successfully use Packer. + +## `ssh` Communicator Reference + +The `ssh` communicator connects to the host via SSH. If you have an SSH agent +configured on the host running Packer, and SSH agent authentication is enabled +in the communicator config, Packer will automatically forward the SSH agent to +the remote host. + +The `ssh` communicator has the following options: + +@include "packer-plugin-sdk/communicator/SSH-not-required.mdx" + +@include "packer-plugin-sdk/communicator/SSH-Key-Pair-Name-not-required.mdx" + +@include "packer-plugin-sdk/communicator/SSH-Agent-Auth-not-required.mdx" + +@include "packer-plugin-sdk/communicator/SSH-Temporary-Key-Pair-not-required.mdx" + +@include "packer-plugin-sdk/communicator/SSH-Private-Key-File-not-required.mdx" + +~> Note: The options `ssh_keypair_name`, `ssh_agent_auth`, +`temporary_key_pair_name`, and `ssh_private_key_file` are supported by the +communicator; however, they may not be supported for every builder. Please refer +to the builder documentation for supported options. + +### `ssh` Communicator Details + +Packer will only use one authentication method, either `publickey` or if +`ssh_password` is used Packer will offer `password` and `keyboard-interactive` +both sending the password. In other words Packer will not work with _sshd_ +configured with more than one configured authentication method using +`AuthenticationMethods`. + +Packer supports the following MACs: + +- hmac-sha1 +- hmac-sha1-96 +- hmac-sha2-256 +- `hmac-sha2-256-etm@openssh.com` + +For more information on the ciphers that Packer supports, check the docs for +the [ssh_ciphers](/packer/docs/communicators/ssh#ssh_ciphers) template option. \ No newline at end of file diff --git a/content/packer/content/docs/communicators/winrm.mdx b/content/packer/content/docs/communicators/winrm.mdx new file mode 100644 index 0000000000..27a5fff951 --- /dev/null +++ b/content/packer/content/docs/communicators/winrm.mdx @@ -0,0 +1,200 @@ +--- +description: | + The `winrm` communicator establishes a WinRM connection, letting Packer upload files or execute scripts on the machine it creates. +page_title: Establish a WinRM Connection +--- + +# Establish a WinRM Connection + +This topic describes how to use the `winrm` communicator to establish a WinRM connection to the machine it creates so that Packer can perform actions, such as upload files and execute scripts. + +## Introduction + +Communicators are the mechanism Packer uses to upload files, execute scripts, +etc. with the machine being created. The `winrm` communicator uses the +Windows Remote Management protocol to do this. + +## Getting Ready to Use the `winrm` Communicator + +The `winrm` communicator is not the default communicator, so you will always have +to set the `"communicator": "winrm",` template option explicitly. In addition, +you will almost always have to provide a pre-run script that enables and +configures WinRM on the guest machine. This will generally be in the form of a +PowerShell script or a batch file. + +If you are building from a brand-new and unconfigured operating system +image, you will need to provide this pre-run script as part of your +Autounattend.xml file, required by Windows for automatic operating system +installation. If you are building in a cloud or from a pre-installed image, your +method for providing this pre-run script will vary based on the builder. Please +refer to each builder's documentation for more information on how to supply the +winrm configuration script. + +If you are unfamiliar with how to use an autounattend file, take a look at our +[quick guides](/packer/guides/automatic-operating-system-installs); knowing +how to automatically initalize your operating system is critical for being able +to successfully use Packer to build from an iso. + +## `winrm` Communicator Options + +@include "packer-plugin-sdk/communicator/WinRM-not-required.mdx" + +## Examples + +### Basics of WinRM Connection + +Please note that WinRM is not a Packer-specific protocol. Microsoft has a great +deal of documentation about WinRM. If you find after reading this guide that +you are still not able to connect via WinRM, check the +[Microsoft documentation](https://docs.microsoft.com/en-us/windows/win32/winrm/installation-and-configuration-for-windows-remote-management) +to make sure there isn't anything you're missing. + +There are some steps that you will normally need to take in order for Packer +to be able to connect via WinRM + +1. Set up a username and password that Packer to connect with. +2. Make any necesary registry edits to enable remote execution + (and remote execution with elevated privileges, if needed) +3. Start WinRM, setting any config needed for allowing basic auth +4. Open ports 5985 and/or 5986 depending on how you're connecting +5. launch WinRM and set it to automatically launch when the computer restarts +6. If necessary, generate a self-signed certificate or provide a real certificate + to the WinRM listener. + +#### Configuring WinRM in VMware + +If you are configuring WinRM using an Autounattend.xml, the simplest way to set +up WinRM is to put the configuration commands directly into the Autounattend +file as shown [here](https://github.com/StefanScherer/packer-windows/blob/6e603e904e9b280eeb97f7eb542940a043954112/answer_files/2008_r2_core/Autounattend.xml#L157-L234) + +Instead of entering each line individually, you can also add a batch file to +your autounattend that contains the commands for configuring winrm. Depending +on your winrm setup, this could be a complex batch file, or a very simple one. + +Below is an example of how we would call a batch file from inside the +Autounattend file. + +```xml + + ... + + cmd.exe /c a:\winrmConfig.bat + Configure WinRM + 3 + true + + ... + +``` + +It is also possible to call PowerShell scripts in a similar manner. + +The winrmConfig.bat referenced above can be as simple as + +```powershell +rem basic config for winrm +cmd.exe /c winrm quickconfig -q + +rem allow unencrypted traffic, and configure auth to use basic username/password auth +cmd.exe /c winrm set winrm/config/service @{AllowUnencrypted="true"} +cmd.exe /c winrm set winrm/config/service/auth @{Basic="true"} + +rem update firewall rules to open the right port and to allow remote administration +cmd.exe /c netsh advfirewall firewall set rule group="remote administration" new enable=yes + +rem restart winrm +cmd.exe /c net stop winrm +cmd.exe /c net start winrm +``` + +Please note that the above batch file is _extremely_ simplistic, and not secure. +It is intended to be an example of the bare minimum configuration. Below, you'll +find a more complicated example of a more secure WinRM configuration process. + +This batch file will only work for HTTP connections, not HTTPS, but will enable +you to connect using only the username and password created earlier in the +Autounattend file. The above batchfile will allow you to connect using a very +simple Packer config: + +```json + "communicator": "winrm", + "winrm_username": "packeruser", + "winrm_password": "SecretPassword" +``` + +A more complex example of a PowerShell script used for configuration can be seen +below. + +```powershell +# A Packer config that works with this example would be: +# +# +# "winrm_username": "Administrator", +# "winrm_password": "SuperS3cr3t!!!", +# "winrm_insecure": true, +# "winrm_use_ssl": true +# +# + +# Create username and password +net user Administrator SuperS3cr3t!!! +wmic useraccount where "name='Administrator'" set PasswordExpires=FALSE + +Set-ExecutionPolicy Unrestricted -Scope LocalMachine -Force -ErrorAction Ignore + +# Don't set this before Set-ExecutionPolicy as it throws an error +$ErrorActionPreference = "stop" + +# Remove HTTP listener +Remove-Item -Path WSMan:\Localhost\listener\listener* -Recurse + +# Create a self-signed certificate to let ssl work +$Cert = New-SelfSignedCertificate -CertstoreLocation Cert:\LocalMachine\My -DnsName "packer" +New-Item -Path WSMan:\LocalHost\Listener -Transport HTTPS -Address * -CertificateThumbPrint $Cert.Thumbprint -Force + +# WinRM +write-output "Setting up WinRM" +write-host "(host) setting up WinRM" + +# Configure WinRM to allow unencrypted communication, and provide the +# self-signed cert to the WinRM listener. +cmd.exe /c winrm quickconfig -q +cmd.exe /c winrm set "winrm/config/service" '@{AllowUnencrypted="true"}' +cmd.exe /c winrm set "winrm/config/client" '@{AllowUnencrypted="true"}' +cmd.exe /c winrm set "winrm/config/service/auth" '@{Basic="true"}' +cmd.exe /c winrm set "winrm/config/client/auth" '@{Basic="true"}' +cmd.exe /c winrm set "winrm/config/service/auth" '@{CredSSP="true"}' +cmd.exe /c winrm set "winrm/config/listener?Address=*+Transport=HTTPS" "@{Port=`"5986`";Hostname=`"packer`";CertificateThumbprint=`"$($Cert.Thumbprint)`"}" + +# Make sure appropriate firewall port openings exist +cmd.exe /c netsh advfirewall firewall set rule group="remote administration" new enable=yes +cmd.exe /c netsh advfirewall firewall add rule name="Port 5986" dir=in action=allow protocol=TCP localport=5986 profile=any + +# Restart WinRM, and set it so that it auto-launches on startup. +cmd.exe /c net stop winrm +cmd.exe /c sc config winrm start= auto +cmd.exe /c net start winrm +``` + +Please note that having WinRM auto-launch on all start ups may not be the right +choice for you, if you don't need the server to recieve WinRM connections in the +future. Clean up after yourself and close unnecesary firewall ports at a final +provisioning step to make sure your image is secure. + +#### Configuring WinRM in the Cloud + +Most clouds allow you to provide a configuration script that runs when the +instance is launched. In AWS, this is the +[user_data_file](/packer/plugins/builders/amazon/ebs#user_data_file). In Google +Cloud, this is provided using the `windows-startup-script-cmd` +[metadata](/packer/plugins/builders/googlecompute#metadata) tag. +[Example](/packer/plugins/builders/googlecompute#windows-example) + +Essentially, these files are powershell or cmd scripts that configure winrm, +without having to be wrapped in an Autounattend. Provide the script in the +format requested by each cloud, and make sure you manually configure any +firewall rules that the cloud doesn't allow you to manage internally. More +specific details for each cloud can be found in the builder sections. + +The above examples will work in cloud prep too, but may be overkill depending on +how much preconfiguration the cloud has done for you. diff --git a/content/packer/content/docs/community-plugins.mdx b/content/packer/content/docs/community-plugins.mdx new file mode 100644 index 0000000000..43a427c3f9 --- /dev/null +++ b/content/packer/content/docs/community-plugins.mdx @@ -0,0 +1,42 @@ +--- +page_title: Community vs HashiCorp Maintained Plugins +description: Packer maintains these core plugins. +--- + +# HashiCorp Maintained Plugins + +The following plugins (i.e. Builders, Provisioners, and Post-Processors) are +maintained by HashiCorp. Any plugins not on this list are maintained by the +community, and not actively contributed to by HashiCorp, although they are +still distributed with Packer. If you are interested in seeing features or +bugfixes to these plugins, please consider making a pull request, or asking the +HashiCorp maintainers for advice on how to get started contributing. + +## Builders + +- Amazon EC2 +- Azure +- Docker +- Google Cloud +- VMware +- VirtualBox + +## Provisioners + +- File +- HCP SBOM +- InSpec +- PowerShell +- Shell +- Windows Restart +- Windows Shell + +## Post-Processors + +- Amazon Import +- Artifice +- Docker +- Local Shell +- Manifest +- Vagrant +- Vagrant Cloud diff --git a/content/packer/content/docs/community-tools.mdx b/content/packer/content/docs/community-tools.mdx new file mode 100644 index 0000000000..edb2f46683 --- /dev/null +++ b/content/packer/content/docs/community-tools.mdx @@ -0,0 +1,76 @@ +--- +page_title: Download Packer Community Projects +description: >- + Packer community contributors have built many Packer tools and projects that demonstrate Packer templates. Learn about Packer community projects you can download. +--- + +# Download Packer Community Projects + +Packer has a vibrant community of contributors who have built a number of great +tools on top of Packer. There are also quite a few projects demonstrating the +power of Packer templates. + +## Third-Party plugins + +To learn more about how to use community plugins, or how to build your own, +refer to the docs on [extending Packer](/packer/docs/plugins/install) + +If you have built a plugin and would like to add it to this community list, +make a pull request so that we can document your contribution. + +Refer to the following topics to learn about plugins built by the community of Packer users and +vendors. We do not test or maintain community plugins. + + +- [Community Builders](/packer/docs/builders/community-supported) +- [Community Provisioners](/packer/docs/provisioners/community-supported) +- [Community Post-Processors](/packer/docs/post-processors/community-supported) + + +## Templates + +- [bento](https://github.com/chef/bento) - Packer templates for building minimal + Vagrant base boxes + +- [boxcutter](https://github.com/boxcutter) - Community-driven templates and + tools for creating cloud, virtual machines, containers and metal operating + system environments + +- [cbednarski/packer-ubuntu](https://github.com/cbednarski/packer-ubuntu) - + Ubuntu LTS Virtual Machines for Vagrant + +- [geerlingguy/packer-ubuntu-1604](https://github.com/geerlingguy/packer-ubuntu-1604) + \- Ubuntu 16.04 minimal Vagrant Box using Ansible provisioner + +- [jakobadam/packer-qemu-templates](https://github.com/jakobadam/packer-qemu-templates) + \- QEMU templates for various operating systems + +- [lucidone/baseliner](https://git.sr.ht/~lucidone/baseliner) - Example of using + QEMU + Ansible with Packer + +- [packer-build](https://github.com/tylert/packer-build) - Build fresh Debian + and Ubuntu virtual machine images for Vagrant, VirtualBox and QEMU + +- [packer-windows](https://github.com/joefitzgerald/packer-windows) - Windows + Packer Templates + +- [packer-baseboxes](https://github.com/taliesins/packer-baseboxes) - Templates + for Packer to build base boxes + +- [vmware/packer-examples-for-vsphere](https://github.com/vmware/packer-examples-for-vsphere) - Examples + to automate the creation of virtual machine images and their guest operating systems on VMware vSphere using HashiCorp Packer + and the Packer Plugin for VMware vSphere. + +- [Parallels/packer-examples](https://github.com/parallels/packer-examples) - Examples in how to use Packer with Parallels Desktop + to automate the creation of virtual machine images and their guest operating systems on macOS, windows and linux. + +## Wrappers + +- [packer-config](https://github.com/ianchesal/packer-config) - a Ruby model that lets you build Packer configurations in Ruby +- [packerlicious](https://github.com/mayn/packerlicious) - a Python library for generating Packer templates +- [packer.py](https://github.com/mayn/packer.py) - a Python library for executing Packer CLI commands +- [racker](https://github.com/aspring/racker) - an opinionated Ruby DSL for generating Packer templates + +## Other + +- [suitcase](https://github.com/tmclaugh/suitcase) - Packer based build system for CentOS OS images diff --git a/content/packer/content/docs/configure.mdx b/content/packer/content/docs/configure.mdx new file mode 100644 index 0000000000..8c8672714d --- /dev/null +++ b/content/packer/content/docs/configure.mdx @@ -0,0 +1,132 @@ +--- +description: | + Learn how to configure Packer's global behavior using environment variables. +page_title: Configure Packer +--- + +# Configure Packer + +This topic describes how to configure Packer. The default configurations are suitable for learning how to get started, but you should refer to this topic for guidance as you become more familiar with Packer. + +## Overview + +You can set Packer-specific environment variables in a configuration file to configure Packer behavior globally. Install configuration files in the Packer configuration directory to implement the configurations. You can also install plugin files and their SHA256SUM files to the plugins subdirectory under the configuration directory. Refer to [Installing Plugins](/packer/docs/plugins/install) for instructions. + +You can also define Packer settings in a JSON configuration file and add it to the execution path. This configuration method is deprecated. + +## Install a configuration file + +You can place internal configuration files into Packer's `config` directory. Packer checks the following paths for configuration files: + +| OS | Path | Notes | +| --- | ---| --- | +| Unix | `${HOME}/.config/packer/` | Packer follows the XDG base directory specification by default. As a result, Packer uses `$HOME/.config/packer` as the default directory when the `PACKER_CONFIG_DIR` environment variable is unset or empty. Packer ignores the XDG specification when the `PACKER_CONFIG_DIR` variable is set or when the installation uses legacy `.packer.d`-style directories, for example `$HOME/.packer.d/`. Refer to the [XDG specification](https://specifications.freedesktop.org/basedir-spec/latest/) for additional information. | +| Windows | `%APPDATA%\packer.d\` | | + + +### Examples + +- **Unix**: If `$PACKER_CONFIG_DIR` is set to `/home/packer`, then the configuration directory is `/home/packer/.packer.d/`. Packer does not check other values. +- **Windows**: If `PACKER_CONFIG_DIR` is set to `C:/`, then the configuration directory is `C:/packer.d/`. Packer does not check other values. + + + + +## Install a JSON configuration file + +This installation method is deprecated since 1.7. Instead, use the [HCL2 +`required_plugins` block](/packer/docs/templates/hcl_templates/blocks/packer#specifying-plugin-requirements) to configure how Packer installs plugins. + +You use a JSON file to set core Packer settings. Packer checks the following paths for the configuration file: + +| Unix | Windows | +| -------------------------------- | --------------------------------- | +| `${PACKER_CONFIG}` | `%PACKER_CONFIG%` | +| `${HOME}/.packerconfig` | `%APPDATA%\packer.config\` | + + +### JSON configuration file reference + +Use basic JSON to configure the file. You can specify the configuration parameters for the core +configuration file. All parameters are optional. None of these are required, since all have defaults. + +- `plugin_min_port`: Number that specifies the lowest port that Packer can use for communicating with plugins. Packer communicates with plugins over TCP or Unix sockets on your local host. Default is `10000`. We recommend setting a wide range between `plugin_min_port` and `plugin_max_port` so that Packer has access to at least 25 ports on a single run. +- `plugin_max_port`: Number that specifies highest port that Packer can for communicating with plugins. Packer communicates with plugins over TCP connections on your local Unix host. Default is `25000`. We recommend setting a wide range between `plugin_min_port` and `plugin_max_port` so that Packer has access to at least 25 ports on a single run. + +The [`packer init`](/packer/docs/commands/init) command takes precedence over JSON-configure settings when installing plugins. + + +## Configure the cache directory + +Packer uses a cache directory to download large files and for logistics around large file download. By default, Packer caches files in the `.packer_cache` folder of the current directory. Use the `PACKER_CACHE_DIR` environment variable to configure the cache directory. Refer to [`PACKER_CACHE_DIR`](/packer/docs/configure#packer_config_dir) for details. + +We recommend using the same Packer cache directory across your builds if multiple builds perform similar actions. This is to avoid downloading the same large file, such as an ISO, multiple times. + + +## Configuration reference + +Packer uses a variety of environmental variables. A listing and description of +each can be found below: + +- `PACKER_CACHE_DIR` - The location of the Packer cache. This defaults to + `./packer_cache/`. Relative paths can be used. Some plugins can cache large + files like ISOs in the cache dir. + +- `PACKER_CONFIG` - The location of the core configuration file. The format + of the configuration file is basic JSON. See [Packer's Config + file](#packer-s-config-file). + +- `PACKER_CONFIG_DIR` - The location for the home directory of Packer. See + [Packer's home directory](#packer-s-home-directory) for more. + +- `PACKER_GETTER_READ_TIMEOUT` - Override the timeout when a packer plugin + tries to fetch a ISO. The default is `30m`. This is specified as a string with + a duration suffix. The plugin needs to be built on + [v0.6.0](https://github.com/hashicorp/packer-plugin-sdk/releases/tag/v0.6.0) + and above of the SDK to be compatible with this variable. + +- `PACKER_GITHUB_API_TOKEN` - When using Packer init on HCL2 templates, Packer + queries the public API from GitHub which has rate-limits on its APIs. + You can set the `PACKER_GITHUB_API_TOKEN` environment variable + with a GitHub Token to make it higher. + +- `PACKER_LOG` - Setting this to any value other than "" (empty string) or + "0" will enable the logger. See the [debugging + page](/packer/docs/debugging). + +- `PACKER_LOG_PATH` - The location of the log file. Note: `PACKER_LOG` must + be set for any logging to occur. See the [debugging + page](/packer/docs/debugging). + +- `PACKER_NO_COLOR` - Setting this to any value will disable color in the + terminal. + +- `PACKER_PLUGIN_MAX_PORT` - The maximum port that Packer uses for + communication with plugins, since plugin communication happens over TCP + connections on your local host. The default is 25,000. This can also be set + using the Packer's config file, see the [config file configuration + reference](#packer-config-file-configuration-reference) for more. + +- `PACKER_PLUGIN_MIN_PORT` - The minimum port that Packer uses for + communication with plugins, since plugin communication happens over TCP + connections on your local host. The default is 10,000. This can also be set + using the Packer's config file, see the [config file configuration + reference](#packer-config-file-configuration-reference) for more. + +- `PACKER_PLUGIN_PATH` - a PATH variable for finding packer plugins. This takes + precedence over `PACKER_CONFIG_DIR/plugins` for plugin discovery if + defined. Plugin installation requires access to temporary files under + `TMPDIR`. If the system's temp directory is non-writable or non-executable, + refer to `TMPDIR` to override the location of the temporary file store used by + Packer. + +- `CHECKPOINT_DISABLE` - When Packer is invoked it sometimes calls out to + [checkpoint.hashicorp.com](https://checkpoint.hashicorp.com/) to look for + new versions of Packer. If you want to disable this for security or privacy + reasons, you can set this environment variable to `1`. + +- `TMPDIR` (Unix) / `TMP`, `TEMP`, `USERPROFILE` (Windows) - This specifies the + directory for temporary files (defaulting to `/tmp` on Linux/Unix and + `%USERPROFILE%\AppData\Local\Temp` on Windows Vista and later). Customizing + this setting might be necessary for systems where the default temporary + directory is either non-writable or non-executable. diff --git a/content/packer/content/docs/datasources/hcp/hcp-packer-artifact.mdx b/content/packer/content/docs/datasources/hcp/hcp-packer-artifact.mdx new file mode 100644 index 0000000000..f1332f6f70 --- /dev/null +++ b/content/packer/content/docs/datasources/hcp/hcp-packer-artifact.mdx @@ -0,0 +1,102 @@ +--- +description: | + The `hcp-packer-artifact` data source retrieves information about an + artifact from the HCP Packer Registry. Use the information to provide a source artifact to Packer builders. +page_title: hcp-packer-artifact data source reference +--- + + + + + + +# `hcp-packer-artifact` + +The `hcp-packer-artifact` data source retrieves information about an +artifact from the HCP Packer Registry. Use this retrieved information to +provide a source artifact to various Packer builders. + +To get started with HCP Packer, refer to the [HCP Packer documentation](/hcp/docs/packer) or try +the [Get Started with HCP Packer tutorials](/packer/tutorials/hcp-get-started). + +~> **Note:** You will receive an error if you try to reference metadata from a deactivated or deleted registry. +An administrator can manually deactivate or delete a registry, and HCP Packer automatically deactivates registries +with billing issues. Contact [HashiCorp Support](https://support.hashicorp.com/) with questions. + +## Revoked Versions + +If an HCP Packer Version is revoked, the `hcp-packer-version` data source will fail and Packer won't proceed with +the build. Building new artifacts from a revoked artifact is not compliant. +Versions that are scheduled to be revoked will still be considered valid until the revocation date. + +## Basic Example + +Below is a fully functioning example. It stores information about an image artifact, +which can then be parsed and accessed as a variable. + +```hcl +data "hcp-packer-artifact" "example" { + bucket_name = "hardened-ubuntu-16-04" + version_fingerprint = "${data.hcp-packer-version.hardened-source.fingerprint}" + platform = "aws" + region = "us-east-1" +} +``` + +## Full Example + +This data source can be used in conjunction with the hcp-packer-version +data source to retrieve a version fingerprint using a channel. You provide the version fingerprint and channel +name to the version data source, then use the version source inside the +artifact data source, then use the artifact data source inside your source block. + +```hcl +# Retrieves information about the HCP Packer Version; a "version" can be +# thought of as all the metadata created by a single call of `packer build`. +data "hcp-packer-version" "hardened-source" { + bucket_name = "hardened-ubuntu-16-04" + channel_name = "dev" +} + +# Retrieves information about the HCP Packer Artifact; an artifact can be thought +# of as all the metadata (including the artifact names) created by a single +# "source" builder; this can include multiple artifacts so we provide a +# region to disambiguate. +data "hcp-packer-artifact" "example" { + bucket_name = "hardened-ubuntu-16-04" + version_fingerprint = data.hcp-packer-version.hardened-source.fingerprint + platform = "aws" + region = "us-east-1" +} + +# This source uses the output from a previous Packer build. By using the +# HCP Packer Registry in this way, you can easily create build pipelines where +# a single base artifact can be customized in multiple secondary layers. +source "amazon-ebs" "packer-secondary" { + source_ami = data.hcp-packer-artifact.example.external_identifier + ... +} +``` + +## Configuration Reference + +Configuration options are organized below into two categories: required and +optional. Within each category, the available options are alphabetized and +described. + +### Required: + +@include 'datasource/hcp-packer-artifact/Config-required.mdx' + +### Optional: + +~> **Note:** This data source only returns the first found artifact's metadata filtered by the given options, +from the returned list of artifacts associated with the specified version. Therefore, if multiple artifacts exist +in the same region, it will only pick one of them. In this case, you can filter artifact by a source build name +(Ex: `amazon-ebs.example`) using the `component_type` option. + +@include 'datasource/hcp-packer-artifact/Config-not-required.mdx' + +### Output Fields: + +@include 'datasource/hcp-packer-artifact/DatasourceOutput.mdx' diff --git a/content/packer/content/docs/datasources/hcp/hcp-packer-image.mdx b/content/packer/content/docs/datasources/hcp/hcp-packer-image.mdx new file mode 100644 index 0000000000..afa38b2b12 --- /dev/null +++ b/content/packer/content/docs/datasources/hcp/hcp-packer-image.mdx @@ -0,0 +1,95 @@ +--- +description: | + The `hcp-packer-image` data source retrieves information about an image from the HCP Packer registry. This data source is deprecated, use the `hcp-packer-artifact` data source instead. +page_title: hcp-packer-image data source reference +--- + + + + + +# `hcp-packer-image` + +~> **This data source is deprecated**. Use the [`hcp-packer-artifact`](/packer/docs/datasources/hcp/hcp-packer-artifact) data source instead. + +The `hcp-packer-image` data source retrieves information about an +image from the HCP Packer registry. This information can be used to +provide a source image to various Packer builders. + +To get started with HCP Packer, refer to the [HCP Packer documentation](/hcp/docs/packer) or try the [Get Started with HCP Packer tutorials](/packer/tutorials/hcp-get-started). + +~> **Note:** You will receive an error if you try to reference metadata from a deactivated or deleted registry. An administrator can manually deactivate or delete a registry, and HCP Packer automatically deactivates registries with billing issues. Contact [HashiCorp Support](https://support.hashicorp.com/) with questions. + +## Revoked Iterations + +If an iteration is revoked, the `hcp-packer-iteration` data source will fail and Packer won't proceed with the build. Building new images from a revoked image is not compliant. +Iterations that are scheduled to be revoked will still be considered valid until the revocation date. + +## Basic Example + +Below is a fully functioning example. It stores information about an image, +which can then be parsed and accessed as a variable. + +```hcl +data "hcp-packer-image" "example" { + bucket_name = "hardened-ubuntu-16-04" + iteration_id = "${data.hcp-packer-iteration.hardened-source.id}" + cloud_provider = "aws" + region = "us-east-1" +} +``` + +## Full Example + +This data source can be used in conjunction with the hcp-packer-iteration +data source to retrieve an image ID using a channel. You provide the channel +name to the iteration data source, then use the iteration source in the image +data source, then use the image data source inside your source block. + +```hcl +# Retrieves information about the HCP Packer "iteration"; an "iteration" can be +# thought of as all the metadata created by a single call of `packer build`. +data "hcp-packer-iteration" "hardened-source" { + bucket_name = "hardened-ubuntu-16-04" + channel = "packer-test" +} + +# Retrieves information about the HCP Packer "image"; an image can be thought +# of as all the metadata (including the artifact names) created by a single +# "source" builder; this can include multiple images so we provide a cloud +# region to disambiguate. +data "hcp-packer-image" "foo" { + bucket_name = "hardened-ubuntu-16-04" + iteration_id = data.hcp-packer-iteration.hardened-source.id + cloud_provider = "aws" + region = "us-east-1" +} + +# This source uses the output from a previous Packer build. By using the +# HCP Packer registry in this way, you can easily create build pipelines where +# a single base image can be customized in multiple secondary layers. +source "amazon-ebs" "packer-secondary" { + source_ami = data.hcp-packer-image.foo.id + # ... +} +``` + +## Configuration Reference + +Configuration options are organized below into two categories: required and +optional. Within each category, the available options are alphabetized and +described. + +### Required: + +@include 'datasource/hcp-packer-image/Config-required.mdx' + +### Optional: + +~> **Note:** This data source only returns the first found image's metadata filtered by the given options, from the returned list of images associated with the specified iteration. Therefore, if multiple images exist in the same region, it will only pick one of them. In this case, you can filter images by a source build name (Ex: `amazon-ebs.example`) using the `component_type` option. + +@include 'datasource/hcp-packer-image/Config-not-required.mdx' + +### Output Fields: + +@include 'datasource/hcp-packer-image/DatasourceOutput.mdx' diff --git a/content/packer/content/docs/datasources/hcp/hcp-packer-iteration.mdx b/content/packer/content/docs/datasources/hcp/hcp-packer-iteration.mdx new file mode 100644 index 0000000000..b28570bf87 --- /dev/null +++ b/content/packer/content/docs/datasources/hcp/hcp-packer-iteration.mdx @@ -0,0 +1,90 @@ +--- +description: | + The `hcp-packer-iteration` data source retrieves information about an iteration from the HCP Packer registry. This data source is deprecated. Use the `hcpe-packer-verion` data source instead. +page_title: hcp-packer-iteration data source reference +--- + + + + + +# `hcp-packer-iteration` + +~> **This data source is deprecated**. Use the [`hcp-packer-version`](/packer/docs/datasources/hcp/hcp-packer-version) data source instead. + +The `HCP Packer Iteration` Data Source retrieves information about an +iteration from the HCP Packer registry. This information can be used to query +HCP for a source image for various Packer builders. + +To get started with HCP Packer, refer to the [HCP Packer documentation](/hcp/docs/packer) or try the [Get Started with HCP Packer tutorials](/packer/tutorials/hcp-get-started). + +~> **Note:** You will receive an error if you try to reference metadata from a deactivated or deleted registry. An administrator can manually deactivate or delete a registry, and HCP Packer automatically deactivates registries with billing issues. Contact [HashiCorp Support](https://support.hashicorp.com/) with questions. + +## Revoked Iterations + +If an iteration is revoked, the `hcp-packer-iteration` data source will fail and Packer won't proceed with the build. Building new images from a revoked image is not compliant. +Iterations that are scheduled to be revoked will still be considered valid until the revocation date. + +## Basic Example + +Below is a fully functioning example. It stores information about an image +iteration, which can then be accessed as a variable. + +```hcl +data "hcp-packer-iteration" "hardened-source" { + bucket_name = "hardened-ubuntu-16-04" + channel = "packer-test" +} +``` + +## Full Example + +This data source can be used in conjunction with the hcp-packer-image +data source to retrieve an image ID using a channel. You provide the channel +name to the iteration data source, then use the iteration source inside the +image data source, then use the image data source inside your source block. + +```hcl +# Retrieves information about the HCP Packer "iteration"; an "iteration" can be +# thought of as all the metadata created by a single call of `packer build`. +data "hcp-packer-iteration" "hardened-source" { + bucket_name = "hardened-ubuntu-16-04" + channel = "packer-test" +} + +# Retrieves information about the HCP Packer "image"; an image can be thought +# of as all the metadata (including the artifact names) created by a single +# "source" builder; this can include multiple images so we provide a cloud +# region to disambiguate. +data "hcp-packer-image" "foo" { + bucket_name = "hardened-ubuntu-16-04" + iteration_id = data.hcp-packer-iteration.hardened-source.id + cloud_provider = "aws" + region = "us-east-1" +} + +# This source uses the output from a previous Packer build. By using the +# HCP Packer registry in this way, you can easily create build pipelines where +# a single base image can be customized in multiple secondary layers. +source "amazon-ebs" "packer-secondary" { + source_ami = data.hcp-packer-image.foo.id + ... +} +``` + +## Configuration Reference + +Configuration options are organized below into two categories: required and +optional. Within each category, the available options are alphabetized and +described. + +### Required: + +@include 'datasource/hcp-packer-iteration/Config-required.mdx' + +There are currently no optional fields for this datasource, though we intend +to add filtering fields in the future. + +### Output Fields: + +@include 'datasource/hcp-packer-iteration/DatasourceOutput.mdx' diff --git a/content/packer/content/docs/datasources/hcp/hcp-packer-version.mdx b/content/packer/content/docs/datasources/hcp/hcp-packer-version.mdx new file mode 100644 index 0000000000..05a9d4b19f --- /dev/null +++ b/content/packer/content/docs/datasources/hcp/hcp-packer-version.mdx @@ -0,0 +1,93 @@ +--- +description: | + The `hcp-packer-version` data source retrieves information about + an HCP Packer version from the HCP Packer registry. Use this information to + get a source's external identifier. +page_title: hcp-packer-version data source reference +--- + + + + + + +# `hcp-packer-version` + +The `hcp-packer-version` dsata source retrieves information about +an HCP Packer version from the HCP Packer registry. You can use the version information to +query HCP for a source's external identifier so that you can use it in various Packer builders. + +To get started with HCP Packer, refer to the [HCP Packer documentation](/hcp/docs/packer) or try the +[Get Started with HCP Packer tutorials](/packer/tutorials/hcp-get-started). + +Packer prints an error if you try to reference metadata from a deactivated or deleted registry. +An administrator can manually deactivate or delete a registry, and HCP Packer automatically deactivates registries with billing issues. Contact [HashiCorp Support](https://support.hashicorp.com/) with questions. + +## Revoked Versions + +If an HCP Packer Version is revoked, the `hcp-packer-version` data source will fail and Packer won't proceed with +the build. Building new artifacts from a revoked artifact is not compliant. +Versions that are scheduled to be revoked will still be considered valid until the revocation date. + +## Basic Example + +Below is a fully functioning example. It stores information about an HCP Packer Version, which can then be accessed as a variable. + +```hcl +data "hcp-packer-version" "hardened-source" { + bucket_name = "hardened-ubuntu-16-04" + channel_name = "dev" +} +``` + +## Full Example + +This data source can be used in conjunction with the `hcp-packer-artifact` +data source to retrieve an artifact identifier. You provide the version fingerprint and channel +name to the version data source, then use the version source inside the +artifact data source, then use the artifact data source inside your source block. + +```hcl +# Retrieves information about the HCP Packer Version; a "version" can be +# thought of as all the metadata created by a single call of `packer build`. +data "hcp-packer-version" "hardened-source" { + bucket_name = "hardened-ubuntu-16-04" + channel_name = "dev" +} + +# Retrieves information about the HCP Packer Artifact; an artifact can be thought +# of as all the metadata (including the artifact names) created by a single +# "source" builder; this can include multiple artifacts so we provide a +# region to disambiguate. +data "hcp-packer-artifact" "example" { + bucket_name = "hardened-ubuntu-16-04" + version_fingerprint = data.hcp-packer-version.hardened-source.fingerprint + platform = "aws" + region = "us-east-1" +} + +# This source uses the output from a previous Packer build. By using the +# HCP Packer Registry in this way, you can easily create build pipelines where +# a single base artifact can be customized in multiple secondary layers. +source "amazon-ebs" "packer-secondary" { + source_ami = data.hcp-packer-artifact.example.external_identifier + ... +} +``` + +## Configuration Reference + +Configuration options are organized below into two categories: required and +optional. Within each category, the available options are alphabetized and +described. + +### Required: + +@include 'datasource/hcp-packer-version/Config-required.mdx' + +There are currently no optional fields for this datasource, though we intend +to add filtering fields in the future. + +### Output Fields: + +@include 'datasource/hcp-packer-version/DatasourceOutput.mdx' diff --git a/content/packer/content/docs/datasources/hcp/index.mdx b/content/packer/content/docs/datasources/hcp/index.mdx new file mode 100644 index 0000000000..74df7791bd --- /dev/null +++ b/content/packer/content/docs/datasources/hcp/index.mdx @@ -0,0 +1,39 @@ +--- +description: | + HCP Packer data sources query data stored to the HCP Packer registry. Use the data sources when your artifact metadata is centralized in HCP. +page_title: HCP Packer Registry Data sources overview +sidebar_title: Overview +--- + + + + + +# HCP Packer Registry data sources overview + +The HCP Packer Registry connects artifact factories and artifact +deployments so that development and security teams can work together to create, +manage, and consume artifacts in a centralized way. + +## Introduction + +The HCP Packer Registry stores metadata about your artifacts, including when they +were created, where the artifacts exists in the cloud, and git commit +information associated with your build. You can use the registry to track +information about the artifacts your Packer builds produce, clearly +designate which artifacts are appropriate for test and production environments, +and query for the right artifacts to use in both Packer and Terraform +configurations. + +The following Packer data sources work together to determine and retrieve information from the +HCP Packer registry: + +- [hcp-packer-version](/packer/docs/datasources/hcp/hcp-packer-version): Retrieves information about an HCP Packer version in the HCP Packer registry. +- [hcp-packer-artifact](/packer/docs/datasources/hcp/hcp-packer-artifact): Retrieves +information about a specific artifact created in the HCP Packer registry. + +The following data source types are deprecated since v1.10.1: + +- [hcp-packer-iteration](/packer/docs/datasources/hcp/hcp-packer-iteration): Retrieves information about an iteration in HCP Packer registry. This data source type is deprecated. Use `hcp-packer-version` instead. +- [hcp-packer-image](/packer/docs/datasources/hcp/hcp-packer-image): Retrieves + information about a specific image created in the HCP Packer registry. This data source type is deprecated. Use `hcp-packer-artifact` instead. \ No newline at end of file diff --git a/content/packer/content/docs/datasources/http.mdx b/content/packer/content/docs/datasources/http.mdx new file mode 100644 index 0000000000..3357d749ae --- /dev/null +++ b/content/packer/content/docs/datasources/http.mdx @@ -0,0 +1,47 @@ +--- +description: | + The `http` data source makes an HTTP `GET` request to the specified URL and exports information about the response. +page_title: http data source reference +--- + + + + + + +# `http` + +The `http` data source makes an HTTP `GET` request to the specified URL and exports information about the response. + + +## Basic Example + +```hcl +data "http" "example" { + url = "https://checkpoint-api.hashicorp.com/v1/check/terraform" + + # Optional request headers + request_headers = { + Accept = "application/json" + } +} +``` + +## Configuration Reference + +Configuration options are organized below into two categories: required and +optional. Within each category, the available options are alphabetized and +described. + +### Required: + +@include 'datasource/http/Config-required.mdx' + +### Not Required: +@include 'datasource/http/Config-not-required.mdx' + +## Datasource outputs + +The outputs for this datasource are as follows: + +@include 'datasource/http/DatasourceOutput.mdx' diff --git a/content/packer/content/docs/datasources/index.mdx b/content/packer/content/docs/datasources/index.mdx new file mode 100644 index 0000000000..1bcf1c8ffa --- /dev/null +++ b/content/packer/content/docs/datasources/index.mdx @@ -0,0 +1,14 @@ +--- +page_title: Data sources overview +description: | + A data source holds data you want to use in the Packer configuration. Define a data source in your configuration so that Packer can use external data during builds. +--- + +# Data sources + +Data sources let Packer fetch data to use in a template, including information defined outside of Packer. + +Refer to the [`data`](/packer/docs/templates/hcl_templates/datasources) block documentation to learn more about working with data sources. The documentation also contains details about each type of data source. + +Data sources are only available in HCL2 templates and require Packer `v1.7.0` and newer. + diff --git a/content/packer/content/docs/debugging.mdx b/content/packer/content/docs/debugging.mdx new file mode 100644 index 0000000000..88b92ca636 --- /dev/null +++ b/content/packer/content/docs/debugging.mdx @@ -0,0 +1,149 @@ +--- +description: | + Learn how to debug issues with Packer builds and plugins using `packer build`, logs, and other troubleshooting tools. +page_title: Debugging Packer +--- + +# Debugging Packer Builds + +Using `packer build -on-error=ask` allows you to inspect failures and try out +solutions before restarting the build. + +For remote builds with cloud providers like Amazon Web Services AMIs, debugging +a Packer build can be eased greatly with `packer build -debug`. This disables +parallelization and enables debug mode. + +Debug mode informs the builders that they should output debugging information. +The exact behavior of debug mode is left to the builder. In general, builders +usually will stop between each step, waiting for keyboard input before +continuing. This will allow you to inspect state and so on. + +In debug mode once the remote instance is instantiated, Packer will emit to the +current directory an ephemeral private SSH key as a .pem file. Using that you +can `ssh -i ` into the remote build instance and see what is going on +for debugging. The key will only be emitted for cloud-based builders. The +ephemeral key will be deleted at the end of the Packer run during cleanup. + +For a local builder, the SSH session initiated will be visible in the detail +provided when `PACKER_LOG=1` environment variable is set prior to a build, and +you can connect to the local machine using the userid and password defined in +the kickstart or preseed associated with initializing the local VM. + +It should be noted that one of the options `-on-error` is to `retry`, the retry +of the step in question has limitations: + +- the template Packer is building is **not** reloaded from file. +- the resources specified from builders **are** reloaded from file. + +Check the specifics on your builder to confirm their behavior. + +### Windows + +As of Packer 0.8.1 the default WinRM communicator will emit the password for a +Remote Desktop Connection into your instance. This happens following the +several minute pause as the instance is booted. Note a .pem key is still +created for securely transmitting the password. Packer automatically decrypts +the password for you in debug mode. + +## Debugging Packer + +Issues occasionally arise where certain things may not work entirely correctly, +or may not appear to work correctly. In these cases, it is sometimes helpful to +see more details about what Packer is actually doing. + +Packer has detailed logs which can be enabled by setting the `PACKER_LOG` +environmental variable to any value but `""` (empty string) and `"0"` like this +`PACKER_LOG=1 packer build `. This will cause detailed logs to +appear on stderr. The logs contain log messages from Packer as well as any +plugins that are being used. Log messages from plugins are prefixed by their +application name. + +Note that because Packer is highly parallelized, log messages sometimes appear +out of order, especially with respect to plugins. In this case, it is important +to pay attention to the timestamp of the log messages to determine order. + +In addition to simply enabling the log, you can set `PACKER_LOG_PATH` in order +to force the log to always go to a specific file when logging is enabled. Note +that even when `PACKER_LOG_PATH` is set, `PACKER_LOG` must be set in order for +any logging to be enabled. + +### Debugging Plugins + +Each packer plugin runs in a separate process and communicates with RPC over a +socket therefore using a debugger will not work (be complicated at least). + +But most of the Packer code is really simple and easy to follow with PACKER_LOG +turned on. If that doesn't work adding some extra debug print outs when you have +homed in on the problem is usually enough. + +### Debugging Packer in Powershell/Windows + +In Windows you can set the detailed logs environmental variable `PACKER_LOG` or +the log variable `PACKER_LOG_PATH` using PowerShell environment variables. For +example: + +```powershell +$env:PACKER_LOG=1 +$env:PACKER_LOG_PATH="packerlog.txt" +``` + +If you find a bug with Packer, please include the detailed log by using a +service such as [gist](https://gist.github.com). + +## Issues Installing Ubuntu Packages + +Issues may arise using and building Ubuntu AMIs where common packages that +_should_ be installed from Ubuntu's Main repository are not found during a +provisioner step: + +```text +amazon-ebs: No candidate version found for build-essential +amazon-ebs: No candidate version found for build-essential +``` + +This, obviously can cause problems where a build is unable to finish +successfully as the proper packages cannot be provisioned correctly. The +problem arises when cloud-init has not finished fully running on the source AMI +by the time that packer starts any provisioning steps. + +Adding the following provisioner to the Packer template, allows for the +cloud-init process to fully finish before packer starts provisioning the source +AMI. + +```json +{ + "type": "shell", + "inline": [ + "cloud-init status --wait" + ] +} +``` + +## Issues when using numerous Builders/Provisioners/Post-Processors + +Packer uses a separate process for each builder, provisioner, post-processor, +and plugin. In certain cases, if you have too many of these, you can run out of +[file descriptors](https://en.wikipedia.org/wiki/File_descriptor). This results +in an error that might look like + +```text +error initializing provisioner 'powershell': fork/exec /files/go/bin/packer: +too many open files +``` + +On Unix systems, you can check what your file descriptor limit is with +`ulimit -Sn`. You should check with your OS vendor on how to raise this limit. + +## Issues when using long temp directory + +Packer uses Unix sockets internally, which are created inside the default +directory for temporary files. Some operating systems place a limit on the +length of the socket name, usually between 80 and 110 characters. If you get an +error like this (for any builder, not just Docker): + +```text +Failed to initialize build 'docker': error initializing builder 'docker': plugin exited before we could connect +``` + +you should try setting your temp directory to something shorter. This can be +done through the `TMPDIR` environment variable. diff --git a/content/packer/content/docs/guides/1.7-plugin-upgrade.mdx b/content/packer/content/docs/guides/1.7-plugin-upgrade.mdx new file mode 100644 index 0000000000..9739d05400 --- /dev/null +++ b/content/packer/content/docs/guides/1.7-plugin-upgrade.mdx @@ -0,0 +1,118 @@ +--- +page_title: Upgrading your plugin to use the Packer plugin sdk +description: >- + Learn how to upgrade your plugin to support v1.7.0 and later. +--- + +# Upgrading your plugin to be compatible with Packer v1.7.0 and later + +In version v1.7.0 the Packer plugin development model was split into two separate packages: the Packer core +and the Packer plugin SDK. Prior to version 1.7.0, the Packer core repository (https://github.com/hashicorp/packer) contained the libraries and interfaces used by Packer plugins. In version 1.7.0, these libraries were moved into their own repository (https://github.com/hashicorp/packer-plugin-sdk). If you authored a plugin prior to the v1.7.0 release using the API version 4, you should switch to the Packer Plugin SDK. + +## Update plugins to use the new SDK + +### Why update to the new SDK? + +The goal of the SDK is to clearly separate the Packer core from the Packer plugins; as a plugin maintainer, you should only have to import the SDK and not the core. The SDK will allow us to use semantic versioning to express the changes to the API that our maintainers are expected to use, and will help us keep a clearer promise to you about the location and functionality of our helper tools. + +### How to update plugins to use the Packer plugin SDK + +We have created a [packer-sdk-migrator](https://github.com/hashicorp/packer-sdk-migrator) cli tool to help you migrate your plugin to use the new import paths. To use it, follow the installation instructions for the migration tool, then call `packer-sdk-migrator migrate` from the root of your plugin directory. More details can be found in the migrator's [README](https://github.com/hashicorp/packer-sdk-migrator/blob/main/README.md). + +Once you have migrated your plugin to the new SDK, your users can continue using your plugin as they always have, manually installing it in their `plugins` directories. However, Packer v1.7 also introduced the `packer init` command, and since you're here anyway it's a great time to upgrade your plugin so it can be found by `packer init`. Keep reading for more details! + +## Upgrade Plugins to use the new multi-component RPC server + +### Why is there a new RPC server? + +There are two main reasons we wrote the new server type. + +First, it enables multiple related components, for example a builder and a post-processor that share a hypervisor or cloud, to live together in the same plugin. This helps maintainers who are experts in a specific technology to focus on that technology without having to maintain several repositories that submodule the same common code. You can think of a multi-component plugin as being in some ways analogous to Terraform providers, in the sense that both frequently bundle common resources or components by hypervisor or cloud. + +Second, the new server provides the Packer core with structured metadata that allows the new `packer init` feature to work. This data includes semantic versioning that we need to implement the new [`required_plugins`](/packer/docs/templates/hcl_templates/blocks/packer#specifying-plugin-requirements) block, which allows us to ensure that users are using the correct _version_ of their plugin for a given Packer template. If you don't upgrade, your users cannot take advantage of the `packer init` tooling. + +### How to upgrade the plugin + +We've created a [scaffolding repository](https://github.com/hashicorp/packer-plugin-scaffolding/) that you can either clone or use as a reference example. If you already have a working plugin, it should look pretty familiar. You'll notice, however, that there are some changes to main.go. + +Previously, you may have had a main.go that looks something like: + +```go +package main + +import ( + "github.com/hashicorp/packer-plugin-sdk/plugin" +) + +func main() { + server, err := plugin.Server() + if err != nil { + panic(err) + } + + // MyProvisioner fulfils the Provisioner interface and is defined elsewhere + server.RegisterProvisioner(new(MyProvisioner)) + server.Serve() +} + +``` + +With this single-component plugin binary you'd install it by putting it into the plugin directory with the name `packer-provisioner-foo`, and to access your provisioner in your Packer template, you would use the type `foo`. + +To use the new multi-component server, you'll want to use the NewSet() function to create a server: + +```go +package main + +import ( + "fmt" + "os" + + "github.com/hashicorp/packer-plugin-sdk/plugin" +) + +func main() { + pps := plugin.NewSet() + pps.RegisterProvisioner(plugin.DEFAULT_NAME, new(MyProvisioner)) + err := pps.Run() + if err != nil { + fmt.Fprintln(os.Stderr, err.Error()) + os.Exit(1) + } +} + +``` + +The implementation is similar, but now we use the "NewSet" function to create the component Server, and call Run() instead of Serve(). + +You build it as you would any Go binary: + +```go +go build -o packer-plugin-bar +``` + +Then you install it by putting it into the plugin directory with the name `packer-plugin-foo`, and reference it in your template using the type "foo". + +Notice that the new naming convention uses the generic name "plugin" as opposed to saying the specific component type like the old style did; `packer-pluign-name` instead of `packer-provisioner-name`. You need to respect this new convention when installing the plugin, or Packer will not register it as a multi-component plugin. + +What does that `plugin.DEFAULT_NAME` do? It tells the Packer core that in your template you'll be using the root plugin name instead of a compound name composed of the plugin name and the component name. If you are only serving a single component, you may want to use `plugin.DEFAULT_NAME` to prevent your templates from stuttering. + +Here's an example using a custom string instead of `plugin.DEFAULT_NAME`: + +```go +pps.RegisterProvisioner("bar", new(MyProvisioner)) +``` + +When you do this, if you install your plugin as `packer-plugin-foo`, the Packer core expects you to access the provisioner in your template using the type `foo-bar`, where "foo" is the last portion of the plugin name as installed and "bar" is the name of the provisioner that you registered. + +## Registering multiple components + +The goal of this guide is to help you upgrade a single-component plugin to use the new SDK and plugin server; check out our [Plugin Development Basics](/packer/docs/plugins/creation#plugin-development-basics) guide for details on how to add new components to your plugin. + +## Distributing migrated plugins + +Once a plugin has been migrated to use the `packer-plugin-sdk` it can be released as it normally would and used by Packer by [installing the plugin](/packer/docs/plugins#installing-plugins) manually into the Packer plugin directory. But this method will not allow your users to take advantage of the `packer init` command. + +If you want Packer to be able to automatically install your plugin for your users via`packer init` -- the preferred method of installation -- you need to make the plugin available on GitHub in a repository named after the multi-component plugin. `https://github.com//packer-plugin-name`. We recognize that this may require you to rename or fork your plugin repository, but we think that it is worth the inconvenience to reduce ambiguity in the `init` call. + +See our documentation on [Creating a GitHub Release](/packer/docs/plugins/creation#creating-a-github-release) for details on the recommended practice for releasing Packer plugins on GitHub. diff --git a/content/packer/content/docs/guides/1.7-template-upgrade.mdx b/content/packer/content/docs/guides/1.7-template-upgrade.mdx new file mode 100644 index 0000000000..19d862c258 --- /dev/null +++ b/content/packer/content/docs/guides/1.7-template-upgrade.mdx @@ -0,0 +1,185 @@ +--- +page_title: Upgrading your template to use Packer init +description: >- + Use packer init and the required_plugins block to better control plugin usage within Packer templates. Learn how to update your template to work with these new features. +--- + +# Upgrading your template to be compatible with `packer init` + +The `packer init` command introduced in version v1.7.0 benefits Packer users by providing better control of plugin +usage within a Packer template configuration. + +In short, `packer init` will look at the `required_plugins` block definition within a template, and download the correct +binaries for that template. For details about how the command works and where the plugins will be installed, please +refer to the [`packer init`](/packer/docs/commands/init) documentation. + +-> **Note:** `packer init` is only supported for HCL templates. You can +upgrade legacy JSON templates to HCL using the [hcl2_upgrade](/packer/docs/commands/hcl2_upgrade) command. + +## Updating your template with the `required_plugins` block + +## FAQs + +### Why do you need to upgrade your template? + +To use the `packer init` command, you must upgrade your template to use the `required_plugins` block. The `init` command +won't work if no `required_plugins` is provided. + +We strongly encourage you to upgrade and start using `required_plugins` block within your templates to manage plugin +installation, but if you prefer not to use the `required_plugins` block you can continue to +[install plugins manually](/packer/docs/plugins#installing-plugins). + +### What if you only use components that are built into the Packer core? + +You don't need `packer init` for this, as of v1.7.0. But it's a good idea to get familiar with the required_plugins +block anyway, because we are going to start splitting popular HashiCorp-maintained components like the amazon-ebs +builder out of the core into their own multi-component plugins. When we do split these plugins out of the core, we will +provide a tool to help you create the correct required_plugins block. + +### When should you upgrade your template? + +The `packer init` command can only install plugins that have been upgraded to use the latest version of the +[Packer Plugin SDK](https://github.com/hashicorp/packer-plugin-sdk), and therefore are compatible with Packer's API +version v5.0. The plugin repository on GitHub also needs to use a specific release format. If you are not sure whether +the plugin you use fits those requirements, you can reach out to your maintainer to ask. You can also look for clues +that the plugin is ready for use with `packer init`: + +- Check the plugin's CHANGELOG for notes about migrating to the packer-plugin-sdk. +- Check the name of the repository the plugin resides in. Chances are that if the repository follows the naming + convention `packer-plugin-*` (e.g. `packer-plugin-comment`), then the plugin has been upgraded to be compatible with + `packer init`. If the repository has a name that references a specific Packer component (for example, + `packer-provisioner-*`, `packer-builder-*`, or `packer-post-processor-*`) then the plugin likely still needs to be + upgraded to be compatible with `packer init`. Reach out to your plugin maintainer to request that they upgrade; the + Packer team has written a maintainer-focused guide [here](/packer/guides/1.7-plugin-upgrade). + +If the plugin(s) have been upgraded, then they can be used with the `required_plugins` block in your upgraded template. + +#### What if the plugin you need is not upgraded to API v5.0? + +Since the SDK is being released at the same time as the `init` command, plugin users may face a gap between when the +Packer core v1.7.0 is released, and the plugins you use are upgraded by their maintainers. The Packer team is getting +in touch with all currently known plugin maintainers to provide support during their upgrade process. + +If you are willing to upgrade your template but found out that the plugin you are using hasn't been upgraded yet, +we suggest you reach out to the plugin maintainers and ask for an upgrade; the Packer team has written a +maintainer-focused guide [here](/packer/guides/1.7-plugin-upgrade). + +Check the table below to better understand whether your plugin is compatible with `packer init`, with manual +installation, or with both for the Packer version you are using. + +| Packer Core Version | Single Component Plugin - API v4 | Single Component Plugin - API v5.0 | Multi Component Plugin - API v5.0 | +| ------------------- | ------------------------------------------------- | ------------------------------------------------- | --------------------------------------------------------- | +| v1.5.0 to v1.6.6 | ✋ Plugin must be manually installed | ⛔ Plugin cannot be used with this Packer version | ⛔ Plugin cannot be used with this Packer version | +| v1.7.0 | ⛔ Plugin cannot be used with this Packer version | ✋ Plugin must be manually installed | 📦 Plugin can be installed manually or with `packer init` | + +### How to upgrade your template + +Let's use the following template as an example: + +```hcl +# file: example.pkr.hcl + +source "null" "basic-example" { + communicator = "none" +} + +build { + sources = ["sources.null.basic-example"] + + provisioner "comment" { + comment = "Hello from comment plugin" + ui = true + bubble_text = true + } +} +``` + +This template uses the example [packer-provisioner-comment](https://github.com/sylviamoss/packer-plugin-comment) plugin. +Until Packer v1.7.0, the binaries were installed manually. + +Since the Packer v1.7.0 release, the plugin has been upgraded by its maintainers and is now called +`packer-plugin-comment`. with its latest version being v0.2.23. Knowing that, our example template can be upgraded to +use the `required_plugins` block. + +```hcl +# file: example.pkr.hcl + +packer { + required_plugins { + comment = { + version = ">=v0.2.23" + source = "github.com/sylviamoss/comment" + } + } +} + +source "null" "basic-example" { + communicator = "none" +} + +build { + sources = ["sources.null.basic-example"] + + provisioner "comment" { + comment = "Hello from comment plugin" + ui = true + bubble_text = true + } +} +``` + +The upgraded template is now telling Packer that it requires a plugin named `comment` stored in a github repository +owned by the user `sylviamoss`. It also says that the template needs to use a version of the plugin equal to or greater +than `v0.2.23`. Finally, the local_name of the plugin will be `comment`, which means it will be referred to as `comment` +in the rest of the template. + +Here it is a brief explanation of each field: + +- `version` - Should follow the [version constraints](/packer/docs/templates/hcl_templates/blocks/packer#version-constraints). +- `source` - Should have the GitHub hostname, the plugin's organizational namespace, and its short name. + Packer will be responsible for determining the full GitHub + address. For example, if the source is `github.com/sylviamoss/comment`, Packer + will download the binaries from `github.com/sylviamoss/packer-plugin-comment`. + To learn more about the source field, check out the [Source + Address](/packer/docs/plugins#source-addresses) documentation. +- `local_name`- Can be replaced with whatever you want, and the new value will become the name of the plugin. + For example: + + ```hcl + packer { + required_plugins { + bubbled_up = { + # ... + } + } + } + + # ... + + build { + # ... + + provisioner "bubbled_up" { + # ... + } + } + ``` + +Once the template is upgraded, you can run `packer init example.pkr.hcl` and the output should tell you the installed +plugin, and the place where it was installed. + +```shell +➜ packerdev init example.pkr.hcl +Installed plugin sylviamoss/comment v0.2.23 in "/Users//.packer.d/plugins/github.com/sylviamoss/comment/packer-plugin-comment_v0.2.23_x5.0_darwin_amd64" +``` + +`packer init` will only download plugin binaries and never delete existing ones. If all the required plugins are installed, +the command won't do anything unless you set the `-upgrade` flag to force download newer versions of the existing plugins. + +You can uninstall a plugin by deleting it from the `.packer.d/plugins/` directory on your computer. + +Now that the required comment plugin is installed via `packer init`, you can run `packer build example.pkr.hcl` as usual +and won't need to run `init` again unless you want to upgrade the plugin version. + +A template with a `required_plugins` block should **always** be initialised at least once with `packer init` before +`packer build`. If the template is built before init, Packer will fail and ask for initialisation. diff --git a/content/packer/content/docs/guides/automatic-operating-system-installs/autounattend_windows.mdx b/content/packer/content/docs/guides/automatic-operating-system-installs/autounattend_windows.mdx new file mode 100644 index 0000000000..e36354dd4a --- /dev/null +++ b/content/packer/content/docs/guides/automatic-operating-system-installs/autounattend_windows.mdx @@ -0,0 +1,89 @@ +--- +page_title: Unattended Windows Installation +description: |- + Learn how to use an autounattend file to automatically answer installation + questions and enable Packer to connect to your Windows instance. +--- + +# Unattended Installation for Windows + +Unattended Windows installation is done via "Answer Files", or "Unattend files". + +These files are generally named "autounattend.xml". They are not +Packer-specific tools, though we do make use of them. + +If, after following this guide, you're still having issues getting an answer +file working, We recommend you read the official documentation on +[answer files](https://docs.microsoft.com/en-us/windows-hardware/manufacture/desktop/update-windows-settings-and-scripts-create-your-own-answer-file-sxs). + +The guide here is hopefully enough to get you started, but isn't a replacement +for the official documentation. + +## When To Use an Answer File + +If you are installing the Windows Operating System from a mounted iso as part of +your Packer build, you will need to use an Answer file. For example, you're +building an image from scratch using the [vmware-iso](/packer/plugins/builders/vmware/iso), +[virtualbox-iso](/packer/plugins/builders/virtualbox/iso), or +[hyperv-iso](/packer/plugins/builders/hyperv/iso) builders. + +If you are not installing the operating system, you won't need to provide an +answer file. If you are using a pre-built image +in a cloud, you don't need to worry about Answer files. + +## How to make an Answer File + +You can either start from an example answer file from a known repo (take a look +at the examples links below), or you can generate one using an answer file +wizard by selecting New File > New Answer file on a Windows machine. +A comprehensive list of all the options you can set in an answer file can be +found [here](https://docs.microsoft.com/en-us/windows-hardware/customize/desktop/unattend/components-b-unattend) + +## Where to put the Answer File + +Windows will automatically look for an autounattend.xml file on mounted drives. +Many users use the `floppy_files` option or a secondary mounted iso for +providing the answer file to their iso builders. + +You can also specify an unattend file to use by using the /unattend: option when +running Windows Setup (setup.exe) in your `boot_command`. + +## What does Packer _need_ the Answer File to do? + +Packer needs the Answer File to handle any questions that would normally be +answered interactively during a Windows installation. + +If you want to be able to use provisioners, the Answer file must also contain +a script that sets up SSH or WinRM so that Packer can connect to the instance. + +Finally, your Packer build will be much smoother if the Answer File handles or +disables Windows updates rather than you trying to run them using a Packer +provisioner. This is because the winrm communicator does not handle the +disconnects caused by automatic reboots in Windows updates well, and the +disconnections can fail a build. + +## Examples + +The chef-maintained bento boxes are a great example of a Windows build that +sets up openssh as part of the unattended installation so that Packer can +connect using the SSH communicator. The functioning answer files for every +modern Windows version can be found [here](https://github.com/chef/bento/tree/master/packer_templates/win_answer_files). + +Stefan Scherer's [packer-windows repo](https://github.com/StefanScherer/packer-windows) +is a great example of Windows builds that set up WinRM as part of the unattended +installation so that Packer can connect using the `winrm` communicator: + +```json +{ + "type": "virtualbox-iso", + "guest_os_type": "Windows2008_64", + "iso_url": "https://download.microsoft.com/download/7/5/E/75EC4E54-5B02-42D6-8879-D8D3A25FBEF7/7601.17514.101119-1850_x64fre_server_eval_en-us-GRMSXEVAL_EN_DVD.iso", + "iso_checksum": "sha256:30832AD76CCFA4CE48CCB936EDEFE02079D42FB1DA32201BF9E3A880C8ED6312", + "shutdown_command": "shutdown /s /t 10 /f /d p:4:1 /c Packer_Provisioning_Shutdown", + "guest_additions_mode": "attach", + "floppy_files": ["./scripts/Autounattend.xml", "./scripts/openssh.ps1"], + "communicator": "winrm", + "winrm_username": "vagrant", + "winrm_password": "vagrant" +} +``` diff --git a/content/packer/content/docs/guides/automatic-operating-system-installs/index.mdx b/content/packer/content/docs/guides/automatic-operating-system-installs/index.mdx new file mode 100644 index 0000000000..872df2d9ea --- /dev/null +++ b/content/packer/content/docs/guides/automatic-operating-system-installs/index.mdx @@ -0,0 +1,23 @@ +--- +page_title: Automatic OS Installs +description: |- + Learn how to use preseed, kickstart, and autounattend files to automatically + answer installation questions and enable Packer to connect to your instance. +--- + +# Automatic OS Installs + +If you are building from a brand-new and unconfigured operating system +image, Packer will need you to perform the operating system install before it +can connect to and configure your image using its provisioners. Most operating +system distributions have a mechanism for performing the normally-interactive +installation in an automated way. For Debian operating systems, this is done +using a preseed file; for Windows, it's done using an Autounattend.xml. We have +compiled some simple guides here for common operating system distributions. + +These guides are meant to give you a quick introduction to how to use automated +installation answer files in order to perfom those installs; we don't mean to +be a comprehensive guide on each operating system, but we hope to give you +enough context to be able to more easily find any further information you need. + +Refer to the instructions for your operating system. diff --git a/content/packer/content/docs/guides/automatic-operating-system-installs/preseed_ubuntu.mdx b/content/packer/content/docs/guides/automatic-operating-system-installs/preseed_ubuntu.mdx new file mode 100644 index 0000000000..5f2bfcd98a --- /dev/null +++ b/content/packer/content/docs/guides/automatic-operating-system-installs/preseed_ubuntu.mdx @@ -0,0 +1,175 @@ +--- +page_title: Unattended Debian/Ubuntu Installation +description: |- + Learn how to use a preseed file to automatically answer installation + questions and enable Packer to connect to your Debian instance. +--- + +# Unattended Installation for Debian + +Unattended Debian/Ubuntu installation is done via "preseed" files + +These files are generally named "preseed.cfg". They are not +Packer-specific tools, though we do make use of them. + +If, after following this guide, you're still having issues getting a preseed +file working, We recommend you read the official documentation on +[preseed files](https://wiki.debian.org/DebianInstaller/Preseed). + +The guide here is hopefully enough to get you started, but isn't a replacement +for the official documentation. + +## When To Use a Preseed File + +If you are installing the operating system from a mounted iso as part of +your Packer build, you will need to use a preseed file. For example, you're +building an image from scratch using the [vmware-iso](/packer/plugins/builders/vmware/iso), +[virtualbox-iso](/packer/plugins/builders/virtualbox/iso), or +[hyperv-iso](/packer/plugins/builders/hyperv/iso) builders. + +If you are not installing the operating system, you won't need to provide a +preseed file. If you are using a pre-built image in a cloud, you don't need to +worry about preseed files. + +## How to make a Preseed File + +You can either start from an example preseed file from a known repo (take a look +at the examples links below), or you can start with the official [example +preseed](https://www.debian.org/releases/stable/example-preseed.txt), and +comment or uncomment the options as you need them. + +## Where to put the preseed file + +The `-iso` builders mentioned above all have an `http_dir` or an `http_content` +option. Any file inside of your `http_dir` or `http_content` will be served on a +local fileserver for your virtual machine to be able to access. One very common +use for this directory is to use it to provide your preseed file. + +You then reference the file using a `boot_command` to kick off the installation. +In the example below, the `preseed/url` command line option is being +used in the `/install/vmlinuz command`. The `{{ .HTTPIP }}` and +`{{ .HTTPPort }}` options are special Packer template options that will get set +by Packer to point to the HTTP server we create, so that your boot command can +access it. For an example of a working boot_command, refer to the Examples section +below. For more information on how boot_command works, refer to the +boot_command section of the docs for whatever builder you are using. + +## What does Packer _need_ the preseed file to do? + +Packer needs the preseed file to handle any questions that would normally be +answered interactively during a Debian installation. + +If you want to be able to use provisioners, the preseed file must also install +SSH so that Packer can connect to the instance. + +## Examples + +A very minimal example of a preseed file can be found below. Much of this was +copied from the official example-preseed shared above. Notice that we are +installing ssh via `d-i pkgsel/include string openssh-server` and configuring +a username so that Packer will be able to connect. + +You need to make sure that your mirror settings are properly configured for your +specific distribution of Debian. + +```shell +# Preseeding only locale sets language, country and locale. +d-i debian-installer/locale string en_US + +# Keyboard selection. +d-i console-setup/ask_detect boolean false +d-i keyboard-configuration/xkb-keymap select us + +choose-mirror-bin mirror/http/proxy string + +### Clock and time zone setup +d-i clock-setup/utc boolean true +d-i time/zone string UTC + +# Avoid that last message about the install being complete. +d-i finish-install/reboot_in_progress note + +# This is fairly safe to set, it makes grub install automatically to the MBR +# if no other operating system is detected on the machine. +d-i grub-installer/only_debian boolean true + +# This one makes grub-installer install to the MBR if it also finds some other +# OS, which is less safe as it might not be able to boot that other OS. +d-i grub-installer/with_other_os boolean true + +### Mirror settings +# If you select ftp, the mirror/country string does not need to be set. +d-i mirror/country string manual +d-i mirror/http/directory string /ubuntu/ +d-i mirror/http/hostname string archive.ubuntu.com +d-i mirror/http/proxy string + +### Partitioning +d-i partman-auto/method string lvm + +# This makes partman automatically partition without confirmation. +d-i partman-md/confirm boolean true +d-i partman-partitioning/confirm_write_new_label boolean true +d-i partman/choose_partition select finish +d-i partman/confirm boolean true +d-i partman/confirm_nooverwrite boolean true + +### Account setup +d-i passwd/user-fullname string vagrant +d-i passwd/user-uid string 1000 +d-i passwd/user-password password vagrant +d-i passwd/user-password-again password vagrant +d-i passwd/username string vagrant + +# The installer will warn about weak passwords. If you are sure you know +# what you're doing and want to override it, uncomment this. +d-i user-setup/allow-password-weak boolean true +d-i user-setup/encrypt-home boolean false + +### Package selection +tasksel tasksel/first standard +d-i pkgsel/include string openssh-server build-essential +d-i pkgsel/install-language-support boolean false + +# disable automatic package updates +d-i pkgsel/update-policy select none +d-i pkgsel/upgrade select full-upgrade +``` + +Here's an example of the vmware-iso builder being used to call this preseed. +In this case, it is assumed that the file is saved as preseed.cfg inside of a +directory called "http", and Packer is being called from the directory +containing the "http" directory. + +```hcl + "builders": [ + { + "boot_command": [ + "", + "", + "", + "/install/vmlinuz", + " initrd=/install/initrd.gz", + " auto-install/enable=true", + " debconf/priority=critical", + " preseed/url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/preseed_2.cfg", + " -- ", + "" + ], + "boot_wait": "10s", + "guest_os_type": "ubuntu-64", + "http_directory": "http", + "iso_checksum": "file:///Users/mmarsh/dev/repro_cases/packer_cache/shasums.txt", + "iso_url": "http://old-releases.ubuntu.com/releases/14.04.1/ubuntu-14.04.1-server-amd64.iso", + "shutdown_command": "echo 'vagrant' | sudo -S shutdown -P now", + "ssh_password": "vagrant", + "ssh_username": "vagrant", + "ssh_wait_timeout": "10000s", + "tools_upload_flavor": "linux", + "type": "vmware-iso" + } + ], +``` + +For more functional examples of a Debian preseeded installation, you can refer to the Chef-maintained [bento](https://github.com/chef/bento#using-packer) box [preseed](https://github.com/chef/bento/blob/6fe9fa20d1f37e916a7babdee87c89ba38ce54a4/packer_templates/http/debian/preseed.cfg), +the Ubuntu [autoinstall config](https://github.com/chef/bento/blob/6fe9fa20d1f37e916a7babdee87c89ba38ce54a4/packer_templates/http/ubuntu/user-data), their [builders](https://github.com/chef/bento/blob/6fe9fa20d1f37e916a7babdee87c89ba38ce54a4/packer_templates/pkr-builder.pkr.hcl). diff --git a/content/packer/content/docs/guides/hcl/component-object-spec.mdx b/content/packer/content/docs/guides/hcl/component-object-spec.mdx new file mode 100644 index 0000000000..9ca8c29a6b --- /dev/null +++ b/content/packer/content/docs/guides/hcl/component-object-spec.mdx @@ -0,0 +1,41 @@ +--- +page_title: Generating code for config spec. +description: Learn how to generate the HCL2 configuration of your component easily. +--- + +# Auto Generate the HCL2 code of a plugin + +From v1.5, Packer can be configured using HCL2. Because Packer has so many +builders, provisioners, and post-processors, we created a on code generation +tool to add the HCL2-enabling code more easily. You can use this code generator +to create the HCL2 spec code of your custom plugin simply. +It's a Go binary package made available through the Packer plugin SDK + +Say you want to configure the `Config` struct of a `Builder` in a package +located in `my/example-plugin/config.go`. Here are some simple steps you can +follow to make it HCL2 enabled: + +- run `go install github.com/hashicorp/packer-plugin-sdk/cmd/packer-sdc@latest` + +- Add `//go:generate packer-sdc mapstructure-to-hcl2 -type Config` at the top of + `config.go` + +- run `go generate ./my/example-plugin/...` + + This will generate a `my/example-plugin/config.hcl2spec.go` file containing + the configuration fields of `Config`. + +- Make sure that all the nested structs of `Config` are also auto generated the + same way. + +- Now we only need to make your Builder implement the interface by adding the + following snippet: + + ```go + func (b *Builder) ConfigSpec() hcldec.ObjectSpec { return b.config.FlatMapstructure().HCL2Spec() } + ``` + + From now on every time you add or change a field of Config you will need to + run the `go generate` command again. + +A good example of this is the [Config struct of the amazon-ebs builder](https://github.com/hashicorp/packer-plugin-amazon/blob/main/builder/ebs/builder.go) diff --git a/content/packer/content/docs/guides/hcl/index.mdx b/content/packer/content/docs/guides/hcl/index.mdx new file mode 100644 index 0000000000..aa8d779b49 --- /dev/null +++ b/content/packer/content/docs/guides/hcl/index.mdx @@ -0,0 +1,90 @@ +--- +page_title: Getting started configuring Packer with HCL2 files +description: >- + Use the HCL2 syntax for your Packer templates. Learn about arguments and blocks, comments, and multi-line strings. +--- + +# Introduction to Packer HCL2 + +@include 'guides/hcl2-beta-note.mdx' + +It is not necessary to know all of the details of the HCL syntax in order to +use Packer, and so this page summarizes the most important details to get you +started. If you are interested, you can find a [full definition of HCL +syntax](https://github.com/hashicorp/hcl2/blob/master/hcl/hclsyntax/spec.md) in +the HCL native syntax specification. + +## Arguments and Blocks + +The HCL syntax is built around two key syntax constructs: arguments and blocks. + +```hcl +# block +source "amazon-ebs" "example" { + + # argument + ami_name = "abc123" +} +``` + +## Comments + +The HCL language supports three different syntaxes for comments: + +- `#` begins a single-line comment, ending at the end of the line. +- `//` also begins a single-line comment, as an alternative to `#`. +- `/*` and `*/` are start and end delimiters for a comment that might + span over multiple lines. + +## Multi-line strings + +A multi-line string value can be provided using heredoc syntax. + +```hcl +variable "long_key" { + type = "string" + default = < +/install/vmlinuz noapic +... +EOF +} +``` + +```hcl +# folder/build.pkr.hcl +build { + sources = [ + "source.amazon-ebs.example-1", + "source.virtualbox-iso.example-2" + ] + + provisioner "shell" { + inline = [ + "echo 'it is alive !'" + ] + } +} +``` diff --git a/content/packer/content/docs/guides/hcl/variables.mdx b/content/packer/content/docs/guides/hcl/variables.mdx new file mode 100644 index 0000000000..b8952c7cdf --- /dev/null +++ b/content/packer/content/docs/guides/hcl/variables.mdx @@ -0,0 +1,373 @@ +--- +page_title: Input and Local Variables guide +description: |- + This page introduces input variables and local variables as a way to + parameterize a configuration. +--- + +# Input Variables and local variables + +@include 'guides/hcl2-beta-note.mdx' + +This page introduces input variables and local variables as a way to +parameterize a configuration. Once defined, input variables are settable from a +default value, environment, special var files, and command line arguments. +Local variables can be a compound of input variables and local variables. + +## Defining Variables and locals + +In the legacy JSON Packer templates, any variables we hadn't already defined in +the "variables" stanza of our json template could simply be passed in via the +command line or a `var-file`; if a variable was never defined it would generally +be interpolated to an empty string. + +_In the HCL2 Packer templates, we must always pre-define our variables in the +HCL equivalent of the "variables" stanza._ + +Another difference between JSON and HCL Packer templates is that in JSON packer +templates, the "variables" stanza, if used, was always in the same .json file +as the builds and builders. In HCL, it can exist in its own file if you want it +to. + +To demonstrate, let's create a file `variables.pkr.hcl` with the following +contents. + +-> **Note**: that the file can be named anything, since Packer loads all +files ending in `.pkr.hcl` in a directory. If you split your configuration +across multiple files, use +`packer build ` to initiate a build. + +```hcl +// variables.pkr.hcl + +// For those variables that you don't provide a default for, you must +// set them from the command line, a var-file, or the environment. + +variable "weekday" {} + +variable "sudo_password" { + type = string + default = "mypassword" + // Sensitive vars are hidden from output as of Packer v1.6.5 + sensitive = true +} + +variable "flavor" { + type = string + default = "strawberry" +} + +variable "exit_codes" { + type = list(number) + default = [0] +} + +locals { + ice_cream_flavor = "${var.flavor}-ice-cream" + foo = "bar" +} +``` + +This defines several variables within your Packer configuration, each showing +off a different way to set them. The first variable, "weekday", is an empty +block `{}`, without a type or a default. + +However, it's generally best to provide the type in your variable definition, +as you can see in variable "flavor", which we have given a type of "string", +and variable "exit_codes", which we have given a type of "list(number)", +meaning it is a list/array of numbers. + +When a variable is passed from the cli or environment and the variable's type +is not set, Packer will expect it to be a string. But if it is passed from a +var-file where Packer can interpret HCL properly it can be a slice or any +supported type. + +In addition to setting the type, the "flavor" and "exit_codes" variables also +provide a default. If you set a default value, then you don't need to set the +variable at run time. Packer will use a provided command-line var, +var-file, or environment var if it exists, but if not Packer will fall back to +this default value. + +If you do not set a default value, Packer will fail immediately when you try to +run a `build` if you have not provided the missing variable via the +command-line, a var-file, or the environment. The `validate`, `inspect` and +`console` commands will work, but variables with unknown values will be +``. + +This also defines two locals: `ice_cream_flavor` and `foo`. + +-> **Note**: that it is _not_ possible to reference a variable in the +definition of another variable. But it _is_ possible to use locals and +variables in the definition of a local, as shown in the ice_cream_flavor +definition. + +## Using Variables and locals in Configuration + +For simplicity's sake, we're going to put a null source in the same file as +we are putting the build configuration. This file demonstrates how to use the +variables we previously defined. + +```hcl +// null_example.pkr.hcl + +source "null" "example" { + communicator = "none" +} + +build { + sources = [ + "source.null.example" + ] + provisioner "shell-local" { + // Note that for options that are documented as template engines, + // we still have to use the Go template engine syntax rather than our + // specialized HCL2 variable syntax. This example shows a combination of + // an HCL2 variable and the Go template engines built into the + // execute_command option + execute_command = ["/bin/sh", "-c", "echo ${var.sudo_password}| {{.Vars}} {{.Script}}"] + environment_vars = ["HELLO_USER=packeruser", "UUID=${build.PackerRunUUID}"] + inline = ["echo the Packer run uuid is $UUID"] + } + provisioner "shell-local" { + inline = ["echo var.flavor is: ${var.flavor}", + "echo local.ice_cream_flavor is: ${local.ice_cream_flavor}"] + valid_exit_codes = var.exit_codes + } +} +``` + +As you can see in the example, you can access your variables directly by +giving them the `var.` or `local.` prefix. If you want to embed the variables +in a string, you can do so with the `${}` HCL interpolation syntax. If you are +using an option that is a template engine, you still need to use the Go +templating engine syntax `{{ .OPTION }}` for those engines. + +## Assigning Variables + +There are multiple ways to assign variables. Below is also the order +in which variable values are chosen. The following is the descending order +of precedence in which variables are considered. So first defined; first used. + +#### Command-line flags + +You can set variables directly on the command-line with the +`-var` flag.: + +```shell-session +$ packer build \ + -var "weekday=Sunday" \ + -var "flavor=chocolate" \ + -var "sudo_password=hunter42" . +``` + +Once again, setting variables this way will not save them, and they'll +have to be input repeatedly as commands are executed. + +If you plan to assign variables via the command line, we strongly recommend that +you at least set a default type instead of using empty blocks; this helps the +HCL parser understand what is being set. Otherwise it will interpret all of your +command line variables as strings. + +#### From a file + +To persist variable values, create a file and assign variables within +this file. A variable definitions file uses the same basic syntax as Packer language files, +but consists only of variable name assignments. + +```hcl +sudo_password = "partyparrot" +weekday = "Sunday" +``` + +When assigning values in a variable definitions file please make sure to quote any string values, +wrap list values within brackets `[...]`, and wrap maps within curly-braces `{...}`. + +Given a file named `variables.pkrvars.hcl` Packer can be instructed to use the +variable definitions file by using the `-var-file` command line flag. + +```shell-session +$ packer build -var-file="variables.pkrvars.hcl" . +``` + +Packer will automatically load any var file that matches the name +`*.auto.pkrvars.hcl`, without the need to pass the file via the command line. +If we rename the above variable definitions file from `variables.pkrvars.hcl` to +`variables.auto.pkrvars.hcl`, then we can run our build simply by calling + +```shell-session +$ packer build . +``` + +You may provide as many -var-file flags as you would like: + +```shell-session +$ packer build \ + -var-file="secret.pkrvars.hcl" \ + -var-file="production.pkrvars.hcl" . +``` + +These files can also be JSON: + +variables.json: + +```json +{ + "weekday": "sunday", + "flavor": "mint" +} +``` + +```shell-session +$ packer build -var-file=variables.json +``` + +We don't recommend saving sensitive information to version control, but you +can create a local secret variables file and use `-var-file` to load it. + +You can use multiple `-var-file` arguments in a single command, with some +checked in to version control and others not checked in. For example: + +```shell-session +$ packer build \ + -var-file="secret.pkrvars.hcl" \ + -var-file="production.pkrvars.hcl" . +``` + +#### From environment variables + +Packer will read environment variables in the form of `PKR_VAR_name` to find +the value for a variable. For example, the `PKR_VAR_access_key` variable can be +set to set the `access_key` variable. + +```shell-session +$ export PKR_VAR_weekday=Monday +$ packer build . +``` + +#### Variable Defaults + +If no value is assigned to a variable via any of these methods and the +variable has a `default` key in its declaration, that value will be used +for the variable. + +If all of your variables have defaults, then you can call a packer build using: + +```shell-session +$ packer build . +``` + +You can make this work for yourself using the variable example file above by +commenting out or removing the "weekday" variable declaration, since it is not +actually used in the example build. + +If your variable definitions are stored in the same file as your source and +build, you can call the build against that specific file: + +```shell-session +$ packer build self_contained_example.pkr.hcl +``` + +#### Unspecified Values Fail + +If you call `packer build` with any variables defined but not set, Packer will +error. + +## Variable Type Reference + +### Lists + +Lists are defined either explicitly or implicitly + +```hcl +# implicitly by using brackets [...] +variable "cidrs" { default = [] } + +# explicitly +variable "cidrs" { type = list } +``` + +You can specify lists in a `variables.pkrvars.hcl` file: + +```hcl +cidrs = [ "10.0.0.0/16", "10.1.0.0/16" ] +``` + +### Maps + +Maps are a way to create variables that are lookup tables. An example +will show this best. Let's extract our AMIs into a map and add +support for the `us-west-2` region as well: + +```hcl +variable "amis" { + type = map(string) + default = { + "us-east-1" = "ami-b374d5a5" + "us-west-2" = "ami-4b32be2b" + } +} +``` + +A variable can have a `map` type assigned explicitly, or it can be implicitly +declared as a map by specifying a default value that is a map. The above +demonstrates both. + +Then we can `lookup` in maps like in the following: + +```hcl +source "amazon-ebs" "example" { + source_ami = "${lookup(var.amis, var.region)}" + instance_type = "t2.micro" +} +``` + +This introduces a new type of interpolation: a function call. The +`lookup` function does a dynamic lookup in a map for a key. The +key is `var.region`, which specifies that the value of the region +variables is the key. + +You can also do a static lookup of a map directly with +`${var.amis["us-east-1"]}`. + +## Assigning Maps + +We set defaults above, but maps can also be set using the `-var` and +`-var-file` values. For example: + +```shell-session +$ packer build -var 'amis={ us-east-1 = "foo", us-west-2 = "bar" }' +# ... +``` + +Here is an example of setting a map's keys from a file. Starting with these +variable definitions: + +```hcl +variable "region" {} +variable "amis" { + type = map +} +``` + +You can specify keys in a `variables.pkrvars.hcl` file: + +```hcl +amis = { + "us-east-1" = "ami-abc123" + "us-west-2" = "ami-def456" +} +``` + +And access them via `lookup()`: + +```hcl +output "ami" { + value = "${lookup(var.amis, var.region)}" +} +``` + +Like so: + +```shell-session +$ packer build -var region=us-west-2 +``` diff --git a/content/packer/content/docs/guides/index.mdx b/content/packer/content/docs/guides/index.mdx new file mode 100644 index 0000000000..347e876c16 --- /dev/null +++ b/content/packer/content/docs/guides/index.mdx @@ -0,0 +1,17 @@ +--- +page_title: Guides +description: |- + Welcome to the Packer guides. If you are just getting started with Packer, + please see the Packer introduction instead and then continue on to the guides. + These guides provide examples for common Packer workflows and actions for + users of Packer. +--- + +# Packer + +Welcome to the Packer guides. If you are just getting started with Packer, +please see the [Packer introduction][intro] instead and then continue on to the +guides. These guides provide examples for common Packer workflows and actions +for users of Packer. + +[intro]: /packer/docs/intro diff --git a/content/packer/content/docs/guides/packer-on-cicd/build-image-in-cicd.mdx b/content/packer/content/docs/guides/packer-on-cicd/build-image-in-cicd.mdx new file mode 100644 index 0000000000..4efc595153 --- /dev/null +++ b/content/packer/content/docs/guides/packer-on-cicd/build-image-in-cicd.mdx @@ -0,0 +1,29 @@ +--- +page_title: Build Images in CI/CD +description: >- + Learn how to build Packer images within a continuous integration / continuous delivery pipeline. +--- + +# Build Images in CI/CD + +The following guides from our partners show how to use their services to build +images with Packer. + +- [How to Build Immutable Infrastructure with Packer and CircleCI Workflows](https://circleci.com/blog/how-to-build-immutable-infrastructure-with-packer-and-circleci-workflows/) +- [Using Packer and Ansible to Build Immutable Infrastructure in CodeShip](https://blog.codeship.com/packer-ansible/) + +The majority of the [Packer Builders](/packer/docs/builders) can run just fine in a +container, a common model used by most CI/CD services. However, while it is +possible to run many builders in containers or nested virtualization, this may +require advanced configuration; examples include the [QEMU +builder](/packer/plugins/builders/qemu) for +[KVM](https://www.linux-kvm.org/page/Main_Page) or +[Xen](https://www.xenproject.org/), the [VirtualBox +builder](/packer/plugins/builders/virtualbox) for OVA or OVF virtual machines, and the +[VMware builder](/packer/plugins/builders/vmware) for use with VMware products that are +all designed to run on a bare-metal machine or within nested virtualization. + +The [Building a VirtualBox Image with Packer in +TeamCity](/packer/guides/packer-on-cicd/build-virtualbox-image) guide shows +how to create a VirtualBox image using TeamCity's support for running scripts +on bare-metal machines. diff --git a/content/packer/content/docs/guides/packer-on-cicd/build-virtualbox-image.mdx b/content/packer/content/docs/guides/packer-on-cicd/build-virtualbox-image.mdx new file mode 100644 index 0000000000..69e4701840 --- /dev/null +++ b/content/packer/content/docs/guides/packer-on-cicd/build-virtualbox-image.mdx @@ -0,0 +1,149 @@ +--- +page_title: Build a VirtualBox Image with Packer in TeamCity +description: >- + Use Packer to build VirtualBox images in TeamCity. Learn how to provision a bare-metal machine, install dependencies, and create and run a build in TeamCity. +--- + +# Build a VirtualBox Image with Packer in TeamCity + +This guide walks through the process of building a VirtualBox image using +Packer on a new TeamCity Agent. Before getting started you should have access +to a TeamCity Server. + +The Packer VirtualBox builder requires access to VirtualBox. Virtualization is +not universally supported on cloud instances, so we recommend you run these +builds on either a bare metal server, or cloud instances which support nested +virtualization, such as Azure or GCP. This is also true for the +[VMware](/packer/plugins/builders/vmware) and the [QEMU](/packer/plugins/builders/qemu) +Packer builders. + +We will use Chef's [Bento boxes](https://github.com/chef/bento) to provision an +Ubuntu image on VirtualBox. For this example, we will use the repository +directly, but you may also fork it for the same results. + +## 1. Provision a Bare-metal Machine + +For the purposes of this example, we will run on a bare-metal instance from +[Packet](https://www.packet.net/). If you are a first time user of Packet, the +Packet team has provided HashiCorp the coupon code `hashi25` which you can use +for `$25` off to test out this guide and up to 30% if you decide to +reserve ongoing servers (email help@packet.net for details). You can use +a `baremetal_0` server type for testing, but for regular use, the `baremetal_1` +instance may be a better option. + +There is also a [Packet +Provider](https://registry.terraform.io/providers/packethost/packet/latest/docs) in +Terraform you can use to provision the project and instance. + +```hcl +provider "packet" { } + +resource "packet_project" "teamcity_agents" { + name = "TeamCity" +} + +resource "packet_device" "agent" { + hostname = "teamcity-agent" + plan = "baremetal_0" + facility = "ams1" + operating_system = "ubuntu_16_04" + billing_cycle = "hourly" + project_id = "${packet_project.teamcity_project.id}" +} +``` + +## 2. Install VirtualBox and TeamCity dependencies + +VirtualBox must be installed on the new instance, and TeamCity requires the JDK +prior to installation. This guide uses Ubuntu as the Linux distribution, so you +may need to adjust these commands for your distribution of choice. + +**Install Teamcity Dependencies** + +```shell-session +$ apt-get upgrade +$ apt-get install -y zip linux-headers-generic linux-headers-4.13.0-16-generic build-essential openjdk-8-jdk +``` + +**Install VirtualBox** + +```shell-session +$ curl -OL "https://download.virtualbox.org/virtualbox/5.2.2/virtualbox-5.2_5.2.2-119230~Ubuntu~xenial_amd64.deb" +$ dpkg -i virtualbox-5.2_5.2.2-119230~Ubuntu~xenial_amd64.deb +``` + +You can also use the [`remote-exec` +provisioner](/terraform/docs/provisioners/remote-exec) in +your Terraform configuration to automatically run these commands when +provisioning the new instance. + +## 3. Install Packer + +The TeamCity Agent machine will also need Packer Installed. You can find the +latest download link from the [Packer +Download](/packer/downloads) page. + +```shell-session +$ curl -OL "https://releases.hashicorp.com/packer/1.1.2/packer_1.1.2_linux_amd64.zip" +$ unzip ./packer_1.1.2_linux_amd64.zip +``` + +Packer is installed at the `/root/packer` path which is used in subsequent +steps. If it is installed elsewhere, take note of the path. + +## 4. Install TeamCity Agent + +This guide assume you already have a running instance of TeamCity Server. The +new TeamCity Agent can be installed by [downloading a zip file and installing +manually](https://confluence.jetbrains.com/display/TCD10//Setting+up+and+Running+Additional+Build+Agents#SettingupandRunningAdditionalBuildAgents-InstallingAdditionalBuildAgents), +or using [Agent +Push](https://confluence.jetbrains.com/display/TCD10//Setting+up+and+Running+Additional+Build+Agents#SettingupandRunningAdditionalBuildAgents-InstallingviaAgentPush). +Once it is installed it should appear in TeamCity as a new Agent. + +Create a new Agent Pool for agents responsible for VirtualBox Packer builds and +assign the new Agent to it. + +## 5. Create a New Build in TeamCity + +In TeamCity Server, create a new build. To use the upstream Bento repository, +we'll choose _From a repository URL_, and enter +`https://github.com/chef/bento.git` as the **Repository URL**. + +![TeamCity screenshot: New Build](/img/guides/teamcity_create_project_from_url-1.png) + +Click **Proceed**. + +![TeamCity screenshot: New Build](/img/guides/teamcity_create_project_from_url-2.png) + +And **Proceed** again. + +We won't use the _Auto-detected Build Steps_. Instead, click _configure build +steps manually_. For the _runner type_, pick **Command Line**, and enter the +following values. Make sure to click _Show advanced options_, as we need to set +the working directory. + +![TeamCity screenshot: Build Step](/img/guides/teamcity_build_configuration.png) + +This will use the `build` command in Packer to build the image defined in +`ubuntu/ubuntu-16.04-amd64.json`. It assumes that the VCS repository you're +using is a fork of [Chef/Bento](https://github.com/chef/bento). Packer defaults +to building VirtualBox machines by launching a GUI that shows the console. +Since this will run in CI/CD, use the [`headless` +variable](/packer/plugins/builders/virtualbox/iso#headless) to instruct Packer to +start the machine without the console. Packer can build multiple image types, +so the [`-only=virtualbox-iso` +option](/packer/docs/commands/build#only-foo-bar-baz) instructs Packer to only +build the builds with the name `virtualbox-iso`. + +## 6. Run a build in TeamCity + +The entire configuration is ready for a new build. Start a new run in TeamCity +by pressing “Run”. + +The new run should be triggered and the virtual box image will be built. + +![TeamCity screenshot: Build log](/img/guides/teamcity_build_log.png) + +Once complete, the build status should be updated to complete and successful. + +![TeamCity screenshot: Build log complete](/img/guides/teamcity_build_log_complete.png) diff --git a/content/packer/content/docs/guides/packer-on-cicd/index.mdx b/content/packer/content/docs/guides/packer-on-cicd/index.mdx new file mode 100644 index 0000000000..d083ed4766 --- /dev/null +++ b/content/packer/content/docs/guides/packer-on-cicd/index.mdx @@ -0,0 +1,17 @@ +--- +page_title: Build Immutable Infrastructure with Packer in CI/CD +description: >- + Learn how to use Packer to build immutable infrastructure within a continuous integration / continuous delivery pipeline. +--- + +# Build Immutable Infrastructure with Packer in CI/CD + +This guide focuses on the following workflow for building immutable +infrastructure. This workflow can be manual or automated and it can be +implemented with a variety of technologies. The goal of this guide is to show +how this workflow can be fully automated using Packer for building images from +a continuous integration/continuous deployment (CI/CD) pipeline. + +1. [Build Images using Packer in CI/CD](/packer/guides/packer-on-cicd/build-image-in-cicd) +2. [Upload the new image to S3](/packer/guides/packer-on-cicd/upload-images-to-artifact) for future deployment or use during development +3. [Create new Terraform Enterprise runs](/packer/guides/packer-on-cicd/trigger-tfe) to provision new instances with the images diff --git a/content/packer/content/docs/guides/packer-on-cicd/pipelineing-builds.mdx b/content/packer/content/docs/guides/packer-on-cicd/pipelineing-builds.mdx new file mode 100644 index 0000000000..e9605d4343 --- /dev/null +++ b/content/packer/content/docs/guides/packer-on-cicd/pipelineing-builds.mdx @@ -0,0 +1,354 @@ +--- +page_title: Packer Build Pipelines +description: >- + Learn how to shorten Packer build times and improve reliability. Start from an ISO, customize using the virtualbox-ovf builder, and improve efficiency. +--- + +# Why Create a Template Pipeline? + +A common issue users face when beginning to create their Packer templates is +that while they may need several specialized images, the early provisioning +steps are all the same. It can feel tedious to copy all of those images' basic +configuraton into each build template. It can feel even more tedious to wait a +long time for similar builds to run duplicate steps. + +This is one reason why Packer recommends breaking your builds into small, +discrete steps. Not only does it allow you to create "base" images that you can +build from to create further customizations, but it also allows you to save +time in your build process because the "base" images are likely to change less +than your customizations. + +It also makes it so that a failing build takes less time to debug and re-run. + +In this example, we will use the Virtualbox builders, but the concepts from +this example can be applied to other builders as well. + +## Starting from an ISO + +Here is an extremely basic virtualbox-iso template: + + + + +```hcl +source "virtualbox-iso" "step_1" { + boot_command = ["", "", "", + "/install/vmlinuz", " initrd=/install/initrd.gz", + " auto-install/enable=true", " debconf/priority=critical", + " preseed/url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ubuntu_preseed.cfg", + " -- ", ""] + disk_size = "40960" + guest_os_type = "Ubuntu_64" + http_directory = "./http" + iso_checksum = "sha256:946a6077af6f5f95a51f82fdc44051c7aa19f9cfc5f737954845a6050543d7c2" + iso_url = "http://old-releases.ubuntu.com/releases/14.04.1/ubuntu-14.04-server-amd64.iso" + shutdown_command = "echo 'vagrant' | sudo -S shutdown -P now" + ssh_password = "vagrant" + ssh_port = 22 + ssh_username = "vagrant" + vm_name = "vbox-example" +} +build { + sources = ["source.virtualbox-iso.step_1"] + + + provisioner "shell" { + inline = ["echo initial provisioning"] + } + post-processor "manifest" { + output = "stage-1-manifest.json" + } +} +``` + + + + +```json +{ + "builders": [ + { + "type": "virtualbox-iso", + "vm_name": "vbox-example", + "boot_command": [ + "", + "", + "", + "/install/vmlinuz", + " initrd=/install/initrd.gz", + " auto-install/enable=true", + " debconf/priority=critical", + " preseed/url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ubuntu_preseed.cfg", + " -- ", + "" + ], + "http_directory": "./http", + + "disk_size": "40960", + "guest_os_type": "Ubuntu_64", + "iso_checksum": "sha256:946a6077af6f5f95a51f82fdc44051c7aa19f9cfc5f737954845a6050543d7c2", + "iso_url": "http://old-releases.ubuntu.com/releases/14.04.1/ubuntu-14.04-server-amd64.iso", + + "shutdown_command": "echo 'vagrant' | sudo -S shutdown -P now", + + "ssh_port": 22, + "ssh_username": "vagrant", + "ssh_password": "vagrant" + } + ], + "provisioners": [ + { + "type": "shell", + "inline": ["echo initial provisioning"] + } + ], + "post-processors": [ + { + "type": "manifest", + "output": "stage-1-manifest.json" + } + ] +} +``` + + + + +In order to build using this template, create a directory named "http" in your +current working directory. Copy the minimal example from our +[preseed guide](/packer/guides/automatic-operating-system-installs/preseed_ubuntu#examples) +into a file in your http directory and name it "ubuntu_preseed.cfg". Copy the +above json template into your current working directory and save it as +"example_virtualbox_iso.json" + +To run the build, call `packer build example_virtualbox_iso.json`. + +This example does not set the output_directory or output_filename, so the file +will be placed in a default name of "output-virtualbox-iso/vbox-example.ovf" -- +the builder will print this file name to the UI output, but in this example the +[manifest](/packer/docs/post-processors/manifest) post-processor +to will store build information, including the names of the output files, in a +json file named "stage-1-manifest.json". From there, you can programmatically +look up the output file information. + +## Customizing the iso using the virtualbox-ovf builder + +That output filename generated in the first stage can be used as the +[source_path](/packer/plugins/builders/virtualbox/ovf#source_path) +for the virtualbox-ovf builder. + + + + +```hcl +source "virtualbox-ovf" "step_2" { + shutdown_command = "echo 'vagrant' | sudo -S shutdown -P now" + source_path = "output-virtualbox-iso/vbox-example.ovf" + ssh_password = "vagrant" + ssh_port = 22 + ssh_username = "vagrant" + vm_name = "virtualbox-example-ovf" +} + +build { + sources = ["source.virtualbox-ovf.step_2"] + + provisioner "shell" { + inline = ["echo secondary provisioning"] + } +} + +``` + + + + +```json +{ + "builders": [ + { + "type": "virtualbox-ovf", + "vm_name": "virtualbox-example-ovf", + + "shutdown_command": "echo 'vagrant' | sudo -S shutdown -P now", + "source_path": "output-virtualbox-iso/vbox-example.ovf", + + "ssh_password": "vagrant", + "ssh_port": 22, + "ssh_username": "vagrant" + } + ], + "provisioners": [ + { + "inline": ["echo secondary provisioning"], + "type": "shell" + } + ] +} +``` + + + + +## More efficiencies + +You may find that you want to run time-consuming import post-processors like +the "amazon-import" post-processor independently of the build that produces +the artifacts you want to process. + +In this case, you can use a null builder +and manually modify the input to the post-processing chain so that you can get +the behavior you want. The below example shows a "vagrant" post-processor +being used with a null builder, and manually sets the artifact from our +stage-2 ovf build: + + + + +```hcl +source "null" "step_3" { + communicator = "none" +} + +build { + sources = ["source.null.step_3"] + + post-processors { + post-processor "artifice" { + files = ["output-virtualbox-ovf/virtualbox-example-ovf.ovf", "output-virtualbox-ovf/virtualbox-example-ovf-disk001.vmdk"] + } + post-processor "vagrant" { + provider_override = "virtualbox" + } + } +} +``` + + + + +```json +{ + "builders": [ + { + "type": "null", + "communicator": "none" + } + ], + "post-processors": [ + [ + { + "type": "artifice", + "files": [ + "output-virtualbox-ovf/virtualbox-example-ovf.ovf", + "output-virtualbox-ovf/virtualbox-example-ovf-disk001.vmdk" + ] + }, + { + "type": "vagrant", + "provider_override": "virtualbox" + } + ] + ] +} +``` + + + + +By using the null builder instead of just running an ovf builder, we can spare +ourselves all of the time Packer would normally spend launching and destroying +VMs. + +## Putting it all together + +Packer templates don't come with a custom "glue" to bind them together. We +recommend using your CI system or wrapping scripts to connect the templates +into a chain. + +## Chaining together several of the same builders to make "save points" + +If you want to use the same builder for several builds in a row, this can feel +tedious to implement in json. We recommend you try using HCL configs so that +you can reuse the same source in several builds: + +HCL templates work by allowing you to draw sources and variables from multiple +different files in a single directory, so the following files are assumed to +exist in their own folder: + +sources.pkr.hcl + +```hcl +// In your sources file, you can create a configuration for a builder that you +// want to reuse between multiple steps in the build. Just leave the source +// and destination images out of this source, and set them specifically in each +// step without having to set all of the other options over and over again. + +source "docker" "example" { + commit = true + // any other configuration you want for your Docker containers +} +``` + +build.pkr.hcl + +```hcl +build { + // Make sure to name your builds so that you can selectively run them one at + // a time. + name = "step1" + + source "source.docker.example" { + image = "ubuntu" + } + + provisioner "shell" { + inline = ["echo example provisioner"] + } + provisioner "shell" { + inline = ["echo another example provisioner"] + } + provisioner "shell" { + inline = ["echo a third example provisioner"] + } + + // Make sure that the output from your build can be used in the next build. + // In this example, we're tagging the Docker image so that the step-2 + // builder can find it without us having to track it down in a manifest. + post-processor "docker-tag" { + repository = "ubuntu" + tag = ["step-1-output"] + } +} + +build { + name = "step2" + + source "source.docker.example" { + // This is the tagged artifact from the stage 1 build. You can retrieve + // this from a manifest file and setting it as a variable on the command + // line, or by making sure you define and know the output of the build, + // if it's something you can define like an output name or directory. + image = "ubuntu:step-1-output" + // disable the pull if your image tag only exists locally + pull = false + } + + provisioner "shell" { + inline = ["echo another provision!"] + } +} +``` + +pipeline.sh + +```sh +#!/bin/bash +set -e # abort if there is an issue with any build +packer build -only='step1.docker.example' . +packer build -only='step2.docker.example' . +``` + +To run the pipeline, call pipeline.sh. You can create as many build steps as +you want. Each can either inhabit one file, or you can put multiple steps in +a single file like shown in the example above. diff --git a/content/packer/content/docs/guides/packer-on-cicd/trigger-tfe.mdx b/content/packer/content/docs/guides/packer-on-cicd/trigger-tfe.mdx new file mode 100644 index 0000000000..100e046554 --- /dev/null +++ b/content/packer/content/docs/guides/packer-on-cicd/trigger-tfe.mdx @@ -0,0 +1,58 @@ +--- +page_title: Trigger Terraform Enterprise runs +description: >- + Learn how to Terraform Enterprise from your CI/CD pipeline to provision infrastructure. +--- + +# Create Terraform Enterprise Runs + +Once an image is built and uploaded to an artifact store, the next step is to +use this new image. In some cases the image will be downloaded by the dev team +and used locally in development, like is often done with VirtualBox images with +Vagrant. In most other cases, the new image will be used to provision new +infrastructure. + +You can use [Terraform](/terraform/cli) to provision new infrastructure with images generated by Packer. When working with teams, you can use [HCP Terraform](/terraform/cloud-docs) or [Terraform +Enterprise](/terraform/enterprise) to automate Terraform runs. + +## Create a Terraform Configuration and Workspace + +The following is a sample Terraform configuration which provisions a new AWS +EC2 instance. The `aws_ami_id` is a variable which will be provided when +running `terraform plan` and `terraform apply`. This variable references the +latest AMI generated with the Packer build in CI/CD. + +```hcl +variable "aws_ami_id" { } + +provider "aws" { + region = "us-west-2" +} + +resource "aws_instance" "web" { + ami = "${var.aws_ami_id}" + instance_type = "t2.micro" +} +``` + +Terraform Enterprise should have a workspace with this terraform configuration +and a placeholder variable `aws_ami_id`. + +## Include Terraform Enterprise in Your CI Builds + +Follow these steps to create a new run from CI/CD after a Packer build is +complete and uploaded. + +1. Add a new step to the CI/CD pipeline. +2. In the new step add a `curl` call to update the variables in the workspace + using the [update variables + API](/terraform/docs/enterprise/api/variables#update-variables), + so that Terraform has a reference to the latest image. For the sample + configuration above, the `aws_ami_id` variable should be updated to the AMI + ID of the latest image. +3. In that same step, add another `curl` call to [create a new run via the + API](/terraform/docs/enterprise/api/run#create-a-run). + A run performs a plan and apply on the last configuration version created, + using the variables set in the workspace. In the previous step we update the + variables, so the new run can be created using the previous configuration + version. diff --git a/content/packer/content/docs/guides/packer-on-cicd/upload-images-to-artifact.mdx b/content/packer/content/docs/guides/packer-on-cicd/upload-images-to-artifact.mdx new file mode 100644 index 0000000000..6728a19338 --- /dev/null +++ b/content/packer/content/docs/guides/packer-on-cicd/upload-images-to-artifact.mdx @@ -0,0 +1,45 @@ +--- +page_title: Upload VirtualBox Image to S3 +description: >- + Learn how to upload a VirtualBox image built with Packer to S3 within a TeamCity build. +--- + +# Upload VirtualBox Image to S3 + +Once the image is generated it will be used by other parts of your operations +workflow. For example, it is common to build VirtualBox images with Packer to +be used as base boxes in Vagrant. + +The exact process for uploading images depends on the artifact store and CI +system you use. TeamCity provides a [Build +Artifacts](https://confluence.jetbrains.com/display/TCD9/Build+Artifact) +feature which can be used to store the newly generated image. Other CI/CD +services also have similar build artifacts features built in, like [Circle CI +Build Artifacts](https://circleci.com/docs/2.0/artifacts/). In addition to the +built in artifact stores in CI/CD tools, there are also dedicated universal +artifact storage services like +[Artifactory](https://confluence.jetbrains.com/display/TCD9/Build+Artifact). +All of these are great options for image artifact storage. + +The following example uses TeamCity and Amazon S3. + +## Example: Uploading to S3 in a TeamCity Build + +On the agent machine responsible for building images, install the [AWS Command +Line Tool](https://aws.amazon.com/cli/). Since this is a one-time operation, +this can be incorporated into the initial agent provisioning step when +installing other dependencies. The AWS Command Line tool may require installing +additional +[dependencies](http://docs.aws.amazon.com/cli/latest/userguide/installing.html) +prior. + +```shell-session +$ pip install awscli +``` + +In your build configuration in TeamCity Server, add an additional **Build Step: +Command Line** and set the **Script content** field to the following: + +```shell-session +$ awscli s3 cp . s3://bucket/ --exclude “*” --include “*.ovf" +``` diff --git a/content/packer/content/docs/hcp/index.mdx b/content/packer/content/docs/hcp/index.mdx new file mode 100644 index 0000000000..7143b2ba01 --- /dev/null +++ b/content/packer/content/docs/hcp/index.mdx @@ -0,0 +1,125 @@ +--- +description: | + Packer can publish metadata for completed builds to the HCP Packer registry. Learn how to connect Packer to the HCP Packer registry. +page_title: Connect to the HCP Packer registry overview +--- + +# Connect to the HCP Packer Registry + +This topic provides an overview of how to connect JSON and HCL2 templates to the HCP Packer registry and provides a full list of HCP Packer environment variables. Refer to the +[Packer Template Configuration](/hcp/docs/packer/store-image-metadata/packer-template-configuration) page in the HCP +Packer documentation for configuration details and examples. + +# Introduction + +The HCP Packer registry bridges the gap between artifact factories and artifact deployments, allowing development and +security teams to work together to create, manage, and consume artifacts in a centralized way. + +The HCP Packer Registry stores metadata about your artifact, including when they were created, where the artifact +exists on the external platform, and what (if any) git commit is associated with your build. You can use the registry +to track information about the artifact your Packer builds produce, clearly designate which artifact are appropriate +for test and production environments, and query for the right artifact to use in both Packer and Terraform +configurations. + +You can use HCP Packer with both JSON and HCL2 templates. If you are using JSON templates, we recommend getting started with +the [HCP Packer environment variables](#hcp-packer-environment-variables) and then migrating to HCL when possible. + +## Requirements + +Packer version 1.9.1 or newer is required to use the `HCP_PROJECT_ID` environment variable, which lets Packer connect to specific projects in HCP. Your builds will fail if you configure them to send mulit-project metadata using Packer versions older than 1.9.1. + +## HCP Packer environment variables + +The following environment variables let you configure Packer to push artifact metadata to an active registry without +changing your template. You can use environment variables with both JSON and HCL2 templates. +Refer to [Basic Configuration With Environment Variables](/hcp/docs/packer/store-image-metadata/packer-template-configuration#basic-configuration-with-environment-variables) +in the HCP Packer documentation for complete instructions and examples. + +You must set authentication environment variables to connect to HCP Packer, you can either directly set a client ID and secret, or (on Packer versions 1.14.2 and later) use an HCP certificate file + +For client ID and secret, you set the following environemnt variables + +- `HCP_CLIENT_ID` - The HCP client ID of a HashiCorp Cloud Platform service principle that Packer can use to +authenticate to an HCP Packer Registry. + +- `HCP_CLIENT_SECRET` - The HCP client secret of the HashiCorp Cloud Platform service principle that Packer +can use to authenticate to an HCP Packer Registry. + +For certificate based auth simply specify the location of the valid HCP certificate file in the `HCP_CRED_FILE` environment variable, or place it in the default location for the HCP SDK, `~/.config/hcp/cred_file.json` + +See the following HCP docs for more information on [Workload Identity Federation](https://developer.hashicorp.com/hcp/docs/hcp/iam/service-principal/workload-identity-federation) and certificate authentication + +- `HCP_PACKER_BUCKET_NAME` - The name of the HCP Packer Bucket where you want HCP Packer to store artifact metadata +from builds associated with your template. HCP Packer automatically creates the bucket if it does not already exist. +If your HCL2 template contains an `hcp_packer_registry` block, the bucket name specified in the configuration will be +overwritten by this environment variable. + +You can set these additional environment variables to control how metadata is pushed to the registry. + +- `HCP_PACKER_BUILD_FINGERPRINT` - A unique identifier assigned to each version. To reuse a fingerprint that is +associated with an existing incomplete version you must set this environment variable. Refer to +[Version Fingerprinting](#version-fingerprinting) for usage details. + +- `HCP_PACKER_REGISTRY` - When set, Packer does not push artifact metadata to HCP Packer from an otherwise +configured template. Allowed values are [0|OFF]. + +- `HCP_ORGANIZATION_ID` - The ID of the HCP organization linked to your service principal. This is environment +variable is not required and available for the sole purpose of keeping parity with the HCP SDK authentication options. +Its use may change in a future release. + +- `HCP_PROJECT_ID` - The ID of the HCP project to use. This is useful if your service principal has access to multiple +projects, as by default Packer will pick the one created first as target. + +-> **Note**: The HCP_PROJECT_ID environment variable must be set if you're authenticating with a project-level service +principal, otherwise Packer will attempt to get the list of projects for an organization and error due to a lack of +permissions for a project-level service principal. This is supported starting with Packer 1.9.3; older versions of +Packer do not support using project-level service principals. + +## HCP Packer registry block + +The only metadata that Packer can infer from a template with the basic configuration are the build name and build fingerprint. +For HCL2 templates, we recommend adding the `hcp_packer_registry` block to your template so that you can customize +the metadata that Packer sends to the registry. + +The `hcp_packer_registry` block is only available for HCL2 Packer templates. There is no [`PACKER_CONFIG`](/packer/docs/configure#packer-s-config-file) equivalent for JSON. + +Refer to [`hcp_packer_registry`](/packer/docs/templates/hcl_templates/blocks/build/hcp_packer_registry) for a full list +of configuration arguments. Refer to [Custom Configuration](/hcp/docs/packer/store-image-metadata/packer-template-configuration#custom-configuration) +in the HCP Packer documentation for information and examples about how to customize artifact metadata. + +## Version fingerprinting + +Packer uses a unique fingerprint for tracking the completion of builds associated to a version. By default a fingerprint +is automatically generated by Packer during each invocation of `packer build`, unless a fingerprint is manually provided +via the `HCP_PACKER_BUILD_FINGERPRINT` environment variable. + +In versions before 1.9.0, this fingerprint was computed from the Git SHA of the current HEAD in which your template is +stored. If you were running builds using a non Git managed template, you had to set the `HCP_PACKER_BUILD_FINGERPRINT` +environment variable prior to invoking `packer build`. +Starting with Packer 1.9.0, fingerprint generation does not rely on Git at all, and instead Packer now generates +a Unique Lexicographically sortable Identifier (ULID) as the fingerprint for every `packer build` invocation. + +### Fingerprints and incomplete versions + +When you build a template with Packer, there's always a chance that it does not succeed because of a network issue, +a provisioning failure, or some upstream error. When that happens, Packer will output the generated fingerprint +associated with the incomplete version so that you can resume building that version using the `HCP_PACKER_BUILD_FINGERPRINT` +environment variable; a version can be resumed until it is marked as complete. This environment variable is necessary +for resuming an incomplete version, otherwise Packer will create a new version for the build. + +There are two alternatives for when and how to set your own fingerprint: + +* You can set it prior to invoking `packer build` for the first time on this template. This will require the +fingerprint to be unique, otherwise Packer will attempt to continue the version with the same fingerprint. +* You can invoke `packer build` on the template, and if it fails, you can then get the fingerprint from the output of +the command and set it for subsequent runs, Packer will then continue building this version. + +The first alternative is recommended for CI environments, as you can use environment variables from the CI to generate +a unique, deterministic, fingerprint, and then re-use this in case the step fails for any reason. This will let you +continue building the same version, rather than creating a new one on each invocation. + +The second alternative is good for local builds, as you can interact with the build environment directly, and therefore +can decide if you want to continue building a version by setting the fingerprint provided by Packer in case of failure. + +Please note that in all cases, a version can only be continued if it has not completed yet. Once a version is +complete, it cannot be modified, and you will have to create a new one. diff --git a/content/packer/content/docs/index.mdx b/content/packer/content/docs/index.mdx new file mode 100644 index 0000000000..d2962f287a --- /dev/null +++ b/content/packer/content/docs/index.mdx @@ -0,0 +1,23 @@ +--- +description: | + The Packer documentation describes how to use Packer and providers reference information for configuring Packer templates. +page_title: Packer documentation +--- + +# Packer Documentation + +Packer is a community tool that lets you to create identical machine images for multiple platforms from a single source template. People commonly use Packer to create golden images that teams across an organization can use in cloud infrastructure. + +To install Packer and learn the standard Packer workflow, try the [Get Started tutorials](/packer/tutorials). + +## HCP Packer + +The HCP Packer registry stores metadata about your artifacts. You can use the registry to track information about +artifacts from your Packer builds, clearly designate which artifacts are appropriate for test and production +environments, and query for the right artifacts to use in both Packer and Terraform configurations. + +Refer to the following resources to get started: + +- [HCP Packer documentation](/hcp/docs/packer) +- [HCP Packer tutorials](/packer/tutorials/hcp-get-started) +- [Log into the HCP portal](https://portal.cloud.hashicorp.com) \ No newline at end of file diff --git a/content/packer/content/docs/intro/index.mdx b/content/packer/content/docs/intro/index.mdx new file mode 100644 index 0000000000..81d1e0f502 --- /dev/null +++ b/content/packer/content/docs/intro/index.mdx @@ -0,0 +1,27 @@ +--- +page_title: Introduction to Packer +description: Packer is a community tool for creating identical machine images for multiple platforms from a single source configuration. Learn about Packer benefits and how to get started. +--- + +# Introduction to Packer + +This introduction describes Packer benefits and how you can get started with it. + +## What is Packer? + +Packer is a community tool for creating identical machine images for multiple +platforms from a single source configuration. Packer is lightweight, runs on +every major operating system, and is highly performant, creating machine images +for multiple platforms in parallel. Packer does not replace configuration +management like Chef or Puppet. In fact, when building images, Packer is able to +use tools like Chef or Puppet to install software onto the image. + +A _machine image_ is a single static unit that contains a pre-configured +operating system and installed software which is used to quickly create new +running machines. Machine image formats change for each platform. Some examples +include AMIs for EC2, VMDK and VMX files for VMware, and OVF exports for VirtualBox. + +## HCP Packer + +For information about using HCP Packer to store metadata about build artifacts, refer to the +[HCP Packer documentation](/hcp/docs/packer) or [sign into HCP](https://portal.cloud.hashicorp.com/sign-in) to explore HCP Packer features. diff --git a/content/packer/content/docs/intro/use-cases.mdx b/content/packer/content/docs/intro/use-cases.mdx new file mode 100644 index 0000000000..6b4a22cc17 --- /dev/null +++ b/content/packer/content/docs/intro/use-cases.mdx @@ -0,0 +1,46 @@ +--- +page_title: Packer use cases +description: |- + Learn about use cases for Packer, such as continuous delivery, dev/prod parity, and appliance and demo creation. +--- + +# Packer use cases + +In this topic describes some of the use cases for Packer. This is a partial list of use cases intended to give you an idea of how Packer may improve your processes. + +## Continuous delivery + +Packer is lightweight, portable, and command-line driven. This makes it the +perfect tool to put in the middle of your continuous delivery pipeline. Packer +can be used to generate new machine images for multiple platforms on every +change to Chef/Puppet. + +As part of this pipeline, the newly created images can then be launched and +tested, verifying the infrastructure changes work. If the tests pass, you can be +confident that the image will work when deployed. This brings a new level of +stability and testability to infrastructure changes. + +## Environment parity + +Packer helps keep development, staging, and production as similar as +possible. Refer to the following external article to learn more about parity between environments: ["X. Dev/prod parity"](http://www.12factor.net/dev-prod-parity). + +You can use Packer to generate images for multiple platforms at the same time. So if you use AWS for production and VMware, perhaps with [Vagrant](https://www.vagrantup.com/), for +development, you can generate both an AMI and a VMware machine using Packer at +the same time from the same template. + +Mix this in with the continuous delivery use case above, and you have a pretty +slick system for consistent work environments from development all the way +through to production. + +## Appliance and demo creation + +Because Packer creates consistent images for multiple platforms in parallel, it is +perfect for creating +[appliances](https://en.wikipedia.org/wiki/Software_appliance) and disposable +product demos. As your software changes, you can automatically create appliances +with the software pre-installed. Potential users can then get started with your +software by deploying it to the environment of their choice. + +Packaging up software with complex requirements has never been so easy. Or +enjoyable, if you ask me. diff --git a/content/packer/content/docs/intro/why.mdx b/content/packer/content/docs/intro/why.mdx new file mode 100644 index 0000000000..39ddf78fcb --- /dev/null +++ b/content/packer/content/docs/intro/why.mdx @@ -0,0 +1,47 @@ +--- +page_title: Why use Packer +description: |- + Learn about the advantages of using Packer to automate the creation of machine images and artifacts. +--- + +# Why use Packer + +This topic describes why you should use Packer to automate the creation of machine images over traditional _pre-baked_ images, which are pre-configured digital images that include the necessary software, settings, and data. + +## Pre-baked images + +Pre-baked machine images have a lot of advantages, but most people have been unable to +benefit from them because images have been too tedious to create and manage. +There are either no existing tools to automate the creation of machine images +or they had too high of a learning curve. The result is that, prior to Packer, +creating machine images threatened the agility of operations teams, and +therefore aren't used, despite the massive benefits. + +Packer changes all of this. Packer automates the creation of +any type of machine image. It embraces modern configuration management by +encouraging you to use a framework such as Chef or Puppet to install and +configure the software within your Packer-made images. + +## Advantages of using Packer + +**_Super fast infrastructure deployment_**. Packer images allow you to launch +completely provisioned and configured machines in seconds, rather than several +minutes or hours. This benefits not only production, but development as well, +since development virtual machines can also be launched in seconds, without +waiting for a typically much longer provisioning time. + +**_Multi-provider portability_**. Because Packer creates identical images for +multiple platforms, you can run production in AWS, staging/QA in a private cloud +like OpenStack, and development in desktop virtualization solutions such as +VMware or VirtualBox. Each environment is running an identical machine image, +giving ultimate portability. + +**_Improved stability_**. Packer installs and configures all the software for a +machine at the time the image is built. If there are bugs in these scripts, +they'll be caught early, rather than several minutes after a machine is +launched. + +**_Greater testability_**. After a machine image is built, that machine image +can be quickly launched and smoke tested to verify that things appear to be +working. If they are, you can be confident that any other machines launched from +that image will function properly. \ No newline at end of file diff --git a/content/packer/content/docs/partnerships.mdx b/content/packer/content/docs/partnerships.mdx new file mode 100644 index 0000000000..8f2c7d48fb --- /dev/null +++ b/content/packer/content/docs/partnerships.mdx @@ -0,0 +1,146 @@ +--- +description: | + The Packer Integration Program helps vendors integrate their products with Packer. Learn how to participate in the Packer integration program. +page_title: Packer integration program +--- + +# Packer integration program + +The HashiCorp Packer Integration Program allows vendors to integrate their products to work with Packer. + +Vendors integrating their solutions via the Packer Integration Process provide their customers a verified and seamless user experience. The Packer Integration Program currently only supports coding with the Go programming language. + +This program is intended to be largely a self-service process with links and guidance to information sources, clearly defined steps, and checkpoints. + +## Types of Packer integrations + +Packer is a community tool for creating identical machine images for multiple platforms from a single source configuration. +Packer is lightweight, runs on every major operating system, and is highly performant, creating machine images for multiple platforms in parallel. Packer does not replace configuration management like Chef or Puppet. In fact, when building images, Packer is able to use tools like Chef or Puppet to install software onto the image. + +A machine image is a single static unit that contains a pre-configured operating system and installed software which is used to quickly create new running machines. Machine image formats change for each platform. Some examples include AMIs for EC2, VMDK/VMX files for VMware, OVF exports for VirtualBox, etc. To know more about use cases of Packer click ([Use Cases - Introduction | Packer by HashiCorp](/packer/docs/intro/use-cases)) + +The diagram below depicts the key Packer integration categories and types. + + + +![Packer Ecosystem Integration Diagram](/img/docs/packer-ecosystem-diagram.png) + + + +Main Packer categories for partners to integrate with include: + +- **Data Sources** + - Data Sources allow users to retrieve values from a remote API and store them as variables in the Packer configuration template. An example is the [AWS secrets manager](/packer/plugins/datasources/amazon/secretsmanager) data source. +- **Builders** + - Builders manage the VM lifecycle. They manage launching a VM/instance, running provisioners against that instance, shutting the instance down, and saving an artifact image from that instance. Your builder handles all of the setup and cleanup costs associated with creating the output image artifact. +- **Provisioners** + - Provisioners are run against the instance launched by the builder. They generally provide an interface for interacting with a particular provisioning tool, such as Ansible, Puppet, or Chef. +- **Post-Processors** + - Post-Processors manage the image artifact after it has been created. This can be something general like running a compression tool against the artifact, or something specific like uploading it to a particular cloud service. + +## Development process + +The Packer integration development process is divided into six steps. By following these steps, Packer integrations can be developed alongside HashiCorp to ensure that the integrations are able to be verified and supported in Packer as quickly as possible. A visual representation of the self-guided steps is depicted below. + +![Packer Integration Program Steps](/img/docs/packer-integration-steps-diagram.png 'Packer Integration Program Steps') + +The individual Packer integration steps include: + +1. Engage: Initial contact between vendor and HashiCorp +1. Enable: Information and articles to aid with the development of the integration +1. Dev/Test: Integration development and test process +1. Review: HashiCorp code review and verification of integration (iterative process) +1. Release: Verified integration made available and listed on the HashiCorp website once the HashiCorp technology partnership agreement has been executed +1. Support: Ongoing maintenance and support of the provider by the vendor. + +### Engage + +Begin by providing some basic information about the integration that is being built via a simple [webform](https://docs.google.com/forms/d/e/1FAIpQLSfgq3HJ9Rfsi7LgPLFln28ZrmarATGlD_6A47-Io-bPUftKUw/viewform) + +This information is recorded and used by HashiCorp to track the integration through various stages. The information is also used to notify the integration developer of any overlapping work, perhaps coming from the community so you may better focus resources. + +Packer has an active community and ecosystem of partners that may have already started working on a similar integration. We'll do our best to connect similar parties to avoid duplicate work. + +### Enable + +While not mandatory, HashiCorp encourages vendors to sign an MNDA (Mutual Non-Disclosure Agreement) to allow for open dialog and sharing of ideas during the integration process. + +In an effort to support our self-serve model we’ve included links to resources, documentation, examples and best practices to guide you through the Packer integration development and testing process. + +- [Writing vendor extension guide](/packer/docs/plugins/creation) +- Sample development implemented by a [partner](https://github.com/exoscale/packer-plugin-exoscale) +- [Scaffolding plugin repository](https://github.com/hashicorp/packer-plugin-scaffolding) to help bootstrap a new contribution: +- Contributing to Packer [guidelines](https://github.com/hashicorp/packer/blob/main/.github/CONTRIBUTING.md) +- [Packer developer community forum](https://discuss.hashicorp.com/c/packer/23) +- [Packer's source code](https://github.com/hashicorp/packer) + +We encourage vendors to closely follow the above guidance. Adopting the same structure and coding patterns helps expedite the review and release cycles. + +### Develop and test + +Packer requires all code-level integrations to be written in the [Go](https://go.dev/) programming language and contain an [MPL-2.0](https://en.wikipedia.org/wiki/Mozilla_Public_License) open source license. The only knowledge necessary to write a plugin is basic command-line skills and knowledge of the Go programming language. When writing in Go, HashiCorp has found the integration development process to be straightforward and simple when vendors pay close attention and follow the resources and by adopting the same structure and coding patterns helps expedite the review and release cycles. Please remember that all integration major steps should contain acceptance testing and the appropriate documentation. + +Data Sources + +- [Custom Data Sources documentation](/packer/docs/plugins/creation/custom-datasources) +- [Example Data Source](https://github.com/hashicorp/packer-plugin-hashicups/tree/main/datasource) + +Builders + +- [Custom Builders documentation](/packer/docs/plugins/creation/custom-builders) +- [Example Builder](https://github.com/hashicorp/packer-plugin-hashicups/tree/main/builder/order) + +Provisioners + +- [Custom Provisioners documentation](/packer/docs/plugins/creation/custom-provisioners) +- [Example Provisioner](https://github.com/hashicorp/packer-plugin-hashicups/tree/main/provisioner/toppings) + +Post-Processors + +- [Custom Post-Processors documentation](/packer/docs/plugins/creation/custom-post-processors) +- [Example Post-Processor](https://github.com/hashicorp/packer-plugin-hashicups/tree/main/post-processor/receipt) + +Packer-Plugin-SDK + +- The [Packer-plugin SDK](https://github.com/hashicorp/packer-plugin-sdk) contains tools to help plugin developers with common needs, like handling SSH connections or basic plugin architecture. + +### Review + +During the review process, HashiCorp will provide feedback on the newly developed integration. This is an important step to allow HashiCorp to review and verify your Packer integration. Please send the integration code and other relevant logs for verification to: [Packer-integration-dev@hashicorp.com](mailto:packer-integration-dev@hashicorp.com). + +In order to document your plugins with Packer, please submit a GitHub pull request (PR) against the [Packer project](https://github.com/hashicorp/packer). See [Registering Plugin Documentation](/packer/docs/plugins/creation#registering-plugin-documentation) for instructions on how to register your remote plugin documentation with Packer. +The review process can take a while to complete and may require some iterations through the code to address any problems identified by the HashiCorp team. + +### Release + +At this stage, it is expected that the integration is fully complete, the necessary documentation has been written, the acceptance tests have all passed, and that HashiCorp has reviewed the integration. Once the plugin has been validated and accepted by HashiCorp, the plugin can be hosted on GitHub so it can easily be [downloaded then installed within Packer](/packer/docs/plugins/creation#creating-a-github-release). + +Once the integration has been released the vendor is requested to sign the HashiCorp Technology Partner Agreement so that we can have their integration be listed on the HashiCorp website. + +### Support + +Many vendors view the release step to be the end of the journey, while at HashiCorp we view it to be the beginning of the journey. Getting the integration built is just the first step in enabling users to leverage it against their infrastructure. Once development is completed, on-going effort is required to support the developed integration, maintain the plugin and address any issues in a timely manner. + +The expectation from the vendor/partner is to create a mechanism for them to track and resolve all issues on an ongoing basis. Vendors who choose to not support their integration will not be considered a verified integration and cannot be listed on the website. + +## Checklist + +Below is an ordered checklist of steps that should be followed during the integration process. This just reiterates the steps already documented in the sections above. + + + +- Fill out the Packer integration [webform](https://docs.google.com/forms/d/e/1FAIpQLSfgq3HJ9Rfsi7LgPLFln28ZrmarATGlD_6A47-Io-bPUftKUw/viewform) +- Execute the HashiCorp MNDA (Mutual Non-Disclosure Agreement) if needed +- Develop and test Packer integration along with the acceptance tests and documentation +- Send email to [packer-integration-dev@hashicorp.com](mailto:packer-integration-dev@hashicorp.com) to schedule an initial review +- Address review feedback and finalize the development process +- Provide HashiCorp with credentials for underlying infrastructure for test purposes +- Demo the integration and/or send the test logs to HashiCorp at: [packer-integration-dev@hashicorp.com](mailto:packer-integration-dev@hashicorp.com) +- Execute HashiCorp Partner Agreement Documents, review logo guidelines, partner listing and more +- Plan to continue supporting the integration with additional functionality and responding to customer issues. + + + +## Contact Us + +For any questions or feedback, please contact us at: packer-integration-dev@hashicorp.com. diff --git a/content/packer/content/docs/plugins/creation/custom-builders.mdx b/content/packer/content/docs/plugins/creation/custom-builders.mdx new file mode 100644 index 0000000000..6f875e91b2 --- /dev/null +++ b/content/packer/content/docs/plugins/creation/custom-builders.mdx @@ -0,0 +1,309 @@ +--- +description: | + You can create custom builders for Packer that extend building functionality. Learn how write custom builders using the Packer plugin interface. +page_title: Create custom builders +--- + +# Create Custom Builders + +Packer builders are responsible for creating a virtual machine, setting the virtual machine up for provisioning, and then turning that provisioned virtual machine into a machine image. We officially maintain and distribute several builders, including builders to create images on Amazon EC2, VMware, Google +Compute Engine, and many more. Refer to the [Builders](/packer/docs/builders) documentation for details. + +This page explains how to use the Packer plugin interface to write custom builders. If you want your builder to support HashiCorp Cloud Platform (HCP) Packer, you should also review the [HCP Packer Support](/packer/docs/plugins/creation/hcp-support) documentation. + +~> **Warning:** This is an advanced topic that requires strong knowledge of Packer and Packer plugins. + +## Before You Begin + +We recommend reviewing the following resources before you begin development: + +- [Developing Plugins - Overview](/packer/docs/plugins/creation) +- The [Go](https://go.dev/) language. You must write custom plugins in Go, so this guide assumes you are familiar with the language. + +## The Interface + +To create your own builder, you must create a struct that implements the +[`packer.Builder`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer#Builder) interface. It is reproduced below for reference. + +```go +type Builder interface { + ConfigSpec() hcldec.ObjectSpec + Prepare(...interface{}) ([]string, []string, error) + Run(context.Context, ui Ui, hook Hook) (Artifact, error) +} +``` + +### The "ConfigSpec" Method + +This method returns a hcldec.ObjectSpec, which is a spec necessary for using +HCL2 templates with Packer. For information on how to use and implement this +function, check our +[object spec docs](/packer/guides/hcl/component-object-spec) + +### The "Prepare" Method + +The `Prepare` method for each builder will be called by the Packer core +at the beginning of the build. Its purpose is to parse and validate the +configuration template provided to Packer with `packer build your_packer_template.json`, but not to execute API calls or begin creating any +resources or artifacts. + +The configuration from your Packer template is passed into the Prepare() method +as an array of `interface{}` types, but is generally `map[string]interface{}`. +The Prepare method is responsible for translating this configuration into an +internal structure, validating it, and returning any errors. + +If multiple parameters are passed into Prepare(), they should be merged together +into the final configuration, with later parameters overwriting any previous +configuration. The exact semantics of the merge are left to the builder author. + +We recommend that you use the +[mapstructure](https://godoc.org/github.com/mitchellh/mapstructure) library to +decode the `interface{}` into a meaningful structure. Mapstructure will take an +`interface{}` and decode it into an arbitrarily complex struct. If there are any +errors, it generates very human friendly errors that can be returned directly +from the prepare method. You can find many usage examples of this library within +the Prepare() methods of HashiCorp-maintained Packer plugins. + +While Packer does not actively enforce this, **no side effects** should occur +from running the `Prepare` method. Specifically: don't create files, don't +launch virtual machines, etc. Prepare's purpose is solely to load the +configuration from the template into a format usable by your builder, to +validate that configuration, and to apply necessary defaults to that +configuration. + +In addition to the configuration provided in the Packer template, Packer will +also supply a [common.PackerConfig](https://github.com/hashicorp/packer-plugin-sdk/blob/8a28198491f70deca3824ce452adf6f9bd507880/common/packer_config.go#L44) +containing meta-information such as the build name, builder type, core version, +etc, and coded into a `map[string]interface{}`. One important piece of +meta information in this map is the `packer.DebugConfigKey` set to boolean +`true` if debug mode is enabled for the build. If this is set to true, then the +builder should enable a debug mode which assists builder developers and +advanced users to introspect what is going on during a build. During debug +builds, parallelism is strictly disabled, so it is safe to request input from +stdin and so on. + +Prepare() returns an array of strings and an error. The array of strings is a +special list of keys for variables created at runtime which your builder will +make accessible to provisioners using the generatedData mechanism (see below for +more details) An example could be an instance ID for the cloud instance created +by your builder. If you do not plan to make use of the generatedData feature, +just return an empty list. The error should be used if there is something wrong +with the user-provided configuration and the build should not proceed. + +### The "Run" Method + +`Run` is executed, often in parallel for multiple builders, to actually build +the machine, provision it, and create the resulting machine image, which is +returned as an implementation of the `packer.Artifact` interface. + +The `Run` method takes three parameters. The context.Context used to cancel the +build. The `packer.Ui` object is used to send output to the console. +`packer.Hook` is used to execute hooks, which are covered in more detail in the +Provisioning section below. + +Because builder runs are typically a complex set of many steps, the +packer-plugin-sdk contains a +[multistep](https://godoc.org/github.com/hashicorp/packer-plugin-sdk/multistep) +module. Multistep allows you to separate your build logic into multiple distinct +"steps" with separate run and cleanup phases, and run them in order. It +supports cancellation mid-step, pausing between steps when debugging, the CLI's +on-error flag, and more. All of the HashiCorp maintained builders make use of +this module, and while it is not required for builder implementation, it will +help you create your builder in a way that matches user and Packer Core +assumptions. The SDK also provides a number of "helper" generic steps that may +prevent you from having to re-implement work that has already been done by the +HashiCorp maintainers. Examples include sending boot commands, connecting to +SSH, and creating virtual CDs to mount on your VM. Take a look at the +[communicator](https://github.com/hashicorp/packer-plugin-sdk/tree/main/communicator) +and +[multistep/commonsteps](https://github.com/hashicorp/packer-plugin-sdk/tree/main/multistep/commonsteps) +modules in the SDK to see what tools are available to you. + +Finally, `Run` should return an implementation of `packer.Artifact`. More +details on creating a `packer.Artifact` are covered in the artifact section +below. If something goes wrong during the build that prevents an artifact from +being correctly created, `Run` should return an error and a nil artifact. Note +that your builder is allowed to produce no artifact and no error, although this +is a rare use case. + +### Cancellation + +#### With the "Cancel" Method ( for plugins for Packer < v1.3 ) + +The `Cancel` method can be called at any time and requests cancellation of any +builder run in progress. This method should block until the run actually stops. +Note that the Cancel method will not be called by Packer versions >= 1.4.0. + +#### Context cancellation ( from Packer v1.4 ) + +The `<-ctx.Done()` can unblock at any time and signifies request for +cancellation of any builder run in progress. + +Cancels are most commonly triggered by external interrupts, such as the user +pressing `Ctrl-C`. Packer will only exit once all the builders clean up, so it +is important that you architect your builder in a way that it is quick to +respond to these cancellations and clean up after itself. If your builder makes +a long-running call, you should consider the possibility that a user may cancel +the build during that call, and make sure that such a cancellation is not +blocked. + +## Creating an Artifact + +The `Run` method is expected to return an implementation of the +`packer.Artifact` interface. Each builder must create its own implementation of +this interface. + +Most of the pieces of an artifact should be fairly self-explanatory by reading +the [packer.Artifact interface +documentation](https://godoc.org/github.com/hashicorp/packer-plugin-sdk/packer#Artifact). + +However one part of an artifact that may be confusing is the `BuilderId` method. +This method must return an absolutely unique ID for the builder. In general, a +reasonable ID would be the github username or organization that created the +builder, followed by the platform it is building for. For example, the builder +ID of the VMware builder is "hashicorp.vmware". + +Post-processors use the builder ID value in order to make some assumptions +about the artifact results and to determine whether they are even able to run +against a given artifact, so it is important that this ID never changes once +the builder is published. + +The builder ID for each builder is included on the associated documentation page. + +## Provisioning + +Packer has built-in support for provisioning using the Provisioner plugins. But +builders themselves, rather than the Packer core, must determine when to invoke +the provisioners since only the builder knows when the machine is running and +ready for communication. + +When the machine is ready to be provisioned, run the `packer.HookProvision` +hook, making sure the communicator is not nil, since this is required for +provisioners. An example of calling the hook is shown below: + +```go +hook.Run(context.Context, packer.HookProvision, ui, comm, nil) +``` + +At this point, Packer will run the provisioners and no additional work is +necessary. + +If you are using the multistep tooling, the Packer plugin SDK contains a +generic StepProvision which handles execution the provision hook for you and +automatically supplies any custom builder generatedData you would like to +provide to provisioners (see below for more details on generatedData.) + +## Template Engine + +~> Note: In HCL2 the JSON template engine and generated data is slowly going to +be deprecated in favor of using HCL2 objects, we will keep on supporting those +but it is recommended to avoid using them if you can. + +### Build variables + +Packer JSON makes it possible to provide custom template engine variables to be +shared with provisioners and post-processors using the `build` function. +JSON template `build` docs are [here](/packer/docs/templates/legacy_json_templates/engine#build) +and HCL template build docs are [here](/packer/docs/templates/hcl_templates/contextual-variables#build-variables). + +As of Packer v1.5.0, builder Prepare() methods return a list of custom variables +which we call `generated data`. We use that list of variables to generate a +custom placeholder map per builder that combines custom variables with the +placeholder map of default build variables created by Packer. Here's an example +snippet telling packer what will be made available by the builder: + +```go +func (b *Builder) Prepare(raws ...interface{}) ([]string, []string, error) { + // ... + + generatedData := []string{"SourceImageName"} + return generatedData, warns, nil +} +``` + +When a user provides a Packer template that uses a `build` function, Packer +validates that the key in the `build` function exists for a given builder using +this generatedData array. If it does not exist, then Packer validation will +fail. + +Once the placeholder is set, it's necessary to pass the variables' real values +when calling the provisioner. This can be done as the example below: + +```go +func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) { + // ... + + // Create map of custom variable + generatedData := map[string]interface{}{"SourceImageName": "the source image name value"} + // Pass map to provisioner + hook.Run(context.Context, packer.HookProvision, ui, comm, generatedData) + + // ... +} +``` + +In order to make these same variables and the Packer default ones also available +to post-processors, your builder will need to add them to its Artifact. +This can be done by adding an attribute of type `map[string]interface{}` to the +Artifact and putting the generated data in it. The post-processor will access +this data later via the Artifact's `State` method. + +The Artifact code should be implemented similar to the below: + +```go +type Artifact struct { + // ... + + // StateData should store data such as GeneratedData + // to be shared with post-processors + StateData map[string]interface{} +} + +// ... + +func (a *Artifact) State(name string) interface{} { + return a.StateData[name] +} + +// ... +``` + +The builder should return the above Artifact containing the generated data and +the code should be similar to the example snippet below: + +```go +func (b *Builder) Run(ctx context.Context, ui packer.Ui, hook packer.Hook) (packer.Artifact, error) { + // ... + + return &Artifact{ + // ... + StateData: map[string]interface{}{"generated_data": state.Get("generated_data")}, + }, nil +} +``` + +The code above assigns the `generated_data` state to the `StateData` map with +the key `generated_data`. + +Here some example of how this data will be used by post-processors: + +```go +func (p *PostProcessor) PostProcess(ctx context.Context, ui packer.Ui, source packer.Artifact) (packer.Artifact, bool, bool, error) { + generatedData := source.State("generated_data") + + // generatedData will then be used for interpolation + + // ... +} +``` + +## Putting it all together + +This page has focused up until now on the implementation details for the Builder +interface. You will need to create a server and store the builder in a binary in +order to make it available to the Packer core as a plugin. We have created a +[scaffolding](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/builder/scaffolding/builder.go) +repo to give you an idea of the relationship between the builder +implementation and the server implementation within a repository, and then read +[basics of how Plugins work](/packer/docs/plugins/install), which breaks down all the +server details. diff --git a/content/packer/content/docs/plugins/creation/custom-datasources.mdx b/content/packer/content/docs/plugins/creation/custom-datasources.mdx new file mode 100644 index 0000000000..38a1c9186b --- /dev/null +++ b/content/packer/content/docs/plugins/creation/custom-datasources.mdx @@ -0,0 +1,94 @@ +--- +description: > + Data sources fetch data to use in the Packer configuration. Learn how to define custom data sources so that Packer can use external data in builds. +page_title: Custom Data Sources - Extending +--- + +# Custom Data Sources + +Packer data sources let Packer fetch data to use within the configuration, including information defined outside of Packer. For example, the [amazon-ami data source](/packer/plugins/datasources/amazon/ami), outputs the data from an Amazon AMI. + +Data Source plugins implement the `packersdk.Datasource` interface and are registered within a plugin Set +with `set.RegisterDatasource(...)` function and served using the `set.Run()`. + +~> **Warning:** This is an advanced topic that requires strong knowledge of Packer and Packer plugins. + +## Before You Begin + +We recommend reviewing the following resources before you begin development: + +- [Developing Plugins - Overview](/packer/docs/plugins/creation) +- The [Go](https://go.dev/) language. You must write custom plugins in Go, so this guide assumes you are familiar with the language. + +## The Interface + +The interface that must be implemented for a datasource is the +`packersdk.Datasource` interface. It is reproduced below for reference. The +actual interface in the source code contains some basic documentation as well +explaining what each method should do. + +```go +type Datasource interface { + ConfigSpec() hcldec.ObjectSpec + OutputSpec() hcldec.ObjectSpec + Configure(...interface{}) error + Execute() (cty.Value, error) +} +``` + +### The "ConfigSpec" Method + +This method returns a hcldec.ObjectSpec, which is a spec necessary for using +HCL2 templates with Packer. For information on how to use and implement this +function, check our +[object spec docs](/packer/guides/hcl/component-object-spec) + +### The "OutputSpec" Method + +This method returns a [hcldec.ObjectSpec](/packer/guides/hcl/component-object-spec) of the data source output. +The object spec can be generated using the command [`packer-sdc mapstructure-to-hcl2`](https://github.com/hashicorp/packer-plugin-sdk/tree/main/cmd/packer-sdc) +just like the configuration spec for the `ConfigSpec` method. + +This method is used in `packer validate` command. Packer will use the spec to assign +unknown values to the data source, instead of executing it and fetching real values. + +### The "Configure" Method + +The `Configure` method is called prior to any runs with the configuration that was given in the template. +This is passed in as an array of `interface{}` types, but is generally `map[string]interface{}`. The Configure +method is responsible for translating this configuration into an internal structure, validating it, +and returning any errors. + +For multiple parameters, they should be merged together into the final +configuration, with later parameters overwriting any previous configuration. +The exact semantics of the merge are left to the builder author. + +For decoding the `interface{}` into a meaningful structure, the +[mapstructure](https://github.com/mitchellh/mapstructure) library is +recommended. Mapstructure will take an `interface{}` and decode it into an +arbitrarily complex struct. If there are any errors, it generates very human +friendly errors that can be returned directly from the Configure method. + +While it is not actively enforced, **no side effects** should occur from +running the `Configure` method. Specifically, don't create files, don't launch +virtual machines, etc. Configure's purpose is solely to configure the data source and +validate the configuration. + +The `Configure` method is called very early in the build process so that errors +may be displayed to the user before anything actually happens. + +### The "Execute" Method + +This method returns an HCL cty.Value and an error. Packer will run the Execute method on `packer build` command +so that the data source output will be available prior to evaluating build and locals blocks. Is expected that the +Execute command will fetch the data and return it as a cty.Value. + +To get the equivalent cty.Value from an output config, we suggest using our +[packer-plugin-sdk hcl2helper functions](https://github.com/hashicorp/packer-plugin-sdk/blob/v0.0.7/hcl2helper/values.go). + +## Scaffolding template + +To make your experience easier when developing your new data source plugin, we provide you a +[Packer plugin scaffolding](https://github.com/hashicorp/packer-plugin-scaffolding) +to use as a template to your plugin repository. The template is structure with the basics and contain the necessary +configuration to start creating your own plugin. diff --git a/content/packer/content/docs/plugins/creation/custom-post-processors.mdx b/content/packer/content/docs/plugins/creation/custom-post-processors.mdx new file mode 100644 index 0000000000..c93fbaccbb --- /dev/null +++ b/content/packer/content/docs/plugins/creation/custom-post-processors.mdx @@ -0,0 +1,104 @@ +--- +description: > + Post-processors compress files, upload files, and perform other tasks that transform artifacts. Learn how to create customm post-processors that extend Packer. +page_title: Create custom post-processors +--- + +# Create custom post-processors + +Packer post-processors transform one artifact into another. For example, a post-processor might compress or upload files. + +In the compression example, the transformation would be taking an artifact with +a set of files, compressing those files, and returning a new artifact with only +a single file (the compressed archive). For the upload example, the +transformation would be taking an artifact with some set of files, uploading +those files, and returning an artifact with a single ID: the URL of the upload. + +Post-processor plugins implement the [`packer.PostProcessor`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer#PostProcessor) interface and are +served using the `plugin.ServePostProcessor` function. + +This page explains how to implement and serve custom post-processors. If you want your post-processor to support HashiCorp Cloud Platform (HCP) Packer, you should also review the [HCP Packer Support](/packer/docs/plugins/creation/hcp-support) documentation. + +~> **Warning:** This is an advanced topic that requires strong knowledge of Packer and Packer plugins. + +## Before You Begin + +We recommend reviewing the following resources before you begin development: +- [Developing Plugins - Overview](/packer/docs/plugins/creation) +- The [Go](https://go.dev/) language. You must write custom plugins in Go, so this guide assumes you are familiar with the language. + +## The Interface + +The interface that must be implemented for a post-processor is the +[`packer.PostProcessor`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer#PostProcessor) interface. It is reproduced below for reference. The +actual interface in the source code contains some basic documentation as well +explaining what each method should do. + +```go +type PostProcessor interface { + ConfigSpec() hcldec.ObjectSpec + Configure(interface{}) error + PostProcess(context.Context, Ui, Artifact) (a Artifact, keep, mustKeep bool, err error) +} +``` + +### The "ConfigSpec" Method + +This method returns a hcldec.ObjectSpec, which is a spec necessary for using +HCL2 templates with Packer. For information on how to use and implement this +function, check our +[object spec docs](/packer/guides/hcl/component-object-spec) + +### The "Configure" Method + +The `Configure` method for each post-processor is called early in the build +process to configure the post-processor. The configuration is passed in as a +raw `interface{}`. The configure method is responsible for translating this +configuration into an internal structure, validating it, and returning any +errors. + +For decoding the `interface{}` into a meaningful structure, the +[mapstructure](https://github.com/mitchellh/mapstructure) library is +recommended. Mapstructure will take an `interface{}` and decode it into an +arbitrarily complex struct. If there are any errors, it generates very +human-friendly errors that can be returned directly from the configure method. + +While it is not actively enforced, **no side effects** should occur from +running the `Configure` method. Specifically, don't create files, don't create +network connections, etc. Configure's purpose is solely to setup internal state +and validate the configuration as much as possible. + +`Configure` being run is not an indication that `PostProcess` will ever run. +For example, `packer validate` will run `Configure` to verify the configuration +validates, but will never actually run the build. + +### The "PostProcess" Method + +The `PostProcess` method is where the real work goes. PostProcess is +responsible for taking one `packer.Artifact` implementation, and transforming +it into another. +A `PostProcess` call can be cancelled at any moment. Cancellation is triggered +when the done chan of the context struct (`<-ctx.Done()`) unblocks . + +When we say "transform," we don't mean actually modifying the existing +`packer.Artifact` value itself. We mean taking the contents of the artifact and +creating a new artifact from that. For example, if we were creating a +"compress" post-processor that is responsible for compressing files, the +transformation would be taking the `Files()` from the original artifact, +compressing them, and creating a new artifact with a single file: the +compressed archive. + +The result signature of this method is `(Artifact, bool, bool, error)`. Each +return value is explained below: + +- `Artifact` - The newly created artifact if no errors occurred. +- `bool` - If keep true, the input artifact will forcefully be kept. By default, + Packer typically deletes all input artifacts, since the user doesn't + generally want intermediary artifacts. However, some post-processors depend + on the previous artifact existing. If this is `true`, it forces packer to + keep the artifact around. +- `bool` - If forceOverride is true, then any user input for + keep_input_artifact is ignored and the artifact is either kept or discarded + according to the value set in `keep`. +- `error` - Non-nil if there was an error in any way. If this is the case, + the other two return values are ignored. diff --git a/content/packer/content/docs/plugins/creation/custom-provisioners.mdx b/content/packer/content/docs/plugins/creation/custom-provisioners.mdx new file mode 100644 index 0000000000..e221dc4a4e --- /dev/null +++ b/content/packer/content/docs/plugins/creation/custom-provisioners.mdx @@ -0,0 +1,119 @@ +--- +description: > + Provisioners install and configure software prior to turning a machine into an image. Learn how to create custom provisioners that extend Packer functionality. + +page_title: Create custom provisioners +--- + +# Create Custom Provisioners + +Packer provisioners install and configure software into a running machine prior to turning that machine into an image. For example, the [shell provisioner](/packer/docs/provisioners/shell) runs shell scripts within machines. + +Provisioner plugins implement the [`packer.Provisioner`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer#Provisioner) interface and are served +using the `plugin.ServeProvisioner` function. + +~> **Warning:** This is an advanced topic that requires strong knowledge of Packer and Packer plugins. + +## Before You Begin + +We recommend reviewing the following resources before you begin development: +- [Developing Plugins - Overview](/packer/docs/plugins/creation) +- The [Go](https://go.dev/) language. You must write custom plugins in Go, so this guide assumes you are familiar with the language. + +## The Interface + +The interface that must be implemented for a provisioner is the +[`packer.Provisioner`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer#Provisioner) interface. It is reproduced below for reference. The +actual interface in the source code contains some basic documentation as well +explaining what each method should do. + +```go +type Provisioner interface { + ConfigSpec() hcldec.ObjectSpec + Prepare(...interface{}) error + Provision(context.Context, Ui, Communicator, map[string]interface{}) error +} +``` + +### The `Prepare` Method + +The `Prepare` method for each provisioner is called prior to any runs with the +configuration that was given in the template. This is passed in as an array of +`interface{}` types, but is generally `map[string]interface{}`. The prepare +method is responsible for translating this configuration into an internal +structure, validating it, and returning any errors. + +For multiple parameters, they should be merged together into the final +configuration, with later parameters overwriting any previous configuration. +The exact semantics of the merge are left to the builder author. + +For decoding the `interface{}` into a meaningful structure, the +[mapstructure](https://github.com/mitchellh/mapstructure) library is +recommended. Mapstructure will take an `interface{}` and decode it into an +arbitrarily complex struct. If there are any errors, it generates very human +friendly errors that can be returned directly from the prepare method. + +While it is not actively enforced, **no side effects** should occur from +running the `Prepare` method. Specifically, don't create files, don't launch +virtual machines, etc. Prepare's purpose is solely to configure the builder and +validate the configuration. + +The `Prepare` method is called very early in the build process so that errors +may be displayed to the user before anything actually happens. + +### The `ConfigSpec` Method + +This method returns a `hcldec.ObjectSpec`, which is a spec necessary for using +HCL2 templates with Packer. For information on how to use and implement this +function, check our +[object spec docs](/packer/guides/hcl/component-object-spec) + +### The `Provision` Method + +The `Provision` method is called when a machine is running and ready to be +provisioned. The provisioner should do its real work here. + +The method takes two parameters: a `packer.Ui` and a `packer.Communicator`. The +UI can be used to communicate with the user what is going on. The communicator +is used to communicate with the running machine, and is guaranteed to be +connected at this point. + +The provision method should not return until provisioning is complete. + +The map[string]interface{} provides users with build-specific information, +like host and IP, provided by the `build` template engine. Provisioners may use +this information however they please, or not use it. + +## Using the Communicator + +The `packer.Communicator` parameter and interface is used to communicate with +running machine. The machine may be local (in a virtual machine or container of +some sort) or it may be remote (in a cloud). The communicator interface +abstracts this away so that communication is the same overall. + +The documentation around the [code +itself](https://godoc.org/github.com/hashicorp/packer-plugin-sdk/packer#Communicator) +is really great as an overview of how to use the interface. You should begin by +reading this. Once you have read it, you can see some example usage below: + +```go +// Build the remote command. +var cmd packer.RemoteCmd +cmd.Command = "echo foo" + +// We care about stdout, so lets collect that into a buffer. Since +// we don't set stderr, that will just be discarded. +var stdout bytes.Buffer +cmd.Stdout = &stdout + +// Start the command +if err := comm.Start(&cmd); err != nil { + panic(err) +} + +// Wait for it to complete +cmd.Wait() + +// Read the stdout! +fmt.Printf("Command output: %s", stdout.String()) +``` diff --git a/content/packer/content/docs/plugins/creation/hcp-support.mdx b/content/packer/content/docs/plugins/creation/hcp-support.mdx new file mode 100644 index 0000000000..182072bb24 --- /dev/null +++ b/content/packer/content/docs/plugins/creation/hcp-support.mdx @@ -0,0 +1,116 @@ +--- +description: | + You can create or modify custom plugins to support HCP Packer. Add support for HCP Packer to let plugins manage image metadata stored in the HCP Packer registry. +page_title: Enable HCP Packer support for custom plugins +--- + +# Enable HCP Packer Support for Custom Plugins + +This page explains how to update a custom plugin so that it can publish image metadata to the [HCP Packer registry](/hcp/docs/packer). Refer to [Custom Builders](/packer/docs/plugins/creation/custom-builders) and [Custom Post-Processors](/packer/docs/plugins/creation/custom-post-processors) for details about creating an external Packer plugin. + +Before pushing metadata to the HCP Packer registry, Packer uses the [`par.artifact.metadata` key](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer/registry/image#pkg-constants) +to query a builder artifact for the image metadata that a particular component would like to have stored in the registry. + +For details and examples of how to manage image metadata, refer to the [HCP Packer GitHub documentation](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer/registry/image). + +HCP Packer is under active development, and we suggest plugin maintainers to keep up with the SDK changes for more on HCP Packer support. + +## Builder Artifact + +To support HCP Packer, changes must be made to the plugin's build artifact. The artifact should keep the appropriate +Image metadata in state under the key `par.artifact.metadata`. The expected key is provided by the [image.ArtifactStateURI](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer/registry/image#pkg-constants) constant. + +To make HCP Packer support easier, we ship an Image package with the [packer-plugin-sdk](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer/registry/image#pkg-overview). +We provide the [`FromArtifact`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer/registry/image#FromArtifact) function +to easily create an Image with the basic information contained in the Artifact. If customization is necessary, the `FromArtifact` accepts +override functions as the second argument, and that can be used to set different values for the metadata. + +The [Image metadata](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer/registry/image#Image) must contain every +metadata HCP Packer should store about this component. The structure is: +```go +// Image represents the metadata for some Artifact in the HCP Packer Registry. +type Image struct { + // ImageID is a unique reference identifier stored on the HCP Packer registry + // that can be used to get back the built artifact of a builder or post-processor. + ImageID string + // ProviderName represents the name of the top level cloud or service where the built artifact resides. + // For example "aws, azure, docker, gcp, and vsphere". + ProviderName string + // ProviderRegion represents the location of the built artifact. + // For cloud providers region usually maps to a cloud region or zone, but for things like the file builder, + // S3 bucket or vsphere cluster region can represent a path on the upstream datastore, or cluster. + ProviderRegion string + // Labels represents additional details about an image that a builder or post-processor may with to provide for a given build. + // Any additional metadata will be made available as build labels within a HCP Packer registry iteration. + Labels map[string]string + // SourceImageID is the cloud image ID of the image that was used as the + // source for this image. If set, the HCP Packer registry will be able + // link the parent and child images for ancestry visualizations and + // dependency tracking. + SourceImageID string +} +``` + +Where `ImageID`, `ProviderName`, and `SourceImageID` are mandatory fields. + +Examples using the `FromArtifact` method: + +- Simple form: +```go +func (a *Artifact) State(name string) interface{} { + if name == registryimage.ArtifactStateURI { + img, err := registryimage.FromArtifact(a) + if err != nil { + log.Printf("[DEBUG] error encountered when creating a registry image %v", err) + return nil + } + return img + } + return a.StateData[name] +} +``` + +- Using overrides: +```go +func (a *Artifact) State(name string) interface{} { + if name == registryimage.ArtifactStateURI { + img, err := registryimage.FromArtifact(a, + registryimage.WithID(a.Name), + registryimage.WithRegion(a.Region.Name()), + registryimage.WithProvider("happy-cloud"), + registryimage.WithSourceID(a.SourceID), + registryimage.SetLabels(a.Labels), + ) + if err != nil { + log.Printf("[DEBUG] error encountered when creating a registry image %v", err) + return nil + } + return img + } + return a.StateData[name] +} +``` + +See [packer-plugin-sdk/packer/registry/image](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer/registry/image#pkg-examples) for additional examples. + +### SDK Version + +- The HCP Packer support is available from packer-plugin-sdk >= v0.2.7 + +## Plugins Example + +The following plugins currently support HCP Packer and are great references for development: + +- [packer-plugin-amazon](https://github.com/hashicorp/packer-plugin-amazon) +- [packer-plugin-azure](https://github.com/hashicorp/packer-plugin-azure) +- [packer-plugin-vsphere](https://github.com/hashicorp/packer-plugin-vsphere) +- [packer-plugin-docker](https://github.com/hashicorp/packer-plugin-docker) +- [packer-plugin-googlecompute](https://github.com/hashicorp/packer-plugin-googlecompute) + +## HCP Packer + +Refer to the following resources to learn about HCP Packer: + +- [HCP Packer documentation](/hcp/docs/packer) +- [HCP Packer tutorials](/packer/tutorials/hcp-get-started). +- [Log into the HCP portal](https://portal.cloud.hashicorp.com) \ No newline at end of file diff --git a/content/packer/content/docs/plugins/creation/index.mdx b/content/packer/content/docs/plugins/creation/index.mdx new file mode 100644 index 0000000000..9a981eb41b --- /dev/null +++ b/content/packer/content/docs/plugins/creation/index.mdx @@ -0,0 +1,283 @@ +--- +description: | + Learn about extending Packer by creating custom plugins for builders, provisioners, and + post-processors. +page_title: Create custom plugins to extend Packer overview +--- + +# Create Custom Plugins to Extend Packer + +Packer is extensible and supports plugins that let you +create and use custom builders, provisioners, post-processors, and data sources. This page explains how to develop Packer plugins. Before you begin, we recommend reviewing the Packer documentation and the instructions for [installing external plugins](/packer/docs/plugins/install). + +~> **Warning** This is an advanced topic. You should have strong knowledge of Packer before you start writing plugins. + +## Language Requirements + +You must write Packer plugins in [Go](https://go.dev/). + + +## Plugin System Architecture + +A Packer plugin is just a Go binary. Instead of loading plugins directly into a +running application, Packer runs each plugin as a _separate application_. +The multiple separate Packer plugin processes communicate with the Core using +an RPC defined in the packer-plugin SDK. The Packer core itself is responsible +launching and cleaning up the plugin processes. + +## Plugin Development Basics + +The components that can be created and used in a Packer plugin are builders, +provisioners, post-processors, and data sources. + +Each of these components has a corresponding [interface](https://go.dev/doc/effective_go.html#interfaces_and_types). + +All you need to do to create a plugin is: + +1. create an implementation of the desired interface, and +2. serve it using the server provided in the [packer-plugin-sdk](https://github.com/hashicorp/packer-plugin-sdk). + +The core and the SDK handle all of the communication details inside the server. + +Your plugin must use two packages from the SDK to implement the server and +interfaces. You're encouraged to use whatever other packages you want in your +plugin implementation. Because plugins are their own processes, there is no +danger of colliding dependencies. + +- [`github.com/hashicorp/packer-plugin-sdk/packer`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/packer) - Contains all the interfaces that you have to implement for any given plugin. + +- [`github.com/hashicorp/packer-plugin-sdk/plugin`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/plugin) - Contains the code to serve the plugin. This handles all the inter-process communication. + +Basic examples of serving your component are shown below. + +```go +// main.go + +import ( + "github.com/hashicorp/packer-plugin-sdk/plugin" +) + +// Assume this implements the packer.Builder interface +type ExampleBuilder struct{} + +// Assume this implements the packer.PostProcessor interface +type FooPostProcessor struct{} + +// Assume this implements the packer.Provisioner interface +type BarProvisioner struct{} + +func main() { + pps := plugin.NewSet() + pps.RegisterBuilder("example", new(ExampleBuilder)) + pps.RegisterBuilder(plugin.DEFAULT_NAME, new(AnotherBuilder)) + pps.RegisterPostProcessor("foo", new(FooPostProcessor)) + pps.RegisterProvisioner("bar", new(BarProvisioner)) + err := pps.Run() + if err != nil { + fmt.Fprintln(os.Stderr, err.Error()) + os.Exit(1) + } +} +``` + +This `plugin.NewSet` invocation handles all the details of communicating with +Packer core and serving your component over RPC. As long as your struct being +registered implements one of the component interfaces, Packer will now be able +to launch your plugin and use it. + +If you register a component with its own name, the component name will be +appended to the plugin name to create a unique name. If you register a component +using the special string constant `plugin.DEFAULT_NAME`, then the component will +be referenced by using only the plugin name. For example: + +If your plugin is named `packer-plugin-my`, the above set definition would make +the following components available: + +- the `my-example` builder +- the `my` builder +- the `my-foo` post-processor +- the `my-bar` provisioner + +Next, build your plugin as you would any other Go application. The resulting +binary is the plugin that can be installed using +[standard installation procedures](https://developer.hashicorp.com/packer/docs/plugins#installing-plugins). + +This documentation explains how to implement each type of plugin interface: builders, data sources, provisioners, and post-processors. + +~> **Lock your dependencies!** Using `go mod` is highly recommended since +the Packer codebase will continue to improve, potentially breaking APIs along +the way until there is a stable release. By locking your dependencies, your +plugins will continue to work with the version of Packer you lock to. + +## Logging and Debugging + +Plugins can use the standard Go `log` package to log. Anything logged using +this will be available in the Packer log files automatically. The Packer log is +visible on stderr when the `PACKER_LOG` environment var is set. + +Packer will prefix any logs from plugins with the path to that plugin to make +it identifiable where the logs come from. Some example logs are shown below: + +```text +2013/06/10 21:44:43 Loading builder: custom +2013/06/10 21:44:43 packer-builder-custom: 2013/06/10 21:44:43 Plugin minimum port: 10000 +2013/06/10 21:44:43 packer-builder-custom: 2013/06/10 21:44:43 Plugin maximum port: 25000 +2013/06/10 21:44:43 packer-builder-custom: 2013/06/10 21:44:43 Plugin address: :10000 +``` + +As you can see, the log messages from the custom builder plugin are prefixed +with "packer-builder-custom". Log output is _extremely_ helpful in debugging +issues and you're encouraged to be as verbose as you need to be in order for +the logs to be helpful. + +### Creating a GitHub Release + +`packer init` does not work using a centralized registry. Instead, it requires +you to publish your plugin in a GitHub repo with the name +`packer-plugin-*` where \* represents the name of your plugin. You also need to +create a GitHub release of your plugin with specific assets for the +`packer init` download to work. We provide a pre-defined release workflow +configuration using +[GitHub Actions](https://docs.github.com/en/free-pro-team@latest/actions). We +strongly encourage maintainers to use this configuration to make sure the +release contains the right assets with the right names for Packer to leverage +`packer init` installation. + +Here's what you need to create releases using GitHub Actions: + +1. Generate a GPG key to be used when signing releases (See [GitHub's detailed instructions](https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/generating-a-new-gpg-key) + for help with this step) +2. Copy the [GoReleaser configuration from the packer-plugin-scaffolding repository](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/.goreleaser.yml) to the root of your repository. + ```sh + curl -L -o .goreleaser.yml \ + https://raw.githubusercontent.com/hashicorp/packer-plugin-scaffolding/main/.goreleaser.yml + ``` +3. Copy the [GitHub Actions workflow from the packer-plugin-scaffolding repository](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/.github/workflows/release.yml) to `.github/workflows/release.yml` in your repository. + ```sh + mkdir -p .github/workflows && + curl -L -o .github/workflows/release.yml \ + https://raw.githubusercontent.com/hashicorp/packer-plugin-scaffolding/main/.github/workflows/release.yml + ``` +4. Go to your repository page on GitHub and navigate to Settings > Secrets. Add + the following secrets: + - `GPG_PRIVATE_KEY` - Your ASCII-armored GPG private key. You can export this with `gpg --armor --export-secret-keys [key ID or email]`. + - `GPG_PASSPHRASE` - The passphrase for your GPG private key. +5. Push a new valid version tag (e.g. `v1.2.3`) to test that the GitHub Actions + releaser is working. The tag must be a valid + [Semantic Version](https://semver.org/) preceded with a `v`. Once the tag is pushed, the github actions you just configured will automatically build release binaries that Packer can download using `packer init`. For more details on how + to install a plugin using `packer init`, see the + [init docs](https://developer.hashicorp.com/packer/docs/commands/init). + +## Registering Plugins + +~> Note: Registering a plugin as an integration requires the documentation to match the [Scaffolding example layout](https://github.com/hashicorp/packer-plugin-scaffolding/tree/main/.web-docs). + +To help with the discovery of Packer plugins, plugins maintainers can choose to register their plugin as a [Packer Integration](https://developer.hashicorp.com/packer/integrations). + +The registration process requires [metadata configuration](https://github.com/hashicorp/integration-template#metadata-configuration) to be added to your plugin repository for configuring the Packer integration pipeline and +a specific directory structure for plugin documentation to be rendered on the [Packer Integrations](https://developer.hashicorp.com/packer/integrations) portal. + +You can execute the following steps to register your plugin as an integration: + +1. Update your plugin documentation structure to match the [Scaffolding example layout](https://github.com/hashicorp/packer-plugin-scaffolding/tree/main/.web-docs). +New plugins generated from this template have the necessary structure in place. If so you can jump to step 3. +1. For the integrations library, only one top-level README per integration is supported. Any top-level index.mdx files that exist +within a plugin's existing documentation will need to migrate to a top-level README. +1. Update your top-level integration README to include a description, plugin installation steps, available components section, and, any, additional sections +needed to inform users on how to work with your integration. Refer to [Packer scaffolding plugin](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/docs/README.md) for an example. +1. Update the top-level README for each of the components within your integration to follow the structure defined in the scaffolding template. +1. Add the integration configuration file [metadata.hcl](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/.web-docs/metadata.hcl) to the plugins `.web-docs` directory. +1. Open a request for integration issue with the Packer team - [Open Request](https://github.com/hashicorp/packer/issues/new?labels=new-plugin-contribution&template=plugin_integration.md). +Provide all the requested information to help expedite the integration request. + +#### [Example] Add integration files to existing plugin repository + +```shell +## Update Plugin repository with integration config, workflows, and scripts +cd packer-plugin-name +mkdir -p .web-docs/scripts + +# Download packer-plugin-scaffolding repo copy files +wget https://github.com/hashicorp/packer-plugin-scaffolding/archive/refs/heads/main.zip +unzip main.zip +cp packer-plugin-scaffolding-main/.web-docs/metadata.hcl .web-docs/ +cp -r packer-plugin-scaffolding-main/.web-docs/scripts/ .web-docs/scripts/ +cp packer-plugin-scaffolding-main/.github/workflows/notify-integration-release-via-* .github/workflows/ + +# Remove downloaded scaffolding project +rm main.zip +rm -rf packer-plugin-scaffolding-main + +# Add the following commands to your plugin GNUmakefile +generate: install-packer-sdc + @go generate ./... + @rm -rf .docs + @packer-sdc renderdocs -src docs -partials docs-partials/ -dst .docs/ + @./.web-docs/scripts/compile-to-webdocs.sh "." ".docs" ".web-docs" "" + @rm -r ".docs" +``` + +By opening an integration request, you are asking a member of the to Packer team to review your plugin integration configuration, plugin documentation, +and, finally, to open an internal pull-request to finalize the integration setup. + +Plugin integrations will be listed as a [Packer Integration](https://developer.hashicorp.com/packer/integrations), with details on how to install and use your the plugin. + +Plugin integrations, once deployed, can be updated manually, or automatically upon a new release, by the plugin authors. Changes to the defined documentation structure +or parent repository should be communicated to the Packer team to ensure a working integration pipeline. + +## Plugin Development Tips and FAQs + +### Working Examples + +Here's a non-exhaustive list of Packer plugins that you can check out: + +- [github.com/hashicorp/packer-plugin-docker](https://github.com/hashicorp/packer-plugin-docker) +- [github.com/exoscale/packer-plugin-exoscale](https://github.com/exoscale/packer-plugin-exoscale) +- [github.com/sylviamoss/packer-plugin-comment](https://github.com/sylviamoss/packer-plugin-comment) + +Looking at their code will give you good examples. + +### Naming Conventions + +It is standard practice to name the resulting plugin application in the format +of `packer-plugin-NAME`. For example, if you're building a new builder for +CustomCloud, it would be standard practice to name the resulting plugin +`packer-plugin-customcloud`. This naming convention helps users identify the +scope of a plugin. + +### Testing Plugins + +To install plugins from local source, you can use the `packer plugins install` command with the `--path` flag: + +```shell-session +$ packer plugins install --path // +``` + +For example let's install happycloud plugin from a locally sourced binary + +```shell-session +$ packer plugins install --path packer-plugin-happycloud github.com/hashicorp/happycloud +``` + +This will install the happycloud plugin from the `packer-plugin-happycloud` binary so Packer can discover it. +If you want to use this in an HCL2 template, you can optionally add it to the `required_plugins` section like so: + +```hcl + required_plugins { + happycloud = { + source = "github.com/hashicorp/happycloud" + version = ">=0.0.1" + } +} +``` + +For further information on how Packer discovers and loads plugins, you may refer to our [documentation](/packer/docs/plugins/creation/plugin-load-spec) on the subject. + +### Distributing Plugins + +We recommend that you use a tool like the GoReleaser in order to cross-compile +your plugin for every platform that Packer supports, since Go applications are +platform-specific. If you have created your plugin from the +[packer-plugin-scaffolding](https://github.com/hashicorp/packer-plugin-scaffolding) +repo, simply tagging a commit and pushing the tag to GitHub will correctly build +and release the binaries using GoReleaser. diff --git a/content/packer/content/docs/plugins/creation/plugin-load-spec.mdx b/content/packer/content/docs/plugins/creation/plugin-load-spec.mdx new file mode 100644 index 0000000000..b845477d38 --- /dev/null +++ b/content/packer/content/docs/plugins/creation/plugin-load-spec.mdx @@ -0,0 +1,167 @@ +--- +description: | + Packer needs plugins in order to build artifacts from a template. + This page explains how plugins are discovered and loaded by Packer. +page_title: Plugin Loading - Specification +--- + +# Packer plugin loading + +This document aims to document how Packer discovers plugins on the local filesystem. +This is not meant for beginners with Packer but instead serves as a technical reference for curiosity and troubleshooting purposes. + +If you are new to Packer, you are advised to start with the [Installing Plugins](/packer/docs/plugins/install) page for details on plugin installation. + +## Plugin sources + +A source is conceptually the distribution point for a plugin. It is a URL to that location. It is intended to document where to get a plugin binary, and for Packer to remotely install plugins from this source, if compatible. +If Packer cannot install from that source remotely, a plugin can still be installed with `packer plugins install --path `. + +As of Packer 1.11.0, sources are mandatory to install plugins. +A source will also reflect where a plugin is installed on the local filesystem, as a series of directories. + +Example: `github.com/hashicorp/hashicups` will result in the following directory tree `$HOME/.config/packer/plugins/github.com/hashicorp/hashicups`. + + +There are some conventions to adopt when declaring a source: + +* No schema (e.g. `https://`) +* No fragment (e.g. `#section`) +* No query (e.g. `?page=1`) +* The URL must contain at least a host and two parts to the URL. +* The URL must not have over 15 parts, in addition to the host. +* `/` is the only separator that can be used for the source. +* The full source URL needs to be compatible with the filesystem you are installing the plugin onto. Packer does not check for unsupported characters within the URL, but will error if a valid directory tree can not be created. + +The last part of the source URL is the plugin's name. +It must always have the raw name of the plugin, without the `packer` or `packer-plugin` prefix. + +**Note**: If your plugin is on GitHub, the repository's name must look like `packer-plugin-` for Packer to be able to find it. However, the source address specified in your template or on the CLI must exclude `packer-plugin-`. + +Example: `https://github.com/hashicorp/packer-plugin-hashicups` will become `github.com/hashicorp/hashicups` when declaring it as a source. + +## Root plugin directory + +The plugins need to be installed under a root plugin directory. +This will default to either of the following: + +**UNIX**: + +* `$HOME/.packer.d/plugins`: this is the old-style installation directory for plugins. `~/.packer.d` will have precedence over the following if it exists. +* `$HOME/.config/packer/plugins`: this replaces `~/.packer.d`, if no existing configuration directory exists, Packer will create this automatically at first use. + +**WINDOWS**: + +* `%APPDATA%/packer.d/plugins`: this is the only default for Windows systems. + +If you want to change that behavior, there are two alternatives: + +* `PACKER_CONFIG_DIR`: this environment variable allows you to customize where the configuration directory is. The plugins are installed in a `plugin` subdirectory of this configuration directory. +* `PACKER_PLUGIN_PATH`: this environment variable allows you to customize where plugins are installed. This points to a root plugins directory, under which the normal directory hierarchy will be enforced. + +**Note**: `PACKER_PLUGIN_PATH` has precedence over `PACKER_CONFIG_DIR`, if it is defined, `PACKER_CONFIG_DIR` will be ignored for plugin installation and loading. + +Refer to [Configuring Packer](/packer/docs/configure) for a full list of environment variables. + +## Plugin installation directories + +All plugins must be installed under a root plugin directory. +Under this directory, the plugins are installed under a series of directories that match the source URL. + +Example: `github.com/hashicorp/hashicups` will translate to a hierarchy like the following: + +```shell-session + +└── github.com + └── hashicorp + └── hashicups +``` + +Plugins are installed in the leaf directory of that example. +Each plugin version must have only one binary per version, accompanied by a matching SHA256SUM file for the said version. The SHA256SUM file's contents are the raw hexdigest of the sha256 sum from the contents of the plugin binary. + +### Plugin binary naming convention + +In order for Packer to discover and load a plugin binary it must conform to the following naming convention : + +`packer-plugin-____[.exe]` + +The sha256sum file must follow the same convention, with a `SHA256SUM` suffix to the name: + +`packer-plugin-____[.exe]_SHA256SUM` + +As for the components of the name, the convention is the following: + +* `name`: the raw name of the plugin, it should match the parent directory's name. Ex: `hashicups`. +* `version`: the semver version of the plugin. It must follow the convention `v..[-]`. Metadata information must not be part of the version string. +* `api_version`: the plugin API version that the plugin was compiled with. Typically it looks like `x.`. +* `os`: the OS the plugin was built for. Ex: `darwin` (macOS), `windows`, `linux`, etc. +* `arch`: the micro-architecture the plugin was built for. Ex: `arm64`, `amd64`, `386`, etc. + +Note: the `.exe` suffix is only used for Windows plugins. Any other OS must not add the suffix to the plugin name, otherwise Packer will ignore it. + +## Loading process + +When running either `packer build` or `packer validate`, Packer will attempt to discover and load plugins to execute the command on a template. + +There are two phases to this: + +1. Load explicitly required plugins. +2. Discover the remainder of the installed plugins. + +Explicitly required plugins are an HCL2 specificity. +They are declared through `required_plugins` blocks. +These allow you to specify an exact source and version constraints on that plugin requirement. + +Each of the plugins declared this way will have precedence over what the second phase will gather. + +The second phase is optimistically attempting to discover the remainder of the plugins installed. +The name of the plugins will be inferred from the binary's name, without the `packer-plugin-` part. + +Ex: If the `github.com/hashicorp/hashicups` plugin is installed, and discovered during this phase, each of the uses of this plugin's components will have their name start with `hashicups`. + +When discovering a plugin, Packer will execute its `describe` command. +The `describe` command showcases the capabilities of a plugin and gives information about its respective version and API version. + +Typically this is what you can see by invoking `describe` on a plugin: + +```shell-session +> $HOME/.packer.d/plugins/github.com/hashicorp/hashicups/packer-plugin-hashicups_v1.0.2_x5.0_linux_amd64 describe +{"version":"1.0.2","sdk_version":"0.5.1","api_version":"x5.0","builders":["order"],"post_processors":["receipt"],"provisioners":["toppings"],"datasources":["coffees","ingredients"]} +``` + +**Note**: the information from the plugin's described output must match the version specified within the name of the plugin. + +To summarise, this is a list of the checks Packer performs before deciding if a plugin should be listed as a candidate: + +* The version reported by `describe` must match the version in the plugin name: i.e. if `describe` reports v1.0.2 while the binary is named `v1.0.1`, Packer rejects it. +* The version must be canonical (the version must be its simplest expression): i.e. v1.00.01 is non-canonical, v1.0.1 would be. Any plugin with this version mismatch will be rejected. +* The API version must match between what the plugin reports, and the name: i.e. if `describe` reports x5.1 and the binary contains `x5.0`, Packer rejects it. +* The version may contain a pre-release fragment if this is non-final. It must however be `-dev`, anything else is rejected. + +When multiple plugins are installed, Packer always chooses the one with the highest version that matches a potential constraint. + +Final releases have precedence over pre-releases if the version radical is equivalent: `v1.0.0 < v1.0.1-dev < v1.0.1`. + +### Known limits + +While explicit discovery ensures you always get what you intend, automatic discovery can lead to cohesion issues. + +For example, if a plugin is installed twice, with a different source, Packer will discover both but the final plugin that will be executed when requesting a component from this plugin is undefined behavior. + +Example: + +```shell-session + +├── github.com +│ └── hashicorp +│ └── hashicups +│ └── packer-plugin-hashicups_v1.0.2_x5.0_linux_amd64 +└── gitlab.com + └── hashicorp + └── hashicups + └── packer-plugin-hashicups_v1.0.2_x5.0_linux_amd64 +``` + +In this case, there's an ambiguity problem as both plugins are `hashicups`, and they define a series of components that may overlap. +Therefore using the `hashicups-coffees` datasource without a `required_plugins` to resolve this ambiguity means one of the two plugins is executed, but there are no guarantees as to which one it will be. diff --git a/content/packer/content/docs/plugins/index.mdx b/content/packer/content/docs/plugins/index.mdx new file mode 100644 index 0000000000..54684f6b90 --- /dev/null +++ b/content/packer/content/docs/plugins/index.mdx @@ -0,0 +1,53 @@ +--- +description: | + This topic provides overview information about installing and loading Packer plugins +page_title: Plugins +--- + +# Plugin installation overview + + + +Starting August 1st, 2025, the source for many official HashiCorp-maintained Packer plugins is moving from GitHub releases to the official HashiCorp release site, [releases.hashicorp.com](https://releases.hashicorp.com). Refer to [Install HashiCorp-maintained plugins](/packer/docs/plugins/install#install-hashicorp-maintained-plugins) for more information. + + + +This topic provides overview information about installing and loading Packer plugins. Plugins are standalone applications that perform additional tasks during each build. + +## Introduction + +The Packer binary includes a set of built-in components that are automatically usable. You can also install external plugins for Packer to run and communicate with throughout the build. These external plugins extend Packer functionality without modifying the core source code. Packer launches one plugin process for each component in the build. + +## Workflows + +To use a plugin with Packer, you must install the plugin code and its SHA256SUM file into the Packer plugins directory. +Refer to [Installing Plugins](/packer/docs/plugins/install) for instructions. If you are unable to use an installed plugin refer to the [Plugin Loading - Specification](/packer/docs/plugins/creation/plugin-load-spec) for details on how Packer discovers and loads plugins. + +## Where Packer stores plugins + +Packer stores external plugins that are downloaded during initialization into the plugins directory. The default plugins directory is `$HOME/.config/packer/plugins` on Unix and `%APPDATA%\packer.d\plugins` on Windows. Refer to [`PACKER_PLUGIN_PATH`](/packer/docs/configure#packer_plugin_path) in the configuration reference for additional information. + +## Guidance + +Refer to the following plugin documentation for details about the plugin installation process: + +### Installing plugins + +Refer to [Installing Plugins](/packer/docs/plugins/install) for instructions on adding external plugins to your Packer template and installing the binaries. + +### External plugins + +Refer to the [Packer Integrations](/packer/integrations) documentation to learn about the available external plugins not included with the Packer binary. + + +### Built-in plugins + +- Packer uses plugins called _builders_ to create machines and images. Refer to [Builders](/packer/docs/builders) for additional information. +- The _data sources_ plugin fetches data to use in a template. Refer to [Data Sources](/packer/docs/datasources) for additional information. +- Plugins called _provisioners_ install and configure machine images after booting. Refer to [Provisioners](/packer/docs/provisioners) for additional information. +- _Post-processors_ perform additional tasks after provisioning. Refer to [Post-Processors](/packer/docs/post-processors) for additional information. + + +### Developing plugins + +Refer to the [Developing Plugins](/packer/docs/plugins/creation) to learn how to create custom external plugins. diff --git a/content/packer/content/docs/plugins/install.mdx b/content/packer/content/docs/plugins/install.mdx new file mode 100644 index 0000000000..fba5e25e38 --- /dev/null +++ b/content/packer/content/docs/plugins/install.mdx @@ -0,0 +1,228 @@ +--- +description: | + You can install plugins to extend Packer functionality. Learn how to install external plugins. +page_title: Install Plugins +--- + +# Install Plugins + + + +Starting August 1st, 2025, the source for many official HashiCorp-maintained Packer plugins is moving from GitHub releases to the official HashiCorp release site, [releases.hashicorp.com](https://releases.hashicorp.com). Refer to [Install HashiCorp-maintained plugins](#install-hashicorp-maintained-plugins) for more information. + + + +This topic describes how to install external plugins for Packer. Refer to [Packer Plugins Overview](/packer/docs/plugins) for additional information about plugins. + +## Overview + +Specify one or more required plugins in a Packer template and run the initialization command. During initialization, Packer queries the source repository for plugins specified in the template that match the version constraints for the host operating system. Packer downloads and installs plugins and other dependencies specified in the template. + +You can specify a remote project on GitHub for Packer to install the plugin from. Refer to [Requirements](#requirements) for additional information about installing plugins from a remote source. + +When you run the Packer template, Packer first checks the directory for installed plugins that match the version requirements specified in the template. If a suitable version of the plugin is not installed, Packer prints an error. Packer downloads the required version from the `source` configuration. + +### Manual installation method + +You can manually install plugins if you are unable to specify plugins in an HCL template file. You may need to use the manual plugin installation method if the plugin source is a non-GitHub project or if your plugins are still in development. + +Manually install a plugin using the `packer plugins install` command. You specify a local or remote plugin source and include a path to the binary to generate the SHA256SUM file required to load the plugin during Packer builds. Refer to [Requirements](#requirements) for additional information about installing plugins from a remote source. + +Note that Packer checks the plugin installation directory against the `required_plugins` block in the Packer template when you build artifacts using the template. Plugins specified in the template that have been installed manually must still comply with the version constraints set in the template. + + +### Installation directory + +By default, Packer installs plugins into the plugins directory at `$HOME/.config/packer/plugins` on Unix and `%APPDATA%\packer.d\plugins` on Windows, but you can specify a different directory using the `PACKER_PLUGIN_PATH` environment variable. + + + +Plugin installation requires access to temporary files under `TMPDIR`. If the system's temp directory is non-writable or non-executable, use TMPDIR to override the location of the temporary file store used by Packer. +Refer to the [Packer configuration reference](/packer/docs/configure) for additional information. + + + +## Requirements + +To install a plugin from a remote source, the plugin must meet the following requirements: + +- The plugin project must be hosted on GitHub. +- The repository must be named `packer-plugin-`. +- The project must have semantic version tags per the following format: `v..`. +- The release linked to the tag must be available with a `shasums` file that indicates which files are available in the release. + +### GitHub API token + +GitHub's public API limits the number of unauthenticated requests per hour from a single IP address. Refer to the [GitHub rate limits documentation](https://docs.github.com/en/developers/apps/rate-limits-for-github-apps#normal-user-to-server-rate-limits) for additional information. + +If you expect to exceed the request rate limit, you can use a GitHub API token to authenticate requests and exceed the rate limit. We recommend setting a GitHub API token when running Packer from a continuous integration server to avoid potential rate limiting errors. Refer to [Authenticate requests to the GitHub API](#authenticate-requests-to-the-github-api) for instructions. + +## Install a plugin + +1. In your Packer template file, add the `required_plugins` block to the `packer` block. +1. Specify the name of the plugin and its `version` and `source` parameters. Setting a correct [version constraint string](/packer/docs/templates/hcl_templates/blocks/packer#version-constraints) is important for +pinning plugin versions for build reproducibility. Refer to the [`packer` block configuration reference](/packer/docs/templates/hcl_templates/blocks/packer) for additional information. + + The following example configures Packer to install a plugin called `happycloud`. When the template is initialized, Packer downloads version 2.7.0 or newer of the plugin from GitHub: + + ```hcl + packer { + required_plugins { + happycloud = { + version = ">= 2.7.0" + source = "github.com/hashicorp/happycloud" + } + } + } + + ``` + +1. Run the `packer init` command. Packer lists all installed plugins then installs the latest plugin version matching the version constraints specified in the `required_plugins` block. Refer to the [`init` command reference](/packer/docs/commands/init) for additional information. + +## Install HashiCorp-maintained plugins + +HashiCorp now makes the following official HashiCorp-maintained plugins available through the [HashiCorp release site](https://releases.hashicorp.com/). + +These plugins include: + +- [Amazon](https://developer.hashicorp.com/packer/integrations/hashicorp/amazon) +- [Ansible](https://developer.hashicorp.com/packer/integrations/hashicorp/ansible) +- [Azure](https://developer.hashicorp.com/packer/integrations/hashicorp/azure) +- [Docker](https://developer.hashicorp.com/packer/integrations/hashicorp/docker) +- [Google Cloud Platform](https://developer.hashicorp.com/packer/integrations/hashicorp/googlecompute) +- [QEMU](https://developer.hashicorp.com/packer/integrations/hashicorp/qemu) +- [Vagrant](https://developer.hashicorp.com/packer/integrations/hashicorp/vagrant) +- [VirtualBox](https://developer.hashicorp.com/packer/integrations/hashicorp/virtualbox) + +Starting in Packer 1.14.0, Packer automatically installs these plugins from the new release source, and you do not need to make any changes to your Packer templates. To continue to automatically receive updates to these plugins, you must upgrade to Packer 1.14.0 or newer. If you cannot upgrade your version of Packer, you can still install new versions of these plugins manually. Refer to [Manually install plugins using the CLI](#manually-install-plugins-using-the-cli) for more information. + +## Manually install plugins using the CLI + +You can use the `packer plugins install` command to manually install plugin binaries. +Use the `--path` flag to specify a local source. Packer then automatically calculates the SHA256SUM file and installs the files into the Packer plugin directory: + +```shell-session +$ packer plugins install --path // +``` + +The following example installs the `happycloud` plugin from a locally-sourced binary: + +```shell-session +$ unzip packer-plugin-happycloud.zip +$ ls -l +-rwxr-xr-x [...] happycloud +$ packer plugins install --path happycloud github.com/hashicorp/happycloud +``` + +Refer to the [`packer plugins install`](/packer/docs/commands/plugins/install) reference for additional information. + +## Upgrade plugins + +To upgrade plugins that are already installed, run the `packer init` with the `--upgrade` flag. Packer retrieves the latest versions of installed plugins specified in the template configuration. + +The following example upgrades plugins according to the template in the current directory: + +```shell-session +$ packer init --upgrade . +``` + +Refer to [`packer init` command](/packer/docs/commands/init) for additional information. + +## Use a plugin under development + +If a development binary, such as a manually-built binary, is available at the specified source, Packer uses it in the build if it is the highest compatible version installed and if no final plugin version with the same version number is installed alongside it. + +In the following example, version `1.1.0` or newer is required: + +```hcl +packer = { + required_plugins = { + amazon = { + source = "github.com/hashicorp/amazon" + version = ">= 1.1.0" + } + } + +. . . + +} +``` + +Packer uses the `-dev` version of the Amazon plugin if the following binaries are available: + +```shell-session +/Users/dev/.packer.d/plugins +└─ github.com + └─ hashicorp + └── amazon + ├── packer-plugin-amazon_v1.1.0_x5.0_darwin_arm64 + ├── packer-plugin-amazon_v1.1.0_x5.0_darwin_arm64_SHA256SUM + ├── packer-plugin-amazon_v1.1.1-dev_x5.0_darwin_arm64 + └── packer-plugin-amazon_v1.1.1-dev_x5.0_darwin_arm64_SHA256SUM +``` + +When a non-development version of 1.1.1 becomes available, the binary takes precedence over the development binary: + +```shell-session +/Users/dev/.packer.d/plugins +└─ github.com + └─ hashicorp + └── amazon + ├── packer-plugin-amazon_v1.1.1-dev_x5.0_darwin_arm64 + ├── packer-plugin-amazon_v1.1.1-dev_x5.0_darwin_arm64_SHA256SUM + ├── packer-plugin-amazon_v1.1.1_x5.0_darwin_arm64 + └── packer-plugin-amazon_v1.1.1_x5.0_darwin_arm64_SHA256SUM +``` + +### Example Docker plugin + +Complete the following steps to build and install a custom version of the Docker plugin as an example: + +1. Clone the plugin's GitHub repository. + + ```shell-session + $ git clone https://github.com/hashicorp/packer-plugin-docker.git + ``` + +1. Change to the plugin directory root and run the `go build` command to build the plugin as a development binary. + + ```shell-session + $ cd packer-plugin-docker + $ go build -ldflags="-X github.com/hashicorp/packer-plugin-docker/version.VersionPrerelease=dev" -o packer-plugin-docker-dev + ``` + +1. Validate the release version. + + ```shell-session + $ ./packer-plugin-docker-dev describe + {"version":"1.0.10-dev","sdk_version":"0.5.2","api_version":"x5.0","builders":["-packer-default-plugin-name-"],"post_processors":["import","push","save","tag"],"provisioners":[],"datasources":[]} + ``` + +1. Use the `packer plugins install` command to install the development binary. + + ```shell-session + packer plugins install --path packer-plugin-docker-dev github.com/hashicorp/docker + Successfully installed plugin github.com/hashicorp/docker from $HOME/Development/packer-plugin-docker/packer-plugin-docker-dev to ~/github.com/hashicorp/docker/packer-plugin-docker_v1.0.10-dev_x5.0_darwin_arm64 + ``` + +1. Run a `packer build` with the newly installed plugin. + + ```shell-session + $ packer build . + ``` + +For convenience, the makefile in the Packer plugin scaffolding repository builds and installs development binaries using `make dev`. + +Refer to the documentation in the [Packer plugin scaffolding repository](https://github.com/hashicorp/packer-plugin-scaffolding?tab=readme-ov-file#build-from-source) for additional information. + +## Authenticate requests to the GitHub API + +You can set the `PACKER_GITHUB_API_TOKEN` environment variable to send more requests per hour than the limits imposed by the GitHub API: + +1. Go to your personal [access token page](https://github.com/settings/tokens) to +generate a new token. +1. Set the `PACKER_GITHUB_API_TOKEN` environment variable to your token value: + + ```shell-session + $ export PACKER_GITHUB_API_TOKEN= + ``` diff --git a/content/packer/content/docs/post-processors/artifice.mdx b/content/packer/content/docs/post-processors/artifice.mdx new file mode 100644 index 0000000000..b6397244c8 --- /dev/null +++ b/content/packer/content/docs/post-processors/artifice.mdx @@ -0,0 +1,162 @@ +--- +description: > + The `artifice` post-processor overrides the artifact list from an upstream builder or post-processor. Use it to build artifacts inside a Packer builder, such as starting an EC2 instance to build a Docker container, but only keep the Docker container's artifacts. +page_title: artifice post-processor reference +--- + + + + + +# `artifice` post-processor + +Artifact BuilderId: `packer.post-processor.artifice` + +The artifice post-processor overrides the artifact list from an upstream +builder or post-processor. All downstream post-processors will see the new +artifacts you specify. + +After overriding the artifact with artifice, you can use it with other +post-processors, including most of the core post-processors and third-party +post-processors. + +A major benefit of this is that you can modify builder +artifacts using shell-local and pass those modified artifacts into +post-processors that may not have worked with the original builder. +For example, maybe you want to export a Docker container from an amazon-ebs +builder and then use Docker-push to put that Docker container into your Docker +Hub account. + +Artifice allows you to use the familiar packer workflow to create a fresh, +stateless build environment for each build on the infrastructure of your +choosing. You can use this to build just about anything: buildpacks, +containers, jars, binaries, tarballs, msi installers, and more. + +Please note that the artifice post-processor will _not_ delete your old artifact +files, even if it removes them from the artifact. If you want to delete the +old artifact files, you can use the shell-local post-processor to do so. + +## Workflow + +Artifice helps you tie together a few other packer features: + +- A builder, which spins up a VM (or container) to build your artifact +- A provisioner, which performs the steps to create your artifact +- A file provisioner, which downloads the artifact from the VM +- The artifice post-processor, which identifies which files have been + downloaded from the VM +- Additional post-processors, which push the artifact to Docker hub, etc. + +You will want to perform as much work as possible inside the VM. Ideally the +only other post-processor you need after artifice is one that uploads your +artifact to the appropriate repository. + +## Configuration + +The configuration allows you to specify which files comprise your artifact. + +### Required: + +- `files` (array of strings) - A list of files that comprise your artifact. + These files must exist on your local disk after the provisioning phase of + packer is complete. These will replace any of the builder's original + artifacts (such as a VM snapshot). + +### Optional: + +- `keep_input_artifact` (boolean) - if true, do not delete the original + artifact files after creating your new artifact. Defaults to true. + +### Example Configuration + +You can use Packer to deploy HashiCorp Consul, which is a service discovery and service mesh solution. Refer to the [Consul website](https://www.hashicorp.com/en/products/consul) for additional information. + +In this example, Packer uses a minimal configuration to perform the following actions: + +1. Spins up a cloned VMware virtual machine +1. Installs a Consul release. +1. Downloads the Consul binary. +1. Packages Consul into a `.tar.gz` file. +1. Uploads Consul to S3. + +VMX is a fast way to build and test locally, but you can substitute a different builder. + +```json +{ + "builders": [ + { + "type": "vmware-vmx", + "source_path": "/opt/ubuntu-1404-vmware.vmx", + "ssh_username": "vagrant", + "ssh_password": "vagrant", + "shutdown_command": "sudo shutdown -h now", + "headless": "true", + "skip_compaction": "true" + } + ], + "provisioners": [ + { + "type": "shell", + "inline": [ + "sudo apt-get install -y python-pip", + "sudo pip install ifs", + "sudo ifs install consul --version=0.5.2" + ] + }, + { + "type": "file", + "source": "/usr/local/bin/consul", + "destination": "consul", + "direction": "download" + } + ], + "post-processors": [ + [ + { + "type": "artifice", + "files": ["consul"] + }, + { + "type": "compress", + "output": "consul-0.5.2.tar.gz" + }, + { + "type": "shell-local", + "inline": [ + "/usr/local/bin/aws s3 cp consul-0.5.2.tar.gz s3://" + ] + } + ] + ] +} +``` + +**Notice that there are two sets of square brackets in the post-processor +section.** This creates a post-processor chain, where the output of the +proceeding artifact is passed to subsequent post-processors. If you use only +one set of square braces the post-processors will run individually against the +build artifact (the vmx file in this case) and it will not have the desired +result. + +```json +{ + "post-processors": [ + [ // <--- Start post-processor chain + { + "type": "artifice", + "files": ["consul"] + }, + { + "type": "compress", + ... + } + ], // <--- End post-processor chain + { + "type":"compress" // <-- Standalone post-processor + } + ] +} +``` + +You can create multiple post-processor chains to handle multiple builders (for +example, building Linux and Windows binaries during the same build). diff --git a/content/packer/content/docs/post-processors/checksum.mdx b/content/packer/content/docs/post-processors/checksum.mdx new file mode 100644 index 0000000000..d454ac4fb0 --- /dev/null +++ b/content/packer/content/docs/post-processors/checksum.mdx @@ -0,0 +1,80 @@ +--- +description: > + The `checksum` post-processor computes the checksum for the artifact list + from an upstream builder or post-processor so that you can verify artifacts. +page_title: checksum post-processor reference +--- + + + + + +# `checksum` post-processor + +Artifact BuilderId: `packer.post-processor.checksum` + +The checksum post-processor computes specified checksum for the artifact list +from an upstream builder or post-processor. All downstream post-processors will +see the new artifacts. The primary use-case is compute checksum for artifact to +verify it later. + +After computes checksum for artifacts, you can use new artifacts with other +post-processors like +[artifice](/packer/docs/post-processors/artifice), +[compress](/packer/docs/post-processors/compress), +[docker-push](/packer/plugins/post-processors/docker/docker-push), or +a third-party post-processor. + +## Basic example + + + + +```hcl +post-processor "checksum" { + checksum_types = ["sha1", "sha256"] + output = "packer_{{.BuildName}}_{{.ChecksumType}}.checksum" +} +``` + + + + +```json +{ + "type": "checksum", + "checksum_types": ["sha1", "sha256"], + "output": "packer_{{.BuildName}}_{{.ChecksumType}}.checksum" +} +``` + + + + +## Configuration Reference + +Optional parameters: + +- `checksum_types` (array of strings) - An array of strings of checksum types + to compute. If empty, Defaults to md5. Allowed values are: + + - md5 + - sha1 + - sha224 + - sha256 + - sha384 + - sha512 + +- `output` (string) - Specify filename to store checksums. This defaults to + `packer_{{.BuildName}}_{{.BuilderType}}_{{.ChecksumType}}.checksum`. For + example, if you had a builder named `database`, you might see the file + written as `packer_database_docker_md5.checksum`. This is treated as a + [template engine](/packer/docs/templates/legacy_json_templates/engine). Therefore, you + may use user variables and template functions in this field. + The following special variables are also available to use in the output + template: + + - `BuildName`: The name of the builder that produced the artifact. + - `BuilderType`: The type of builder used to produce the artifact. + - `ChecksumType`: The type of checksums the file contains. This should be + used if you have more than one value in `checksum_types`. diff --git a/content/packer/content/docs/post-processors/community-supported.mdx b/content/packer/content/docs/post-processors/community-supported.mdx new file mode 100644 index 0000000000..0821d86168 --- /dev/null +++ b/content/packer/content/docs/post-processors/community-supported.mdx @@ -0,0 +1,13 @@ +--- +description: > + Community-maintained post-processors are not part of the core Packer binary. You can run community post-processors with Packer post-processors. +page_title: Community post-processors overview +--- + +# Community Post-processors Overview + +The following post-processors are developed and maintained by various members of the +Packer community, not by HashiCorp. For more information on how to use community +post-processors, refer to our documentation on [extending Packer](/packer/docs/plugins/creation). + +@include 'post-processors/community_post-processors.mdx' diff --git a/content/packer/content/docs/post-processors/compress.mdx b/content/packer/content/docs/post-processors/compress.mdx new file mode 100644 index 0000000000..a75039f9df --- /dev/null +++ b/content/packer/content/docs/post-processors/compress.mdx @@ -0,0 +1,85 @@ +--- +description: | + The `compress` post-processor compresses an artifact that contains multiple files, such as VMware of VirtualBox builds, into a single archive. +page_title: compress post-processor reference +--- + + + + + +# `compress` post-processor + +Artifact BuilderId: `packer.post-processor.compress` + +The Packer compress post-processor takes an artifact with files, such as from +VMware or VirtualBox, and compresses the artifact into a single archive. + +## Configuration + +### Optional: + +By default, packer will build archives in `.tar.gz` format with the following +filename: `packer_{{.BuildName}}_{{.BuilderType}}`. If you want to change this +you will need to specify the `output` option. + +- `output` (string) - The path to save the compressed archive. The archive + format is inferred from the filename. E.g. `.tar.gz` will be a gzipped + tarball. `.zip` will be a zip file. If the extension can't be detected + packer defaults to `.tar.gz` behavior but will not change the filename. + + This is treated as a + [template engine](/packer/docs/templates/legacy_json_templates/engine). Therefore, you + may use user variables and template functions in this field. + The following special variables are also available to use in the output + template: + + - `{{.BuildName}}` + - `{{.BuilderType}}` + + If you are executing multiple builders in parallel you should make sure + `output` is unique for each one. For example `packer_{{.BuildName}}.zip`. + +- `format` (string) - Disable archive format autodetection and use provided + string. + +- `compression_level` (number) - Specify the compression level, for + algorithms that support it, from 1 through 9 inclusive. Typically higher + compression levels take longer but produce smaller files. Defaults to `6` + +- `keep_input_artifact` (boolean) - if `true`, keep both the source files and + the compressed file; if `false`, discard the source files. Defaults to + `false` + +### Supported Formats + +Supported file extensions include `.zip`, `.tar`, `.gz`, `.tar.gz`, `.lz4` and +`.tar.lz4`. Note that `.gz` and `.lz4` will fail if you have multiple files to +compress. + +## Examples + +Some minimal examples are shown below, showing only the post-processor +configuration: + +```json +{ + "type": "compress", + "output": "archive.tar.lz4" +} +``` + +```json +{ + "type": "compress", + "output": "{{.BuildName}}_bundle.zip" +} +``` + +```json +{ + "type": "compress", + "output": "log_{{.BuildName}}.gz", + "compression_level": 9 +} +``` diff --git a/content/packer/content/docs/post-processors/index.mdx b/content/packer/content/docs/post-processors/index.mdx new file mode 100644 index 0000000000..2348cc4a26 --- /dev/null +++ b/content/packer/content/docs/post-processors/index.mdx @@ -0,0 +1,13 @@ +--- +description: | + Post-processors are programs that you can run after Packer builds and provisions an image. Use post-processors to upload artifacts and repackage files. +page_title: Post-processors overview +--- + +# Post-processors overview + +Post-processors run after builders and provisioners. Post-processors are optional, and you can use them to upload artifacts, re-package files, and more. The documentation includes a page for each type of post-processor. + +Refer to the [`post-processor`](/packer/docs/templates/hcl_templates/blocks/build/post-processor) and +[`post-processors`](/packer/docs/templates/hcl_templates/blocks/build/post-processors) +blocks documentation to learn more about working with post-processors. diff --git a/content/packer/content/docs/post-processors/manifest.mdx b/content/packer/content/docs/post-processors/manifest.mdx new file mode 100644 index 0000000000..0f8feb3113 --- /dev/null +++ b/content/packer/content/docs/post-processors/manifest.mdx @@ -0,0 +1,205 @@ +--- +description: > + The `manifest` post-processor creates a JSON file that contains data about a Packer build's artifacts, letting you track a run's outputs. +page_title: manifest post-processor reference +--- + + + + + +# `manifest` post-processor + +Artifact BuilderId: `packer.post-processor.manifest` + +The manifest post-processor writes a JSON file with a list of all of the +artifacts packer produces during a run. If your Packer template includes +multiple builds, this helps you keep track of which output artifacts (files, +AMI IDs, Docker containers, etc.) correspond to each build. + +The manifest post-processor is invoked each time a build completes and +_updates_ data in the manifest file. Builds are identified by name and type, +and include their build time, artifact ID, and file list. + +If packer is run with the `-force` flag the manifest file will be truncated +automatically during each packer run. Otherwise, subsequent builds will be +added to the file. You can use the timestamps to see which is the latest +artifact. + +You can specify manifest more than once and write each build to its own file, +or write all builds to the same file. For simple builds manifest only needs to +be specified once (see below) but you can also chain it together with other +post-processors such as Docker and Artifice. + +## Configuration + +### Optional: + +@include 'post-processor/manifest/Config-not-required.mdx' + +~> **Note**: Unlike most other post-processors, the keep_input_artifact option doesn't apply for the manifest +post-processor. We will always retain the input artifact for manifest, since deleting the files we just recorded +is not a behavior anyone should ever expect. + +### Example Configuration + +The minimal way to use the manifest post-processor is by just writing its definition, like: + + + + +```hcl +post-processor "manifest" {} +``` + + + + +```json +{ + "post-processors": [ + { + "type": "manifest" + } + ] +} +``` + + + + +A more complete example: + + + + +```hcl +post-processor "manifest" { + output = "manifest.json" + strip_path = true + custom_data = { + my_custom_data = "example" + } +} +``` + + + + +```json +{ + "post-processors": [ + { + "type": "manifest", + "output": "manifest.json", + "strip_path": true, + "custom_data": { + "my_custom_data": "example" + } + } + ] +} +``` + + + + +An example manifest file looks like: + +```json +{ + "builds": [ + { + "name": "docker", + "builder_type": "docker", + "build_time": 1507245986, + "files": [ + { + "name": "packer_example", + "size": 102219776 + } + ], + "artifact_id": "Container", + "packer_run_uuid": "6d5d3185-fa95-44e1-8775-9e64fe2e2d8f", + "custom_data": { + "my_custom_data": "example" + } + } + ], + "last_run_uuid": "6d5d3185-fa95-44e1-8775-9e64fe2e2d8f" +} +``` + +If the build is run again, the new build artifacts will be added to the +manifest file rather than replacing it. It is possible to grab specific build +artifacts from the manifest by using `packer_run_uuid`. + +The above manifest was generated with the following template: + + + + +```hcl +source "docker" "docker"{ + image = "ubuntu:latest" + export_path = "packer_example" + run_command = ["-d", "-i", "-t", "--entrypoint=/bin/bash", "{{.Image}}"] +} + +build { + sources = ["docker.docker"] + + post-processor "manifest" { + output = "manifest.json" + strip_path = true + custom_data = { + my_custom_data = "example" + } + } +} +``` + + + + +```json +{ + "builders": [ + { + "type": "docker", + "image": "ubuntu:latest", + "export_path": "packer_example", + "run_command": ["-d", "-i", "-t", "--entrypoint=/bin/bash", "{{.Image}}"] + } + ], + "post-processors": [ + { + "type": "manifest", + "output": "manifest.json", + "strip_path": true, + "custom_data": { + "my_custom_data": "example" + } + } + ] +} +``` + + + + +Example usage: + +The manifest can be very useful for cleaning up old artifacts, or printing +important values to logs. The following example uses jq, a command-line tool for +parsing json output, to find and echo the AWS ami-id of an AMI created by a +build. + +```bash + +#!/bin/bash + +AMI_ID=$(jq -r '.builds[-1].artifact_id | split(":") | .[1]' manifest.json) +echo $AMI_ID + +``` diff --git a/content/packer/content/docs/post-processors/shell-local.mdx b/content/packer/content/docs/post-processors/shell-local.mdx new file mode 100644 index 0000000000..002f9d6aa3 --- /dev/null +++ b/content/packer/content/docs/post-processors/shell-local.mdx @@ -0,0 +1,646 @@ +--- +description: | + The `shell-local` post-processor starts a local shell, letting you automate post-build actions after Packer builds your artifacts. +page_title: shell-local post-processor reference +--- + + + + + +# `shell-local` post-processor + +The `shell-local` post processor executes scripts locally during the post +processing stage. Shell local provides a convenient way to automate executing +some task with packer outputs and variables. + +## Basic example + +The example below is a fully functional self-contained build. + + + + +```hcl +source "file" "example" { + content = "example content" +} + +build { + source "source.file.example" { + target = "./test_artifact.txt" + } + + post-processor "shell-local" { + inline = ["echo foo"] + } +} +``` + + + + +```json +{ + "builders": [ + { + "type": "file", + "name": "example", + "target": "./test_artifact.txt", + "content": "example content" + } + ], + "post-processors": [ + { + "type": "shell-local", + "inline": ["echo foo"] + } + ] +} +``` + + + + +## Configuration Reference + +The reference of available configuration options is listed below. The only +required element is either "inline" or "script". Every other option is +optional. + +Exactly _one_ of the following is required: + +- `command` (string) - This is a single command to execute. It will be + written to a temporary file and run using the `execute_command` call below. + +- `inline` (array of strings) - This is an array of commands to execute. The + commands are concatenated by newlines and turned into a single file, so + they are all executed within the same context. This allows you to change + directories in one command and use something in the directory in the next + and so on. Inline scripts are the easiest way to pull off simple tasks + within the machine. + +- `script` (string) - The path to a script to execute. This path can be + absolute or relative. If it is relative, it is relative to the working + directory when Packer is executed. + +- `scripts` (array of strings) - An array of scripts to execute. The scripts + will be executed in the order specified. Each script is executed in + isolation, so state such as variables from one script won't carry on to the + next. + +Optional parameters: + +- `env` (map of strings) - A map of key/value pairs to inject prior to the + execute_command. Packer injects some environmental variables by default into + the environment, as well, which are covered in the section below. Duplicate + `env` settings override `environment_vars` settings. + +- `environment_vars` (array of strings) - An array of key/value pairs to + inject prior to the `execute_command`. The format should be `key=value`. + Packer injects some environmental variables by default into the + environment, as well, which are covered in the section below. + +- `env_var_format` (string) - When we parse the environment_vars that you + provide, this gives us a string template to use in order to make sure that + we are setting the environment vars correctly. By default on Windows hosts + this format is `set %s=%s &&` and on Unix, it is `%s='%s'`. You probably + won't need to change this format, but you can see usage examples for where + it is necessary below. + +- `execute_command` (array of strings) - The command used to execute the + script. By default, on \*nix systems this is: + + ```text + ["/bin/sh", "-c", "{{.Vars}} {{.Script}}"] + ``` + + While on Windows, `execute_command` defaults to: + + ```text + ["cmd", "/V", "/C", "{{.Vars}}", "call", "{{.Script}}"] + ``` + + This is treated as a [template engine](/packer/docs/templates/legacy_json_templates/engine). + There are several available variables: `Script`, which is the path to the + script to run, and `Vars`, which is the list of `environment_vars`, if + configured. In addition, you may access any of the variables stored in the + generated data using the [build](/packer/docs/templates/legacy_json_templates/engine) template + function. If you choose to set this option, make sure that the first + element in the array is the shell program you want to use (for example, + "sh" or "/usr/local/bin/zsh" or even "powershell.exe" although anything + other than a flavor of the shell command language is not explicitly + supported and may be broken by assumptions made within Packer). It's + worth noting that if you choose to try to use shell-local for + Powershell or other Windows commands, the environment variables will + not be set properly for your environment. + + For backwards compatibility, `execute_command` will accept a string instead + of an array of strings. If a single string or an array of strings with only + one element is provided, Packer will replicate past behavior by appending + your `execute_command` to the array of strings `["sh", "-c"]`. For example, + if you set `"execute_command": "foo bar"`, the final `execute_command` that + Packer runs will be \["sh", "-c", "foo bar"\]. If you set + `"execute_command": ["foo", "bar"]`, the final execute_command will remain + `["foo", "bar"]`. + + Again, the above is only provided as a backwards compatibility fix; we + strongly recommend that you set execute_command as an array of strings. + +- `inline_shebang` (string) - The + [shebang](http://en.wikipedia.org/wiki/Shebang_%28Unix%29) value to use + when running commands specified by `inline`. By default, this is + `/bin/sh -e`. If you're not using `inline`, then this configuration has no + effect. **Important:** If you customize this, be sure to include something + like the `-e` flag, otherwise individual steps failing won't fail the + provisioner. + +- `keep_input_artifact` (boolean) - Unlike most other post-processors, the + keep_input_artifact option will have no effect for the shell-local + post-processor. Packer will always retain the input artifact for + shell-local, since the shell-local post-processor merely passes forward the + artifact it receives. If your shell-local post-processor produces a file or + files which you would like to have replace the input artifact, you may + overwrite the input artifact using the [artifice](/packer/docs/post-processors/artifice) + post-processor after your shell-local processor has run. + +- `only_on` (array of strings) - This is an array of [runtime operating + systems](https://go.dev/doc/install/source#environment) where + `shell-local` will execute. This allows you to execute `shell-local` _only_ + on specific operating systems. By default, shell-local will always run if + `only_on` is not set." + +- `use_linux_pathing` (bool) - This is only relevant to Windows hosts. If you + are running Packer in a Windows environment with the Windows Subsystem for + Linux feature enabled, and would like to invoke a bash script rather than + invoking a Cmd script, you'll need to set this flag to true; it tells + Packer to use the Linux subsystem path for your script rather than the + Windows path. (e.g. /mnt/c/path/to/your/file instead of + C:/path/to/your/file). Please see the example below for more guidance on + how to use this feature. If you are not on a Windows host, or you do not + intend to use the shell-local post-processor to run a bash script, please + ignore this option. If you set this flag to true, you still need to provide + the standard Windows path to the script when providing a `script`. This is + a beta feature. + +- `valid_exit_codes` (list of ints) - Valid exit codes for the script. By + default this is `0`. + +## Execute Command + +To many new users, the `execute_command` is puzzling. However, it provides an +important function: customization of how the command is executed. The most +common use case for this is dealing with **sudo password prompts**. You may +also need to customize this if you use a non-POSIX shell, such as `tcsh` on +FreeBSD. + +### The Windows Linux Subsystem + +The shell-local post-processor was designed with the idea of allowing you to +run commands in your local operating system's native shell. For Windows, we've +assumed in our defaults that this is Cmd. However, it is possible to run a bash +script as part of the Windows Linux Subsystem from the shell-local +post-processor, by modifying the `execute_command` and the `use_linux_pathing` +options in the post-processor config. + +The example below is a fully functional test config. + +One limitation of this offering is that "inline" and "command" options are not +available to you; please limit yourself to using the "script" or "scripts" +options instead. + +Please note that this feature is still in beta, as the underlying WSL is also +still in beta. There will be some limitations as a result. For example, it will +likely not work unless both Packer and the scripts you want to run are both on +the C drive. + + + + +```hcl +source "null" "example" { + communicator = "none" +} + +build { + sources = [ + "source.null.example" + ] + + post-processor "shell-local"{ + environment_vars = ["PROVISIONERTEST=ProvisionerTest1"] + execute_command = ["bash", "-c", "{{.Vars}} {{.Script}}"] + use_linux_pathing = true + scripts = ["C:/Users/me/scripts/example_bash.sh"] + } + post-processor "shell-local"{ + environment_vars = ["PROVISIONERTEST=ProvisionerTest2"] + execute_command = ["bash", "-c", "{{.Vars}} {{.Script}}"] + use_linux_pathing = true + script = "C:/Users/me/scripts/example_bash.sh" + } +} +``` + + + + +```json +{ + "builders": [ + { + "type": "null", + "communicator": "none" + } + ], + "post-processors": [ + { + "type": "shell-local", + "environment_vars": ["PROVISIONERTEST=ProvisionerTest1"], + "execute_command": ["bash", "-c", "{{.Vars}} {{.Script}}"], + "use_linux_pathing": true, + "scripts": ["C:/Users/me/scripts/example_bash.sh"] + }, + { + "type": "shell-local", + "environment_vars": ["PROVISIONERTEST=ProvisionerTest2"], + "execute_command": ["bash", "-c", "{{.Vars}} {{.Script}}"], + "use_linux_pathing": true, + "script": "C:/Users/me/scripts/example_bash.sh" + } + ] +} +``` + + + + +## Default Environmental Variables + +In addition to being able to specify custom environmental variables using the +`environment_vars` configuration, the provisioner automatically defines certain +commonly useful environmental variables: + +- `PACKER_BUILD_NAME` is set to the [name of the + build](/packer/docs/templates/legacy_json_templates/builders#named-builds) that Packer is running. + This is most useful when Packer is making multiple builds and you want to + distinguish them slightly from a common provisioning script. + +- `PACKER_BUILDER_TYPE` is the type of the builder that was used to create + the machine that the script is running on. This is useful if you want to + run only certain parts of the script on systems built with certain + builders. + +## Safely Writing A Script + +Whether you use the `inline` option, or pass it a direct `script` or `scripts`, +it is important to understand a few things about how the shell-local +post-processor works to run it safely and easily. This understanding will save +you much time in the process. + +### Once Per Builder + +The `shell-local` script(s) you pass are run once per builder. This means that +if you have an `amazon-ebs` builder and a `docker` builder, your script will be +run twice. If you have 3 builders, it will run 3 times, once for each builder. + +### Interacting with Build Artifacts + +In order to interact with build artifacts, you may want to use the [manifest +post-processor](/packer/docs/post-processors/manifest). This will write the list +of files produced by a `builder` to a json file after each `builder` is run. + +For example, if you wanted to package a file from the file builder into a +tarball, you might write this: + + + + +```json +{ + "builders": [ + { + "content": "Lorem ipsum dolor sit amet", + "target": "dummy_artifact", + "type": "file" + } + ], + "post-processors": [ + [ + { + "output": "manifest.json", + "strip_path": true, + "type": "manifest" + }, + { + "inline": [ + "jq \".builds[].files[].name\" manifest.json | xargs tar cfz artifacts.tgz" + ], + "type": "shell-local" + } + ] + ] +} +``` + + + + +```hcl +source "file" "example" { + content = "Lorem ipsum dolor sit amet" + target = "dummy_artifact.txt" +} +build { + sources = [ + "source.file.example" + ] + post-processor "manifest" { + output = "manifest.json" + strip_path = true + } + + post-processor "shell-local" { + inline = [ + "jq \".builds[].files[].name\" manifest.json | xargs tar cfz artifacts.tgz" + ] + } +} +``` + + + + +This uses the [jq](https://stedolan.github.io/jq/) tool to extract all of the +file names from the manifest file and passes them to tar. + +### Always Exit Intentionally + +If any post-processor fails, the `packer build` stops and all interim artifacts +are cleaned up. + +For a shell script, that means the script **must** exit with a zero code. You +_must_ be extra careful to `exit 0` when necessary. + +## Usage Examples: + +### Windows Host + +Example of running a .cmd file on Windows: + + + + +```hcl +post-processor "shell-local" { + environment_vars = ["SHELLLOCALTEST=ShellTest1"] + scripts = ["./scripts/test_cmd.cmd"] +} +``` + + + + +```json +{ + "type": "shell-local", + "environment_vars": ["SHELLLOCALTEST=ShellTest1"], + "scripts": ["./scripts/test_cmd.cmd"] +} +``` + + + + +Contents of `test_cmd.cmd`: + + echo %SHELLLOCALTEST% + +Example of running an inline command on Windows: Required customization: +tempfile_extension + + + + +```hcl +post-processor "shell-local" { + environment_vars = ["SHELLLOCALTEST=ShellTest2"], + tempfile_extension = ".cmd", + inline = ["echo %SHELLLOCALTEST%"] +} +``` + + + + +```json +{ + "type": "shell-local", + "environment_vars": ["SHELLLOCALTEST=ShellTest2"], + "tempfile_extension": ".cmd", + "inline": ["echo %SHELLLOCALTEST%"] +} +``` + + + + +Example of running a bash command on Windows using WSL: Required +customizations: `use_linux_pathing` and `execute_command`: + + + + +```hcl +post-processor "shell-local" { + environment_vars = ["SHELLLOCALTEST=ShellTest3"], + execute_command = ["bash", "-c", "{{.Vars}} {{.Script}}"] + use_linux_pathing = true + script = "./scripts/example_bash.sh" +} +``` + + + + +```json +{ + "type": "shell-local", + "environment_vars": ["SHELLLOCALTEST=ShellTest3"], + "execute_command": ["bash", "-c", "{{.Vars}} {{.Script}}"], + "use_linux_pathing": true, + "script": "./scripts/example_bash.sh" +} +``` + + + + +Contents of `example_bash.sh`: + + #!/bin/bash + echo $SHELLLOCALTEST + +Example of running a PowerShell script on Windows: +Required customizations: `env_var_format` and `execute_command`. + + + + +```hcl +post-processor "shell-local" { + environment_vars = ["SHELLLOCALTEST=ShellTest4"] + execute_command = ["powershell.exe", "{{.Vars}} {{.Script}}"] + env_var_format = "$env:%s=\"%s\"; " + script = "./scripts/example_ps.ps1" +} +``` + + + + +```json +{ + "type": "shell-local", + "environment_vars": ["SHELLLOCALTEST=ShellTest4"], + "execute_command": ["powershell.exe", "{{.Vars}} {{.Script}}"], + "env_var_format": "$env:%s=\"%s\"; ", + "script": "./scripts/example_ps.ps1" +} +``` + + + + +Example of running a PowerShell script on Windows as "inline": Required +customizations: `env_var_format`, `tempfile_extension`, and `execute_command` + + + + +```hcl +post-processor "shell-local" { + tempfile_extension = ".ps1" + environment_vars = ["SHELLLOCALTEST=ShellTest5"] + execute_command = ["powershell.exe", "{{.Vars}} {{.Script}}"] + env_var_format = "$env:%s=\"%s\"; " + inline = ["write-output $env:SHELLLOCALTEST"] +} +``` + + + + +```json +{ + "type": "shell-local", + "tempfile_extension": ".ps1", + "environment_vars": ["SHELLLOCALTEST=ShellTest5"], + "execute_command": ["powershell.exe", "{{.Vars}} {{.Script}}"], + "env_var_format": "$env:%s=\"%s\"; ", + "inline": ["write-output $env:SHELLLOCALTEST"] +} +``` + + + + +### Unix Host + +Example of running a Shell script on Unix: + + + + +```hcl +post-processor "shell-local" { + environment_vars = ["PROVISIONERTEST=ProvisionerTest1"] + scripts = ["./scripts/example_bash.sh"] +} +``` + + + + +```json +{ + "type": "shell-local", + "environment_vars": ["PROVISIONERTEST=ProvisionerTest1"], + "scripts": ["./scripts/example_bash.sh"] +} +``` + + + + +Example of running a bash "inline" on Unix: + + + + +```hcl +post-processor "shell-local" { + environment_vars = ["PROVISIONERTEST=ProvisionerTest2"] + inline = ["echo hello", "echo $PROVISIONERTEST"] +} + +``` + + + + +```json +{ + "type": "shell-local", + "environment_vars": ["PROVISIONERTEST=ProvisionerTest2"], + "inline": ["echo hello", "echo $PROVISIONERTEST"] +} +``` + + + + +Example of running a Python script on Unix: + + + + +```hcl +post-processor "shell-local" { + script = "hello.py" + environment_vars = ["HELLO_USER=packeruser"] + execute_command = [ + "/bin/sh", + "-c", + "{{.Vars}} /usr/local/bin/python {{.Script}}" + ] +} + +``` + + + + +```json +{ + "type": "shell-local", + "script": "hello.py", + "environment_vars": ["HELLO_USER=packeruser"], + "execute_command": [ + "/bin/sh", + "-c", + "{{.Vars}} /usr/local/bin/python {{.Script}}" + ] +} +``` + + + + +```text +Where "hello.py" contains: + + import os + + print('Hello, %s!' % os.getenv("HELLO_USER")) +``` diff --git a/content/packer/content/docs/provisioners/breakpoint.mdx b/content/packer/content/docs/provisioners/breakpoint.mdx new file mode 100644 index 0000000000..2cde938639 --- /dev/null +++ b/content/packer/content/docs/provisioners/breakpoint.mdx @@ -0,0 +1,103 @@ +--- +description: > + The `breakpoint` provisioner pauses a build until you press the Enter key. Use the `breakpoint` provisioner to help you debug errors. +page_title: breakpoint provisioner reference +--- + + + + + +# `breakpoint` provisioner + +The `breakpoint` provisioner pauses the build operation until you press the Enter key to resume the build. Use the `breakpoint` provisioner to help you debug errors. + +Alternatively, you can add the [`-debug` flag](/packer/docs/commands/build#debug) when running your build to halt the operation at every step and between every provisioner. + +## Basic Example + + + + +```hcl +source "null" "example" { + communicator = "none" +} + +build { + sources = ["source.null.example"] + + provisioner "shell-local" { + inline = ["echo hi"] + } + provisioner "breakpoint" { + disable = false + note = "this is a breakpoint" + } + provisioner "shell-local" { + inline = ["echo hi 2"] + } +} +``` + + + + +```json +{ + "builders": [ + { + "type": "null", + "communicator": "none" + } + ], + "provisioners": [ + { + "type": "shell-local", + "inline": "echo hi" + }, + { + "type": "breakpoint", + "disable": false, + "note": "this is a breakpoint" + }, + { + "type": "shell-local", + "inline": "echo hi 2" + } + ] +} +``` + + + + +## Configuration Reference + +### Optional + +- `disable` (boolean) - If `true`, skip the breakpoint. Useful for when you + have set multiple breakpoints and want to toggle them off or on. Default: + `false` + +- `note` (string) - a string to include explaining the purpose or location of + the breakpoint. For example, you may find it useful to number your + breakpoints or label them with information about where in the build they + occur + +@include 'provisioners/common-config.mdx' + +## Usage + +Insert this provisioner wherever you want the build to pause. You'll see ui +output prompting you to press "enter" to continue the build when you are ready. + +For example: + +```shell-session +==> docker: Pausing at breakpoint provisioner with note "foo bar baz". +==> docker: Press enter to continue. +``` + +Once you press enter, the build will resume and run normally until it either +completes or errors. diff --git a/content/packer/content/docs/provisioners/community-supported.mdx b/content/packer/content/docs/provisioners/community-supported.mdx new file mode 100644 index 0000000000..f2bdf70e9d --- /dev/null +++ b/content/packer/content/docs/provisioners/community-supported.mdx @@ -0,0 +1,23 @@ +--- +description: | + You can use provisioners built and maintained by the Packer developer community to extend Packer functionality. Learn about community-supported provisioners. +page_title: Community provisioners reference +--- + +# Community Provisioners + +The following provisioners are developed and maintained by various members of the +Packer community, not by HashiCorp. For more information on how to use community +provisioners, refer to [Developing Plugins](/packer/docs/plugins/creation). + +- [Comment Provisioner](https://github.com/SwampDragons/packer-provisioner-comment) - + Example provisioner that allows you to annotate your build with bubble-text + comments. + +- [Windows Update provisioner](https://github.com/rgl/packer-plugin-windows-update) - + A provisioner for gracefully handling Windows updates and the reboots they + cause. + +- [S3 Provisioner](https://github.com/spacechunks/packer-plugin-s3) - + A provisioner that retrieves objects from S3 and stores them at a given destination. + diff --git a/content/packer/content/docs/provisioners/file.mdx b/content/packer/content/docs/provisioners/file.mdx new file mode 100644 index 0000000000..5918b12737 --- /dev/null +++ b/content/packer/content/docs/provisioners/file.mdx @@ -0,0 +1,192 @@ +--- +description: | + The `file` Packer provisioner uploads files to machines built by Packer. Learn how to use the `file` provisioner and about the Packer provisioner workflow. +page_title: file provisioner reference +--- + + + + + +# `file` provisioner + +The `file` Packer provisioner uploads files to machines built by Packer. We recommend implementing the following workflow: + +1. Use the `file` provisioner to upload files +1. Use the [`shell` provisioner](/packer/docs/provisioners/shell) to move the files into the proper place, set permissions, and perform other tasks. + +You can only upload files to locations that the provisioning user +(generally not root) has permission to access. Creating files in /tmp and +using a shell provisioner to move them into the final location is the only +way to upload files to root owned locations. + +The file provisioner can upload both single files and complete directories. + +## Basic Example + + + + +```hcl +provisioner "file" { + source = "app.tar.gz" + destination = "/tmp/app.tar.gz" +} +``` + + + + +```json +{ + "type": "file", + "source": "app.tar.gz", + "destination": "/tmp/app.tar.gz" +} +``` + + + + +## Configuration Reference + +The available configuration options are listed below. + +## Configuration Reference + +Required Parameters: + +@include 'provisioner/file/Config-required.mdx' + +Optional Parameters: + +@include '/provisioner/file/Config-not-required.mdx' + +@include 'provisioners/common-config.mdx' + +## Directory Uploads + +The file provisioner is also able to upload a complete directory to the remote +machine. When uploading a directory, there are a few important things you +should know. + +First, the destination directory must already exist. If you need to create it, +use a shell provisioner just prior to the file provisioner in order to create +the directory. If the destination directory does not exist, the file +provisioner may succeed, but it will have undefined results. + +Next, the existence of a trailing slash on the source path will determine +whether the directory name will be embedded within the destination, or whether +the destination will be created. An example explains this best: + +If the source is `/foo` (no trailing slash), and the destination is `/tmp`, +then the contents of `/foo` on the local machine will be uploaded to `/tmp/foo` +on the remote machine. The `foo` directory on the remote machine will be +created by Packer. + +If the source, however, is `/foo/` (a trailing slash is present), and the +destination is `/tmp`, then the contents of `/foo` will be uploaded into `/tmp` +directly. + +This behavior was adopted from the standard behavior of rsync. Note that under +the covers, rsync may or may not be used. + +**Note:** Packer does not infer behaviors based on the guest operating system, +as this would add unnecessary complexity and reduce maintainability. To ensure +consistent behavior across all platforms, use a trailing forward slash when +specifying directories, even when working with Windows guests. + +## Uploading files that don't exist before Packer starts + +In general, local files used as the source **must** exist before Packer is run. +This is great for catching typos and ensuring that once a build is started, +that it will succeed. However, this also means that you can't generate a file +during your build and then upload it using the file provisioner later. A +convenient workaround is to upload a directory instead of a file. The directory +still must exist, but its contents don't. You can write your generated file to +the directory during the Packer run, and have it be uploaded later. + +## Symbolic link uploads + +The behavior when uploading symbolic links depends on the communicator. The +Docker communicator will preserve symlinks, but all other communicators will +treat local symlinks as regular files. If you wish to preserve symlinks when +uploading, it's recommended that you use `tar`. Below is an example of what +that might look like: + +```shell-session +$ ls -l files +total 16 +drwxr-xr-x 3 mwhooker staff 102 Jan 27 17:10 a +lrwxr-xr-x 1 mwhooker staff 1 Jan 27 17:10 b -> a +-rw-r--r-- 1 mwhooker staff 0 Jan 27 17:10 file1 +lrwxr-xr-x 1 mwhooker staff 5 Jan 27 17:10 file1link -> file1 +$ ls -l toupload +total 0 +-rw-r--r-- 1 mwhooker staff 0 Jan 27 17:10 files.tar +``` + + + + +```json +{ + "provisioners": [ + { + "type": "shell-local", + "command": "tar cf toupload/files.tar files" + }, + { + "destination": "/tmp/", + "source": "./toupload", + "type": "file" + }, + { + "inline": [ + "cd /tmp && tar xf toupload/files.tar", + "rm toupload/files.tar" + ], + "type": "shell" + } + ] +} +``` + + + + +```hcl +build { + sources = [ + "source.docker.example" + ] + + provisioner "shell-local" { + command = "tar cf toupload/files.tar files" + } + provisioner "file" { + destination = "/tmp/" + source = "./toupload" + } + provisioner "shell" { + inline = [ + "cd /tmp && tar xf toupload/files.tar", + "rm toupload/files.tar" + ] + } +} +``` + + + + +## Slowness when transferring large files over WinRM. + +Because of the way our WinRM transfers works, it can take a very long time to +upload and download even moderately sized files. If you're experiencing slowness +using the file provisioner on Windows, it's suggested that you set up an SSH +server and use the [ssh communicator](/packer/docs/communicators/ssh). If you only want +to transfer files to your guest, and if your builder supports it, you may also +use the `http_directory` or `http_content` directives. This will cause that +directory to be available to the guest over HTTP, and set the environment +variable `PACKER_HTTP_ADDR` to the address. diff --git a/content/packer/content/docs/provisioners/hcp-sbom.mdx b/content/packer/content/docs/provisioners/hcp-sbom.mdx new file mode 100644 index 0000000000..a165c5d24f --- /dev/null +++ b/content/packer/content/docs/provisioners/hcp-sbom.mdx @@ -0,0 +1,135 @@ +--- +description: | + The `hcp-sbom` Packer provisioner uploads a CycloneDX- or SPDX JSON-formatted software bill of materials record to HCP Packer. Learn how to use the `hcp-sbom` provisioner. +page_title: hcp-sbom provisioner reference +--- + + + + + +# `hcp-sbom` provisioner + +The `hcp-sbom` provisioner uploads software bill of materials (SBOM) files from artifacts built by Packer to HCP Packer. You must format SBOM files you want to upload as JSON and follow either the [SPDX](https://spdx.github.io/spdx-spec/latest) or [CycloneDX](https://cyclonedx.org/) specification. HCP Packer ties these SBOM files to the version of the artifact that Packer builds. + +## Example + +The following example uploads an SBOM from the local `/tmp` directory and stores a copy at `./sbom/sbom_cyclonedx.json` on the local machine. + + + + +```hcl +provisioner "hcp-sbom" { + source = "/tmp/sbom_cyclonedx.json" + destination = "./sbom/sbom_cyclonedx.json" + sbom_name = "sbom-cyclonedx" +} +``` + + + + +```json +{ + "type": "hcp-sbom", + "source": "/tmp/sbom_cyclonedx.json", + "destination": "./sbom/sbom_cyclonedx.json", + "sbom_name": "sbom-cyclonedx" +} +``` + + + + +## Configuration reference + +You can specify the following configuration options. + +Required parameters: + +@include 'provisioner/hcp-sbom/Config-required.mdx' + +Optional parameters: + +@include '/provisioner/hcp-sbom/Config-not-required.mdx' + +## Example usage + + + + +```hcl +packer { + required_plugins { + docker = { + version = ">= 1.0.0" + source = "github.com/hashicorp/docker" + } + } +} + +source "docker" "ubuntu" { + image = "ubuntu:20.04" + commit = true +} + +build { + sources = ["source.docker.ubuntu"] + + hcp_packer_registry { + bucket_name = "test-bucket" + } + + + provisioner "shell" { + inline = [ + "apt-get update -y", + "apt-get install -y curl gpg", + "bash -c \"$(curl -sSL https://install.mondoo.com/sh)\"", + "cnquery sbom --output cyclonedx-json --output-target /tmp/sbom_cyclonedx.json", + ] + } + + provisioner "hcp-sbom" { + source = "/tmp/sbom_cyclonedx.json" + destination = "./sbom" + sbom_name = "sbom-cyclonedx" + } +} +``` + + + + +```json +{ + "builders": [ + { + "type": "docker", + "image": "ubuntu:20.04", + "commit": true + } + ], + "provisioners": [ + { + "type": "shell", + "inline": [ + "apt-get update -y", + "apt-get install -y curl", + "bash -c \"$(curl -sSL https://install.mondoo.com/sh)\"", + "cnquery sbom --output cyclonedx-json --output-target /tmp/sbom_cyclonedx.json" + ] + }, + { + "type": "hcp-sbom", + "source": "/tmp/sbom_cyclonedx.json", + "destination": "./sbom", + "sbom_name": "sbom-cyclonedx" + } + ] +} +``` + + + \ No newline at end of file diff --git a/content/packer/content/docs/provisioners/index.mdx b/content/packer/content/docs/provisioners/index.mdx new file mode 100644 index 0000000000..f64e25f468 --- /dev/null +++ b/content/packer/content/docs/provisioners/index.mdx @@ -0,0 +1,39 @@ +--- +description: | + Provisioners use builtin and third-party software to install and configure the + machine image after booting. Learn about Packer provisioners. +page_title: Provisioners overview +--- + +# Provisioners overview + +Provisioners use built-in and third-party software to install and configure the machine image after +booting. Provisioners prepare the system, so you may want to use them for the following use cases: + +- installing packages +- patching the kernel +- creating users +- downloading application code + +The following provisioners are included with Packer: + +- [`breakpoint`](/packer/docs/provisioners/breakpoint) - pause until the user presses `Enter` to resume + a build. +- [`file`](/packer/docs/provisioners/file) - upload files to machines image during a build. +- [`hcp-sbom`](/packer/docs/provisioners/hcp-sbom) - upload an SBOM and associate it with an artifact + version in the HCP Packer registry. +- [`shell`](/packer/docs/provisioners/shell) - run shell scripts on the machines image during a build. +- [`shell-local`](/packer/docs/provisioners/shell-local) - run shell scripts on the host running Packer + during a build. +- [`powershell`](/packer/docs/provisioners/powershell) - run PowerShell scripts on Windows machine + images during a build. +- [`windows-shell`](/packer/docs/provisioners/windows-shell) - run commands using `cmd` on Windows + machine images during a build. +- [`windows-restart`](/packer/docs/provisioners/windows-restart) - initiate a reboot on a Windows + machine images during a build. + +Additional [Community Supported](/packer/docs/provisioners/community-supported) provisioners are +developed and maintained by various members of the community. + +Refer to the [`provisioner`](/packer/docs/templates/hcl_templates/blocks/build/provisioner) block +documentation to learn more about working with provisioners. diff --git a/content/packer/content/docs/provisioners/powershell.mdx b/content/packer/content/docs/provisioners/powershell.mdx new file mode 100644 index 0000000000..d0cffb8904 --- /dev/null +++ b/content/packer/content/docs/provisioners/powershell.mdx @@ -0,0 +1,478 @@ +--- +description: | + The `powershell` Packer provisioner runs PowerShell scripts on Windows machines. + Learn how to use the provisioner with WinRM and SSH communicators. +page_title: powershell provisioner reference +--- + + + + + +# `powershell` provisioner + +The `powershell` Packer provisioner runs PowerShell scripts on Windows machines. +The provisioner is designed for machines using WinRM as the communicator, but you can also use the provisioner with the SSH communicator. Refer to the [Combining the PowerShell Provisioner with the SSH Communicator](#combining-the-powershell-provisioner-with-the-ssh-communicator) for +details. + +`@include 'path/separator-note.mdx'` + +## Basic Example + +The example below is fully functional. + + + + +```hcl +provisioner "powershell" { + inline = ["dir c:/"] +} +``` + + + + +```json +{ + "type": "powershell", + "inline": ["dir c:/"] +} +``` + + + + +## Configuration Reference + +@include 'provisioners/shell-config.mdx' + +- `debug_mode` - If set, sets PowerShell's [PSDebug mode](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.core/set-psdebug?view=powershell-7) + in order to make script debugging easier. For instance, setting the value to 1 + results in adding this to the execute command: + + ```powershell + Set-PSDebug -Trace 1 + ``` + +- `elevated_execute_command` (string) - The command to use to execute the + elevated script. By default this is as follows: + + ```powershell + powershell -executionpolicy bypass "& { if (Test-Path variable:global:ProgressPreference){$ProgressPreference='SilentlyContinue'};. {{.Vars}}; &'{{.Path}}'; exit $LastExitCode }" + ``` + + This is a [template engine](/packer/docs/templates/legacy_json_templates/engine). + Therefore, you may use user variables and template functions in this field. In + addition, you may use two extra variables: + + - `Path`: The path to the script to run + - `Vars`: The location of a temp file containing the list of + `environment_vars`, if configured. + +- `env` (map of strings) - A map of key/value pairs to inject prior to the + execute_command. Packer injects some environmental variables by default into + the environment, as well, which are covered in the section below. Duplicate + `env` settings override `environment_vars` settings. This is not a JSON + template engine enabled function. HCL interpolation works as usual. + +- `environment_vars` (array of strings) - An array of key/value pairs to + inject prior to the execute_command. The format should be `key=value`. + Packer injects some environmental variables by default into the + environment, as well, which are covered in the section below. + +- `use_pwsh` (boolean) - Run `pwsh.exe` instead of `powershell.exe`. Defaults to false. + + This is a [template engine](/packer/docs/templates/legacy_json_templates/engine). Therefore, you + may use user variables and template functions in this field. If you are + running on AWS, Azure, Google Compute, or OpenStack and would like to access + the autogenerated password that Packer uses to connect to the instance via + WinRM, you can use the `build` template engine to inject it using + `` {{ build `Password` }} ``. In HCL templates, you can do the same thing by + accessing the `build` variables For example: + + + + +```hcl +provisioner "powershell" { + environment_vars = ["WINRMPASS=${build.Password}"] + inline = ["Write-Host \"Automatically generated aws password is: $Env:WINRMPASS\""] +} +``` + + + + +```json +{ + "type": "powershell", + "environment_vars": ["WINRMPASS={{ build `Password` }}"], + "inline": ["Write-Host \"Automatically generated aws password is: $Env:WINRMPASS\""] +}, +``` + + + + +- `execute_command` (string) - The command to use to execute the script. By + default this is as follows: + + ```powershell + powershell -executionpolicy bypass "& { if (Test-Path variable:global:ProgressPreference){$ProgressPreference='SilentlyContinue'};. {{.Vars}}; &'{{.Path}}'; exit $LastExitCode }" + ``` + + This is a [template engine](/packer/docs/templates/legacy_json_templates/engine). + Therefore, you may use user variables and template functions in this field. In + addition, you may use two extra variables: + + - `Path`: The path to the script to run + - `Vars`: The location of a temp file containing the list of + `environment_vars`, if configured. + The value of both `Path` and `Vars` can be manually configured by setting + the values for `remote_path` and `remote_env_var_path` respectively. + + If you use the SSH communicator and have changed your default shell, you + may need to modify your `execute_command` to make sure that the command is + valid and properly escaped; the default assumes that you have not changed + the default shell away from cmd. + +- `elevated_user` and `elevated_password` (string) - If specified, the + PowerShell script will be run with elevated privileges using the given + Windows user. + + This is a [template engine](/packer/docs/templates/legacy_json_templates/engine). Therefore, you + may use user variables and template functions in this field. If you are + running on AWS, Azure, Google Compute, or OpenStack and would like to access + the autogenerated password that Packer uses to connect to the instance via + WinRM, you can use the `build` template engine to inject it using + `` {{ build `Password` }} ``. In HCL templates, you can do the same thing by + accessing the `build` variables For example: + + + + +```hcl +provisioner "powershell" { + elevated_user = "Administrator" + elevated_password = build.Password +} +``` + + + + +```json +{ + "type": "powershell", + "elevated_user": "Administrator", + "elevated_password": "{{ build `Password` }}", + ... +}, +``` + + + + +If you specify an empty `elevated_password` value then the PowerShell +script is run as a service account. For example: + + + + +```hcl +provisioner "powershell" { + elevated_user = "SYSTEM" + elevated_password = "" +} +``` + + + + +```json +{ + "type": "powershell", + "elevated_user": "SYSTEM", + "elevated_password": "", + ... +}, +``` + + + + +- `execution_policy` - To run ps scripts on Windows, Packer defaults this to + "bypass" and wraps the command to run. Setting this to "none" will prevent + wrapping, allowing to see exit codes on Docker for Windows. Possible values + are `bypass`, `allsigned`, `default`, `remotesigned`, `restricted`, + `undefined`, `unrestricted`, and `none`. + +- `remote_path` (string) - The path where the PowerShell script will be + uploaded to within the target build machine. This defaults to + `C:/Windows/Temp/script-UUID.ps1` where UUID is replaced with a dynamically + generated string that uniquely identifies the script. + + This setting allows users to override the default upload location. The + value must be a writable location and any parent directories must already + exist. + +- `remote_env_var_path` (string) - Environment variables required within the + remote environment are uploaded within a PowerShell script and then enabled + by 'dot sourcing' the script immediately prior to execution of the main + command or script. + + The path the environment variables script will be uploaded to defaults to + `C:/Windows/Temp/packer-ps-env-vars-UUID.ps1` where UUID is replaced with a + dynamically generated string that uniquely identifies the script. + + This setting allows users to override the location the environment variable + script is uploaded to. The value must be a writable location and any parent + directories must already exist. + +- `skip_clean` (bool) - Whether to clean scripts up after executing the provisioner. + Defaults to false. When true any script created by a non-elevated Powershell + provisioner will be removed from the remote machine. Elevated scripts, + along with the scheduled tasks, will always be removed regardless of the + value set for `skip_clean`. + +- `start_retry_timeout` (string) - The amount of time to attempt to _start_ + the remote process. By default this is `5m` or 5 minutes. This setting + exists in order to deal with times when SSH may restart, such as a system + reboot. Set this to a higher value if reboots take a longer amount of time. + +- `pause_after` (string) - Wait the amount of time after provisioning a PowerShell + script, this pause be taken if all previous steps were successful. + +@include 'provisioners/common-config.mdx' + +## Default Environmental Variables + +In addition to being able to specify custom environmental variables using the +`environment_vars` configuration, the provisioner automatically defines certain +commonly useful environmental variables: + +- `PACKER_BUILD_NAME` is set to the [name of the + build](/packer/docs/templates/legacy_json_templates/builders#named-builds) that Packer is running. + This is most useful when Packer is making multiple builds and you want to + distinguish them slightly from a common provisioning script. + +- `PACKER_BUILDER_TYPE` is the type of the builder that was used to create + the machine that the script is running on. This is useful if you want to + run only certain parts of the script on systems built with certain + builders. + +- `PACKER_HTTP_ADDR` If using a builder that provides an HTTP server for file + transfer (such as `hyperv`, `parallels`, `qemu`, `virtualbox`, and `vmware`), this + will be set to the address. You can use this address in your provisioner to + download large files over HTTP. This may be useful if you're experiencing + slower speeds using the default file provisioner. A file provisioner using + the `winrm` communicator may experience these types of difficulties. + +## Combining the PowerShell Provisioner with the SSH Communicator + +The good news first. If you are using the [Microsoft port of +OpenSSH](https://github.com/PowerShell/Win32-OpenSSH/wiki) then the provisioner +should just work as expected - no extra configuration effort is required. + +Now the caveats. If you are using an alternative configuration, and your SSH +connection lands you in a \*nix shell on the remote host, then you will most +likely need to manually set the `execute_command`; The default +`execute_command` used by Packer will not work for you. When configuring the +command you will need to ensure that any dollar signs or other characters that +may be incorrectly interpreted by the remote shell are escaped accordingly. + +The following example shows how the standard `execute_command` can be +reconfigured to work on a remote system with +[Cygwin/OpenSSH](https://cygwin.com/) installed. The `execute_command` has each +dollar sign backslash escaped so that it is not interpreted by the remote Bash +shell - Bash being the default shell for Cygwin environments. + + + + +```hcl +provisioner "powershell" { + execute_command = "powershell -executionpolicy bypass \"& { if (Test-Path variable:global:ProgressPreference){\\$ProgressPreference='SilentlyContinue'};. {{.Vars}}; &'{{.Path}}'; exit \\$LastExitCode }\"" + inline = [ "Write-Host \"Hello from PowerShell\""] +} +``` + + + + +```json +"provisioners": [ + { + "type": "powershell", + "execute_command": "powershell -executionpolicy bypass \"& { if (Test-Path variable:global:ProgressPreference){\\$ProgressPreference='SilentlyContinue'};. {{.Vars}}; &'{{.Path}}'; exit \\$LastExitCode }\"", + "inline": ["Write-Host \"Hello from PowerShell\""] + } +] +``` + + + + +## Packer's Handling of Characters Special to PowerShell + +The escape character in PowerShell is the `backtick`, also sometimes referred +to as the `grave accent`. When, and when not, to escape characters special to +PowerShell is probably best demonstrated with a series of examples. + +### When To Escape... + +Users need to deal with escaping characters special to PowerShell when they +appear _directly_ in commands used in the `inline` PowerShell provisioner and +when they appear _directly_ in the users own scripts. Note that where double +quotes appear within double quotes, the addition of a backslash escape is +required for the JSON template to be parsed correctly. + + + + +```hcl +provisioner "powershell" { + inline = [ + "Write-Host \"A literal dollar `$ must be escaped\"", + "Write-Host \"A literal backtick `` must be escaped\"", + "Write-Host \"Here `\"double quotes`\" must be escaped\"", + "Write-Host \"Here `'single quotes`' don`'t really need to be\"", + "Write-Host \"escaped... but it doesn`'t hurt to do so.\"", + ] +} +``` + + + + +```json + "provisioners": [ + { + "type": "powershell", + "inline": [ + "Write-Host \"A literal dollar `$ must be escaped\"", + "Write-Host \"A literal backtick `` must be escaped\"", + "Write-Host \"Here `\"double quotes`\" must be escaped\"", + "Write-Host \"Here `'single quotes`' don`'t really need to be\"", + "Write-Host \"escaped... but it doesn`'t hurt to do so.\"" + ] + } + ] +``` + + + + +The above snippet should result in the following output on the Packer console: + +```shell-session +==> amazon-ebs: Provisioning with Powershell... +==> amazon-ebs: Provisioning with PowerShell script: /var/folders/15/d0f7gdg13rnd1cxp7tgmr55c0000gn/T/packer-powershell-provisioner508190439 + amazon-ebs: A literal dollar $ must be escaped + amazon-ebs: A literal backtick ` must be escaped + amazon-ebs: Here "double quotes" must be escaped + amazon-ebs: Here 'single quotes' don't really need to be + amazon-ebs: escaped... but it doesn't hurt to do so. +``` + +### When Not To Escape... + +Special characters appearing in user environment variable values and in the +`elevated_user` and `elevated_password` fields will be automatically dealt with +for the user. There is no need to use escapes in these instances. + + + + +```hcl +variable "psvar" { + type = string + default = "My$tring" +} + +build { + sources = ["source.amazon-ebs.example"] + + provisioner "powershell" { + elevated_user = "Administrator" + elevated_password = "Super$3cr3t!" + inline = ["Write-Output \"The dollar in the elevated_password is interpreted correctly\""] + } + provisioner "powershell" { + environment_vars = [ + "VAR1=A$Dollar", + "VAR2=A`Backtick", + "VAR3=A'SingleQuote", + "VAR4=A\"DoubleQuote", + "VAR5=${var.psvar}", + ] + inline = [ + "Write-Output \"In the following examples the special character is interpreted correctly:\"", + "Write-Output \"The dollar in VAR1: $Env:VAR1\"", + "Write-Output \"The backtick in VAR2: $Env:VAR2\"", + "Write-Output \"The single quote in VAR3: $Env:VAR3\"", + "Write-Output \"The double quote in VAR4: $Env:VAR4\"", + "Write-Output \"The dollar in VAR5 (expanded from a user var): $Env:VAR5\"", + ] + } +} +``` + + + + +```json +{ + "variables": { + "psvar": "My$tring" + }, + ... + "provisioners": [ + { + "type": "powershell", + "elevated_user": "Administrator", + "elevated_password": "Super$3cr3t!", + "inline": "Write-Output \"The dollar in the elevated_password is interpreted correctly\"" + }, + { + "type": "powershell", + "environment_vars": [ + "VAR1=A$Dollar", + "VAR2=A`Backtick", + "VAR3=A'SingleQuote", + "VAR4=A\"DoubleQuote", + "VAR5={{user `psvar`}}" + ], + "inline": [ + "Write-Output \"In the following examples the special character is interpreted correctly:\"", + "Write-Output \"The dollar in VAR1: $Env:VAR1\"", + "Write-Output \"The backtick in VAR2: $Env:VAR2\"", + "Write-Output \"The single quote in VAR3: $Env:VAR3\"", + "Write-Output \"The double quote in VAR4: $Env:VAR4\"", + "Write-Output \"The dollar in VAR5 (expanded from a user var): $Env:VAR5\"" + ] + } + ] + ... +} +``` + + + + +The above snippet should result in the following output on the Packer console: + +```shell-session +==> amazon-ebs: Provisioning with Powershell... +==> amazon-ebs: Provisioning with PowerShell script: /var/folders/15/d0f7gdg13rnd1cxp7tgmr55c0000gn/T/packer-powershell-provisioner961728919 + amazon-ebs: The dollar in the elevated_password is interpreted correctly +==> amazon-ebs: Provisioning with Powershell... +==> amazon-ebs: Provisioning with PowerShell script: /var/folders/15/d0f7gdg13rnd1cxp7tgmr55c0000gn/T/packer-powershell-provisioner142826554 + amazon-ebs: In the following examples the special character is interpreted correctly: + amazon-ebs: The dollar in VAR1: A$Dollar + amazon-ebs: The backtick in VAR2: A`Backtick + amazon-ebs: The single quote in VAR3: A'SingleQuote + amazon-ebs: The double quote in VAR4: A"DoubleQuote + amazon-ebs: The dollar in VAR5 (expanded from a user var): My$tring +``` diff --git a/content/packer/content/docs/provisioners/shell-local.mdx b/content/packer/content/docs/provisioners/shell-local.mdx new file mode 100644 index 0000000000..578fc46e86 --- /dev/null +++ b/content/packer/content/docs/provisioners/shell-local.mdx @@ -0,0 +1,570 @@ +--- +description: | + The `shell-local` Packer provisioner runs shell scripts on the machine where Packer is running. Learn how to use the `shell-local` provisioner to run a shell script on your build server, desktop, or other local machine. +page_title: shell-local provisioner reference +--- + + + + + +# `shell-local` provisioner + +The `shell-local` provisioner runs shell scripts on the machine where +Packer is running. Use the provisioner to run a shell script on your build server, desktop, or other local machine instead of the remote or guest machine you are +using Packer to provision. + +The [remote shell](/packer/docs/provisioners/shell) provisioner executes shell +scripts on a remote machine. + +## Basic Example + +The example below is fully functional. + + + + +```hcl +source "file" "example" { + content = "example content" +} + +build { + source "source.file.example" { + target = "./test_artifact.txt" + } + + provisioner "shell-local" { + inline = ["echo foo"] + } +} +``` + + + + +```json +{ + "builders": [ + { + "type": "file", + "name": "example", + "target": "./test_artifact.txt", + "content": "example content" + } + ], + "provisioners": [ + { + "type": "shell-local", + "inline": ["echo foo"] + } + ] +} +``` + + + + +## Configuration Reference + +The reference of available configuration options is listed below. The only +required element is `command`. + +Exactly _one_ of the following is required: + +- `command` (string) - This is a single command to execute. It will be + written to a temporary file and run using the `execute_command` call below. + If you are building a Windows VM on AWS, Azure, Google Compute, or OpenStack + and would like to access the generated password that Packer uses to connect + to the instance via WinRM, you can use the template variable `{{.WinRMPassword}}` + to set this as an environment variable. + +- `inline` (array of strings) - This is an array of commands to execute. The + commands are concatenated by newlines and turned into a single file, so + they are all executed within the same context. This allows you to change + directories in one command and use something in the directory in the next + and so on. Inline scripts are the easiest way to pull off simple tasks + within the machine in which Packer is running. + +- `script` (string) - The path to a script to execute. This path can be + absolute or relative. If it is relative, it is relative to the working + directory when Packer is executed. + +- `scripts` (array of strings) - An array of scripts to execute. The scripts + will be executed in the order specified. Each script is executed in + isolation, so state such as variables from one script won't carry on to the + next. + +Optional parameters: + +- `env` (map of strings) - A map of key/value pairs to inject prior to the + execute_command. Packer injects some environmental variables by default into + the environment, as well, which are covered in the section below. Duplicate + `env` settings override `environment_vars` settings. + +- `environment_vars` (array of strings) - An array of key/value pairs to + inject prior to the `execute_command`. The format should be `key=value`. + Packer injects some environmental variables by default into the + environment, as well, which are covered in the section below. If you are + building a Windows VM on AWS, Azure, Google Compute, or OpenStack and would + like to access the generated password that Packer uses to connect to the + instance via WinRM, you can use the template variable `{{.WinRMPassword}}` + to set this as an environment variable. For example: + `"environment_vars": "WINRMPASS={{.WinRMPassword}}"` + +- `env_var_format` (string) - When we parse the environment_vars that you + provide, this gives us a string template to use in order to make sure that + we are setting the environment vars correctly. By default on Windows hosts + this format is `set %s=%s &&` and on Unix, it is `%s='%s'`. You probably + won't need to change this format, but you can see usage examples for where + it is necessary below. + +- `execute_command` (array of strings) - The command used to execute the + script. By default this is `["/bin/sh", "-c", "{{.Vars}}", "{{.Script}}"]` + on Unix and `["cmd", "/c", "{{.Vars}}", "{{.Script}}"]` on Windows. This is + treated as a [template engine](/packer/docs/templates/legacy_json_templates/engine). There are two + available variables: `Script`, which is the path to the script to run, and + `Vars`, which is the list of `environment_vars`, if configured. + + If you choose to set this option, make sure that the first element in the + array is the shell program you want to use (for example, "sh"), and a later + element in the array must be `{{.Script}}`. + + This option provides you a great deal of flexibility. You may choose to + provide your own shell program, for example "/usr/local/bin/zsh" or even + "powershell.exe". However, with great power comes great responsibility - + these commands are not officially supported and things like environment + variables may not work if you use a different shell than the default. + + For backwards compatibility, you may also use `{{.Command}}`, but it is + decoded the same way as `{{.Script}}`. We recommend using `{{.Script}}` for the + sake of clarity, as even when you set only a single `command` to run, + Packer writes it to a temporary file and then runs it as a script. + + If you are building a Windows VM on AWS, Azure, Google Compute, or OpenStack + and would like to access the generated password that Packer uses to connect + to the instance via WinRM, you can use the template variable `{{.WinRMPassword}}` + to set this as an environment variable. + +- `inline_shebang` (string) - The + [shebang](http://en.wikipedia.org/wiki/Shebang_%28Unix%29) value to use + when running commands specified by `inline`. By default, this is + `/bin/sh -e`. If you're not using `inline`, then this configuration has no + effect. **Important:** If you customize this, be sure to include something + like the `-e` flag, otherwise individual steps failing won't fail the + provisioner. + +- `only_on` (array of strings) - This is an array of [runtime operating + systems](https://go.dev/doc/install/source#environment) where + `shell-local` will execute. This allows you to execute `shell-local` _only_ + on specific operating systems. By default, shell-local will always run if + `only_on` is not set." + +- `use_linux_pathing` (bool) - This is only relevant to Windows hosts. If you + are running Packer in a Windows environment with the Windows Subsystem for + Linux feature enabled, and would like to invoke a bash script rather than + invoking a Cmd script, you'll need to set this flag to true; it tells + Packer to use the Linux subsystem path for your script rather than the + Windows path. (e.g. /mnt/c/path/to/your/file instead of + C:/path/to/your/file). Please see the example below for more guidance on + how to use this feature. If you are not on a Windows host, or you do not + intend to use the shell-local provisioner to run a bash script, please + ignore this option. + +- `valid_exit_codes` (list of ints) - Valid exit codes for the script. By + default this is `0`. + +@include 'provisioners/common-config.mdx' + +## Execute Command + +To many new users, the `execute_command` is puzzling. However, it provides an +important function: customization of how the command is executed. The most +common use case for this is dealing with **sudo password prompts**. You may +also need to customize this if you use a non-POSIX shell, such as `tcsh` on +FreeBSD. + +### The Windows Linux Subsystem + +The shell-local provisioner was designed with the idea of allowing you to run +commands in your local operating system's native shell. For Windows, we've +assumed in our defaults that this is Cmd. However, it is possible to run a bash +script as part of the Windows Linux Subsystem from the shell-local provisioner, +by modifying the `execute_command` and the `use_linux_pathing` options in the +provisioner config. + +The example below is a fully functional test config. + +One limitation of this offering is that "inline" and "command" options are not +available to you; please limit yourself to using the "script" or "scripts" +options instead. + +Please note that the WSL is a beta feature, and this tool is not guaranteed to +work as you expect it to. + + + + +```hcl +source "null" "example" { + communicator = "none" +} + +build { + sources = [ + "source.null.example" + ] + + provisioner "shell-local"{ + environment_vars = ["PROVISIONERTEST=ProvisionerTest1"] + execute_command = ["bash", "-c", "{{.Vars}} {{.Script}}"] + use_linux_pathing = true + scripts = ["C:/Users/me/scripts/example_bash.sh"] + } + provisioner "shell-local"{ + environment_vars = ["PROVISIONERTEST=ProvisionerTest2"] + execute_command = ["bash", "-c", "{{.Vars}} {{.Script}}"] + use_linux_pathing = true + script = "C:/Users/me/scripts/example_bash.sh" + } +} +``` + + + + +```json +{ + "builders": [ + { + "type": "null", + "communicator": "none" + } + ], + "provisioners": [ + { + "type": "shell-local", + "environment_vars": ["PROVISIONERTEST=ProvisionerTest1"], + "execute_command": ["bash", "-c", "{{.Vars}} {{.Script}}"], + "use_linux_pathing": true, + "scripts": ["C:/Users/me/scripts/example_bash.sh"] + }, + { + "type": "shell-local", + "environment_vars": ["PROVISIONERTEST=ProvisionerTest2"], + "execute_command": ["bash", "-c", "{{.Vars}} {{.Script}}"], + "use_linux_pathing": true, + "script": "C:/Users/me/scripts/example_bash.sh" + } + ] +} +``` + + + + +## Default Environmental Variables + +In addition to being able to specify custom environmental variables using the +`environment_vars` configuration, the provisioner automatically defines certain +commonly useful environmental variables: + +- `PACKER_BUILD_NAME` is set to the name of the build that Packer is running. + This is most useful when Packer is making multiple builds and you want to + distinguish them slightly from a common provisioning script. + +- `PACKER_BUILDER_TYPE` is the type of the builder that was used to create + the machine that the script is running on. This is useful if you want to + run only certain parts of the script on systems built with certain + builders. + +- `PACKER_HTTP_ADDR` If using a builder that provides an HTTP server for file + transfer (such as `hyperv`, `parallels`, `qemu`, `virtualbox`, and `vmware`), this + will be set to the address. You can use this address in your provisioner to + download large files over HTTP. This may be useful if you're experiencing + slower speeds using the default file provisioner. A file provisioner using + the `winrm` communicator may experience these types of difficulties. + +## Safely Writing A Script + +Whether you use the `inline` option, or pass it a direct `script` or `scripts`, +it is important to understand a few things about how the `shell-local` +provisioner works to run it safely and easily. This understanding will save you +much time in the process. + +### Once Per Builder + +The `shell-local` script(s) you pass are run once per builder. That means that +if you have an `amazon-ebs` builder and a `docker` builder, your script will be +run twice. If you have 3 builders, it will run 3 times, once for each builder. + +### Always Exit Intentionally + +If any provisioner fails, the `packer build` stops and all interim artifacts +are cleaned up. + +For a shell script, that means the script **must** exit with a zero code. You +_must_ be extra careful to `exit 0` when necessary. + +## Usage Examples: + +### Windows Host + +Example of running a .cmd file on Windows: + + + + +```hcl +provisioner "shell-local" { + environment_vars = ["SHELLLOCALTEST=ShellTest1"] + scripts = ["./scripts/test_cmd.cmd"] +} +``` + + + + +```json +{ + "type": "shell-local", + "environment_vars": ["SHELLLOCALTEST=ShellTest1"], + "scripts": ["./scripts/test_cmd.cmd"] +} +``` + + + + +Contents of "test_cmd.cmd": + + echo %SHELLLOCALTEST% + +Example of running an inline command on Windows: Required customization: +tempfile_extension + + + + +```hcl +provisioner "shell-local" { + environment_vars = ["SHELLLOCALTEST=ShellTest2"], + tempfile_extension = ".cmd", + inline = [echo "%SHELLLOCALTEST%"] +} +``` + + + + +```json +{ + "type": "shell-local", + "environment_vars": ["SHELLLOCALTEST=ShellTest2"], + "tempfile_extension": ".cmd", + "inline": ["echo %SHELLLOCALTEST%"] +} +``` + + + + +Example of running a bash command on Windows using WSL: Required +customizations: use_linux_pathing and execute_command + + + + +```hcl +provisioner "shell-local" { + environment_vars = ["SHELLLOCALTEST=ShellTest3"], + execute_command = ["bash", "-c", "{{.Vars}} {{.Script}}"] + use_linux_pathing = true + script = "./scripts/example_bash.sh" +} +``` + + + + +```json +{ + "type": "shell-local", + "environment_vars": ["SHELLLOCALTEST=ShellTest3"], + "execute_command": ["bash", "-c", "{{.Vars}} {{.Script}}"], + "use_linux_pathing": true, + "script": "./scripts/example_bash.sh" +} +``` + + + + +Contents of `example_bash.sh`: + + #!/bin/bash + echo $SHELLLOCALTEST + +Example of running a PowerShell script on Windows: Required customizations: +`env_var_format` and `execute_command` + + + + +```hcl +provisioner "shell-local" { + environment_vars = ["SHELLLOCALTEST=ShellTest4"] + execute_command = ["powershell.exe", "{{.Vars}} {{.Script}}"] + env_var_format = "$env:%s=\"%s\"; " + script = "./scripts/example_ps.ps1" +} +``` + + + + +```json +{ + "type": "shell-local", + "environment_vars": ["SHELLLOCALTEST=ShellTest4"], + "execute_command": ["powershell.exe", "{{.Vars}} {{.Script}}"], + "env_var_format": "$env:%s=\"%s\"; ", + "script": "./scripts/example_ps.ps1" +} +``` + + + + +Example of running a PowerShell script on Windows as "inline": Required +customizations: `env_var_format`, `tempfile_extension`, and `execute_command` + + + + +```hcl +provisioner "shell-local" { + tempfile_extension = ".ps1" + environment_vars = ["SHELLLOCALTEST=ShellTest5"] + execute_command = ["powershell.exe", "{{.Vars}} {{.Script}}"] + env_var_format = "$env:%s=\"%s\"; " + inline = ["write-output $env:SHELLLOCALTEST"] +} +``` + + + + +```json +{ + "type": "shell-local", + "tempfile_extension": ".ps1", + "environment_vars": ["SHELLLOCALTEST=ShellTest5"], + "execute_command": ["powershell.exe", "{{.Vars}} {{.Script}}"], + "env_var_format": "$env:%s=\"%s\"; ", + "inline": ["write-output $env:SHELLLOCALTEST"] +} +``` + + + + +### Unix Host + +Example of running a Shell script on Unix: + + + + +```hcl +provisioner "shell-local" { + environment_vars = ["PROVISIONERTEST=ProvisionerTest1"] + scripts = ["./scripts/example_bash.sh"] +} +``` + + + + +```json +{ + "type": "shell-local", + "environment_vars": ["PROVISIONERTEST=ProvisionerTest1"], + "scripts": ["./scripts/example_bash.sh"] +} +``` + + + + +Example of running a Shell script "inline" on Unix: + + + + +```hcl +provisioner "shell-local" { + environment_vars = ["PROVISIONERTEST=ProvisionerTest2"] + inline = ["echo hello", "echo $PROVISIONERTEST"] +} + +``` + + + + +```json +{ + "type": "shell-local", + "environment_vars": ["PROVISIONERTEST=ProvisionerTest2"], + "inline": ["echo hello", "echo $PROVISIONERTEST"] +} +``` + + + + +Example of running a Python script on Unix: + + + + +```hcl +provisioner "shell-local" { + script = "hello.py" + environment_vars = ["HELLO_USER=packeruser"] + execute_command = [ + "/bin/sh", + "-c", + "{{.Vars}} /usr/local/bin/python {{.Script}}" + ] +} + +``` + + + + +```json +{ + "type": "shell-local", + "script": "hello.py", + "environment_vars": ["HELLO_USER=packeruser"], + "execute_command": [ + "/bin/sh", + "-c", + "{{.Vars}} /usr/local/bin/python {{.Script}}" + ] +} +``` + + + + +```text +Where "hello.py" contains: + + import os + + print('Hello, %s!' % os.getenv("HELLO_USER")) +``` diff --git a/content/packer/content/docs/provisioners/shell.mdx b/content/packer/content/docs/provisioners/shell.mdx new file mode 100644 index 0000000000..97ad0f3792 --- /dev/null +++ b/content/packer/content/docs/provisioners/shell.mdx @@ -0,0 +1,407 @@ +--- +description: | + The `shell` Packer provisioner runs shell scripts on the machine Packer builds. Use the `shell` provisioner to install and configure software on a machine. +page_title: shell provisioner reference +--- + + + + + +# `shell` provisioner + +The `shell` Packer provisioner runs shell scripts on the machine Packer builds. Use the `shell` provisioner to install and configure software on a machine. + +-> **Building Windows images?** You probably want to use the +[PowerShell](/packer/docs/provisioners/powershell) or [Windows +Shell](/packer/docs/provisioners/windows-shell) provisioners. + +## Basic Example + +The example below is fully functional. + + + + +```hcl +provisioner "shell" { + inline = ["echo foo"] +} +``` + + + + +```json +{ + "type": "shell", + "inline": ["echo foo"] +} +``` + + + + +## Configuration Reference + +@include 'provisioners/shell-config.mdx' + +- `env` (map of strings) - A map of key/value pairs to inject prior to the + execute_command. Packer injects some environmental variables by default into + the environment, as well, which are covered in the section below. Duplicate + `env` settings override `environment_vars` settings. + +- `environment_vars` (array of strings) - An array of key/value pairs to + inject prior to the execute_command. The format should be `key=value`. + Packer injects some environmental variables by default into the + environment, as well, which are covered in the section below. + +- `env_var_format` (string) - When we parse the environment_vars that you + provide, this gives us a string template to use in order to make sure that + we are setting the environment vars correctly. By default it is `"%s='%s' "`. + When used in conjunction with `use_env_var_file` the default is `"export %s='%s'\n"` + +- `use_env_var_file` (boolean) - If true, Packer will write your environment + variables to a tempfile and source them from that file, rather than + declaring them inline in our execute_command. The default + `execute_command` will be + `chmod +x {{.Path}}; . {{.EnvVarFile}} && {{.Path}}`. This option is + unnecessary for most cases, but if you have extra quoting in your custom + `execute_command`, then this may be required for proper script + execution. Default: `false`. + +- `execute_command` (string) - The command to use to execute the script. By + default this is `chmod +x {{ .Path }}; {{ .Vars }} {{ .Path }}`, unless the + user has set `"use_env_var_file": true` -- in that case, the default + `execute_command` is `chmod +x {{.Path}}; . {{.EnvVarFile}} && {{.Path}}`. + This is a [template engine](/packer/docs/templates/legacy_json_templates/engine). Therefore, you may + use user variables and template functions in this field. In addition, there + are three available extra variables: + + - `Path` is the path to the script to run + - `Vars` is the list of `environment_vars`, if configured. + - `EnvVarFile` is the path to the file containing env vars, if + `use_env_var_file` is true. + +- `expect_disconnect` (boolean) - Defaults to `false`. When `true`, allow the + server to disconnect from Packer without throwing an error. A disconnect + might happen if you restart the SSH server or reboot the host. + +- `inline_shebang` (string) - The + [shebang](https://en.wikipedia.org/wiki/Shebang_%28Unix%29) value to use + when running commands specified by `inline`. By default, this is + `/bin/sh -e`. If you're not using `inline`, then this configuration has no + effect. **Important:** If you customize this, be sure to include something + like the `-e` flag, otherwise individual steps failing won't fail the + provisioner. + +- `remote_folder` (string) - The folder where the uploaded script will reside + on the machine. This defaults to `/tmp`. + +- `remote_file` (string) - The filename the uploaded script will have on the + machine. This defaults to `script_nnn.sh`. + +- `remote_path` (string) - The full path to the uploaded script will have on + the machine. By default this is `remote_folder/remote_file`, if set this + option will override both `remote_folder` and `remote_file`. + +- `skip_clean` (boolean) - If true, specifies that the helper scripts + uploaded to the system will not be removed by Packer. This defaults to + `false` (clean scripts from the system). + +- `start_retry_timeout` (string) - The amount of time to attempt to _start_ + the remote process. By default this is `5m` or 5 minutes. This setting + exists in order to deal with times when SSH may restart, such as a system + reboot. Set this to a higher value if reboots take a longer amount of time. + +- `pause_after` (string) - Wait the amount of time after provisioning a shell + script, this pause be taken if all previous steps were successful. + +@include 'provisioners/common-config.mdx' + +## Execute Command Example + +To many new users, the `execute_command` is puzzling. However, it provides an +important function: customization of how the command is executed. The most +common use case for this is dealing with **sudo password prompts**. You may +also need to customize this if you use a non-POSIX shell, such as `tcsh` on +FreeBSD. + +### Sudo Example + +Some operating systems default to a non-root user. For example if you login as +`ubuntu` and can sudo using the password `packer`, then you'll want to change +`execute_command` to be: + +```text +"echo 'packer' | sudo -S sh -c '{{ .Vars }} {{ .Path }}'" +``` + +The `-S` flag tells `sudo` to read the password from stdin, which in this case +is being piped in with the value of `packer`. + +The above example won't work if your environment vars contain spaces or single +quotes; in these cases try removing the single quotes: + +```text +"echo 'packer' | sudo -S env {{ .Vars }} {{ .Path }}" +``` + +By setting the `execute_command` to this, your script(s) can run with root +privileges without worrying about password prompts. + +### FreeBSD Example + +FreeBSD's default shell is `tcsh`, which deviates from POSIX semantics. In +order for packer to pass environment variables you will need to change the +`execute_command` to: + +```text +chmod +x {{ .Path }}; env {{ .Vars }} {{ .Path }} +``` + +Note the addition of `env` before `{{ .Vars }}`. + +## Default Environmental Variables + +In addition to being able to specify custom environmental variables using the +`environment_vars` configuration, the provisioner automatically defines certain +commonly useful environmental variables: + +- `PACKER_BUILD_NAME` is set to the [name of the + build](/packer/docs/templates/legacy_json_templates/builders#named-builds) that Packer is running. + This is most useful when Packer is making multiple builds and you want to + distinguish them slightly from a common provisioning script. + +- `PACKER_BUILDER_TYPE` is the type of the builder that was used to create + the machine that the script is running on. This is useful if you want to + run only certain parts of the script on systems built with certain + builders. + +- `PACKER_HTTP_ADDR` If using a builder that provides an HTTP server for file + transfer (such as `hyperv`, `parallels`, `qemu`, `virtualbox`, and `vmware`), this + will be set to the address. You can use this address in your provisioner to + download large files over HTTP. This may be useful if you're experiencing + slower speeds using the default file provisioner. A file provisioner using + the `winrm` communicator may experience these types of difficulties. + +## Handling Reboots + +Provisioning sometimes involves restarts, usually when updating the operating +system. Packer is able to tolerate restarts via the shell provisioner. + +Packer handles this by retrying to start scripts for a period of time before +failing. This allows time for the machine to start up and be ready to run +scripts. The amount of time the provisioner will wait is configured using +`start_retry_timeout`, which defaults to a few minutes. + +Sometimes, when executing a command like `reboot`, the shell script will return +and Packer will start executing the next one before SSH actually quits and the +machine restarts. For this, use `pause_before` to make Packer wait before +executing the next script: + + + + +```hcl +provisioner "shell" { + script = "script.sh" + pause_before = "10s" + timeout = "10s" +} +``` + + + + +```json +{ + "type": "shell", + "script": "script.sh", + "pause_before": "10s", + "timeout": "10s" +} +``` + + + + +Some OS configurations don't properly kill all network connections on reboot, +causing the provisioner to hang despite a reboot occurring. In this case, make +sure you shut down the network interfaces on reboot or in your shell script. +For example, on Gentoo: + +```text +/etc/init.d/net.eth0 stop +``` + +## SSH Agent Forwarding + +Some provisioning requires connecting to remote SSH servers from within the +packer instance. The below example is for pulling code from a private git +repository utilizing openssh on the client. Make sure you are running +`ssh-agent` and add your git repo SSH keys into it using +`ssh-add /path/to/key`. When the Packer instance needs access to the SSH keys +the agent will forward the request back to your `ssh-agent`. + +Note: when provisioning via git you should add the git server keys into the +`~/.ssh/known_hosts` file otherwise the git command could hang awaiting input. +This can be done by copying the file in via the [file +provisioner](/packer/docs/provisioners/file) (more secure) or using `ssh-keyscan` +to populate the file (less secure). An example of the latter accessing github +would be: + + + + +```hcl +provisioner "shell" { + inline = [ + "sudo apt-get install -y git", + "ssh-keyscan github.com >> ~/.ssh/known_hosts", + "git clone git@github.com:exampleorg/myprivaterepo.git" + ] +} +``` + + + + +```json +{ + "type": "shell", + "inline": [ + "sudo apt-get install -y git", + "ssh-keyscan github.com >> ~/.ssh/known_hosts", + "git clone git@github.com:exampleorg/myprivaterepo.git" + ] +} +``` + + + + +## Troubleshooting + +_My shell script doesn't work correctly on Ubuntu_ + +- On Ubuntu, the `/bin/sh` shell is + [dash](https://en.wikipedia.org/wiki/Debian_Almquist_shell). If your script + has [bash]()-specific + commands in it, then put `#!/bin/bash -e` at the top of your script. + Differences between dash and bash can be found on the + [DashAsBinSh](https://wiki.ubuntu.com/DashAsBinSh) Ubuntu wiki page. + +_My shell works when I login but fails with the shell provisioner_ + +- See the above tip. More than likely, your login shell is using `/bin/bash` + while the provisioner is using `/bin/sh`. + +_My installs hang when using `apt-get` or `yum`_ + +- Make sure you add a `-y` to the command to prevent it from requiring user + input before proceeding. + +_How do I tell what my shell script is doing?_ + +- Adding a `-x` flag to the shebang at the top of the script (`#!/bin/sh -x`) + will echo the script statements as it is executing. + +_My builds don't always work the same_ + +- Some distributions start the SSH daemon before other core services which + can create race conditions. Your first provisioner can tell the machine to + wait until it completely boots. + + + + +```hcl +provisioner "shell" { + inline = ["sleep 10"] +} +``` + + + + +```json +{ + "type": "shell", + "inline": ["sleep 10"] +} +``` + + + + +## Quoting Environment Variables + +Packer manages quoting for you, so you shouldn't have to worry about it. Below +is an example of Packer template inputs and what you should expect to get out: + + + + +```hcl +provisioner "shell" { + environment_vars = [ + "FOO=foo", + "BAR=bar's", + "BAZ=baz=baz", + "QUX==qux", + "FOOBAR=foo bar", + "FOOBARBAZ='foo bar baz'", + "QUX2=\"qux\"" + ] + inline = [ + "echo \"FOO is $FOO\"", + "echo \"BAR is $BAR\"", + "echo \"BAZ is $BAZ\"", + "echo \"QUX is $QUX\"", + "echo \"FOOBAR is $FOOBAR\"", + "echo \"FOOBARBAZ is $FOOBARBAZ\"", + "echo \"QUX2 is $QUX2\"" + ] +} +``` + + + + +```json +"provisioners": [ + { + "type": "shell", + "environment_vars": ["FOO=foo", + "BAR=bar's", + "BAZ=baz=baz", + "QUX==qux", + "FOOBAR=foo bar", + "FOOBARBAZ='foo bar baz'", + "QUX2=\"qux\""], + "inline": ["echo \"FOO is $FOO\"", + "echo \"BAR is $BAR\"", + "echo \"BAZ is $BAZ\"", + "echo \"QUX is $QUX\"", + "echo \"FOOBAR is $FOOBAR\"", + "echo \"FOOBARBAZ is $FOOBARBAZ\"", + "echo \"QUX2 is $QUX2\""] + } +] +``` + + + + +Output: + +```text +docker: FOO is foo +docker: BAR is bar's +docker: BAZ is baz=baz +docker: QUX is =qux +docker: FOOBAR is foo bar +docker: FOOBARBAZ is 'foo bar baz' +docker: QUX2 is "qux" +``` diff --git a/content/packer/content/docs/provisioners/windows-restart.mdx b/content/packer/content/docs/provisioners/windows-restart.mdx new file mode 100644 index 0000000000..8a45636fda --- /dev/null +++ b/content/packer/content/docs/provisioners/windows-restart.mdx @@ -0,0 +1,110 @@ +--- +description: | + The `windows-restart` provisioner restarts a Windows machine and waits for it to + come back up. Learn how to use the `windows-restart` provisioner. +page_title: windows-restart provisioner reference +--- + + + + + +# `windows-restart` provisioner + +The `windows-restart` provisioner initiates a reboot on a Windows machine and +waits for the machine to come back online. + +The Windows provisioning process often requires multiple reboots, and this +provisioner helps to ease that process. + +Packer expects the machine to be ready to continue provisioning after it +reboots. Packer detects that the reboot has completed by making an RPC call +through the Windows Remote Management (WinRM) service, not by ACPI functions, +so Windows must be completely booted in order to continue. + +## Basic Example + +The example below is fully functional. + + + + +```hcl +provisioner "windows-restart" {} +``` + + + + +```json +{ + "type": "windows-restart" +} +``` + + + + +## Configuration Reference + +The reference of available configuration options is listed below. + +Optional parameters: + +- `check_registry` (bool) - if `true`, checks for several registry keys that + indicate that the system is going to reboot. This is useful if an + installation kicks off a reboot and you want the provisioner to wait for + that reboot to complete before reconnecting. Please note that this option + is a beta feature, and we generally recommend that you finish installs that + auto-reboot (like Windows Updates) during your _autounattend_ phase before + the `winrm` provisioner connects. + +- `registry_keys` (array of strings) - if `check-registry` is `true`, + `windows-restart` will not reconnect until after all of the listed keys are + no longer present in the registry. + +``` + default: + + var DefaultRegistryKeys = []string{ + "HKLM:SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Component Based Servicing\\RebootPending", + "HKLM:SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Component Based Servicing\\PackagesPending", + "HKLM:Software\\Microsoft\\Windows\\CurrentVersion\\Component Based Servicing\\RebootInProgress", + } +``` + +- `restart_command` (string) - The command to execute to initiate the + restart. By default this is `shutdown /r /f /t 0 /c "packer restart"`. + +- `restart_check_command` (string) - The command to run after executing `restart_command` + to check if the guest machine has restarted. This command will retry until the connection + to the guest machine has been restored or `restart_timeout` has exceeded. + + + + +```hcl +provisioner "windows-restart" { + restart_check_command = "powershell -command \"& {Write-Output 'restarted.'}\"" +} +``` + + + + +```json +{ + "type": "windows-restart", + "restart_check_command": "powershell -command \"& {Write-Output 'restarted.'}\"" +} +``` + + + + +- `restart_timeout` (string) - The timeout to wait for the restart. By + default this is 5 minutes. Example value: `5m`. If you are installing + updates or have a lot of startup services, you will probably need to + increase this duration. + +@include 'provisioners/common-config.mdx' diff --git a/content/packer/content/docs/provisioners/windows-shell.mdx b/content/packer/content/docs/provisioners/windows-shell.mdx new file mode 100644 index 0000000000..60ada72443 --- /dev/null +++ b/content/packer/content/docs/provisioners/windows-shell.mdx @@ -0,0 +1,99 @@ +--- +description: | + The `windows-shell` provisioner runs commands on Windows using the `cmd` + shell. Learn how to use the `windows-shell` provisioner. +page_title: windows-shell provisioner reference +--- + + + + + +# `windows-shell` provisioner + +The `windows-shell` Packer provisioner runs commands on a Windows machine using +`cmd`. The provisioner is designed to communicate with machines running WinRM. + +## Basic Example + +The example below is fully functional. + + + + +```hcl +provisioner "windows-shell" { + inline = ["dir c:\\"] +} +``` + + + + +```json +{ + "type": "windows-shell", + "inline": ["dir c:\\"] +} +``` + + + + +## Configuration Reference + +@include 'provisioners/shell-config.mdx' + +- `env` (map of strings) - A map of key/value pairs to inject prior to the + execute_command. Packer injects some environmental variables by default into + the environment, as well, which are covered in the section below. Duplicate + `env` settings override `environment_vars` settings. + +- `environment_vars` (array of strings) - An array of key/value pairs to + inject prior to the execute_command. The format should be `key=value`. + Packer injects some environmental variables by default into the + environment, as well, which are covered in the section below. + +- `execute_command` (string) - The command to use to execute the script. By + default this is `{{ .Vars }}"{{ .Path }}"`. The value of this is treated as + [template engine](/packer/docs/templates/legacy_json_templates/engine). This is a + [template engine](/packer/docs/templates/legacy_json_templates/engine). Therefore, you may + use user variables and template functions in this field. In addition, there + are two available extra variables: + + - `Path` is the path to the script to run + - `Vars` is the list of `environment_vars`, if configured. + +- `remote_path` (string) - The path where the script will be uploaded to in + the machine. This defaults to "c:/Windows/Temp/script.bat". This value must + be a writable location and any parent directories must already exist. + +- `start_retry_timeout` (string) - The amount of time to attempt to _start_ + the remote process. By default this is "5m" or 5 minutes. This setting + exists in order to deal with times when SSH may restart, such as a system + reboot. Set this to a higher value if reboots take a longer amount of time. + +@include 'provisioners/common-config.mdx' + +## Default Environmental Variables + +In addition to being able to specify custom environmental variables using the +`environment_vars` configuration, the provisioner automatically defines certain +commonly useful environmental variables: + +- `PACKER_BUILD_NAME` is set to the [name of the + build](/packer/docs/templates/legacy_json_templates/builders#named-builds) that Packer is running. + This is most useful when Packer is making multiple builds and you want to + distinguish them slightly from a common provisioning script. + +- `PACKER_BUILDER_TYPE` is the type of the builder that was used to create + the machine that the script is running on. This is useful if you want to + run only certain parts of the script on systems built with certain + builders. + +- `PACKER_HTTP_ADDR` If using a builder that provides an HTTP server for file + transfer (such as `hyperv`, `parallels`, `qemu`, `virtualbox`, and `vmware`), this + will be set to the address. You can use this address in your provisioner to + download large files over HTTP. This may be useful if you're experiencing + slower speeds using the default file provisioner. A file provisioner using + the `winrm` communicator may experience these types of difficulties. diff --git a/content/packer/content/docs/release-notes/v1_11.mdx b/content/packer/content/docs/release-notes/v1_11.mdx new file mode 100644 index 0000000000..fe4c3c401a --- /dev/null +++ b/content/packer/content/docs/release-notes/v1_11.mdx @@ -0,0 +1,39 @@ +--- +page_title: Packer v1.11 release notes +description: Learn about the changes in Packer v 1.11 +--- + +# Packer v1.11 release notes + +This page describes changes to Packer in v1.11. Refer to the [Packer repository](https://github.com/hashicorp/packer/releases) for information about all releases. + +## New features + +This release includes the following updates. + +### Predictable plugin loading + +We have simplified the way Packer discovers and loads plugins. Packer no longer loads plugin binaries installed outside its plugin directory. Additionally, Packer now only loads binaries that have an accompanying SHA256SUM file. We have also updated the commands for installing plugins so that most installation workflows can proceed without manually modifying the filesystem. We also focused on providing clear instructions so that you can understand the processes and how to address potential errors. + +Previous versions of Packer allowed you to install and load plugins in several locations, resulting in various inconsistencies. For example, configuring the `required_plugins` block in HCL2 templates resulted in limited plugin discoverability. This was not only confusing, the number of alternative paths to support also increased the maintenance burden. Because Packer checked several locations before starting to execute, many commands suffered unnecessary performance issues. + +Refer to the following topics for additional information: + +- [Installing Plugins](/packer/docs/plugins/install) for details about how to load plugins. +- [Upgrade to v1.11](/packer/docs/upgrade/v1_11) for information about how these changes may affect your existing workflows. + +## Deprecations + +The following features, workflows, and components are deprecated and are no longer supported. + +### Legacy single-component plugins + +Packer no longer supports single-component plugins, such as legacy plugins that follow the deprecated naming convention of `packer-builder-` or `packer-provisioner-`. Packer only supports multi-component plugins, such as the Docker plugin for Packer. + +### Loading plugin binaries named `packer-plugin-` + +Packer no longer loads plugin binaries following the naming convention of `packer-plugin-`. Packer loads only plugins stored under `PACKER_PLUGIN_PATH` using the expected namespaced directory and CHECKSUM files. + +## Bug fixes + +Fixed an issue with how Packer discovers plugins to ensure proper version ordering. Packer no longer orders plugin versions lexicographically. Instead, Packer now loads versions that have a greater semantic number. For example, version `1.0.9` is lexicographically greater than `1.0.10`, but `1.0.10` is semantically greater than `1.0.9`. This fix results in Packer loading `1.0.10` instead of `1.0.9`. \ No newline at end of file diff --git a/content/packer/content/docs/release-notes/v1_12.mdx b/content/packer/content/docs/release-notes/v1_12.mdx new file mode 100644 index 0000000000..475b75a1d7 --- /dev/null +++ b/content/packer/content/docs/release-notes/v1_12.mdx @@ -0,0 +1,71 @@ +--- +page_title: Packer v1.12 release notes +description: Learn about the changes in Packer v 1.12 +--- + +# Packer v1.12 release notes + +This page describes changes to Packer in v1.12. Refer to the [Packer repository](https://github.com/hashicorp/packer/releases) for information about all releases. + +## New features + +This release includes the following updates. + +### HCP Packer SBOM Support + +In Packer 1.12, you can upload software bill of materials (SBOMs) to HCP Packer and associate it with an artifact version. +SBOMs are a standardized way to export information about a system. +In Packer's case, the generally useful information that you may find in a SBOM for an artifact is the list of installed packages, along with extra information on the system built: OS, version, kernel, architecture, etc. + +While we support uploading SBOMs to HCP Packer as part of a build, we do not offer a special-purpose provisioner or tooling to produce them. +Instead we encourage you to use a third-party scanner to produce the SBOM on the VM you are provisioning, and then you can use the [`hcp-sbom` provisioner](https://developer.hashicorp.com/packer/docs/provisioners/hcp-sbom) to upload it when your Packer build completes. + +### Data source and Locals evaluation order changes + +Older versions of Packer used a phase-based approach, where it evaluated data sources first, then local variables. +This made it impossible for a data source to reference a local variable. + +Packer 1.12 introduces a Directed Acyclic Graph (DAG) approach to evaluating data sources and locals. +This loosens the dependency order between those components, and now you can reference them from both contexts. + +This change is a step in the direction of a complete pivot to using a DAG for evaluating everything in a Packer build, along with the other improvements this can yield in future releases. + +### Serialisation format upgrades for Packer-Plugin communication + +More than one year ago, a dependency of ours (`go-cty`) dropped support for `gob` encoding. +This made it impossible for plugin developers to upgrade to more recent versions of the HCL2 libraries, because otherwise their plugin became incapable to commuinicate with Packer. +We temporarily addressed this issue by forking the `go-cty` repository, and introduced replacement directives to every Packer plugin. +While this fix was functional, it was not desirable as a long-term solution, and instead we were looking for a more permanent fix. + +Now, when Packer communicates with plugins, it swaps to using a protobuf/msgpack hybrid approach instead of relying on `gob`. +We are introducing this change now in a non-breaking way: all the currently supported plugins are expected to continue working with Packer for the time being, and changing to using this new serialization approach will be transparent to you. + +### New HCL2 functions + +As part of Packer 1.12, we have introduced more functions that can be used in HCL2 templates, and one (`aws_secretsmanager_raw`) that can be used both in legacy JSON and HCL2 templates. + +* `anytrue`: check that a collection contains at least one `true` value. +* `alltrue`: check that a collection contains only `true` values. +* `aws_secretsmanager_raw`: get a raw secret from AWS Secrets Manager. Unlike `aws_secretsmanager`, this works with all types. +* `base64gzip`: gzip compress a binary blob and expose it as a base64-encoded `string` to be used elsewhere in a template. +* `strcontains`: checks that a string contains another. + +### `HTTP` data source support methods other than `GET` + +The HTTP data source, embedded with Packer, lets you retrieve data over HTTP from a remote server. +Previous versions of Packer only supported `GET` to do so. Packer 1.12 loosens this by allowing for: `HEAD`, `GET`, `POST`, `PUT`, `DELETE`, `OPTIONS` and `PATCH`. + +## Bug fixes + +### Include LC_UUID in macOS binaries + +Users of macOS started having permission-related problems when using Packer, after upgrading their OS versions. +This problem was caused by an update to macOS's network-usage policies, where binaries that want to use the local loop interface to communicate over the network must now include a valid UUID. +Starting with Packer 1.12, all macOS binaries include a valid LC_UUID, fixing this. + +### Less duplication of error messages in HCL2 templates + +If a template has an error in its top-level HCL2 template, Packer produces a parsing error. +This is expected behavior when you write a Packer template: the tool helps you by pointing out grammar violations so you can remediate them. +However, for a subset of HCL-related errors, older versions of Packer displayed the same message up to five times. +Thanks to a community contribution, starting with Packer 1.12 we now no longer experience this. diff --git a/content/packer/content/docs/release-notes/v1_13.mdx b/content/packer/content/docs/release-notes/v1_13.mdx new file mode 100644 index 0000000000..83a4ba2ce4 --- /dev/null +++ b/content/packer/content/docs/release-notes/v1_13.mdx @@ -0,0 +1,57 @@ +--- +page_title: Packer v1.13 release notes +description: Learn about the changes in Packer v 1.13 +--- + +# Packer v1.13 release notes + +This page describes changes to Packer in v1.13. Refer to the [Packer repository](https://github.com/hashicorp/packer/releases) for information about all releases. + +## New features + +This release includes the following updates. + +### Shell Provisioner Enhancement + +The shell provisioner’s inline field now supports executing scripts directly, including those with their own shebang (#!) lines. +This enhancement allows you to provide a full script as an inline command, enabling dynamic scripts via tools like templatefile. + +### New functions + +As part of Packer 1.13, we have introduced below functions. + +* `sum`: takes a list or set of numbers and returns the sum of those numbers.. +* `startswith`: takes two values: a string to check and a prefix string. The function returns true if the string begins with that exact prefix. +* `endswith`: takes two values: a string to check and a suffix string. The function returns true if the first string ends with that exact suffix. + +### Packer fmt command to format multiple HCL files + +Added the ability for the packer fmt command to format multiple HCL files at once. +This enhancement allows users to format several files in a single command invocation. + +### Top-Level hcp_packer_registry Block Support + +Added support for defining the hcp_packer_registry block at the top level of the Packer configuration. Enables improved management and referencing of HCP Packer registries directly in the root schema. +Also added validation around the multiple hcp_packer_registry blocks at top level. + +## Bug fixes + +### Documentation Update + +Updated the documentation to explicitly state that a trailing slash is required when uploading directories, regardless of the guest OS. +This clarification is intended to ensure consistent behavior and prevent confusion, especially for Windows users. + +### --use-sequential-evaluation Flag Now Works in Packer Console + +Fixed an issue where using the --use-sequential-evaluation flag with packer console resulted in an error. +The flag is now properly included in the supported options for packer console and works as intended. +Removed this flag from the fix subcommand, where it was not used or required. + +### Inline PowerShell Command + +Added a wrapper around inline PowerShell commands executed by the provisioner to catch errors and ensure the Packer build fails if any command results in an error. + +### Preserve the user-defined order of variable files + +Ensures that Packer now preserves the user-defined order of variable files as input by the user. +This change maintains the intended precedence of variables, honoring the order specified on the command line. diff --git a/content/packer/content/docs/security.mdx b/content/packer/content/docs/security.mdx new file mode 100644 index 0000000000..887f7ff380 --- /dev/null +++ b/content/packer/content/docs/security.mdx @@ -0,0 +1,16 @@ +--- +page_title: Security +description: >- + Packer takes security very seriously. Please responsibly disclose any security + vulnerabilities found and we'll handle it quickly. +--- + +# Packer Security + +We understand that many users place a high level of trust in HashiCorp and the tools we build. We apply best practices and focus on security to make sure we can maintain the trust of the community. + +We deeply appreciate any effort to disclose vulnerabilities responsibly. + +If you would like to report a vulnerability, please see the [HashiCorp security page](https://www.hashicorp.com/security), which has the proper email to communicate with as well as our PGP key. Please **do not create a GitHub issue for security concerns**. + +If you are not reporting a security sensitive vulnerability, please open an issue on the [Packer GitHub repository](https://github.com/hashicorp/packer). diff --git a/content/packer/content/docs/templates/hcl_templates/blocks/build/hcp_packer_registry.mdx b/content/packer/content/docs/templates/hcl_templates/blocks/build/hcp_packer_registry.mdx new file mode 100644 index 0000000000..503ba23d49 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/blocks/build/hcp_packer_registry.mdx @@ -0,0 +1,81 @@ +--- +description: > + The `hcp_packer_registry` block specifies details for new or updated images in the HCP Packer registry. Learn how to configure the `hcp_packer_registry` block. +page_title: hcp_packer_registry block reference +--- + +# `hcp_packer_registry` block + +This topic provides reference information about the `hcp_packer_registry` block. + +## Description + +The `hcp_packer_registry` block configures details about an image Packer creates or updates in the HCP Packer registry. Use the `hcp_packer_registry` block to customize the metadata Packer sends to HCP Packer Registry. + +To get started with HCP Packer, refer to the [HCP Packer documentation](/hcp/docs/packer) or try the [Get Started with HCP Packer tutorials](/packer/tutorials/hcp-get-started). + +## Usage + +This block is available from version 1.7.7 of Packer. + +Add a `hcp_packer_registry` block to your template file to enable HCP +Packer mode. When in HCP Packer mode, Packer pushes metadata for all builds in the template to the remote +registry if the appropriate HCP credentials are set (`HCP_CLIENT_ID` and +`HCP_CLIENT_SECRET`). If no HCP credentials are set, Packer will fail the build +and exit immediately to avoid any potential artifact drift between the defined +builders (source blocks) and the HCP Packer registry. + +~> **Note:** You will receive an error if you try to reference metadata from a deactivated or deleted registry. An administrator can manually deactivate or delete a registry, and HCP Packer automatically deactivates registries with billing issues. Contact [HashiCorp Support](https://support.hashicorp.com/) with questions. + +```hcl +# file: builds.pkr.hcl +source "happycloud" "macos" { + os = "macos_amd64" +} + +build { + sources = ["source.happycloud.macos"] +} +hcp_packer_registry { + bucket_name = "ios-dev" + + description = < packer build ./folder +Build 'a.null.first-example' finished. +Build 'a.null.second-example' finished. +Build 'null.second-example' finished. + +==> Builds finished. The artifacts of successful builds are: +--> a.null.first-example: Did not export anything. This is the null builder +--> a.null.second-example: Did not export anything. This is the null builder +--> null.second-example: Did not export anything. This is the null builder +``` + +### Running only specific builds + +The `-only`/`-except` flags will match a source's `type.name` and run 'only' +matching **builders** (source) or all referenced builders 'except' the matching +ones, for example with the same config file: + +```shell-session +> packer build -only "*.second-example" ./folder +Build 'null.second-example' finished. +Build 'a.null.second-example' finished. + +==> Builds finished. The artifacts of successful builds are: +--> a.null.second-example: Did not export anything. This is the null builder +--> null.second-example: Did not export anything. This is the null builder +``` + +Here `'a.null.first-example'` was skipped. + +-> Note: It is not yet possible to match a named `build` block to do this, but +this is soon going to be possible. So here "a.\*" will match nothing. + +## Related + +- Refer to the [community builders reference](/packer/docs/community-tools#community-builders) for information about builders maintained by the community. +- Refer to [Create Custom Builders](/packer/docs/plugins/creation/custom-builders) for instructions on how to create your own builders. \ No newline at end of file diff --git a/content/packer/content/docs/templates/hcl_templates/blocks/build/post-processor.mdx b/content/packer/content/docs/templates/hcl_templates/blocks/build/post-processor.mdx new file mode 100644 index 0000000000..d3d0fc7ba2 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/blocks/build/post-processor.mdx @@ -0,0 +1,110 @@ +--- +description: | + The `post-processor` block defines post-processor behavior after a `packer build` command. Learn how to configure `post-processor` blocks in your Packer templates. +page_title: post-processor block reference +--- + +# `post-processor` block + +This topic provides reference information about the `post-processor` block. + +## Description + +The `post-processor` block defines how a post-processor is configured. + +```hcl +# builds.pkr.hcl +build { + # ... build image + post-processor "checksum" { # checksum image + checksum_types = [ "md5", "sha512" ] # checksum the artifact + keep_input_artifact = true # keep the artifact + } + + post-processor "amazon-import" { # upload image to amazon + } +} +``` + +Each `post-processor` runs after each defined build. A post-processor takes the +`Artifact` from a build. Post-processors are optional, and they can be used to +upload artifacts, re-package, or more. The list of available post-processors +can be found in the [post-processors](/packer/docs/post-processors) section. + +A `post-processor` can also take the `Artifact` from another post-processor +when it is defined in a [`post-processors` +block](/packer/docs/templates/hcl_templates/blocks/build/post-processor) list, that is a list of +chained post processors. + +-> Note: The input 'artifact' received by a post-processor will be automatically +deleted by default. + +# Keep an input artifact + +To prevent an input artifact from being deleted, you can set the +`keep_input_artifact` field to true to make Packer keep both artifacts. For +example if we want to checksum an artifact and keep the artifact: + +```hcl +# builds.pkr.hcl +build { + # ... + post-processor "checksum" { + checksum_types = [ "md5", "sha512" ] + keep_input_artifact = true + } +} +``` + +# Run on Specific Builds + +You can use the `only` or `except` configurations to run a post-processor only +with specific sources. These two configurations do what you expect: `only` will +only run the post-processor on the specified sources and `except` will run the +post-processor on anything other than the specified sources. + +An example of `only` being used is shown below, but the usage of `except` is +effectively the same: + +```hcl +# builds.pkr.hcl +build { + # ... + post-processor "checksum" { + checksum_types = [ "md5", "sha512" ] + keep_input_artifact = true + only = ["amazon-ebs.example"] + } +} +``` + +The values within `only` or `except` are _source names_, not builder types. + +If you want to exclude certain post-processors when running `packer build` +from the command line, you can do so: + +`packer build --except=checksum mytemplate.pkr.hcl` will not run the checksum +post-processor. This command line exclusion works by referencing post-processor +_name_. By default, the post-processor is named after is its type, as +demonstrated above in the `checksum` example. You can make a post-processor's +name unique by adding a "name" field to each post-processor block. + +While the `-except` flag can be used to filter out post-processors on the +command line, the `-only` flag does not work for post-processors. If you wish +to only run a post-processor for a given source build you must use the +`only=[source]` syntax inside of your hcl templates, as described above. + + +## Build Contextual Variables + +Packer allows to access connection information and basic instance state +information from a post-processor. These information are stored in the `build` +variable. Check out the [Contextual +Variables](/packer/docs/templates/hcl_templates/contextual-variables) documentation to learn more +about and see some examples of how to use them. + +### Related + +- The [`post-processors` block](/packer/docs/templates/hcl_templates/blocks/build/post-processors) + allows to define one or more chain of `post-processor`s that will take the + output from the build and provision steps. diff --git a/content/packer/content/docs/templates/hcl_templates/blocks/build/post-processors.mdx b/content/packer/content/docs/templates/hcl_templates/blocks/build/post-processors.mdx new file mode 100644 index 0000000000..23b4f7fc80 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/blocks/build/post-processors.mdx @@ -0,0 +1,84 @@ +--- +description: > + Add the `post-processors` block to a `build` block to define a list of post-processors to apply to an artifact. Learn how to configure nested `post-processors` blocks. +page_title: post-processors block reference +--- + +# `post-processors` block + +This topic provides reference information about the `post-processors` block. + +## Description + +Add a `post-processors` block to your `build` block to include a list of [post-processor](/packer/docs/templates/hcl_templates/blocks/build/post-processor) configurations. Packer runs the post-processors for an artifact after each build. + +```hcl +# builds.pkr.hcl +build { + # ... + post-processors { + post-processor "shell-local" { # create an artifice.txt file containing "hello" + inline = [ "echo hello > artifice.txt" ] + } + post-processor "artifice" { # tell packer this is now the new artifact + files = ["artifice.txt"] + } + post-processor "checksum" { # checksum artifice.txt + checksum_types = [ "md5", "sha512" ] # checksum the artifact + keep_input_artifact = true # keep the artifact + } + } + +} +``` + +The [`post-processor` block](/packer/docs/templates/hcl_templates/blocks/build/post-processor) +allows to define multiple post-processors that will run from the `Artifact` of +each build. Read the `post-processor` documentation to know how to use a +post-processor. + +### Difference between a `post-processor` and a `post-processors` block + +These two templates are doing the same thing: + +```hcl +# builds.pkr.hcl +build { + # ... build image + post-processor "checksum" { # checksum image + checksum_types = [ "md5", "sha512" ] # checksum the artifact + } + + post-processor "amazon-import" { # upload image to AWS + } + + post-processor "googlecompute-import" { # upload image to GCP + } +} +``` + +```hcl +# builds.pkr.hcl +build { + # ... build image + post-processors { + post-processor "checksum" { # checksum image + checksum_types = [ "md5", "sha512" ] # checksum the artifact + } + } + + post-processors { + post-processor "amazon-import" { # upload image to AWS + } + } + + post-processors { + post-processor "googlecompute-import" { # upload image to GCP + } + } +} +``` + +Each of these `post-processors` will start after each build -- that is, after +each provision step has run on each source --. In all cases the source image is +going to be deleted. diff --git a/content/packer/content/docs/templates/hcl_templates/blocks/build/provisioner.mdx b/content/packer/content/docs/templates/hcl_templates/blocks/build/provisioner.mdx new file mode 100644 index 0000000000..7c19c1767b --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/blocks/build/provisioner.mdx @@ -0,0 +1,262 @@ +--- +description: | + The `provisioner` block defines how to configure a provisioner. Learn how to configure `provisioner` blocks in your templates. +page_title: provisioner block reference +--- + +# `provisioner` block + +This topic providers reference information about the `provisioner` block. + +## Description + +The `provisioner` block defines how to configure a provisioner. Provisioners use builtin and third-party software to install and configure the +machine image after booting. Provisioners prepare the system for use. + +The list of available provisioners can be found in the +[Provisioners](/packer/docs/provisioners) section. + +```hcl +# builds.pkr.hcl +build { + # ... + provisioner "shell" { + inline = [ + "echo provisioning all the things", + "echo the value of 'foo' is '${var.foo}'", + ] + } +} +``` + +## Run on Specific Sources + +You can use the `only` or `except` configurations to run a provisioner only +with specific sources. These two configurations do what you expect: `only` will +only run the provisioner on the specified sources and `except` will run the +provisioner on anything other than the specified sources. + +An example of `only` being used is shown below, but the usage of `except` is +effectively the same: + +```hcl +# builds.pkr.hcl + +source "amazon-ebs" "first-example" { + #... +} + +source "amazon-ebs" "second-example" { + #... +} + +build { + sources = [ + "source.amazon-ebs.first-example", + "source.amazon-ebs.second-example", + ] + provisioner "shell" { + # This provisioner only runs for the 'first-example' source. + only = ["amazon-ebs.first-example"] + + inline = [ + "echo provisioning all the things", + "echo the value of 'foo' is '${var.foo}'", + ] + } + provisioner "shell" { + # This runs with all sources. + inline = [ + "echo Hi World!" + ] + } +} +``` + +The source names within `only` and `except` must suppress `source.` prefix. + +The values within `only` or `except` are _source names_, not builder types nor build names. + +-> Note: In the cli `only` and `except` will match against **build names** (for +example:`my_build.amazon-ebs.first-example`) but in a provisioner they will +match on the **source name** (for example:`amazon-ebs.third-example`). + +## Build-Specific Overrides + +While the goal of Packer is to produce identical machine images, it sometimes +requires periods of time where the machines are different before they +eventually converge to be identical. In these cases, different configurations +for provisioners may be necessary depending on the build. This can be done +using build-specific overrides. + +An example of where this might be necessary is when building both an EC2 AMI +and a VMware machine. The source EC2 AMI may setup a user with administrative +privileges by default, whereas the VMware machine doesn't have these +privileges. In this case, the shell script may need to be executed differently. +Of course, the goal is that hopefully the shell script converges these two +images to be identical. However, they may initially need to be run differently. + +This example is shown below: + +```hcl +source "null" "example1" { + communicator = "none" +} + +source "null" "example2" { + communicator = "none" +} + +source "null" "example3" { + communicator = "none" +} + +build { + sources = ["source.null.example2", "source.null.example3"] + + source "source.null.example1" { + // Give a name to this source + name = "renamed" + } + + provisioner "shell-local" { + inline = ["echo not overridden"] + override = { + example3 = { + inline = ["echo overrides for example3"] + } + // Refer to the source with the given name + renamed = { + inline = ["echo overrides for renamed"] + } + } + } +} +``` + +As you can see, the `override` key is used. The value of this key is another +HCL attribute map where the key is the name of a [source +definition](/packer/docs/templates/hcl_templates/blocks/source). The value of this is in turn +another HCL attribute map. This HCL attribute map simply contains the provisioner +configuration as normal. This configuration is merged into the default +provisioner configuration. + +## On Error Provisioner + +You can optionally create a single specialized provisioner called an +`error-cleanup-provisioner`. This provisioner will not run unless the normal +provisioning run fails. If the normal provisioning run does fail, this special +error provisioner will run _before the instance is shut down_. This allows you +to make last minute changes and clean up behaviors that Packer may not be able +to clean up on its own. + +For examples, users may use this provisioner to make sure that the instance is +properly unsubscribed from any services that it connected to during the build +run. + +Toy usage example for the error cleanup script: + +```hcl +source "null" "example" { + communicator = "none" +} + +build { + sources = ["source.null.example"] + + provisioner "shell-local" { + inline = ["exit 2"] + } + + error-cleanup-provisioner "shell-local" { + inline = ["echo 'rubber ducky'> ducky.txt"] + } +} +``` + +## Pausing Before Running + +With certain provisioners it is sometimes desirable to pause for some period of +time before running it. Specifically, in cases where a provisioner reboots the +machine, you may want to wait for some period of time before starting the next +provisioner. + +Every provisioner definition in a Packer template can take a special +configuration `pause_before` that is the amount of time to pause before running +that provisioner. By default, there is no pause. An example is shown below: + +```hcl +# builds.pkr.hcl +build { + # ... + provisioner "shell" { + inline = [ + "echo provisioning all the things", + "echo the value of 'foo' is '${var.foo}'", + ] + pause_before = "10s" + } +} +``` + +For the above provisioner, Packer will wait 10 seconds before uploading and +executing the shell script. + +## Retry on error + +With certain provisioners it is sometimes desirable to retry when it fails. +Specifically, in cases where the provisioner depends on external processes that are not done yet. + +Every provisioner definition in a Packer template can take a special +configuration `max_retries` that is the maximum number of times a provisioner will retry on error. +By default, there `max_retries` is zero and there is no retry on error. An example is shown below: + +```hcl +# builds.pkr.hcl +build { + # ... + provisioner "shell" { + inline = [ + "echo provisioning all the things", + "echo the value of 'foo' is '${var.foo}'", + ] + max_retries = 5 + } +} +``` + +For the above provisioner, Packer will retry maximum five times until stops failing. +If after five retries the provisioner still fails, then the complete build will fail. + +## Timeout + +Sometimes a command can take much more time than expected + +Every provisioner definition in a Packer template can take a special +configuration `timeout` that is the amount of time to wait before +considering that the provisioner failed. By default, there is no timeout. An +example is shown below: + +```hcl +# builds.pkr.hcl +build { + # ... + provisioner "shell" { + inline = [ + "echo provisioning all the things", + "echo the value of 'foo' is '${var.foo}'", + ] + timeout = "5m" + } +} +``` + +For the above provisioner, Packer will cancel the script if it takes more than +5 minutes. + +Timeout has no effect in debug mode. + +## Build Contextual Variables + +Packer allows to access connection information and basic instance state information from a provisioner. These information are stored in the `build` variable. +Check out the [Contextual Variables](/packer/docs/templates/hcl_templates/contextual-variables) documentation to learn more about and see some examples of how to use them. diff --git a/content/packer/content/docs/templates/hcl_templates/blocks/build/source.mdx b/content/packer/content/docs/templates/hcl_templates/blocks/build/source.mdx new file mode 100644 index 0000000000..e006dc9248 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/blocks/build/source.mdx @@ -0,0 +1,56 @@ +--- +description: > + A `source` block nested in a `build` block reuses a builder defined + elsewhere in the configuration. Learn how to configure a nested `source` block. +page_title: source block reference +--- + +# `source` block + +This topic provides reference information about `source` blocks nested in a `build` block. + +## Description + +Add a `source` block to your `build` block to reuse a builder defined elsewhere in the configuration. Packer fills in the source fields that are not already set in the top-level `source` block. + +Build-level source blocks are implemented by merging their contents with the +corresponding top-level source block, and a `packer build` command fails when it +encounters the ambiguity that arises when a source parameter is defined twice. + +For example, in the below example, if the top-level `lxd.arch` source block +also defined an `output_image` field or if one of the build-level source blocks +redefined and image field, Packer would error. + +```hcl +# file: builds.pkr.hcl +source "lxd" "arch" { + image = "archlinux" +} + +build { + # Use the singular `source` block set specific fields. + # Note that fields cannot be overwritten, in other words, you cannot + # set the 'image' field from the top-level source block in here, as well as + # the 'name' and 'output_image' fields cannot be set in the top-level source block. + source "lxd.arch" { + # Setting the name field allows to rename the source only for this build section. + name = "nomad" + output_image = "nomad" + } + + provisioner "shell" { + inline = [ "echo installing nomad" ] + } +} + +build { + source "lxd.arch" { + name = "consul" + output_image = "consul" + } + + provisioner "shell" { + inline = [ "echo installing consul" ] + } +} +``` diff --git a/content/packer/content/docs/templates/hcl_templates/blocks/data.mdx b/content/packer/content/docs/templates/hcl_templates/blocks/data.mdx new file mode 100644 index 0000000000..f503a5a9cd --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/blocks/data.mdx @@ -0,0 +1,13 @@ +--- +description: > + The `data` block defines data sources within your Packer configuration. Learn how to configure `data` blocks. +page_title: data block referenece +--- + +# `data` block + +The `data` block defines data sources within your Packer configuration. The following example instructs Packer to query the `amazon-ami` data source for information to use in the template. + +`@include 'from-1.5/datasources/example-block.mdx'` + +Refer to [Data Sources](/packer/docs/templates/hcl_templates/datasources) for instructions about using the `data` block in your configuration. diff --git a/content/packer/content/docs/templates/hcl_templates/blocks/index.mdx b/content/packer/content/docs/templates/hcl_templates/blocks/index.mdx new file mode 100644 index 0000000000..3be0371c6f --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/blocks/index.mdx @@ -0,0 +1,61 @@ +--- +page_title: Built-in blocks overview +description: A block is a container for configuration in a Packer template. Learn about the types of configration blocks built into HCL for Packer. +--- + +# Built-in blocks overview + +This topic provides an overview of the configuration blocks built into the Packer language that you can use to write Packer templates in HCL2. + +## Introduction + +A block is a container for configuration. You can use the following types of blocks in your Packer templates: + +- `build` blocks contain configuration for a specific combination of builders, + provisioners, and post-processors used to create a specific image artifact. +- `source` blocks contain configuration for builder plugins. Once defined, + sources can be used and further configured by the "build" block. +- `provisioner` blocks contain configuration for provisioner plugins. These + blocks are nested inside of a build block. +- `post-processor` and `post-processors` blocks contain configuration for + post-processor plugins and post-processor plugin sequences. They are also + nested within `build` blocks. +- `variable` blocks contain configuration for variables that can either be + defaulted in configuration or set by the user at runtime. +- `locals` blocks contain configuration for variables that can be created using + HCL functions or data sources, or composited from variables created in the + variables blocks. + +The documentation contains information for each block type. + +Other blocks, such as the `packer` block, provide information to the Packer core +about what version it is allowed to run. The `required_plugins` block helps the +Packer core + +Blocks can be defined in multiple files and `packer build folder` will build +using solely the files from a directory named `folder`. + +Packer does not support user-defined blocks and so only the blocks built in to +the language are available for use. The documentation includes all of the available built-in HCL2 blocks. + +## Configuration examples + +`@include 'from-1.5/variables/foo-block.mdx'` + +- [Variable block documentation](/packer/docs/templates/hcl_templates/blocks/variable). + +`@include 'from-1.5/locals/example-block.mdx'` + +- [Locals block documentation](/packer/docs/templates/hcl_templates/blocks/locals). + +`@include 'from-1.5/sources/example-block.mdx'` + +- [source block documentation](/packer/docs/templates/hcl_templates/blocks/source). + +`@include 'from-1.5/builds/example-block.mdx'` + +- [build block documentation](/packer/docs/templates/hcl_templates/blocks/build). + +`@include 'from-1.5/datasources/example-block.mdx'` + +- [data block documentation](/packer/docs/templates/hcl_templates/blocks/data). diff --git a/content/packer/content/docs/templates/hcl_templates/blocks/locals.mdx b/content/packer/content/docs/templates/hcl_templates/blocks/locals.mdx new file mode 100644 index 0000000000..07fecf813c --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/blocks/locals.mdx @@ -0,0 +1,22 @@ +--- +description: > + The `locals` and `local-variable` blocks declare local variables in + your Packer configuration. Learn how to use these blocks to declare local variables in your template. +page_title: locals block reference +--- + +# `locals` block + +This topic provides reference information about the `locals` block. + +## Description + +The `locals` and `local-variable` blocks declare local variables in +your Packer configuration. + +`@include 'from-1.5/locals/example-block.mdx'` + +Refer to the following topics for additional information about using the `locals` and `local-variable` blocks: + +- [Local Variables](/packer/docs/templates/hcl_templates/locals) +- [Input Variables and local variables](/packer/guides/hcl/variables) diff --git a/content/packer/content/docs/templates/hcl_templates/blocks/packer.mdx b/content/packer/content/docs/templates/hcl_templates/blocks/packer.mdx new file mode 100644 index 0000000000..fd5c7eb1f4 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/blocks/packer.mdx @@ -0,0 +1,144 @@ +--- +page_title: packer block reference +description: |- + The `packer` block configures Packer behavior. Learn how to configure the `packer` in Packer templates written in HCL. +--- + +# `packer` block + +This topic provides reference information about the `packer` block. + +## Description + +The `packer` block configures Packer version requirements and specifies which plugins to install upon initialization. + +## Packer Block Syntax + +Packer settings are gathered together into `packer` blocks: + +```hcl +packer { + # ... +} +``` + +Each `packer` block can contain a number of settings related to Packer's +behavior. Within a `packer` block, only constant values can be used; +arguments may not refer to named objects such as resources, input variables, +etc, and may not use any of the Packer language built-in functions. + +The various options supported within a `packer` block are described in the +following sections. + +## Specifying a Required Packer Version + +The `required_version` setting accepts a [version constraint +string,](#version-constraints) which specifies which versions of Packer +can be used with your configuration. + +If the running version of Packer doesn't match the constraints specified, +Packer will produce an error and exit without taking any further actions. + +Use Packer version constraints in a collaborative environment to +ensure that everyone is using a specific Packer version, or using at least +a minimum Packer version that has behavior expected by the configuration. + +## Specifying Plugin Requirements + +-> **Note:** The `required_plugins` block is only available in Packer v1.7.0 and +later. + +The `required_plugins` block specifies all of the plugins required by the +current template, mapping each local plugin name to a source address and a +version constraint. + +```hcl +packer { + required_plugins { + happycloud = { + version = ">= 2.7.0" + source = "github.com/hashicorp/happycloud" + } + } +} +``` + +Using the `required_plugins` block to codify the plugins required for invoking a +`packer build` on your template(s) is a great way to ensure consistent builds across +different platforms or hosts. + +The block codifies two pieces of information that we've found critical to ensure reproducible builds: +1. **The source of a required plugin** indicates to users where a plugin where a plugin was downloaded from +and where to reach out if there are issues with the plugin. +1. **The version of a required plugin** indicates to users the exact or minimum version needed for the build. +This is a great way to pin approved versions of a plugin that can be installed across your environment or team. + +For more information on plugins, refer to [Plugins](/packer/docs/plugins). + +### Define plugin source +Specify the path to the plugin source code in the `source` field using the following format: + +`/[SUBFOLDER/]/` + +- ``: Specifies the hostname of the location or service that distributes the plugin. Packer only supports using `github.com` as the hostname when you install plugins using the `packer init` command. You can point to non-GitHub addresses, such as an internal proxy or plugin binary store, but Packer only pins the required version if you install the plugin using the `packer plugins install –path` command. Refer to the following documentation for additional information. + - [`init` command](/packer/docs/commands/init) + - [`plugins install`](/packer/docs/commands/plugins/install) +- ``: The subfolder path segment is an optional part of the address that enables you to download plugin sources from custom addresses. You can specify a source containing up to 13 total path segments. +- ``: An organizational namespace within the specified host. This often is the organization that publishes the plugin. + Type: A short name for the platform or system the plugin manages. The type is usually the plugin's preferred local name. + +For example, the value of the `source` field for a plugin named `myawesomecloud` is `github.com/hashicorp/myawesomecloud` if it belongs to the `hashicorp` namespace on the host `github.com`. + +The actual repository that `myawesomecloud` comes from must always have the name format `github.com/hashicorp/packer-plugin-myawesomecloud`, but the `required_plugins` block omits the redundant `packer-plugin-` repository prefix for brevity. + +## Version Constraints + +Anywhere that Packer lets you specify a range of acceptable versions for +something, it expects a specially formatted string known as a version +constraint. + +### Version Constraint Syntax + +Packer's syntax for version constraints is very similar to the syntax used by +other dependency management systems like Bundler and NPM. + +```hcl +required_version = ">= 1.2.0, < 2.0.0" +``` + +A version constraint is a [string literal](/packer/docs/templates/hcl_templates/expressions#string-literals) +containing one or more conditions, which are separated by commas. + +Each condition consists of an operator and a version number. + +Version numbers should be a series of numbers separated by periods (like +`1.2.0`), optionally with a suffix to indicate a beta release. + +The following operators are valid: + +- `=` (or no operator): Allows only one exact version number. Cannot be combined + with other conditions. + +- `!=`: Excludes an exact version number. + +- `>`, `>=`, `<`, `<=`: Comparisons against a specified version, allowing + versions for which the comparison is true. "Greater-than" requests newer + versions, and "less-than" requests older versions. + +- `~>`: Allows the specified version, plus newer versions that only + increase the _most specific_ segment of the specified version number. For + example, `~> 0.9` is equivalent to `>= 0.9, < 1.0`, and `~> 0.8.4`, is + equivalent to `>= 0.8.4, < 0.9`. This is usually called the pessimistic + constraint operator. + +### Version Constraint Behavior + +A version number that meets every applicable constraint is considered acceptable. + +Packer consults version constraints to determine whether it has acceptable +versions of itself. + +A prerelease version is a version number that contains a suffix introduced by +a dash, like `1.2.0-beta`. A prerelease version can be selected only by an +_exact_ version constraint (the `=` operator or no operator). Prerelease +versions do not match inexact operators such as `>=`, `~>`, etc. diff --git a/content/packer/content/docs/templates/hcl_templates/blocks/source.mdx b/content/packer/content/docs/templates/hcl_templates/blocks/source.mdx new file mode 100644 index 0000000000..aa38b68445 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/blocks/source.mdx @@ -0,0 +1,61 @@ +--- +description: | + The `source` block defines reusable builder configuration blocks. Learn how to define reusable builders using the `source` block. +page_title: source block reference +--- + +# `source` block + +This topic providers reference information about the `source` block. + +## Description + +The `source` block defines reusable builder configuration blocks. Builders are commonly used in custom plugins. + +## Example + +The following example defines a source for a builder type called `happycloud` with the name `foo`: + +`@include 'from-1.5/sources/example-block.mdx'` + +There is only one `source.happycloud.foo` top-level source block, but it can be used more than once. + +You can start builders by referring to `source` blocks from a [`build` +block](/packer/docs/templates/hcl_templates/blocks/build) : + +```hcl +build { + sources = [ + # Here Packer will use a default ami_name when saving the image. + "source.happycloud.example", + "source.happycloud.foo", + ] +} +``` + +The build-level [`source` +block](/packer/docs/templates/hcl_templates/blocks/build/source) allows to set specific +source fields. Each field must be defined only once and it is currently not +allowed to override a value. + +```hcl +build { + source "source.happycloud.example" { + # Here Packer will use the provided image_name instead of defaulting it. + # Note that fields cannot be overwritten, in other words, you cannot + # set the 'image_name' field in the top-level source block and here at the + # same time + image_name = "build_specific_field" + } +} +``` + +`@include 'from-1.5/contextual-source-variables.mdx'` + +## Related + +- Refer to the [builders reference overview](/packer/docs/builders) for information about Packer builders. + +- Refer to the [community builders reference](/packer/docs/community-tools#community-builders) for information about builders maintained by the community. + +- Refer to [Create Custom Builders](/packer/docs/plugins/creation/custom-builders) for instructions on how to create your own builders. diff --git a/content/packer/content/docs/templates/hcl_templates/blocks/variable.mdx b/content/packer/content/docs/templates/hcl_templates/blocks/variable.mdx new file mode 100644 index 0000000000..8815035b12 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/blocks/variable.mdx @@ -0,0 +1,39 @@ +--- +description: > + The `variable` and `input-variable` blocks define variables in your Packer configuration. Learn how to use these blocks to declare variables in your template. +page_title: variable block reference +--- + +# `variable` block + +This topic provides reference information about the `variable` block. + +## Description + +The `variable` and `input-variable` blocks define variables +within your Packer configuration. You cannot use the `input-variable` block in another `input-variable` block. We recommend using the [`locals`](/packer/docs/templates/hcl_templates/blocks/locals) to nest variables instead. + +`@include 'from-1.5/variables/foo-block.mdx'` + +## Default value + +If a default value is set, the variable is optional. Otherwise, the variable +**must** be set. + +`@include 'from-1.5/variables/assignment.mdx'` + +`@include 'from-1.5/variables/custom-validation.mdx'` + +Example of a variable assignment from a file: + +`@include 'from-1.5/variables/foo-pkrvar.mdx'` + +`@include 'from-1.5/variables/must-be-set.mdx'` + +`@include 'from-1.5/variables/sensitive.mdx'` + +# More on variables + +- Read the [full variables](/packer/docs/templates/hcl_templates/variables) description for a more + thorough read. +- Read the [variables guide](/packer/guides/hcl/variables) for more examples. diff --git a/content/packer/content/docs/templates/hcl_templates/contextual-variables.mdx b/content/packer/content/docs/templates/hcl_templates/contextual-variables.mdx new file mode 100644 index 0000000000..c6a6b48781 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/contextual-variables.mdx @@ -0,0 +1,234 @@ +--- +page_title: Contextual variables reference +description: >- + Source variables, build variables, and other special variables are context dependent. Learn about contextual variables that return connection and instance state information. +--- + +# Contextual variables + +This topic provides reference information about special variables that return connection and basic instance state information based on the configuration context. + +`@include 'from-1.5/contextual-source-variables.mdx'` + +## Build Variables + +Build variables will allow you to access connection information and basic instance state information for a builder. +All special build variables are stored in the `build` variable: + +```hcl +source "null" "first-example" { + communicator = "none" +} + +build { + name = "my-build-name" + sources = ["null.first-example"] + + provisioner "shell-local" { + environment_vars = ["TESTVAR=${build.PackerRunUUID}"] + inline = ["echo source.name is ${source.name}.", + "echo build.name is ${build.name}.", + "echo build.PackerRunUUID is $TESTVAR"] + } +} +``` + +Here is the list of available build variables: + +- **name** Represents the name of the build block being run. This is different + than the name of the source block being run. + +- **ID**: Represents the VM being provisioned. For example, in Amazon it is the instance ID; in DigitalOcean, + it is the Droplet ID; in VMware, it is the VM name. + +- **Host**, **Port**, **User** and **Password**: The host, port, user, and password that Packer uses to access the machine. + Useful for using the shell local provisioner to run Ansible or Inspec against the provisioned instance. + +- **ConnType**: Type of communicator being used. For example, for SSH communicator this will be "ssh". + +- **PackerRunUUID**: Current build's unique ID. Can be used to specify build artifacts. + An example of that, is when multiple builds runs at the same time producing the same artifact. + It's possible to differentiate these artifacts by naming them with the builds' unique IDs. + +- **PackerHTTPIP**, **PackerHTTPPort**, and **PackerHTTPAddr**: HTTP IP, port, and address of the file server Packer creates to serve items in the "http" dir to the VM. The HTTP address is displayed in the format `IP:PORT`. + +- **SSHPublicKey** and **SSHPrivateKey**: The public and private key that Packer uses to connect to the instance. + These are unique to the SSH communicator and are unset when using other communicators. + **SSHPublicKey** and **SSHPrivateKey** can have escape sequences and special characters so their output should be single quoted to avoid surprises. For example: + + ```hcl + provisioner "shell" { + inline = ["echo '${build.SSHPrivateKey}' > /tmp/packer-session.pem"] + } + ``` + +For backwards compatibility, `WinRMPassword` is also available through this +engine, though it is no different than using the more general `Password`. + +All build variables are valid to use with any of the [HCL2 functions](/packer/docs/templates/hcl_templates/functions). +Example of using [upper](/packer/docs/templates/hcl_templates/functions/string/upper) to upper case the build ID: + +```hcl + post-processor "shell-local" { + inline = ["echo ${upper(build.ID)}"] + } +``` + +For builder-specific builder variables, please also refer to the builder docs: + +- Amazon EC2: [chroot](/packer/plugins/builders/amazon/chroot#build-shared-information-variables), + [EBS Volume](/packer/plugins/builders/amazon/ebsvolume#build-shared-information-variables), + [EBS](/packer/plugins/builders/amazon/ebs#build-shared-information-variables), + [EBS Surrogate](/packer/plugins/builders/amazon/ebssurrogate#build-shared-information-variables), + [Instance](/packer/plugins/builders/amazon/instance#build-shared-information-variables). + +The HCL2 Special Build Variables is in beta; please report any issues or requests on the Packer +issue tracker on GitHub. + +## Packer Version + +This variable is set to the Packer version currently running. + +```hcl +source "null" "first-example" { + communicator = "none" +} + +build { + sources = ["null.first-example"] + + provisioner "shell-local" { + inline = ["echo packer_version is '${packer.version}'"] + } +} +``` + +If you are running a development version of Packer, the version variable will +contain the released version number, dev flag, and current commit. + +```shell-session +PACKER_LOG=0 packer build packer_version_demo.pkr.hcl +null.first-example: output will be in this color. + +==> null.first-example: Running local shell script: /var/folders/8t/0yb5q0_x6mb2jldqq_vjn3lr0000gn/T/packer-shell083160352 + null.first-example: packer_version is 1.6.5-dev (a69392129+CHANGES) +``` + +If you are running a released version of Packer, the version variable will +contain the released version number only: + +```shell-session +PACKER_LOG=0 packer build packer_version_demo.pkr.hcl +null.first-example: output will be in this color. + +==> null.first-example: Running local shell script: /var/folders/8t/0yb5q0_x6mb2jldqq_vjn3lr0000gn/T/packer-shell718995312 + null.first-example: packer_version is 1.6.5 +``` + +Make sure to wrap your variable in single quotes in order to escape the +string that is returned; if you are running a dev version of packer the +parenthesis may through off your shell escaping otherwise. + +## HCP Packer Iteration ID + +~> **Note**: The `packer.iterationID` variable is now deprecated and will be removed in a future version of Packer. HCP Packer Versions should be accessed with their fingerprint instead. The `packer.versionFingerprint` variable is now exposed to be used in its stead with the new HCP Packer data sources. + +If your build is pushing metadata to the HCP Packer registry, this variable is +set to the value of the Iteration ID associated with this run. + +```hcl +source "null" "example" { + communicator = "none" +} + +data "hcp-packer-version" "hardened-source" { + bucket_name = "example" + channel_name = "latest" +} + +data "hcp-packer-artifact" "file" { + bucket_name = "example" + version_fingerprint = "${data.hcp-packer-version.hardened-source.fingerprint}" + platform = "aws" + region = "us-east-1" +} + +build { + hcp_packer_registry { + bucket_name = "simple" + } + sources = [ + "source.null.example" + ] + + provisioner "shell-local" { + inline = [ + "echo data is ${packer.iterationID}" + ] + } +} + +``` + +```shell-session +==> mybuild.null.example: Running local shell script: /var/folders/cz/q3cr3tld2457gtlgw7qs1kqc0000gq/T/packer-shell842419427 + mybuild.null.example: data is 01HN3KCRPVKR5PBQ28TS6B12W0 +``` + +## HCP Packer Version Fingerprint + +If your build is pushing metadata to the HCP Packer registry, this variable is +set to the value of the Version Fingerprint associated with this run. + +```hcl +source "null" "example" { + communicator = "none" +} + +data "hcp-packer-version" "hardened-source" { + bucket_name = "example" + channel_name = "latest" +} + +data "hcp-packer-artifact" "file" { + bucket_name = "example" + version_fingerprint = "${data.hcp-packer-version.hardened-source.fingerprint}" + platform = "aws" + region = "us-east-1" +} + +build { + hcp_packer_registry { + bucket_name = "simple" + } + sources = [ + "source.null.example" + ] + + provisioner "shell-local" { + inline = [ + "echo data is ${packer.versionFingerprint}" + ] + } +} + +``` + +```shell-session +==> mybuild.null.example: Running local shell script: /var/folders/cz/q3cr3tld2457gtlgw7qs1kqc0000gq/T/packer-shell842419427 + mybuild.null.example: data is 01HN3KCRPVKR5PBQ28TS6B12W0 +``` + +You can also add this value to post-processors, for example to add to a manifest file: + +```hcl + post-processor "manifest" { + output = "manifest.json" + strip_path = true + custom_data = { + version_fingerprint = "${packer.versionFingerprint}" + iteration = "${packer.iterationID}" + } + } + +``` diff --git a/content/packer/content/docs/templates/hcl_templates/datasources.mdx b/content/packer/content/docs/templates/hcl_templates/datasources.mdx new file mode 100644 index 0000000000..794841c3ef --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/datasources.mdx @@ -0,0 +1,81 @@ +--- +page_title: Data Sources +description: >- + A `data` block defines a data source that instructs Packer to query data defined outside of Packer for use in builds and sources. Learn how to configure `data` blocks to define data sources. +--- + +# Data sources reference + +This topic describes how to use the `data` block to configure data sources in your HCL Packer templates. The `data` block instructs Packer to fetch or compute data for use in [`locals` blocks](/packer/docs/templates/hcl_templates/blocks/locals) and +[`source` blocks](/packer/docs/templates/hcl_templates/blocks/source) so that builders can use of information defined outside of Packer. + +## Using data sources + +A data source is declared using a data block, and the configuration looks like the following: + +```hcl +data "amazon-ami" "example" { + filters = { + virtualization-type = "hvm" + name = "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*" + root-device-type = "ebs" + } + owners = ["099720109477"] + most_recent = true +} +``` + +A `data` block requests that Packer read from a given data source (`"amazon-ami"`) and export the result under the given +local name (`"example"`). The name is used to refer to this data source from elsewhere in the same Packer configuration. + +The data block creates a data instance of the given _type_ (first block label) and _name_ (second block label). +The combination of the type and name must be unique within a configuration. + +Within the block (the `{ }`) is the configuration for the data instance. The configuration is dependent on the data source type, +and is documented for each data source. For example the configuration of the `amazon-ami` data source can be found at [plugins/datasources/amazon/ami](/packer/plugins/datasources/amazon/ami#configuration-reference). + +A data source can output one or more attributes, which can be used by adding their key name to the data source unique +identifier, like `data...`. + +The output from the `amazon-ami.example` above can be accessed as follows: + +Output data: + +``` +"data.amazon-ami.example" { + id = "ami12345" + name = "MyAMI" + creation_date = "01/01/2021" + owner = "123456789" + owner_name = "Some Name" + tags = {"tag1": "value"} +} +``` + +Usage: + +```hcl +// in a local +locals { + source_ami_id = data.amazon-ami.example.id + source_ami_name = data.amazon-ami.example.name + } +``` + +```hcl +// in a source +source "amazon-ebs" "basic-example" { + source_ami = local.source_ami + // ... +} +``` + +## Known limitations +`@include 'datasources/local-dependency-limitation.mdx'` + + +## Related + +- The list of available data sources can be found in the [data sources](/packer/docs/datasources) + section. +- Create your own [custom data source](/packer/docs/plugins/creation/custom-datasources) ! diff --git a/content/packer/content/docs/templates/hcl_templates/expressions.mdx b/content/packer/content/docs/templates/hcl_templates/expressions.mdx new file mode 100644 index 0000000000..4361ec8fd0 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/expressions.mdx @@ -0,0 +1,583 @@ +--- +page_title: HCL expressions reference +description: |- + HCL expressions provide access to data exported by data sources and transforms and combines the data into other values. Learn how to use HCL expressions in Packer templates. +--- + +# HCL expressions reference + +This topic provides reference information about expressions you can use in HCL templates for Packer. + +## Introduction + +_Expressions_ are used to refer to or compute values within a configuration. +The simplest expressions are just literal values, like `"hello"` or `5`, but +HCL also allows more complex expressions such as references to data exported by +sources, arithmetic, conditional evaluation, and a number of built-in +functions. + +Expressions can be used in a number of places in HCL, but some contexts limit +which expression constructs are allowed, such as requiring a literal value of a +particular type or forbidding. Each language feature's documentation describes +any restrictions it places on expressions. + +The rest of this page describes all of the features of Packer's +expression syntax. + +## Types and Values + +The result of an expression is a _value_. All values have a _type_, which +dictates where that value can be used and what transformations can be +applied to it. + +HCL uses the following types for its values: + +- `string`: a sequence of Unicode characters representing some text, like + `"hello"`. +- `number`: a numeric value. The `number` type can represent both whole + numbers like `15` and fractional values like `6.283185`. +- `bool`: either `true` or `false`. `bool` values can be used in conditional + logic. +- `list` (or `tuple`): a sequence of values, like + `["us-west-1a", "us-west-1c"]`. Elements in a list or tuple are identified by + consecutive whole numbers, starting with zero. +- `map` (or `object`): a group of values identified by named labels, like + `{name = "Mabel", age = 52}`. + +Strings, numbers, and bools are sometimes called _primitive types._ +Lists/tuples and maps/objects are sometimes called _complex types,_ _structural +types,_ or _collection types._ + +Finally, there is one special value that has _no_ type: + +- `null`: a value that represents _absence_ or _omission._ If you set an + argument of a source or module to `null`, Packer behaves as though you + had completely omitted it — it will use the argument's default value if it has + one, or raise an error if the argument is mandatory. `null` is most useful in + conditional expressions, so you can dynamically omit an argument if a + condition isn't met. + +### Advanced Type Details + +In most situations, lists and tuples behave identically, as do maps and objects. +Whenever the distinction isn't relevant, the Packer documentation uses each +pair of terms interchangeably (with a historical preference for "list" and +"map"). + +However, plugin authors should understand the differences between these similar +types (and the related `set` type), since they offer different ways to restrict +the allowed values for input variables and source arguments. + +### Type Conversion + +Expressions are most often used to set values for arguments. In these cases, +the argument has an expected type and the given expression must produce a value +of that type. + +Where possible, Packer automatically converts values from one type to +another in order to produce the expected type. If this isn't possible, Packer +will produce a type mismatch error and you must update the configuration with a +more suitable expression. + +Packer automatically converts number and bool values to strings when needed. +It also converts strings to numbers or bools, as long as the string contains a +valid representation of a number or bool value. + +- `true` converts to `"true"`, and vice-versa +- `false` converts to `"false"`, and vice-versa +- `15` converts to `"15"`, and vice-versa + +## Literal Expressions + +A _literal expression_ is an expression that directly represents a particular +constant value. Packer has a literal expression syntax for each of the value +types described above: + +- Strings are usually represented by a double-quoted sequence of Unicode + characters, `"like this"`. There is also a "heredoc" syntax for more complex + strings. String literals are the most complex kind of literal expression in + Packer, and have additional documentation on this page: + - See [String Literals](#string-literals) below for information about escape + sequences and the heredoc syntax. + - See [String Templates](#string-templates) below for information about + interpolation and template directives. +- Numbers are represented by unquoted sequences of digits with or without a + decimal point, like `15` or `6.283185`. +- Bools are represented by the unquoted symbols `true` and `false`. +- The null value is represented by the unquoted symbol `null`. +- Lists/tuples are represented by a pair of square brackets containing a + comma-separated sequence of values, like `["a", 15, true]`. + + List literals can be split into multiple lines for readability, but always + require a comma between values. A comma after the final value is allowed, + but not required. Values in a list can be arbitrary expressions. + +- Maps/objects are represented by a pair of curly braces containing a series of + ` = ` pairs: + + ```hcl + { + name = "John" + age = 52 + } + ``` + + Key/value pairs can be separated by either a comma or a line break. Values + can be arbitrary expressions. Keys are strings; they can be left unquoted if + they are a valid [identifier](/packer/docs/templates/hcl_templates/syntax#identifiers), but must be quoted + otherwise. You can use a non-literal expression as a key by wrapping it in + parentheses, like `(var.business_unit_tag_name) = "SRE"`. + +## References to Named Values + +Packer makes one named values available. + +The following named values are available: + +- `source..` is an object representing a + [source](/packer/docs/templates/hcl_templates/blocks/source) of the given type + and name. + +### Available Functions + +For a full list of available functions, see [the function +reference](/packer/docs/templates/hcl_templates/functions). + +## Conditional Expressions + +A conditional expression uses the value of a boolean expression to select one of +two values. This is a compact way to express `if-then-else` logic inside an +expression. + +The syntax for a conditional expression is: + +```hcl +condition ? true_val : false_val +``` + +If `condition` is `true`, the expression returns `true_val`. If `condition` is +`false`, it returns `false_val`. The `true_val` and `false_val` arguments must +be of compatible types. + +### Examples + +Here are some examples of how to use conditional expressions. + +1. Setting a Default Value + + You can use a conditional to provide a default value for a variable that + can also be customized. + + ```hcl + locals { + # Set the region to the value of var.region if it is not empty. + # Otherwise, use a default value. + region = var.region != "" ? var.region : "us-east-1" + } + ``` + +2. Dynamically Omitting an Argument + + You can use a `null` value to dynamically omit an argument from a source. + This is a common pattern to effectively "turn off" or prevent an argument + from being set. + + ```hcl + variable "use_http" { + type = bool + description = "Whether to use HTTP for file transfer." + default = true + } + + source "example" "foo" { + # The http_directory argument is only set if var.use_http is true. + # If false, http_directory will be unset. + http_directory = var.use_http ? "/path/to/files" : null + } + ``` + + This can also be used to set one of two mutually exclusive arguments: + + ```hcl + source "builder" "example" { + # If var.use_http is true, http_content is set and cd_content is null. + # If var.use_http is false, http_content is null and cd_content is set. + http_content = var.use_http ? local.http_files : null + cd_content = !var.use_http ? local.cd_files : null + } + ``` + +### Readability and Recommended Practices + +While powerful, conditional expressions can lead to complex or hard-to-read +configurations if not used carefully. + + * **Prioritize Clarity:** When a conditional expression becomes very long or + involves multiple nested conditions, consider breaking it down. + + * **Leverage `locals` for Complexity:** For more intricate conditional logic, + define the result in a `local` variable. This improves readability in the + main `source` or `provisioner` blocks where the `local` is consumed. + + **Example:** + + ```hcl + locals { + # Determine the kickstart file based on the build environment. + kickstart_config_path = var.environment == "production" ? + "config/prod-ks.cfg" : + (var.environment == "staging" ? + "config/stage-ks.cfg" : + "config/dev-ks.cfg") + } + + source "builder" "example" { + # ... + # Use the pre-determined kickstart path. + http_content = file(local.kickstart_config_path) + # ... + } + ``` + + * **Use `null` to Omit Arguments:** As demonstrated in the examples, + using `null` is the idiomatic way to prevent an argument from being set or + to effectively "turn off" a setting based on a condition. + +## `for` Expressions + +A _`for` expression_ creates a complex type value by transforming +another complex type value. Each element in the input value +can correspond to either one or zero values in the result, and an arbitrary +expression can be used to transform each input element into an output element. + +For example, if `var.list` is a list of strings, then the following expression +produces a list of strings with all-uppercase letters: + +```hcl +[for s in var.list : upper(s)] +``` + +This `for` expression iterates over each element of `var.list`, and then +evaluates the expression `upper(s)` with `s` set to each respective element. +It then builds a new tuple value with all of the results of executing that +expression in the same order. + +The type of brackets around the `for` expression decide what type of result +it produces. The above example uses `[` and `]`, which produces a tuple. If +`{` and `}` are used instead, the result is an object, and two result +expressions must be provided separated by the `=>` symbol: + +```hcl +{for s in var.list : s => upper(s)} +``` + +This expression produces an object whose attributes are the original elements +from `var.list` and their corresponding values are the uppercase versions. + +A `for` expression can also include an optional `if` clause to filter elements +from the source collection, which can produce a value with fewer elements than +the source: + +```text +[for s in var.list : upper(s) if s != ""] +``` + +The source value can also be an object or map value, in which case two +temporary variable names can be provided to access the keys and values +respectively: + +```text +[for k, v in var.map : length(k) + length(v)] +``` + +Finally, if the result type is an object (using `{` and `}` delimiters) then +the value result expression can be followed by the `...` symbol to group +together results that have a common key: + +```text +{for s in var.list : substr(s, 0, 1) => s... if s != ""} +``` + +## Splat Expressions + +A _splat expression_ provides a more concise way to express a common operation +that could otherwise be performed with a `for` expression. + +If `var.list` is a list of objects that all have an attribute `id`, then a list +of the IDs could be produced with the following `for` expression: + +```hcl +[for o in var.list : o.id] +``` + +This is equivalent to the following _splat expression:_ + +```hcl +var.list[*].id +``` + +The special `[*]` symbol iterates over all of the elements of the list given to +its left and accesses from each one the attribute name given on its right. A +splat expression can also be used to access attributes and indexes from lists +of complex types by extending the sequence of operations to the right of the +symbol: + +```hcl +var.list[*].interfaces[0].name +``` + +The above expression is equivalent to the following `for` expression: + +```hcl +[for o in var.list : o.interfaces[0].name] +``` + +Splat expressions are for lists only (and thus cannot be used [to reference +resources created with +`for_each`](/terraform/docs/configuration/resources#referring-to-instances), which +are represented as maps). However, if a splat expression is applied to a value +that is _not_ a list or tuple then the value is automatically wrapped in a +single-element list before processing. + +For example, `var.single_object[*].id` is equivalent to +`[var.single_object][*].id`, or effectively `[var.single_object.id]`. This +behavior is not interesting in most cases, but it is particularly useful when +referring to resources that may or may not have `count` set, and thus may or +may not produce a tuple value: + +```hcl +aws_instance.example[*].id +``` + +The above will produce a list of IDs whether `aws_instance.example` has `count` +set or not, avoiding the need to revise various other expressions in the +configuration when a particular resource switches to and from having `count` +set. + +## `dynamic` blocks + +Within top-level block constructs like [source](/packer/docs/templates/hcl_templates/blocks/source), expressions +can usually be used only when assigning a value to an argument using the `name = expression` or `key = expression` form. +This covers many uses, but some source types include repeatable _nested blocks_ in their arguments, which do not accept expressions: + +```hcl +source "amazon-ebs" "example" { + name = "pkr-test-name" # can use expressions here + + tag { + # but the "tag" block is always a literal block + } +} +``` + +You can dynamically construct repeatable nested blocks like `tag` using a +special `dynamic` block type, which is supported in top-level block constructs, example: + +```hcl +locals { + standard_tags = { + Component = "user-service" + Environment = "production" + } +} + +source "amazon-ebs" "example" { + # ... + + tag { + key = "Name" + value = "example-asg-name" + } + + dynamic "tag" { + for_each = local.standard_tags + + content { + key = tag.key + value = tag.value + } + } +} +``` + +A `dynamic` block acts much like a `for` expression, but produces nested blocks +instead of a complex typed value. It iterates over a given complex value, and +generates a nested block for each element of that complex value. + +- The label of the dynamic block (`"tag"` in the example above) specifies + what kind of nested block to generate. +- The `for_each` argument provides the complex value to iterate over. +- The `iterator` argument (optional) sets the name of a temporary variable + that represents the current element of the complex value. If omitted, the name + of the variable defaults to the label of the `dynamic` block (`"tag"` in + the example above). +- The `labels` argument (optional) is a list of strings that specifies the block + labels, in order, to use for each generated block. You can use the temporary + iterator variable in this value. +- The nested `content` block defines the body of each generated block. You can + use the temporary iterator variable inside this block. + +Since the `for_each` argument accepts any collection or structural value, +you can use a `for` expression or splat expression to transform an existing +collection. + +The iterator object (`tag` in the example above) has two attributes: + +- `key` is the map key or list element index for the current element. If the + `for_each` expression produces a _set_ value then `key` is identical to + `value` and should not be used. +- `value` is the value of the current element. + +A `dynamic` block can only generate arguments for nested blocks that belong to +the source type, data source, or provisioner being configured. + +The `for_each` value must be a map or set with one element per desired nested +block. If you need to declare resource instances based on a nested data +structure or combinations of elements from multiple data structures you can use +expressions and functions to derive a suitable value. For some common examples +of such situations, see the +[`flatten`](/packer/docs/templates/hcl_templates/functions/collection/flatten) and +[`setproduct`](/packer/docs/templates/hcl_templates/functions/collection/setproduct) +functions. + +### Best Practices for `dynamic` Blocks + +Overuse of `dynamic` blocks can make configuration hard to read and maintain, +so we recommend using them only when you need to hide details in order to build +a clean user interface for a re-usable code. Always write nested blocks out +literally where possible. + +## String Literals + +HCL has two different syntaxes for string literals. The +most common is to delimit the string with quote characters (`"`), like +`"hello"`. In quoted strings, the backslash character serves as an escape +sequence, with the following characters selecting the escape behavior: + +| Sequence | Replacement | +| ------------ | ----------------------------------------------------------------------------- | +| `\n` | Newline | +| `\r` | Carriage Return | +| `\t` | Tab | +| `\"` | Literal quote (without terminating the string) | +| `\\` | Literal backslash | +| `\uNNNN` | Unicode character from the basic multilingual plane (NNNN is four hex digits) | +| `\UNNNNNNNN` | Unicode character from supplementary planes (NNNNNNNN is eight hex digits) | + +The alternative syntax for string literals is the so-called Here Documents or +"heredoc" style, inspired by Unix shell languages. This style allows multi-line +strings to be expressed more clearly by using a custom delimiter word on a line +of its own to close the string: + +```hcl +< local.install_packages + +> #!/bin/bash +if [ 3 -eq 0 ]; then + echo "No packages to install." + exit 1 +fi +apt-get update + apt-get install -y git + apt-get install -y curl + apt-get install -y vim +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/alltrue.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/alltrue.mdx new file mode 100644 index 0000000000..498eacc238 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/alltrue.mdx @@ -0,0 +1,25 @@ +--- +page_title: alltrue function reference +description: |- + The `alltrue` function returns `true` if all elements in a collection + are `true`, `"true"`, or if the collection is empty. Learn how to use the `alltrue` function in Packer templates. +--- + +# `alltrue` Function + +`alltrue` returns `true` if all elements in a given collection are `true` +or `"true"`. It also returns `true` if the collection is empty. + +```hcl +alltrue(list) +``` + +## Examples + +```command +> alltrue(["true", true]) +true +> alltrue([true, false]) +false +``` + diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/anytrue.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/anytrue.mdx new file mode 100644 index 0000000000..020d6eb261 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/anytrue.mdx @@ -0,0 +1,28 @@ +--- +page_title: anytrue function reference +description: |- + The `anytrue` function returns `true` if any element of a collection + is `true` or `"true"`. Learn how to use the `anytrue` function in Packer templates. +--- + +# `anytrue` Function + +`anytrue` returns `true` if any element in a given collection is `true` +or `"true"`. It also returns `false` if the collection is empty. + +```hcl +anytrue(list) +``` + +## Examples + +```command +> anytrue(["true"]) +true +> anytrue([true]) +true +> anytrue([true, false]) +true +> anytrue([]) +false +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/chunklist.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/chunklist.mdx new file mode 100644 index 0000000000..cea1284422 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/chunklist.mdx @@ -0,0 +1,34 @@ +--- +page_title: chunklist function reference +description: |- + The `chunklist` function splits a single list into fixed-size chunks and returns + a list of lists. Learn how to use the `chunklist` function in Packer templates. +--- + +# `chunklist` Function + +`chunklist` splits a single list into fixed-size chunks, returning a list +of lists. + +```hcl +chunklist(list, chunk_size) +``` + +## Examples + +```shell-session +> chunklist(["a", "b", "c", "d", "e"], 2) +[ + ["a", "b"], + ["c", "d"], + ["e"] +] +> chunklist(["a", "b", "c", "d", "e"], 1) +[ + ["a"], + ["b"], + ["c"], + ["d"], + ["e"] +] +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/coalesce.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/coalesce.mdx new file mode 100644 index 0000000000..d18b20311b --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/coalesce.mdx @@ -0,0 +1,34 @@ +--- +page_title: coalesce function reference +description: |- + The `coalesce` function returns the first argument that is not empty or `null` from a list of arguments. Learn to use the `coalesce` function in Packer templates. +--- + +# `coalesce` Function + +`coalesce` takes any number of arguments and returns the first one +that isn't null or an empty string. + +## Examples + +```shell-session +> coalesce("a", "b") +a +> coalesce("", "b") +b +> coalesce(1,2) +1 +``` + +To perform the `coalesce` operation with a list of strings, use the `...` +symbol to expand the list as arguments: + +```shell-session +> coalesce(["", "b"]...) +b +``` + +## Related Functions + +- [`coalescelist`](/packer/docs/templates/hcl_templates/functions/collection/coalescelist) performs a similar operation with + list arguments rather than individual arguments. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/coalescelist.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/coalescelist.mdx new file mode 100644 index 0000000000..ea13ccb5c3 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/coalescelist.mdx @@ -0,0 +1,41 @@ +--- +page_title: coalescelist function reference +description: |- + The `coalescelist` function returns the first non-empty argument in a list of arguments. Learn how to use the `coalescelist` function in Packer templates. +--- + +# `coalescelist` Function + +`coalescelist` takes any number of list arguments and returns the first one +that isn't empty. + +## Examples + +```shell-session +> coalescelist(["a", "b"], ["c", "d"]) +[ + "a", + "b", +] +> coalescelist([], ["c", "d"]) +[ + "c", + "d", +] +``` + +To perform the `coalescelist` operation with a list of lists, use the `...` +symbol to expand the outer list as arguments: + +```shell-session +> coalescelist([[], ["c", "d"]]...) +[ + "c", + "d", +] +``` + +## Related Functions + +- [`coalesce`](/packer/docs/templates/hcl_templates/functions/collection/coalesce) performs a similar operation with string + arguments rather than list arguments. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/compact.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/compact.mdx new file mode 100644 index 0000000000..4938aace07 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/compact.mdx @@ -0,0 +1,20 @@ +--- +page_title: compact function reference +description: The `compact` function removes empty string elements from a list. Learn how to use the `compact` function in Packer templates. +--- + +# `compact` Function + +`compact` takes a list of strings and returns a new list with any empty string +elements removed. + +## Examples + +```shell-session +> compact(["a", "", "b", "c"]) +[ + "a", + "b", + "c", +] +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/concat.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/concat.mdx new file mode 100644 index 0000000000..365485fce5 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/concat.mdx @@ -0,0 +1,20 @@ +--- +page_title: concat function reference +description: The `concat` function combines two or more lists into a single list. Learn how to use the `concat` function in Packer templates. +--- + +# `concat` Function + +`concat` takes two or more lists and combines them into a single list. + +## Examples + +```shell-session +> concat(["a", ""], ["b", "c"]) +[ + "a", + "", + "b", + "c", +] +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/contains.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/contains.mdx new file mode 100644 index 0000000000..f9892128cf --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/contains.mdx @@ -0,0 +1,22 @@ +--- +page_title: contains function reference +description: The `contains` function determines whether a list or set contains a given value. Learn how to use the `contains` function in Packer templates. +--- + +# `contains` Function + +`contains` determines whether a given list or set contains a given single value +as one of its elements. + +```hcl +contains(list, value) +``` + +## Examples + +```shell-session +> contains(["a", "b", "c"], "a") +true +> contains(["a", "b", "c"], "d") +false +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/distinct.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/distinct.mdx new file mode 100644 index 0000000000..33dec1e1eb --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/distinct.mdx @@ -0,0 +1,24 @@ +--- +page_title: distinct function reference +description: The `distinct` function removes duplicate elements from a list. Learn how to use the `distince` function in Packer templates. +--- + +# `distinct` Function + +`distinct` takes a list and returns a new list with any duplicate elements +removed. + +The first occurrence of each value is retained and the relative ordering of +these elements is preserved. + +## Examples + +```shell-session +> distinct(["a", "b", "a", "c", "d", "b"]) +[ + "a", + "b", + "c", + "d", +] +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/element.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/element.mdx new file mode 100644 index 0000000000..178ecaf4d3 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/element.mdx @@ -0,0 +1,38 @@ +--- +page_title: element function reference +description: The `element` function retrieves a single element from a list. Learn how to use the `element` function in Packer templates. +--- + +# `element` Function + +`element` retrieves a single element from a list. + +```hcl +element(list, index) +``` + +The index is zero-based. This function produces an error if used with an +empty list. + +Use the built-in index syntax `list[index]` in most cases. Use this function +only for the special additional "wrap-around" behavior described below. + +## Examples + +```shell-session +> element(["a", "b", "c"], 1) +b +``` + +If the given index is greater than the length of the list then the index is +"wrapped around" by taking the index modulo the length of the list: + +```shell-session +> element(["a", "b", "c"], 3) +a +``` + +## Related Functions + +- [`index`](/packer/docs/templates/hcl_templates/functions/collection/index-fn) finds the index for a particular element value. +- [`lookup`](/packer/docs/templates/hcl_templates/functions/collection/lookup) retrieves a value from a _map_ given its _key_. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/flatten.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/flatten.mdx new file mode 100644 index 0000000000..9aca98a0b9 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/flatten.mdx @@ -0,0 +1,108 @@ +--- +page_title: flatten function reference +description: The `flatten` function eliminates nested lists from a list. Learn how to use the `flatten` function in Packer templates. +--- + +# `flatten` Function + +`flatten` takes a list and replaces any elements that are lists with a +flattened sequence of the list contents. + +## Examples + +```shell-session +> flatten([["a", "b"], [], ["c"]]) +["a", "b", "c"] +``` + +If any of the nested lists also contain directly-nested lists, these too are +flattened recursively: + +```shell-session +> flatten([[["a", "b"], []], ["c"]]) +["a", "b", "c"] +``` + +Indirectly-nested lists, such as those in maps, are _not_ flattened. + +## Flattening nested structures for `for_each` + +The +[resource `for_each`](/terraform/docs/configuration/resources#for_each-multiple-resource-instances-defined-by-a-map-or-set-of-strings) +and +[`dynamic` block](/packer/docs/templates/hcl_templates/expressions#dynamic-blocks) +language features both require a collection value that has one element for +each repetition. + +Sometimes your input data structure isn't naturally in a suitable shape for +use in a `for_each` argument, and `flatten` can be a useful helper function +when reducing a nested data structure into a flat one. + +For example, consider a folder that declares a variable like the following: + +```hcl +variable "networks" { + type = map(object({ + cidr_block = string + subnets = map(object({ + cidr_block = string + }) + }) +} +``` + +The above is a reasonable way to model objects that naturally form a tree, +such as top-level networks and their subnets. The repetition for the top-level +networks can use this variable directly, because it's already in a form +where the resulting instances match one-to-one with map elements: + +```hcl +resource "aws_vpc" "example" { + for_each = var.networks + + cidr_block = each.value.cidr_block +} +``` + +However, in order to declare all of the _subnets_ with a single `resource` +block, we must first flatten the structure to produce a collection where each +top-level element represents a single subnet: + +```hcl +locals { + # flatten ensures that this local value is a flat list of objects, rather + # than a list of lists of objects. + network_subnets = flatten([ + for network_key, network in var.networks : [ + for subnet_key, subnet in network.subnets : { + network_key = network_key + subnet_key = subnet_key + network_id = aws_vpc.example[network_key].id + cidr_block = subnet.cidr_block + } + ] + ]) +} + +resource "aws_subnet" "example" { + # local.network_subnets is a list, so we must now project it into a map + # where each key is unique. We'll combine the network and subnet keys to + # produce a single unique key per instance. + for_each = { + for subnet in local.network_subnets : "${subnet.network_key}.${subnet.subnet_key}" => subnet + } + + vpc_id = each.value.network_id + availability_zone = each.value.subnet_key + cidr_block = each.value_cidr_block +} +``` + +The above results in one subnet instance per subnet object, while retaining +the associations between the subnets and their containing networks. + +## Related Functions + +- [`setproduct`](/packer/docs/templates/hcl_templates/functions/collection/setproduct) finds all of the combinations of multiple + lists or sets of values, which can also be useful when preparing collections + for use with `for_each` constructs. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/index-fn.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/index-fn.mdx new file mode 100644 index 0000000000..e3d1343015 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/index-fn.mdx @@ -0,0 +1,27 @@ +--- +page_title: index function reference +description: The `index` function finds the element index for a given value in a list. Learn how to use the `index` function in Packer templates. +--- + +# `index` Function + +`index` finds the element index for a given value in a list. + +```hcl +index(list, value) +``` + +The returned index is zero-based. This function produces an error if the given +value is not present in the list. + +## Examples + +```shell-session +> index(["a", "b", "c"], "b") +1 +``` + +## Related Functions + +- [`element`](/packer/docs/templates/hcl_templates/functions/collection/element) retrieves a particular element from a list given + its index. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/keys.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/keys.mdx new file mode 100644 index 0000000000..5c91613892 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/keys.mdx @@ -0,0 +1,26 @@ +--- +page_title: keys function reference +description: The `keys` function returns a list of keys in a given map. Learn how to use the `key` function in Packer templates. +--- + +# `keys` Function + +`keys` takes a map and returns a list containing the keys from that map. + +The keys are returned in lexicographical order, ensuring that the result will +be identical as long as the keys in the map don't change. + +## Examples + +```shell-session +> keys({a=1, c=2, d=3}) +[ + "a", + "c", + "d", +] +``` + +## Related Functions + +- [`values`](/packer/docs/templates/hcl_templates/functions/collection/values) returns a list of the _values_ from a map. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/length.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/length.mdx new file mode 100644 index 0000000000..8fafc7c25a --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/length.mdx @@ -0,0 +1,39 @@ +--- +page_title: length function reference +description: The `length` function determines the length of a collection or string. Learn how to use the `length` function in Packer templates. +--- + +# `length` Function + +`length` determines the length of a given list, map, or string. + +If given a list or map, the result is the number of elements in that collection. +If given a string, the result is the number of characters in the string. + +## Examples + +```shell-session +> length([]) +0 +> length(["a", "b"]) +2 +> length({"a" = "b"}) +1 +> length("hello") +5 +``` + +When given a string, the result is the number of characters, rather than the +number of bytes or Unicode sequences that form them: + +```shell-session +> length("👾🕹️") +2 +``` + +A "character" is a _grapheme cluster_, as defined by +[Unicode Standard Annex #29](http://unicode.org/reports/tr29/). Note that +remote APIs may have a different definition of "character" for the purpose of +length limits on string arguments; a Packer provider is responsible for +translating Packer's string representation into that used by its respective +remote system and applying any additional validation rules to it. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/lookup.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/lookup.mdx new file mode 100644 index 0000000000..f56f7dcb4d --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/lookup.mdx @@ -0,0 +1,26 @@ +--- +page_title: lookup function reference +description: The `lookup` function retrieves an element value from a map using the map key. Learn how to use the `lookup` function in Packer templates. +--- + +# `lookup` Function + +`lookup` retrieves the value of a single element from a map, given its key. +If the given key does not exist, the given default value is returned instead. + +```hcl +lookup(map, key, default) +``` + +## Examples + +```shell-session +> lookup({a="ay", b="bee"}, "a", "what?") +ay +> lookup({a="ay", b="bee"}, "c", "what?") +what? +``` + +## Related Functions + +- [`element`](/packer/docs/templates/hcl_templates/functions/collection/element) retrieves a value from a _list_ given its _index_. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/merge.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/merge.mdx new file mode 100644 index 0000000000..19c24d5cb5 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/merge.mdx @@ -0,0 +1,24 @@ +--- +page_title: merge function reference +description: |- + The `merge` function returns a single map by merging the keys in multiple maps. Learn how to use the `merge` function in Packer templates. +--- + +# `merge` Function + +`merge` takes an arbitrary number of maps and returns a single map that +contains a merged set of elements from all of the maps. + +If more than one given map defines the same key then the one that is later +in the argument sequence takes precedence. + +## Examples + +```shell-session +> merge({"a"="b", "c"="d"}, {"e"="f", "c"="z"}) +{ + "a" = "b" + "c" = "z" + "e" = "f" +} +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/range.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/range.mdx new file mode 100644 index 0000000000..fe326dd5a5 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/range.mdx @@ -0,0 +1,139 @@ +--- +page_title: range function reference +description: The `range` function generates sequences of numbers. Learn how to use the `range` function in Packer templates. +--- + +# `range` Function + +`range` generates a list of numbers using a start value, a limit value, +and a step value. + +```hcl +range(max) +range(start, limit) +range(start, limit, step) +``` + +The `start` and `step` arguments can be omitted, in which case `start` defaults +to zero and `step` defaults to either one or negative one depending on whether +`limit` is greater than or less than `start`. + +The resulting list is created by starting with the given `start` value and +repeatedly adding `step` to it until the result is equal to or beyond `limit`. + +The interpretation of `limit` depends on the direction of `step`: for a positive +step, the sequence is complete when the next number is greater than or equal +to `limit`. For a negative step, it's complete when less than or equal. + +The sequence-building algorithm follows the following pseudocode: + +```text +let num = start +while num < limit: (or, for negative step, num >= limit) + append num to the sequence + num = num + step +return the sequence +``` + +Because the sequence is created as a physical list in memory, Packer imposes +an artificial limit of 1024 numbers in the resulting sequence in order to avoid +unbounded memory usage if, for example, a very large value were accidentally +passed as the limit or a very small value as the step. If the algorithm above +would append the 1025th number to the sequence, the function immediately exits +with an error. + +We recommend iterating over existing collections where possible, rather than +creating ranges. However, creating small numerical sequences can sometimes +be useful when combined with other collections in collection-manipulation +functions or `for` expressions. + +## Examples + +```shell-session +> range(3) +[ + 0, + 1, + 2, +] + +> range(1, 4) +[ + 1, + 2, + 3, +] + +> range(1, 8, 2) +[ + 1, + 3, + 5, + 7, +] + +> range(1, 4, 0.5) +[ + 1, + 1.5, + 2, + 2.5, + 3, + 3.5, +] + +> range(4, 1) +[ + 4, + 3, + 2, +] + +> range(10, 5, -2) +[ + 10, + 8, + 6, +] +``` + +The `range` function is primarily useful when working with other collections +to produce a certain number of instances of something. For example: + +```hcl +variable "name_counts" { + type = map(number) + default = { + "foo" = 2 + "bar" = 4 + } +} + +locals { + expanded_names = { + for name, count in var.name_counts : name => [ + for i in range(count) : format("%s%02d", name, i) + ] + } +} + +output "expanded_names" { + value = local.expanded_names +} + +# Produces the following expanded_names value when run with the default +# "name_counts": +# +# { +# "bar" = [ +# "bar00", +# "bar01", +# "bar02", +# "bar03", +# ] +# "foo" = [ +# "foo00", +# "foo01", +# ] +# } +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/reverse.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/reverse.mdx new file mode 100644 index 0000000000..b844e6a718 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/reverse.mdx @@ -0,0 +1,24 @@ +--- +page_title: reverse function reference +description: The `reverse` function reverses the values in a sequence. Learn how to use the `reverse` function in Packer templates. +--- + +# `reverse` Function + +`reverse` takes a sequence and produces a new sequence of the same length +with all of the same elements as the given sequence but in reverse order. + +## Examples + +```shell-session +> reverse([1, 2, 3]) +[ + 3, + 2, + 1, +] +``` + +## Related Functions + +- [`strrev`](/packer/docs/templates/hcl_templates/functions/string/strrev) reverses a string. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/setintersection.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/setintersection.mdx new file mode 100644 index 0000000000..94411508af --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/setintersection.mdx @@ -0,0 +1,36 @@ +--- +page_title: setintersection function reference +description: |- + The `setintersection` function creates one set from multiple sets that contains only elements common to all input sets. Learn how to use `setintersection` in Packer templates. +--- + +# `setintersection` Function + +The `setintersection` function takes multiple sets and produces a single set +containing only the elements that all of the given sets have in common. +The new set is referred to as the _intersection_. + +```hcl +setintersection(sets...) +``` + +## Examples + +```shell-session +> setintersection(["a", "b"], ["b", "c"], ["b", "d"]) +[ + "b", +] +``` + +The given arguments are converted to sets, so the result is also a set and +the ordering of the given elements is not preserved. + +## Related Functions + +- [`contains`](/packer/docs/templates/hcl_templates/functions/collection/contains) tests whether a given list or set contains + a given element value. +- [`setproduct`](/packer/docs/templates/hcl_templates/functions/collection/setproduct) computes the _Cartesian product_ of multiple + sets. +- [`setunion`](/packer/docs/templates/hcl_templates/functions/collection/setunion) computes the _union_ of + multiple sets. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/setproduct.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/setproduct.mdx new file mode 100644 index 0000000000..6ccea5cfcb --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/setproduct.mdx @@ -0,0 +1,222 @@ +--- +page_title: setproduct function reference +description: |- + The `setproduct` function finds all possible combinations of elements + from the given sets. Learn how to use the `setproduct` function in Packer templates. +--- + +# `setproduct` Function + +The `setproduct` function finds all of the possible combinations of elements +from all of the given sets by computing the +[Cartesian product](https://en.wikipedia.org/wiki/Cartesian_product). + +```hcl +setproduct(sets...) +``` + +This function is particularly useful for finding the exhaustive set of all +combinations of members of multiple sets, such as per-application-per-environment +resources. + +```shell-session +> setproduct(["development", "staging", "production"], ["app1", "app2"]) +[ + [ + "development", + "app1", + ], + [ + "development", + "app2", + ], + [ + "staging", + "app1", + ], + [ + "staging", + "app2", + ], + [ + "production", + "app1", + ], + [ + "production", + "app2", + ], +] +``` + +You must past at least two arguments to this function. + +Although defined primarily for sets, this function can also work with lists. +If all of the given arguments are lists then the result is a list, preserving +the ordering of the given lists. Otherwise the result is a set. In either case, +the result's element type is a list of values corresponding to each given +argument in turn. + +## Examples + +There is an example of the common usage of this function above. There are some +other situations that are less common when hand-writing but may arise in +reusable folder situations. + +If any of the arguments is empty then the result is always empty itself, +similar to how multiplying any number by zero gives zero: + +```shell-session +> setproduct(["development", "staging", "production"], []) +[] +``` + +Similarly, if all of the arguments have only one element then the result has +only one element, which is the first element of each argument: + +```shell-session +> setproduct(["a"], ["b"]) +[ + [ + "a", + "b", + ], +] +``` + +Each argument must have a consistent type for all of its elements. If not, +Packer will attempt to convert to the most general type, or produce an +error if such a conversion is impossible. For example, mixing both strings and +numbers results in the numbers being converted to strings so that the result +elements all have a consistent type: + +```shell-session +> setproduct(["staging", "production"], ["a", 2]) +[ + [ + "staging", + "a", + ], + [ + "staging", + "2", + ], + [ + "production", + "a", + ], + [ + "production", + "2", + ], +] +``` + +## Finding combinations for `for_each` + +The +[resource `for_each`](/terraform/docs/configuration/resources#for_each-multiple-resource-instances-defined-by-a-map-or-set-of-strings) +and +[`dynamic` block](/packer/docs/templates/hcl_templates/expressions/#dynamic-blocks) +language features both require a collection value that has one element for +each repetition. + +Sometimes your input data comes in separate values that cannot be directly +used in a `for_each` argument, and `setproduct` can be a useful helper function +for the situation where you want to find all unique combinations of elements in +a number of different collections. + +For example, consider a folder that declares variables like the following: + +```hcl +variable "networks" { + type = map(object({ + base_cidr_block = string + })) +} + +variable "subnets" { + type = map(object({ + number = number + })) +} +``` + +If the goal is to create each of the defined subnets per each of the defined +networks, creating the top-level networks can directly use `var.networks` +because it's already in a form where the resulting instances match one-to-one +with map elements: + +```hcl +resource "aws_vpc" "example" { + for_each = var.networks + + cidr_block = each.value.base_cidr_block +} +``` + +However, in order to declare all of the _subnets_ with a single `resource` +block, we must first produce a collection whose elements represent all of +the combinations of networks and subnets, so that each element itself +represents a subnet: + +```hcl +locals { + # setproduct works with sets and lists, but our variables are both maps + # so we'll need to convert them first. + networks = [ + for key, network in var.networks : { + key = key + cidr_block = network.cidr_block + } + ] + subnets = [ + for key, subnet in var.subnets : { + key = key + number = subnet.number + } + ] + + network_subnets = [ + # in pair, element zero is a network and element one is a subnet, + # in all unique combinations. + for pair in setproduct(local.networks, local.subnets) : { + network_key = pair[0].key + subnet_key = pair[1].key + network_id = aws_vpc.example[pair[0].key].id + + # The cidr_block is derived from the corresponding network. See the + # cidrsubnet function for more information on how this calculation works. + cidr_block = cidrsubnet(pair[0].cidr_block, 4, pair[1].number) + } + ] +} + +resource "aws_subnet" "example" { + # local.network_subnets is a list, so we must now project it into a map + # where each key is unique. We'll combine the network and subnet keys to + # produce a single unique key per instance. + for_each = { + for subnet in local.network_subnets : "${subnet.network_key}.${subnet.subnet_key}" => subnet + } + + vpc_id = each.value.network_id + availability_zone = each.value.subnet_key + cidr_block = each.value_cidr_block +} +``` + +The above results in one subnet instance per combination of network and subnet +elements in the input variables. + +## Related Functions + +- [`contains`](/packer/docs/templates/hcl_templates/functions/collection/contains) tests whether a given list or set contains + a given element value. +- [`flatten`](/packer/docs/templates/hcl_templates/functions/collection/flatten) is useful for flattening hierarchical data + into a single list, for situations where the relationships between two + object types are defined explicitly. +- [`setintersection`](/packer/docs/templates/hcl_templates/functions/collection/setintersection) computes the _intersection_ of + multiple sets. +- [`setunion`](/packer/docs/templates/hcl_templates/functions/collection/setunion) computes the _union_ of multiple + sets. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/setunion.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/setunion.mdx new file mode 100644 index 0000000000..1a429627dc --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/setunion.mdx @@ -0,0 +1,40 @@ +--- +page_title: setunion function reference +description: |- + The `setunion` function creates one set containing the elements from multiple given sets. Learn how to use the `setunion` function in Packer templates. +--- + +# `setunion` Function + +The `setunion` function takes multiple sets and produces a single set +containing the elements from all of the given sets. In other words, it +computes the [union]() of +the sets. + +```hcl +setunion(sets...) +``` + +## Examples + +```shell-session +> setunion(["a", "b"], ["b", "c"], ["d"]) +[ + "d", + "b", + "c", + "a", +] +``` + +The given arguments are converted to sets, so the result is also a set and +the ordering of the given elements is not preserved. + +## Related Functions + +- [`contains`](/packer/docs/templates/hcl_templates/functions/collection/contains) tests whether a given list or set contains + a given element value. +- [`setintersection`](/packer/docs/templates/hcl_templates/functions/collection/setintersection) computes the _intersection_ of + multiple sets. +- [`setproduct`](/packer/docs/templates/hcl_templates/functions/collection/setproduct) computes the _Cartesian product_ of multiple + sets. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/slice.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/slice.mdx new file mode 100644 index 0000000000..f230f4351f --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/slice.mdx @@ -0,0 +1,31 @@ +--- +page_title: slice function reference +description: The `slice` function extracts consecutive elements from a list. Learn how to use the `slice` function in Packer templates. +--- + +# `slice` Function + +`slice` extracts some consecutive elements from within a list. + +```hcl +slice(list, startindex, endindex) +``` + +`startindex` is inclusive, while `endindex` is exclusive. This function returns +an error if either index is outside the bounds of valid indices for the given +list. + +## Examples + +```shell-session +> slice(["a", "b", "c", "d"], 1, 3) +[ + "b", + "c", +] +``` + +## Related Functions + +- [`substr`](/packer/docs/templates/hcl_templates/functions/string/substr) performs a similar function for characters in a + string, although it uses a length instead of an end index. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/sort.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/sort.mdx new file mode 100644 index 0000000000..78a55cbe33 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/sort.mdx @@ -0,0 +1,25 @@ +--- +page_title: sort function reference +description: |- + The `sort` function returns a new lexicographically sorted list from a list of strings. Learn how to use the `sort` function in Packer templates. +--- + +# `sort` Function + +`sort` takes a list of strings and returns a new list with those strings +sorted lexicographically. + +The sort is in terms of Unicode codepoints, with higher codepoints appearing +after lower ones in the result. + +## Examples + +```shell-session +> sort(["e", "d", "a", "x"]) +[ + "a", + "d", + "e", + "x", +] +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/values.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/values.mdx new file mode 100644 index 0000000000..445cfdba48 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/values.mdx @@ -0,0 +1,28 @@ +--- +page_title: values function reference +description: The `values` function returns a list of the element values in a given map. Learn how to use the `values` function in Packer templates. +--- + +# `values` Function + +`values` takes a map and returns a list containing the values of the elements +in that map. + +The values are returned in lexicographical order by their corresponding _keys_, +so the values will be returned in the same order as their keys would be +returned from [`keys`](/packer/docs/templates/hcl_templates/functions/collection/keys). + +## Examples + +```shell-session +> values({a=3, c=2, d=1}) +[ + 3, + 2, + 1, +] +``` + +## Related Functions + +- [`keys`](/packer/docs/templates/hcl_templates/functions/collection/keys) returns a list of the _keys_ from a map. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/collection/zipmap.mdx b/content/packer/content/docs/templates/hcl_templates/functions/collection/zipmap.mdx new file mode 100644 index 0000000000..a3ae795534 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/collection/zipmap.mdx @@ -0,0 +1,33 @@ +--- +page_title: zipmap function reference +description: |- + The `zipmap` function constructs a map from a list of keys and a corresponding + list of values. Learn how to use the `zipmap` function in Packer templates. +--- + +# `zipmap` Function + +`zipmap` constructs a map from a list of keys and a corresponding list of +values. + +```hcl +zipmap(keyslist, valueslist) +``` + +Both `keyslist` and `valueslist` must be of the same length. `keyslist` must +be a list of strings, while `valueslist` can be a list of any type. + +Each pair of elements with the same index from the two lists will be used +as the key and value of an element in the resulting map. If the same value +appears multiple times in `keyslist` then the value with the highest index +is used in the resulting map. + +## Examples + +```shell-session +> zipmap(["a", "b"], [1, 2]) +{ + "a" = 1, + "b" = 2, +} +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/contextual/aws_secretsmanager.mdx b/content/packer/content/docs/templates/hcl_templates/functions/contextual/aws_secretsmanager.mdx new file mode 100644 index 0000000000..543ca5ffe5 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/contextual/aws_secretsmanager.mdx @@ -0,0 +1,91 @@ +--- +page_title: aws_secretsmanager function reference +description: >- + The `aws_secretsmanager` function retrieves secrets from Amazon Secrets Manager + stores. Learn how to use the `aws_secretsmanager` function in Packer templates. +--- + +# `aws_secretsmanager` Function + +Secrets can be read from the [AWS Secrets +Manager](https://aws.amazon.com/secrets-manager/) and used within your template +as locals. + +~> Note: Support for AWS secrets will always obtain the latest version of a secret, essentially +AWSCURRENT. Support for previous versions of a secret is not supported. + +```hcl +aws_secretsmanager(name, key) +``` + +When key is not set (`null` or empty: `""`) then `aws_secretsmanager` returns +the first secret key stored in secret `name`. + +You can either use this function in a `locals` block or directly inline where +you want to use the value. + +```hcl +locals { + secret = aws_secretsmanager("my_secret", null) +} + +source "null" "first-example" { + communicator = "none" +} + +build { + name = "my-build-name" + sources = ["null.first-example"] + + provisioner "shell-local" { + environment_vars = ["TESTVAR=${build.PackerRunUUID}"] + inline = ["echo my_secret is '${local.secret}'", + "echo my_secret using an inline call is '${aws_secretsmanager("my_secret", null)}'."] + } +} +``` + +This will load the key stored behind `my_secret` from aws secrets manager. + +The retrieval of single key secrets or plaintext secrets can be obtained by specifying (`null` or empty: `""`) as the `key`. + +When obtaining secrets that have multiple keys you can set `key` to the specific key you would like +to fetch. For example, given the following secret with two keys if `key` is set to "shell" `aws_secretsmanager` will +return only its value. + +```json +{ + "test": "kitchen", + "shell": "powershell" +} +``` + +```hcl +locals { + secret = aws_secretsmanager("multikey/secret", "shell") +} + +source "null" "first-example" { + communicator = "none" +} + +build { + name = "my-build-name" + sources = ["null.first-example"] + + provisioner "shell-local" { + environment_vars = ["TESTVAR=${build.PackerRunUUID}"] + inline = ["echo my_secret is '${local.secret}'"] + } +} +``` + +This will load the value `"powershell"` stored in the key `"shell"` behind `multikey/secret`. + +In order to use this function you have to configure valid AWS credentials using +one of the following methods: + +- [Environment Variables](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html) +- [CLI Configuration Files](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html) +- [Container Credentials](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html) +- [Instance Profile Credentials](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html) diff --git a/content/packer/content/docs/templates/hcl_templates/functions/contextual/consul.mdx b/content/packer/content/docs/templates/hcl_templates/functions/contextual/consul.mdx new file mode 100644 index 0000000000..9c47185a24 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/contextual/consul.mdx @@ -0,0 +1,38 @@ +--- +page_title: consul function reference +description: The `consul` function retrieves secrets from HashiCorp Consul KV stores. Learn how to use the `consul` function in Packer templates. +--- + +# `consul_key` Function + +Retrieves secrets from a HashiCorp Consul KV store. You can use the `consul_key` function secrets to add them to your template. Refer to the [Consul documentation](/consul/docs/dynamic-app-config/kv) for additional information about the Consul KV. + +You can either use this function in a locals block or directly inline where you +want to use the value. + +```hcl +locals { + my_version = "${consul_key("myservice/version")}" +} + +source "null" "first-example" { + communicator = "none" +} + +build { + name = "my-build-name" + sources = ["null.first-example"] + + provisioner "shell-local" { + environment_vars = ["TESTVAR=${build.PackerRunUUID}"] + inline = ["echo my_version is '${local.my_version}'", + "echo version is '${consul_key("myservice/version")}'."] + } +} +``` + +This will load the key stored at the path `myservice/version` from consul. + +The configuration for consul (address, tokens, ...) must be specified as +environment variables, as specified in the +[Documentation](/consul/commands#environment-variables). diff --git a/content/packer/content/docs/templates/hcl_templates/functions/contextual/env.mdx b/content/packer/content/docs/templates/hcl_templates/functions/contextual/env.mdx new file mode 100644 index 0000000000..aa7696e270 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/contextual/env.mdx @@ -0,0 +1,48 @@ +--- +page_title: env function reference +description: The `env` function retrieves environment values for input variables. Learn how to use the `env` function in Packer templates. +--- + +# `env` Function + +The `env` function gets the value for an environment variable inside input +variables. This is the only function that is callable from a `variable` +block. You can only use the `env` function in the default input. + +```hcl +variable "aws_region" { + default = env("AWS_DEFAULT_REGION") +} +``` + +In the previous example, the value of `aws_region` will be what's stored in the +`AWS_DEFAULT_REGION` env var, unless aws_region is also set in a [manner that takes +precedence](/packer/docs/templates/hcl_templates/variables#variable-definition-precedence). + +-> **Why can't I use environment variables elsewhere?** User variables are the +single source of configurable input. We felt that having environment variables +used _anywhere_ in a configuration would confuse the user about the possible inputs +to a template. By allowing environment variables only within default values for +input variables, input variables remain as the single source of input to a +template that a user can easily discover using `packer inspect`. + +When the environment variable is not set at all -- not even with the empty +string -- the value returned by `env` will be an empty string. It will still +be possible to set it using other means but you could use [custom validation +rules](/packer/docs/templates/hcl_templates/variables#custom-validation-rules) to error in that case +to make sure it is set, for example: + +```hcl +variable "aws_region" { + default = env("AWS_DEFAULT_REGION") + + validation { + condition = length(var.aws_region) > 0 + error_message = <`. See [Local Values](/packer/docs/templates/hcl_templates/locals) for more details. + + +## Usage + +In order for the Vault function to work, you must set the environment variables `VAULT_TOKEN` +and `VAULT_ADDR` to valid values. + +-> **NOTE:** HCL functions can be used in local variable definitions or inline +with a provisioner/post-processor. They cannot be used in global variable definitions. + +The api tool we use allows for more custom configuration of the Vault client via +environment variables. + +The full list of available environment variables is: + +```text +"VAULT_ADDR" +"VAULT_AGENT_ADDR" +"VAULT_CACERT" +"VAULT_CAPATH" +"VAULT_CLIENT_CERT" +"VAULT_CLIENT_KEY" +"VAULT_CLIENT_TIMEOUT" +"VAULT_SKIP_VERIFY" +"VAULT_NAMESPACE" +"VAULT_TLS_SERVER_NAME" +"VAULT_WRAP_TTL" +"VAULT_MAX_RETRIES" +"VAULT_TOKEN" +"VAULT_MFA" +"VAULT_RATE_LIMIT" +``` + +and detailed documentation for usage of each of those variables can be found +[here](/vault/docs/commands#environment-variables). diff --git a/content/packer/content/docs/templates/hcl_templates/functions/conversion/can.mdx b/content/packer/content/docs/templates/hcl_templates/functions/conversion/can.mdx new file mode 100644 index 0000000000..8115f6ccdc --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/conversion/can.mdx @@ -0,0 +1,56 @@ +--- +page_title: can function reference +description: |- + The `can` function evaluates an expression and indicates whether it succeeds. +--- + +# `can` Function + +`can` evaluates the given expression and returns a boolean value indicating +whether the expression produced a result without any errors. + +This is a special function that is able to catch errors produced when evaluating +its argument. For most situations where you could use `can` it's better to use +[`try`](/packer/docs/templates/hcl_templates/functions/conversion/try) instead, because it allows for more concise definition of +fallback values for failing expressions. + +The `can` function can only catch and handle _dynamic_ errors resulting from +access to data that isn't known until runtime. It will not catch errors +relating to expressions that can be proven to be invalid for any input, such +as a malformed resource reference. + +~> **Warning:** The `can` function is intended only for simple tests in +variable validation rules. Although it can technically accept any sort of +expression and be used elsewhere in the configuration, we recommend against +using it in other contexts. For error handling elsewhere in the configuration, +prefer to use [`try`](/packer/docs/templates/hcl_templates/functions/conversion/try). + +## Examples + +```shell-session +> local.foo +{ + "bar" = "baz" +} +> can(local.foo.bar) +true +> can(local.foo.boop) +false +``` + +The `can` function will _not_ catch errors relating to constructs that are +provably invalid even before dynamic expression evaluation, such as a malformed +reference or a reference to a top-level object that has not been declared: + +```shell-session +> can(local.nonexist) + +Error: Reference to undeclared local value + +A local value with the name "nonexist" has not been declared. +``` + +## Related Functions + +- [`try`](/packer/docs/templates/hcl_templates/functions/conversion/try), which tries evaluating a sequence of expressions and + returns the result of the first one that succeeds. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/conversion/convert.mdx b/content/packer/content/docs/templates/hcl_templates/functions/conversion/convert.mdx new file mode 100644 index 0000000000..952126f6cb --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/conversion/convert.mdx @@ -0,0 +1,52 @@ +--- +page_title: convert function reference +description: The `convert` function converts a value or an expression to a given type. +--- + +# `convert` Function + +`convert` converts a value or an expression to a given type. + +Explicit type conversions are rarely necessary in HCL because it will convert +types automatically where required. Use the explicit type conversion functions +only to normalize types returned in outputs. + +Only numbers and strings containing decimal representations of numbers can be +converted to number. All other values will produce an error. + +Only boolean values and the exact strings "true" and "false" can be converted +to boolean. All other values will produce an error. + +Only the primitive types (string, number, and bool) can be converted to string. +All other values will produce an error. + +`convert(value, type_constraint)` + +## Examples + +```shell-session +> convert(3, string) +"3" +> convert("3", number) +3 +> convert("false", bool) +false +> convert(false, string) +"false" +> convert(["a", "b", 3], list) +[ + "a", + "b", + "3", +] +> convert(["c", "b", "b"], set) +[ + "b", + "c", +] +> convert({"a" = "foo", "b" = true}, map) +{ + "a" = "foo" + "b" = "true" +} +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/conversion/try.mdx b/content/packer/content/docs/templates/hcl_templates/functions/conversion/try.mdx new file mode 100644 index 0000000000..94e70d0ad5 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/conversion/try.mdx @@ -0,0 +1,109 @@ +--- +page_title: try function reference +description: |- + The `try` function evaluates a sequence of expressions and returns the result of the first expression that does not produce any errors. +--- + +# `try` Function + +`try` evaluates all of its argument expressions in turn and returns the result +of the first one that does not produce any errors. + +This is a special function that is able to catch errors produced when evaluating +its arguments, which is particularly useful when working with complex data +structures whose shape is not well-known at implementation time. + +For example, if some data is retrieved from an external system in JSON or YAML +format and then decoded, the result may have attributes that are not guaranteed +to be set. We can use `try` to produce a normalized data structure which has +a predictable type that can therefore be used more conveniently elsewhere in +the configuration: + +```hcl +locals { + raw_value = yamldecode("${path.folder}/example.yaml") + normalized_value = { + name = tostring(try(local.raw_value.name, null)) + groups = try(local.raw_value.groups, []) + } +} +``` + +With the above local value expressions, configuration elsewhere in the folder +can refer to `local.normalized_value` attributes without the need to repeatedly +check for and handle absent attributes that would otherwise produce errors. + +We can also use `try` to deal with situations where a value might be provided +in two different forms, allowing us to normalize to the most general form: + +```hcl +variable "example" { + type = any +} + +locals { + example = try( + [tostring(var.example)], + tolist(var.example), + ) +} +``` + +The above permits `var.example` to be either a list or a single string. If it's +a single string then it'll be normalized to a single-element list containing +that string, again allowing expressions elsewhere in the configuration to just +assume that `local.example` is always a list. + +This second example contains two expressions that can both potentially fail. +For example, if `var.example` were set to `{}` then it could be converted to +neither a string nor a list. If `try` exhausts all of the given expressions +without any succeeding, it will return an error describing all of the problems +it encountered. + +We strongly suggest using `try` only in special local values whose expressions +perform normalization, so that the error handling is confined to a single +location in the folder and the rest of the folder can just use straightforward +references to the normalized structure and thus be more readable for future +maintainers. + +The `try` function can only catch and handle _dynamic_ errors resulting from +access to data that isn't known until runtime. It will not catch errors +relating to expressions that can be proven to be invalid for any input, such +as a malformed resource reference. + +~> **Warning:** The `try` function is intended only for concise testing of the +presence of and types of object attributes. Although it can technically accept +any sort of expression, we recommend using it only with simple attribute +references and type conversion functions as shown in the examples above. +Overuse of `try` to suppress errors will lead to a configuration that is hard +to understand and maintain. + +## Examples + +```shell-session +> local.foo +{ + "bar" = "baz" +} +> try(local.foo.bar, "fallback") +baz +> try(local.foo.boop, "fallback") +fallback +``` + +The `try` function will _not_ catch errors relating to constructs that are +provably invalid even before dynamic expression evaluation, such as a malformed +reference or a reference to a top-level object that has not been declared: + +```shell-session +> try(local.nonexist, "fallback") + +Error: Reference to undeclared local value + +A local value with the name "nonexist" has not been declared. +``` + +## Related Functions + +- [`can`](/packer/docs/templates/hcl_templates/functions/conversion/can), which tries evaluating an expression and returns a + boolean value indicating whether it succeeded. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/crypto/bcrypt.mdx b/content/packer/content/docs/templates/hcl_templates/functions/crypto/bcrypt.mdx new file mode 100644 index 0000000000..f7e6ceff08 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/crypto/bcrypt.mdx @@ -0,0 +1,36 @@ +--- +page_title: bcrypt function reference +description: |- + The `bcrypt` function computes a hash of the given string using the Blowfish + cipher. Learn how to use the `bcrypt` function in Packer templates. +--- + +# `bcrypt` Function + +`bcrypt` computes a hash of the given string using the Blowfish cipher, +returning a string in [the _Modular Crypt +Format_](https://passlib.readthedocs.io/en/stable/modular_crypt_format.html) +usually expected in the shadow password file on many Unix systems. + +```hcl +bcrypt(string, cost) +``` + +The `cost` argument is optional and will default to 10 if unspecified. + +Since a bcrypt hash value includes a randomly selected salt, each call to this +function will return a different value, even if the given string and cost are +the same. Using this function directly with resource arguments will therefore +cause spurious diffs. We recommend using this function only in `provisioner` or +`post-processor` blocks, or in data resources whose results are only used in +those blocks. + +The version prefix on the generated string (e.g. `$2a$`) may change in future +versions of Packer. + +## Examples + +```shell-session +> bcrypt("hello world") +$2a$10$D5grTTzcsqyvAeIAnY/mYOIqliCoG7eAMX0/oFcuD.iErkksEbcAa +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/crypto/md5.mdx b/content/packer/content/docs/templates/hcl_templates/functions/crypto/md5.mdx new file mode 100644 index 0000000000..0db691adf9 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/crypto/md5.mdx @@ -0,0 +1,26 @@ +--- +page_title: md5 function reference +description: |- + The `md5` function computes the MD5 hash of a given string and encodes it + with hexadecimal digits. Learn how to use the `md5` function in Packer templates. +--- + +# `md5` Function + +`md5` computes the MD5 hash of a given string and encodes it with +hexadecimal digits. + +The given string is first encoded as UTF-8 and then the MD5 algorithm is applied +as defined in [RFC 1321](https://tools.ietf.org/html/rfc1321). The raw hash is +then encoded to lowercase hexadecimal digits before returning. + +Before using this function for anything security-sensitive, refer to +[RFC 6151](https://tools.ietf.org/html/rfc6151) for updated security +considerations applying to the MD5 algorithm. + +## Examples + +```shell-session +> md5("hello world") +5eb63bbbe01eeed093cb22bb8f5acdc3 +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/crypto/rsadecrypt.mdx b/content/packer/content/docs/templates/hcl_templates/functions/crypto/rsadecrypt.mdx new file mode 100644 index 0000000000..7e4a5ad843 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/crypto/rsadecrypt.mdx @@ -0,0 +1,31 @@ +--- +page_title: rsadecrypt function reference +description: The `rsadecrypt` function decrypts an RSA-encrypted message. Learn how to usethe `rsadecrypt` function in Packer templates. +--- + +# `rsadecrypt` Function + +`rsadecrypt` decrypts an RSA-encrypted ciphertext, returning the corresponding +cleartext. + +```hcl +rsadecrypt(ciphertext, privatekey) +``` + +`ciphertext` must be a base64-encoded representation of the ciphertext, using +the PKCS #1 v1.5 padding scheme. Packer uses the "standard" Base64 alphabet +as defined in [RFC 4648 section 4](https://tools.ietf.org/html/rfc4648#section-4). + +`privatekey` must be a PEM-encoded RSA private key that is not itself +encrypted. + +Packer has no corresponding function for _encrypting_ a message. Use this +function to decrypt ciphertexts returned by remote services using a keypair +negotiated out-of-band. + +## Examples + +```shell-session +> rsadecrypt(base64(file("${path.folder}/ciphertext")), file("privatekey.pem")) +Hello, world! +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/crypto/sha1.mdx b/content/packer/content/docs/templates/hcl_templates/functions/crypto/sha1.mdx new file mode 100644 index 0000000000..c0cc5cd639 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/crypto/sha1.mdx @@ -0,0 +1,26 @@ +--- +page_title: sha1 function reference +description: |- + The `sha1` function computes the SHA1 hash of a given string and encodes it + with hexadecimal digits. Learn how to use the `sha1` function in Packer templates. +--- + +# `sha1` Function + +`sha1` computes the SHA1 hash of a given string and encodes it with +hexadecimal digits. + +The given string is first encoded as UTF-8 and then the SHA1 algorithm is applied +as defined in [RFC 3174](https://tools.ietf.org/html/rfc3174). The raw hash is +then encoded to lowercase hexadecimal digits before returning. + +Collision attacks have been successfully performed against this hashing +function. Before using this function for anything security-sensitive, review +relevant literature to understand the security implications. + +## Examples + +```shell-session +> sha1("hello world") +2aae6c35c94fcfb415dbe95f408b9ce91ee846ed +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/crypto/sha256.mdx b/content/packer/content/docs/templates/hcl_templates/functions/crypto/sha256.mdx new file mode 100644 index 0000000000..3abec6bb50 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/crypto/sha256.mdx @@ -0,0 +1,22 @@ +--- +page_title: sha256 function reference +description: |- + The `sha256` function computes the SHA256 hash of a given string and encodes it + with hexadecimal digits. Learn how to use the `sha256` function in Packer templates. +--- + +# `sha256` Function + +`sha256` computes the SHA256 hash of a given string and encodes it with +hexadecimal digits. + +The given string is first encoded as UTF-8 and then the SHA256 algorithm is applied +as defined in [RFC 4634](https://tools.ietf.org/html/rfc4634). The raw hash is +then encoded to lowercase hexadecimal digits before returning. + +## Examples + +```shell-session +> sha256("hello world") +b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9 +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/crypto/sha512.mdx b/content/packer/content/docs/templates/hcl_templates/functions/crypto/sha512.mdx new file mode 100644 index 0000000000..3a8cfeacaa --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/crypto/sha512.mdx @@ -0,0 +1,22 @@ +--- +page_title: sha512 function reference +description: |- + The `sha512` function computes the SHA512 hash of a given string and encodes it + with hexadecimal digits. Learn how to use the `sha512` function in Packer templates. +--- + +# `sha512` Function + +`sha512` computes the SHA512 hash of a given string and encodes it with +hexadecimal digits. + +The given string is first encoded as UTF-8 and then the SHA512 algorithm is applied +as defined in [RFC 4634](https://tools.ietf.org/html/rfc4634). The raw hash is +then encoded to lowercase hexadecimal digits before returning. + +## Examples + +```shell-session +> sha512("hello world") +309ecc489c12d6eb4cc40f50c902f2b4d0ed77ee511a7c7a9bcd3ca86d4cd86f989dd35bc5ff499670da34255b45b0cfd830e81f605dcf7dc5542e93ae9cd76f +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/datetime/formatdate.mdx b/content/packer/content/docs/templates/hcl_templates/functions/datetime/formatdate.mdx new file mode 100644 index 0000000000..1cc091cd88 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/datetime/formatdate.mdx @@ -0,0 +1,105 @@ +--- +page_title: formatdate function reference +description: The `formatdate` function converts a timestamp into a specified time format. Learn how to use `formatdate` functions in Packer templates. +--- + +# `formatdate` Function + +`formatdate` converts a timestamp into a different time format. + +```hcl +formatdate(spec, timestamp) +``` + +In Packer, timestamps are conventionally represented as strings using [RFC +3339](https://tools.ietf.org/html/rfc3339) "Date and Time format" syntax. +`formatdate` requires the `timestamp` argument to be a string conforming to +this syntax. + +## Examples + +```shell-session +> formatdate("DD MMM YYYY hh:mm ZZZ", "2018-01-02T23:12:01Z") +02 Jan 2018 23:12 UTC +> formatdate("EEEE, DD-MMM-YY hh:mm:ss ZZZ", "2018-01-02T23:12:01Z") +Tuesday, 02-Jan-18 23:12:01 UTC +> formatdate("EEE, DD MMM YYYY hh:mm:ss ZZZ", "2018-01-02T23:12:01-08:00") +Tue, 02 Jan 2018 23:12:01 -0800 +> formatdate("MMM DD, YYYY", "2018-01-02T23:12:01Z") +Jan 02, 2018 +> formatdate("HH:mmaa", "2018-01-02T23:12:01Z") +11:12pm +``` + +## Specification Syntax + +The format specification is a string that includes formatting sequences from +the following table. This function is intended for producing common +_machine-oriented_ timestamp formats such as those defined in RFC822, RFC850, +and RFC1123. It is not suitable for truly human-oriented date formatting +because it is not locale-aware. In particular, it can produce month and day +names only in English. + +The specification may contain the following sequences: + +| Sequence | Result | +| -------- | ----------------------------------------------------------------------- | +| `YYYY` | Four (or more) digit year, like "2006". | +| `YY` | The year modulo 100, zero padded to at least two digits, like "06". | +| `MMMM` | English month name unabbreviated, like "January". | +| `MMM` | English month name abbreviated to three letters, like "Jan". | +| `MM` | Month number zero-padded to two digits, like "01" for January. | +| `M` | Month number with no padding, like "1" for January. | +| `DD` | Day of month number zero-padded to two digits, like "02". | +| `D` | Day of month number with no padding, like "2". | +| `EEEE` | English day of week name unabbreviated, like "Monday". | +| `EEE` | English day of week name abbreviated to three letters, like "Mon". | +| `hh` | 24-hour number zero-padded to two digits, like "02". | +| `h` | 24-hour number unpadded, like "2". | +| `HH` | 12-hour number zero-padded to two digits, like "02". | +| `H` | 12-hour number unpadded, like "2". | +| `AA` | Hour AM/PM marker in uppercase, like "AM". | +| `aa` | Hour AM/PM marker in lowercase, like "am". | +| `mm` | Minute within hour zero-padded to two digits, like "05". | +| `m` | Minute within hour unpadded, like "5". | +| `ss` | Second within minute zero-padded to two digits, like "09". | +| `s` | Second within minute, like "9". | +| `ZZZZZ` | Timezone offset with colon separating hours and minutes, like "-08:00". | +| `ZZZZ` | Timezone offset with just sign and digit, like "-0800". | +| `ZZZ` | Like `ZZZZ` but with a special case "UTC" for UTC. | +| `Z` | Like `ZZZZZ` but with a special case "Z" for UTC. | + +Any non-letter characters, such as punctuation, are reproduced verbatim in the +output. To include literal letters in the format string, enclose them in single +quotes `'`. To include a literal quote, escape it by doubling the quotes. + +```shell-session +> formatdate("h'h'mm", "2018-01-02T23:12:01-08:00") +23h12 +> formatdate("H 'o''clock'", "2018-01-02T23:12:01-08:00") +11 o'clock +``` + +This format specification syntax is intended to make it easy for a reader +to guess which format will result even if they are not experts on the syntax. +Therefore there are no predefined shorthands for common formats, but format +strings for various RFC-specified formats are given below to be copied into your +configuration as needed: + +- [RFC 822](https://tools.ietf.org/html/rfc822#section-5) and + [RFC RFC 2822](https://tools.ietf.org/html/rfc2822#section-3.3): + `"DD MMM YYYY hh:mm ZZZ"` +- [RFC 850](https://tools.ietf.org/html/rfc850#section-2.1.4): + `"EEEE, DD-MMM-YY hh:mm:ss ZZZ"` +- [RFC 1123](https://tools.ietf.org/html/rfc1123#section-5.2.14): + `"EEE, DD MMM YYYY hh:mm:ss ZZZ"` +- [RFC 3339](https://tools.ietf.org/html/rfc3339): + `"YYYY-MM-DD'T'hh:mm:ssZ"` (but this is also the input format, so such a + conversion is redundant.) + +## Related Functions + +- [`format`](/packer/docs/templates/hcl_templates/functions/string/format) is a more general formatting function for arbitrary + data. +- [`timestamp`](/packer/docs/templates/hcl_templates/functions/datetime/timestamp) returns the current date and time in a format + suitable for input to `formatdate`. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/datetime/legacy_isotime.mdx b/content/packer/content/docs/templates/hcl_templates/functions/datetime/legacy_isotime.mdx new file mode 100644 index 0000000000..b52e9805cb --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/datetime/legacy_isotime.mdx @@ -0,0 +1,41 @@ +--- +page_title: legacy_isotime function reference +description: |- + The `legacy_isotime` function returns a string representation of the current date + and time. Learn how to use the `legacy_isotime` function in Packer templates. +--- + +# `legacy_isotime` Function + +The `legacy_isotime` function returns the current date and time using the given format string. +The format string follows Go's datetime formatting. See +[isotime-function-format-reference](/packer/docs/templates/legacy_json_templates/engine#isotime-function-format-reference) +for more details. + +This function has been provided to create backwards compatibility with +Packer's legacy JSON templates. However, we recommend that you upgrade your +HCL Packer template to use [`timestamp`](/packer/docs/templates/hcl_templates/functions/datetime/timestamp) +and [`formatdate`](/packer/docs/templates/hcl_templates/functions/datetime/formatdate) together as soon as is convenient. + +-> **Note:** If you are using a large number of builders, provisioners +or post-processors, the isotime may be slightly different for each one +because it is from when the plugin is launched not the initial Packer +process. In order to avoid this and make the timestamp consistent across all +plugins, set it as a user variable and then access the user variable within +your plugins. + +## Examples + +```shell-session +> legacy_isotime("2006-01-02") +2021-04-19 +``` + +## Related Functions + +- [`format`](/packer/docs/templates/hcl_templates/functions/string/format) is a more general formatting function for arbitrary +data. +- [`timestamp`](/packer/docs/templates/hcl_templates/functions/datetime/timestamp) returns the current date and time in a format +suitable for input to `formatdate`. +- [`formatdate`](/packer/docs/templates/hcl_templates/functions/datetime/formatdate) can convert the resulting timestamp to + other date and time formats. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/datetime/legacy_strftime.mdx b/content/packer/content/docs/templates/hcl_templates/functions/datetime/legacy_strftime.mdx new file mode 100644 index 0000000000..262c4a6c7e --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/datetime/legacy_strftime.mdx @@ -0,0 +1,40 @@ +--- +page_title: legacy_strftime function reference +description: |- + The `legacy_strftime` function returns a string representation of the current date + and time. Learn how to use the `legacy_strftime` function in Packer templates. +--- + +# `legacy_strftime` — UTC time, formated using the ISO C standard format + +The `legacy_strftime` function returns the current date and time using the given +format string. The format string follows strftime's datetime formatting. + +This function has been provided to create backwards compatibility with Packer's +legacy JSON templates. However, we recommend that you upgrade your HCL Packer +template to use +[`timestamp`](/packer/docs/templates/hcl_templates/functions/datetime/timestamp) and +[`formatdate`](/packer/docs/templates/hcl_templates/functions/datetime/formatdate) +together as soon as is convenient. + +-> **Note:** If you are using a large number of builders, provisioners or +post-processors, the strftime may be slightly different for each one because it +is from when the plugin is launched not the initial Packer process. In order to +avoid this and make the timestamp consistent across all plugins, set it as a +user variable and then access the user variable within your plugins. + +## Examples + +```shell-session +> legacy_strftime("%Y-%m") +2021-08 +``` + +## Related Functions + +- [`format`](/packer/docs/templates/hcl_templates/functions/string/format) is a more general formatting function for arbitrary +data. +- [`timestamp`](/packer/docs/templates/hcl_templates/functions/datetime/timestamp) returns the current date and time in a format +suitable for input to `formatdate`. +- [`formatdate`](/packer/docs/templates/hcl_templates/functions/datetime/formatdate) can convert the resulting timestamp to + other date and time formats. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/datetime/timeadd.mdx b/content/packer/content/docs/templates/hcl_templates/functions/datetime/timeadd.mdx new file mode 100644 index 0000000000..d54847bde2 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/datetime/timeadd.mdx @@ -0,0 +1,40 @@ +--- +page_title: timeadd function reference +description: |- + The `timeadd` function adds time to a timestamp and returns a new + timestamp. Learn how to use the `timeadd` function in Packer templates. +--- + +# `timeadd` Function + +`timeadd` adds a duration to a timestamp, returning a new timestamp. + +```hcl +timeadd(timestamp, duration) +``` + +In the Packer language, timestamps are conventionally represented as +strings using [RFC 3339](https://tools.ietf.org/html/rfc3339) +"Date and Time format" syntax. `timeadd` requires the `timestamp` argument +to be a string conforming to this syntax. + +`duration` is a string representation of a time difference, consisting of +sequences of number and unit pairs, like `"1.5h"` or `"1h30m"`. The accepted +units are `"ns"`, `"us"` (or `"µs"`), `"ms"`, `"s"`, `"m"`, and `"h"`. The first +number may be negative to indicate a negative duration, like `"-2h5m"`. + +The result is a string, also in RFC 3339 format, representing the result +of adding the given direction to the given timestamp. + +-> **Breaking change note:** Packer previously let you decide your own "Date +and Time format" syntax. With HCL2 and for parity with Terraform, Packer will +be using the [RFC 3339](https://tools.ietf.org/html/rfc3339) "Date and Time +format" syntax. As a string. [`formatdate`](/packer/docs/templates/hcl_templates/functions/datetime/formatdate) still allows you +to format a date. + +## Examples + +```shell-session +> timeadd("2017-11-22T00:00:00Z", "10m") +2017-11-22T00:10:00Z +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/datetime/timestamp.mdx b/content/packer/content/docs/templates/hcl_templates/functions/datetime/timestamp.mdx new file mode 100644 index 0000000000..ce31a973d5 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/datetime/timestamp.mdx @@ -0,0 +1,37 @@ +--- +page_title: timestamp function reference +description: |- + The `timestamp` function returns a string representation of the current date + and time. Learn how to use the `timestamp` function in Packer templates. +--- + +# `timestamp` Function + +`timestamp` returns the current date and time. + +In the Packer language, timestamps are conventionally represented as +strings using [RFC 3339](https://tools.ietf.org/html/rfc3339) +"Date and Time format" syntax, and so `timestamp` returns a string +in this format. + +The result of this function will change every second, so using this function +directly with resource attributes will cause a diff to be detected on every +Packer run. + +-> **Breaking change note:** Packer previously let you decide your own "Date +and Time format" syntax. With HCL2 and for parity with Terraform, Packer will +be using the [RFC 3339](https://tools.ietf.org/html/rfc3339) "Date and Time +format" syntax. As a string. [`formatdate`](/packer/docs/templates/hcl_templates/functions/datetime/formatdate) still allows you +to format a date. + +## Examples + +```shell-session +> timestamp() +2018-05-13T07:44:12Z +``` + +## Related Functions + +- [`formatdate`](/packer/docs/templates/hcl_templates/functions/datetime/formatdate) can convert the resulting timestamp to + other date and time formats. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/encoding/base64decode.mdx b/content/packer/content/docs/templates/hcl_templates/functions/encoding/base64decode.mdx new file mode 100644 index 0000000000..1ccc24e961 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/encoding/base64decode.mdx @@ -0,0 +1,36 @@ +--- +page_title: base64decode function reference +description: The `base64decode` function decodes a base64-encoded string. +--- + +# `base64decode` Function + +`base64decode` takes a string containing a Base64 character sequence and +returns the original string. + +Packer uses the "standard" Base64 alphabet as defined in +[RFC 4648 section 4](https://tools.ietf.org/html/rfc4648#section-4). + +Strings in the Packer language are sequences of unicode characters rather +than bytes, so this function will also interpret the resulting bytes as +UTF-8. If the bytes after Base64 decoding are _not_ valid UTF-8, this function +produces an error. + +While we do not recommend manipulating large, raw binary data in the Packer +language, Base64 encoding is the standard way to represent arbitrary byte +sequences, and so resource types that accept or return binary data will use +Base64 themselves, which avoids the need to encode or decode it directly in +most cases. Various other functions with names containing "base64" can generate +or manipulate Base64 data directly. + +## Examples + +```shell-session +> base64decode("SGVsbG8gV29ybGQ=") +Hello World +``` + +## Related Functions + +- [`base64encode`](/packer/docs/templates/hcl_templates/functions/encoding/base64encode) performs the opposite operation, + encoding the UTF-8 bytes for a string as Base64. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/encoding/base64encode.mdx b/content/packer/content/docs/templates/hcl_templates/functions/encoding/base64encode.mdx new file mode 100644 index 0000000000..a80e5bc1a3 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/encoding/base64encode.mdx @@ -0,0 +1,37 @@ +--- +page_title: base64encode function reference +description: The `base64encode` function encodes a string in base64 format. +--- + +# `base64encode` Function + +`base64encode` applies Base64 encoding to a string. + +Packer uses the "standard" Base64 alphabet as defined in +[RFC 4648 section 4](https://tools.ietf.org/html/rfc4648#section-4). + +Strings in the Packer language are sequences of unicode characters rather +than bytes, so this function will first encode the characters from the string +as UTF-8, and then apply Base64 encoding to the result. + +The Packer language applies Unicode normalization to all strings, and so +passing a string through `base64decode` and then `base64encode` may not yield +the original result exactly. + +While we do not recommend manipulating large, raw binary data in the Packer +language, Base64 encoding is the standard way to represent arbitrary byte +sequences, and so resource types that accept or return binary data will use +Base64 themselves, and so this function exists primarily to allow string +data to be easily provided to resource types that expect Base64 bytes. + +## Examples + +```shell-session +> base64encode("Hello World") +SGVsbG8gV29ybGQ= +``` + +## Related Functions + +- [`base64decode`](/packer/docs/templates/hcl_templates/functions/encoding/base64decode) performs the opposite operation, + decoding Base64 data and interpreting it as a UTF-8 string. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/encoding/base64gzip.mdx b/content/packer/content/docs/templates/hcl_templates/functions/encoding/base64gzip.mdx new file mode 100644 index 0000000000..ce7dbe6bbb --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/encoding/base64gzip.mdx @@ -0,0 +1,31 @@ +--- +page_title: base64gzip function reference +description: The `base64gzip` function compresses the given string using gzip and then encodes the result in base64. +--- + +# `base64gzip` Function + +`base64gzip` compresses a string with gzip and then encodes the result in +Base64 encoding. + +Packer uses the "standard" Base64 alphabet as defined in +[RFC 4648 section 4](https://tools.ietf.org/html/rfc4648#section-4). + +Strings in the Packer language are sequences of unicode characters rather +than bytes, so this function will first encode the characters from the string +as UTF-8, and then apply Base64 encoding to the result. + +The Packer language applies Unicode normalization to all strings, and so +passing a string through `base64decode` and then `base64encode` may not yield +the original result exactly. + +While we do not recommend manipulating large, raw binary data in the Packer +language, this function can be used to compress reasonably sized text strings +generated within the Packer language. For example, the result of this +function can be used to create a compressed object in Amazon S3 as part of +an S3 website. + +## Related Functions + +- [`base64decode`](/packer/docs/templates/hcl_templates/functions/encoding/base64decode) performs the opposite operation, + decoding Base64 data and interpreting it as a UTF-8 string. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/encoding/csvdecode.mdx b/content/packer/content/docs/templates/hcl_templates/functions/encoding/csvdecode.mdx new file mode 100644 index 0000000000..a558a5c085 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/encoding/csvdecode.mdx @@ -0,0 +1,95 @@ +--- +page_title: csvdecode function reference +description: The `csvdecode` function decodes CSV data into a list of maps. +--- + +# `csvdecode` Function + +`csvdecode` decodes a string containing CSV-formatted data and produces a +list of maps representing that data. + +CSV is _Comma-separated Values_, an encoding format for tabular data. There +are many variants of CSV, but this function implements the format defined +in [RFC 4180](https://tools.ietf.org/html/rfc4180). + +The first line of the CSV data is interpreted as a "header" row: the values +given are used as the keys in the resulting maps. Each subsequent line becomes +a single map in the resulting list, matching the keys from the header row +with the given values by index. All lines in the file must contain the same +number of fields, or this function will produce an error. + +## Examples + +```shell-session +> csvdecode("a,b,c\n1,2,3\n4,5,6") +[ + { + "a" = "1" + "b" = "2" + "c" = "3" + }, + { + "a" = "4" + "b" = "5" + "c" = "6" + } +] +``` + +## Use with the `for_each` meta-argument + +You can use the result of `csvdecode` with +[the `for_each` meta-argument](/terraform/docs/configuration/resources#for_each-multiple-resource-instances-defined-by-a-map-or-set-of-strings) +to describe a collection of similar objects whose differences are +described by the rows in the given CSV file. + +There must be one column in the CSV file that can serve as a unique ID for each +row, which we can then use as the tracking key for the individual instances in +the `for_each` expression. For example: + +```hcl +locals { + # We've included this inline to create a complete example, but in practice + # this is more likely to be loaded from a file using the "file" function. + csv_data = <<-CSV + local_id,instance_type,ami + foo1,t2.micro,ami-54d2a63b + foo2,t2.micro,ami-54d2a63b + foo3,t2.micro,ami-54d2a63b + bar1,m3.large,ami-54d2a63b + CSV + + instances = csvdecode(local.csv_data) +} + +resource "aws_instance" "example" { + for_each = { for inst in local.instances : inst.local_id => inst } + + instance_type = each.value.instance_type + ami = each.value.ami +} +``` + +The `for` expression in our `for_each` argument transforms the list produced +by `csvdecode` into a map using the `local_id` as a key, which tells +Packer to use the `local_id` value to track each instance it creates. +Packer will create and manage the following instance addresses: + +- `aws_instance.example["foo1"]` +- `aws_instance.example["foo2"]` +- `aws_instance.example["foo3"]` +- `aws_instance.example["bar1"]` + +If you modify a row in the CSV on a subsequent plan, Packer will interpret +that as an update to the existing object as long as the `local_id` value is +unchanged. If you add or remove rows from the CSV then Packer will plan to +create or destroy associated instances as appropriate. + +If there is no reasonable value you can use as a unique identifier in your CSV +then you could instead use +[the `count` meta-argument](/terraform/docs/configuration/resources#count-multiple-resource-instances-by-count) +to define an object for each CSV row, with each one identified by its index into +the list returned by `csvdecode`. However, in that case any future updates to +the CSV may be disruptive if they change the positions of particular objects in +the list. We recommend using `for_each` with a unique ID column to make +behavior more predictable on future changes. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/encoding/jsondecode.mdx b/content/packer/content/docs/templates/hcl_templates/functions/encoding/jsondecode.mdx new file mode 100644 index 0000000000..c55783de04 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/encoding/jsondecode.mdx @@ -0,0 +1,45 @@ +--- +page_title: jsondecode function reference +description: |- + The `jsondecode` function decodes a JSON string into its corresponding Packer value. +--- + +# `jsondecode` Function + +`jsondecode` interprets a given string as JSON, returning a representation +of the result of decoding that string. + +The JSON encoding is defined in [RFC 7159](https://tools.ietf.org/html/rfc7159). + +This function maps JSON values to +[Packer language values](/packer/docs/templates/hcl_templates/expressions#types-and-values) +in the following way: + +| JSON type | Packer type | +| --------- | ------------------------------------------------------------ | +| String | `string` | +| Number | `number` | +| Boolean | `bool` | +| Object | `object(...)` with attribute types determined per this table | +| Array | `tuple(...)` with element types determined per this table | +| Null | The Packer language `null` value | + +The Packer language automatic type conversion rules mean that you don't +usually need to worry about exactly what type is produced for a given value, +and can just use the result in an intuitive way. + +## Examples + +```shell-session +> jsondecode("{\"hello\": \"world\"}") +{ + "hello" = "world" +} +> jsondecode("true") +true +``` + +## Related Functions + +- [`jsonencode`](/packer/docs/templates/hcl_templates/functions/encoding/jsonencode) performs the opposite operation, _encoding_ + a value as JSON. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/encoding/jsonencode.mdx b/content/packer/content/docs/templates/hcl_templates/functions/encoding/jsonencode.mdx new file mode 100644 index 0000000000..4304ff96dd --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/encoding/jsonencode.mdx @@ -0,0 +1,43 @@ +--- +page_title: jsonencode function reference +description: The `jsonencode` function encodes a given value as a JSON string. +--- + +# `jsonencode` Function + +`jsonencode` encodes a given value to a string using JSON syntax. + +The JSON encoding is defined in [RFC 7159](https://tools.ietf.org/html/rfc7159). + +This function maps +[Packer language values](/packer/docs/templates/hcl_templates/expressions#types-and-values) +to JSON values in the following way: + +| Packer type | JSON type | +| ------------- | --------- | +| `string` | String | +| `number` | Number | +| `bool` | Bool | +| `list(...)` | Array | +| `set(...)` | Array | +| `tuple(...)` | Array | +| `map(...)` | Object | +| `object(...)` | Object | +| Null value | `null` | + +Since the JSON format cannot fully represent all of the Packer language +types, passing the `jsonencode` result to `jsondecode` will not produce an +identical value, but the automatic type conversion rules mean that this is +rarely a problem in practice. + +## Examples + +```shell-session +> jsonencode({"hello"="world"}) +{"hello":"world"} +``` + +## Related Functions + +- [`jsondecode`](/packer/docs/templates/hcl_templates/functions/encoding/jsondecode) performs the opposite operation, _decoding_ + a JSON string to obtain its represented value. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/encoding/textdecodebase64.mdx b/content/packer/content/docs/templates/hcl_templates/functions/encoding/textdecodebase64.mdx new file mode 100644 index 0000000000..7a0b060538 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/encoding/textdecodebase64.mdx @@ -0,0 +1,28 @@ +--- +page_title: testdecodebase64 function reference +description: The `testdecodebase64` function converts a base64-encoded string into a UTF-8 string. +--- + +# `textdecodebase64` Function + +Encodes the input string from a speicified encoding into UTF-8. +The input is base64-encoded to account for HCL's string encoding limitations: they must be UTF-8, NFC-normalised. + +Packer uses the "standard" Base64 alphabet as defined in +[RFC 4648 section 4](https://tools.ietf.org/html/rfc4648#section-4). + +The `encoding_name` argument must contain one of the encoding names or aliases recorded in +[the IANA character encoding registry](https://www.iana.org/assignments/character-sets/character-sets.xhtml). + +## Examples + +```shell-session +# Usage: textencodebase64(input_base64, encoding_name) +> textdecodebase64("SABlAGwAbABvACAAVwBvAHIAbABkAA==", "UTF-16LE") +Hello World +``` + +## Related Functions + +- [`base64encode`](/packer/docs/templates/hcl_templates/functions/encoding/base64encode) performs the opposite operation, + encoding the UTF-8 bytes for a string as Base64. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/encoding/textencodebase64.mdx b/content/packer/content/docs/templates/hcl_templates/functions/encoding/textencodebase64.mdx new file mode 100644 index 0000000000..7a10a57162 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/encoding/textencodebase64.mdx @@ -0,0 +1,28 @@ +--- +page_title: testencodebase64 function reference +description: The `testencodebase64` function converts a UTF-8 NFC string into a base64 blob to match the destination encoding. +--- + +# `textencodebase64` Function + +Encodes the input string to the destination encoding. +The output is base64-encoded to account for HCL's string encoding limitations: they must be UTF-8, NFC-normalised. + +Packer uses the standard base64 alphabet as defined in +[RFC 4648 section 4](https://tools.ietf.org/html/rfc4648#section-4). + +The `encoding_name` argument must contain one of the encoding names or aliases recorded in +[the IANA character encoding registry](https://www.iana.org/assignments/character-sets/character-sets.xhtml). + +## Examples + +```shell-session +# Usage: textencodebase64(input_string, encoding_name) +> textencodebase64("Hello World", "UTF-16LE") +SABlAGwAbABvACAAVwBvAHIAbABkAA== +``` + +## Related Functions + +- [`base64encode`](/packer/docs/templates/hcl_templates/functions/encoding/base64encode) performs the opposite operation, + encoding the UTF-8 bytes for a string as Base64. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/encoding/urlencode.mdx b/content/packer/content/docs/templates/hcl_templates/functions/encoding/urlencode.mdx new file mode 100644 index 0000000000..fbe7d7bf50 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/encoding/urlencode.mdx @@ -0,0 +1,31 @@ +--- +page_title: urlencode function reference +description: The `urlencode` function applies URL encoding to a given string. +--- + +# `urlencode` Function + +`urlencode` applies URL encoding to a given string. + +This function identifies characters in the given string that would have a +special meaning when included as a query string argument in a URL and +escapes them using +[RFC 3986 "percent encoding"](https://tools.ietf.org/html/rfc3986#section-2.1). + +The exact set of characters escaped may change over time, but the result +is guaranteed to be interpolatable into a query string argument without +inadvertently introducing additional delimiters. + +If the given string contains non-ASCII characters, these are first encoded as +UTF-8 and then percent encoding is applied separately to each UTF-8 byte. + +## Examples + +```shell-session +> urlencode("Hello World") +Hello%20World +> urlencode("☃") +%E2%98%83 +> "http://example.com/search?q=${urlencode("packer urlencode")}" +http://example.com/search?q=packer%20urlencode +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/encoding/yamldecode.mdx b/content/packer/content/docs/templates/hcl_templates/functions/encoding/yamldecode.mdx new file mode 100644 index 0000000000..c397d6fb73 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/encoding/yamldecode.mdx @@ -0,0 +1,98 @@ +--- +page_title: yamldecode function reference +description: |- + The `yamldecode` function decodes a YAML string into its corresponding Packer value. +--- + +# `yamldecode` Function + +`yamldecode` parses a string as a subset of YAML, and produces a representation +of its value. + +This function supports a subset of [YAML 1.2](https://yaml.org/spec/1.2/spec.html), +as described below. + +This function maps YAML values to +[Packer language values](/packer/docs/templates/hcl_templates/expressions#types-and-values) +in the following way: + +| YAML type | Packer type | +| ------------- | ------------------------------------------------------------------ | +| `!!str` | `string` | +| `!!float` | `number` | +| `!!int` | `number` | +| `!!bool` | `bool` | +| `!!map` | `object(...)` with attribute types determined per this table | +| `!!seq` | `tuple(...)` with element types determined per this table | +| `!!null` | The Packer language `null` value | +| `!!timestamp` | `string` in [RFC 3339](https://tools.ietf.org/html/rfc3339) format | +| `!!binary` | `string` containing base64-encoded representation | + +The Packer language automatic type conversion rules mean that you don't +usually need to worry about exactly what type is produced for a given value, +and can just use the result in an intuitive way. + +Note though that the mapping above is ambiguous -- several different source +types map to the same target type -- and so round-tripping through `yamldecode` +and then `yamlencode` cannot produce an identical result. + +YAML is a complex language and it supports a number of possibilities that the +Packer language's type system cannot represent. Therefore this YAML decoder +supports only a subset of YAML 1.2, with restrictions including the following: + +- Although aliases to earlier anchors are supported, cyclic data structures + (where a reference to a collection appears inside that collection) are not. + If `yamldecode` detects such a structure then it will return an error. + +- Only the type tags shown in the above table (or equivalent alternative + representations of those same tags) are supported. Any other tags will + result in an error. + +- Only one YAML document is permitted. If multiple documents are present in + the given string then this function will return an error. + +## Examples + +```shell-session +> yamldecode("{\"hello\": \"world\"}") +{ + "hello" = "world" +} + +> yamldecode("true") +true + +> yamldecode("{a: &foo [1, 2, 3], b: *foo}") +{ + "a" = [ + 1, + 2, + 3, + ] + "b" = [ + 1, + 2, + 3, + ] +} + +> yamldecode("{a: &foo [1, *foo, 3]}") + +Error: Error in function call + +Call to function "yamldecode" failed: cannot refer to anchor "foo" from inside +its own definition. + +> yamldecode("{a: !not-supported foo}") + +Error: Error in function call + +Call to function "yamldecode" failed: unsupported tag "!not-supported". +``` + +## Related Functions + +- [`jsondecode`](/packer/docs/templates/hcl_templates/functions/encoding/jsondecode) is a similar operation using JSON instead + of YAML. +- [`yamlencode`](/packer/docs/templates/hcl_templates/functions/encoding/yamlencode) performs the opposite operation, _encoding_ + a value as YAML. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/encoding/yamlencode.mdx b/content/packer/content/docs/templates/hcl_templates/functions/encoding/yamlencode.mdx new file mode 100644 index 0000000000..0b8f5c9e3e --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/encoding/yamlencode.mdx @@ -0,0 +1,85 @@ +--- +page_title: yamlencode function reference +description: The `yamlencode` function encodes a given value as a YAML string. +--- + +# `yamlencode` Function + +`yamlencode` encodes a given value to a string using +[YAML 1.2](https://yaml.org/spec/1.2/spec) block syntax. + +~> **Warning:** This function is currently **experimental** and its exact +result format may change in future versions of Packer, based on feedback. +Do not use `yamldecode` to construct a value for any resource argument where +changes to the result would be disruptive. To get a consistent string +representation of a value use [`jsonencode`](/packer/docs/templates/hcl_templates/functions/encoding/jsonencode) instead; its +results are also valid YAML because YAML is a JSON superset. + + + +This function maps +[Packer language values](/packer/docs/templates/hcl_templates/expressions#types-and-values) +to YAML tags in the following way: + +| Packer type | YAML type | +| ------------- | -------------------- | +| `string` | `!!str` | +| `number` | `!!float` or `!!int` | +| `bool` | `!!bool` | +| `list(...)` | `!!seq` | +| `set(...)` | `!!seq` | +| `tuple(...)` | `!!seq` | +| `map(...)` | `!!map` | +| `object(...)` | `!!map` | +| Null value | `!!null` | + +`yamlencode` uses the implied syntax for all of the above types, so it does +not generate explicit YAML tags. + +Because the YAML format cannot fully represent all of the Packer language +types, passing the `yamlencode` result to `yamldecode` will not produce an +identical value, but the Packer language automatic type conversion rules +mean that this is rarely a problem in practice. + +## Examples + +```shell-session +> yamlencode({"a":"b", "c":"d"}) +"a": "b" +"c": "d" + +> yamlencode({"foo":[1, 2, 3], "bar": "baz"}) +"bar": "baz" +"foo": +- 1 +- 2 +- 3 + +> yamlencode({"foo":[1, {"a":"b","c":"d"}, 3], "bar": "baz"}) +"bar": "baz" +"foo": +- 1 +- "a": "b" + "c": "d" +- 3 +``` + +`yamlencode` always uses YAML's "block style" for mappings and sequences, unless +the mapping or sequence is empty. To generate flow-style YAML, use +[`jsonencode`](/packer/docs/templates/hcl_templates/functions/encoding/jsonencode) instead: YAML flow-style is a superset +of JSON syntax. + +## Related Functions + +- [`jsonencode`](/packer/docs/templates/hcl_templates/functions/encoding/jsonencode) is a similar operation using JSON instead + of YAML. +- [`yamldecode`](/packer/docs/templates/hcl_templates/functions/encoding/yamldecode) performs the opposite operation, _decoding_ + a YAML string to obtain its represented value. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/file/abspath.mdx b/content/packer/content/docs/templates/hcl_templates/functions/file/abspath.mdx new file mode 100644 index 0000000000..87c8074c4a --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/file/abspath.mdx @@ -0,0 +1,24 @@ +--- +page_title: abspath function reference +description: The `abspath` function converts an argument to an absolute filesystem path. Learn how to use the `abspath` function in Packer templates. +--- + +# `abspath` Function + +`abspath` takes a string containing a filesystem path and converts it +to an absolute path. That is, if the path is not absolute, it will be joined +with the current working directory. + +Referring directly to filesystem paths in resource arguments may cause spurious +diffs if the same configuration is applied from multiple systems or on +different host operating systems. We recommend using filesystem paths only for +transient values, such as the argument to +[`file`](/packer/docs/templates/hcl_templates/functions/file) (where only the contents are then +stored). + +## Examples + +```shell-session +> abspath(path.root) +/home/user/some/packer/root +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/file/basename.mdx b/content/packer/content/docs/templates/hcl_templates/functions/file/basename.mdx new file mode 100644 index 0000000000..4ed4df1336 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/file/basename.mdx @@ -0,0 +1,40 @@ +--- +page_title: basename function reference +description: |- + The `basename` function removes parts of a filesystem path except for the last portion. Learn how to use the `basename` function in Packer templates. +--- + +# `basename` Function + +`basename` takes a string containing a filesystem path and removes all except +the last portion from it. + +This function works only with the path string and does not access the +filesystem itself. It is therefore unable to take into account filesystem +features such as symlinks. + +If the path is empty then the result is `"."`, representing the current +working directory. + +The behavior of this function depends on the host platform. On Windows systems, +it uses backslash `\` as the path segment separator. On Unix systems, the slash +`/` is used. + +Referring directly to filesystem paths in resource arguments may cause +spurious diffs if the same configuration is applied from multiple systems or on +different host operating systems. We recommend using filesystem paths only +for transient values, such as the argument to [`file`](/packer/docs/templates/hcl_templates/functions/file) (where +only the contents are then stored) or in `connection` and `provisioner` blocks. + +## Examples + +```shell-session +> basename("foo/bar/baz.txt") +baz.txt +``` + +## Related Functions + +- [`dirname`](/packer/docs/templates/hcl_templates/functions/file/dirname) returns all of the segments of a filesystem path + _except_ the last, discarding the portion that would be returned by + `basename`. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/file/dirname.mdx b/content/packer/content/docs/templates/hcl_templates/functions/file/dirname.mdx new file mode 100644 index 0000000000..b76b056ba6 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/file/dirname.mdx @@ -0,0 +1,39 @@ +--- +page_title: dirname function reference +description: The `dirname` function removes the last portion from a filesystem path. Learn how to use the `dirname` function in Packer templates. +--- + +# `dirname` Function + +`dirname` takes a string containing a filesystem path and removes the last +portion from it. + +This function works only with the path string and does not access the +filesystem itself. It is therefore unable to take into account filesystem +features such as symlinks. + +If the path is empty then the result is `"."`, representing the current +working directory. + +The behavior of this function depends on the host platform. On Windows systems, +it uses backslash `\` as the path segment separator. On Unix systems, the slash +`/` is used. The result of this function is normalized, so on a Windows system +any slashes in the given path will be replaced by backslashes before returning. + +Referring directly to filesystem paths in resource arguments may cause +spurious diffs if the same configuration is applied from multiple systems or on +different host operating systems. We recommend using filesystem paths only +for transient values, such as the argument to [`file`](/packer/docs/templates/hcl_templates/functions/file) (where +only the contents are then stored) or in `connection` and `provisioner` blocks. + +## Examples + +```shell-session +> dirname("foo/bar/baz.txt") +foo/bar +``` + +## Related Functions + +- [`basename`](/packer/docs/templates/hcl_templates/functions/file/basename) returns _only_ the last portion of a filesystem + path, discarding the portion that would be returned by `dirname`. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/file/file.mdx b/content/packer/content/docs/templates/hcl_templates/functions/file/file.mdx new file mode 100644 index 0000000000..873ecd8a4f --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/file/file.mdx @@ -0,0 +1,36 @@ +--- +page_title: file function reference +description: |- + The `file` function reads the contents of a file and returns it as a string. Learn how to use the `file` function in Packer templates. +--- + +# `file` Function + +`file` reads the contents of a file at the given path and returns them as +a string. + +```hcl +file(path) +``` + +Strings in the Packer language are sequences of Unicode characters, so +this function will interpret the file contents as UTF-8 encoded text and +return the resulting Unicode characters. If the file contains invalid UTF-8 +sequences then this function will produce an error. + +This function can be used only with files that already exist on disk +at the beginning of a Packer run. Functions do not participate in the +dependency graph, so this function cannot be used with files that are generated +dynamically during a Packer operation. + +## Examples + +```shell-session +> file("${path.folder}/hello.txt") +Hello World +``` + +## Related Functions + +- [`fileexists`](/packer/docs/templates/hcl_templates/functions/file/fileexists) determines whether a file exists + at a given path. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/file/filebase64.mdx b/content/packer/content/docs/templates/hcl_templates/functions/file/filebase64.mdx new file mode 100644 index 0000000000..1b60fcc7fb --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/file/filebase64.mdx @@ -0,0 +1,46 @@ +--- +page_title: filebase64 - Functions - Configuration Language +description: |- + The filebase64 function reads the contents of the file at the given path and + returns them as a base64-encoded string. +--- + +# `filebase64` Function + +`filebase64` reads the contents of a file at the given path and returns them as +a base64-encoded string. + +```hcl +filebase64(path) +``` + +The result is a Base64 representation of the raw bytes in the given file. +Strings in the Packer language are sequences of Unicode characters, so +Base64 is the standard way to represent raw binary data that cannot be +interpreted as Unicode characters. Resource types that operate on binary +data will accept this data encoded in Base64, thus avoiding the need to +decode the result of this function. + +Packer uses the "standard" Base64 alphabet as defined in +[RFC 4648 section 4](https://tools.ietf.org/html/rfc4648#section-4). + +This function can be used only with functions that already exist as static +files on disk at the beginning of a Packer run. Language functions do not +participate in the dependency graph, so this function cannot be used with +files that are generated dynamically during a Packer operation. + +## Examples + +``` +> filebase64("${path.module}/hello.txt") +SGVsbG8gV29ybGQ= +``` + +## Related Functions + +* [`file`](/packer/docs/templates/hcl_templates/functions/file/file) also reads the contents of a given file, + but interprets the data as UTF-8 text and returns the result directly + as a string, without any further encoding. +* [`base64decode`](/packer/docs/templates/hcl_templates/functions/encoding/base64decode) can decode a Base64 string representing + bytes in UTF-8, but in practice `base64decode(filebase64(...))` is equivalent + to the shorter expression `file(...)`. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/file/fileexists.mdx b/content/packer/content/docs/templates/hcl_templates/functions/file/fileexists.mdx new file mode 100644 index 0000000000..6d4f302d38 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/file/fileexists.mdx @@ -0,0 +1,35 @@ +--- +page_title: fileexists function reference +description: The `fileexists` function determines whether a file exists at a given path. Learn how to use the `fileexists` function in Packer templates. +--- + +# `fileexists` Function + +`fileexists` determines whether a file exists at a given path. + +```hcl +fileexists(path) +``` + +Functions are evaluated during configuration parsing rather than at apply time, +so this function can only be used with files that are already present on disk +before Packer takes any actions. + +This function works only with regular files. If used with a directory, FIFO, +or other special mode, it will return an error. + +## Examples + +```shell-session +> fileexists("${path.folder}/hello.txt") +true +``` + +```hcl +fileexists("custom-section.sh") ? file("custom-section.sh") : local.default_content +``` + +## Related Functions + +- [`file`](/packer/docs/templates/hcl_templates/functions/file/file) reads the contents + of a file at a given path. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/file/fileset.mdx b/content/packer/content/docs/templates/hcl_templates/functions/file/fileset.mdx new file mode 100644 index 0000000000..da6d44ece2 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/file/fileset.mdx @@ -0,0 +1,102 @@ +--- +page_title: fileset function reference +description: The `fileset` function enumerates a set of regular file names according to a pattern. Learn how to use the `fileset` function in Packer templates. +--- + +# `fileset` Function + +`fileset` enumerates a set of regular file names given a path and pattern. +The path is automatically removed from the resulting set of file names and any +result still containing path separators always returns forward slash (`/`) as +the path separator for cross-system compatibility. + +```hcl +fileset(path, pattern) +``` + +Supported pattern matches: + +- `*` - matches any sequence of non-separator characters +- `**` - matches any sequence of characters, including separator characters +- `?` - matches any single non-separator character +- `{alternative1,...}` - matches a sequence of characters if one of the comma-separated alternatives matches +- `[CLASS]` - matches any single non-separator character inside a class of characters (see below) +- `[^CLASS]` - matches any single non-separator character outside a class of characters (see below) + +Character classes support the following: + +- `[abc]` - matches any single character within the set +- `[a-z]` - matches any single character within the range + +Functions are evaluated during configuration parsing rather than at apply time, +so this function can only be used with files that are already present on disk +before Packer takes any actions. + +## Examples + +```shell-session +> tree pkr-consul +pkr-consul +├── build-linux.pkr.hcl +└── linux + ├── files + │ ├── hello.txt + │ └── world.txt + └── scripts + ├── script-1-install.sh + └── script-2-setup.sh + +3 directories, 5 files + +> fileset(".", "*") | packer console pkr-consul +[ + "build-linux.pkr.hcl", +] + +> echo 'fileset(".", "linux/scripts/*")' | packer console pkr-consul +[ + "linux/scripts/script-1-install.sh", + "linux/scripts/script-2-setup.sh", +] + +> echo 'fileset("linux", "files/{hello,world}.txt")' | packer console pkr-consul +[ + "files/hello.txt", + "files/world.txt", +] + +> echo 'fileset("./linux/files", "*")' | packer console pkr-consul +[ + "hello.txt", + "world.txt", +] + +> echo 'fileset("./linux", "**")' | packer console pkr-consul +[ + "files/hello.txt", + "files/world.txt", + "scripts/script-1-install.sh", + "scripts/script-2-setup.sh", +] +``` + +A common use of `fileset` is to set the `scripts` field of a `shell` +provisioner with a list of matching scripts to run. + +```hcl +build { + sources = [ + "source.amazon-ebs.linux", + ] + + provisioner "shell" { + scripts = fileset(".", "linux/scripts/*") + } +``` + +List of provisioners with a `scripts` field: + +- [`shell`](/packer/docs/provisioners/shell) +- [`powershell`](/packer/docs/provisioners/powershell) +- [`shell-local`](/packer/docs/provisioners/shell-local) +- [`windows-shell`](/packer/docs/provisioners/windows-shell) diff --git a/content/packer/content/docs/templates/hcl_templates/functions/file/pathexpand.mdx b/content/packer/content/docs/templates/hcl_templates/functions/file/pathexpand.mdx new file mode 100644 index 0000000000..77db1db8d4 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/file/pathexpand.mdx @@ -0,0 +1,54 @@ +--- +page_title: pathexpand function reference +description: |- + The `pathexpand` function replaces a leading tilde `~` character with the current user's + home directory. Learn how to use the `pathexpand` function in Packer templates. +--- + +# `pathexpand` Function + +`pathexpand` takes a filesystem path that might begin with a `~` segment, +and if so it replaces that segment with the current user's home directory +path. + +This function works only with the path string and does not access the +filesystem itself. It is therefore unable to take into account filesystem +features such as symlinks. + +If the leading segment in the path is not `~` then the given path is returned +unmodified. + +Using this function in resource arguments will cause spurious diffs if the +same configuration is run by multiple users with different home directory +paths, or used on different host operating systems. We recommend using this +function only for transient values, such as in `connection` and `provisioner` +blocks to locate SSH keys, etc. + +The rules for determining the "home directory" for the current user vary +depending on host operating system. + +**For Unix systems**, the following sources are consulted, in order of preference: + +- The `HOME` environment variable. +- The result of running `getent passwd` followed by the Packer process uid. +- The result of running `cd && pwd` in `sh`. + +**For Windows systems**, there is not really the concept of a home directory +in the same sense as on Unix, but the following sources are consulted in +order of preference: + +- The `HOME` environment variable. +- The `HOMEDRIVE` and `HOMEPATH` environment variables, if both are set. +- The `USERPROFILE` environment variable. + +The exact rules employed for each operating system may change in future +releases of Packer. + +## Examples + +```shell-session +> pathexpand("~/.ssh/id_rsa") +/home/steve/.ssh/id_rsa +> pathexpand("/etc/resolv.conf") +/etc/resolv.conf +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/file/templatefile.mdx b/content/packer/content/docs/templates/hcl_templates/functions/file/templatefile.mdx new file mode 100644 index 0000000000..1a5707cc61 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/file/templatefile.mdx @@ -0,0 +1,133 @@ +--- +page_title: templatefile function reference +description: |- + The `templatefile` function reads a file and renders its + content as a template using a set of template variables. Learn how to use the `templatefile` function. +--- + +# `templatefile` Function + +The `templatefile` function reads the file at the given path and renders its content as a +template using a supplied set of template variables. + +```hcl +templatefile(path, vars) +``` + +We recommend using the `.pkrtpl.hcl` file extension when using the `templatefile` function to create templates. Packer treats HCL templates as combination of file and template. As a result, HCL template files have slightly different set of features than the templates with a `.pkr.hcl` extension. Using the `.pkr.hcl` extension also enables syntax highlighters to properly understand your file. + +The template syntax is the same as for string templates in the main HCL2 +language, including interpolation sequences delimited with `${ ... }`. This +function just allows longer template sequences to be factored out into a +separate file for readability. + +The "vars" argument must be a map. Within the template file, each of the keys in +the map is available as a variable for interpolation. The template may also use +any other function available in Packer, except that recursive calls to +templatefile are not permitted. Variable names must each start with a letter, +followed by zero or more letters, digits, or underscores. + +Strings in HCL2 are sequences of Unicode characters, so this function will +interpret the file contents as UTF-8 encoded text and return the resulting +Unicode characters. If the file contains invalid UTF-8 sequences then this +function will produce an error. + +This function can be used only with files that already exist on disk at the +beginning of a run. + +## Examples + +### Lists + +Given a template file backends.tpl with the following content: + +```hcl +%{ for addr in ip_addrs ~} +backend ${addr}:${port} +%{ endfor ~} +``` + +The templatefile function renders the template: + +```shell-session +> templatefile("${path.root}/backends.tmpl", { port = 8080, ip_addrs = ["10.0.0.1", "10.0.0.2"] }) +backend 10.0.0.1:8080 +backend 10.0.0.2:8080 +``` + +### Maps + +Given a template file config.tmpl with the following content: + +```hcl +%{ for config_key, config_value in config } +set ${config_key} = ${config_value} +%{ endfor ~} +``` + +The templatefile function renders the template: + +```shell-session +> templatefile( + "${path.root}/config.tmpl", + { + config = { + "x" = "y" + "foo" = "bar" + "key" = "value" + } + } + ) +set foo = bar +set key = value +set x = y +``` + +### Generating JSON or YAML from a template + +If the string you want to generate will be in JSON or YAML syntax, it's often +tricky and tedious to write a template that will generate valid JSON or YAML +that will be interpreted correctly when using lots of individual interpolation +sequences and directives. + +Instead, you can write a template that consists only of a single interpolated +call to either jsonencode or yamlencode, specifying the value to encode using +normal expression syntax as in the following examples: + +```hcl +${jsonencode({ + "backends": [for addr in ip_addrs : "${addr}:${port}"], +})} +${yamlencode({ + "backends": [for addr in ip_addrs : "${addr}:${port}"], +})} +``` + +Given the same input as the `backends.tmpl` example in the previous section, +this will produce a valid JSON or YAML representation of the given data +structure, without the need to manually handle escaping or delimiters. In the +latest examples above, the repetition based on elements of ip_addrs is achieved +by using a for expression rather than by using template directives. + +``` +{"backends":["10.0.0.1:8080","10.0.0.2:8080"]} +``` + +If the resulting template is small, you can choose instead to write jsonencode or yamlencode calls inline in your main configuration files, and avoid creating separate template files at all: + +```hcl +locals { + backend_config_json = jsonencode({ + "backends": [for addr in ip_addrs : "${addr}:${port}"], + }) +} +``` + +For more information, see the main documentation for jsonencode and yamlencode. + +## Related Functions + +- [`file`](/packer/docs/templates/hcl_templates/functions/file/file) reads the contents + of a file at a given path. +- [`fileexists`](/packer/docs/templates/hcl_templates/functions/file/fileexists) + determines whether a file exists at a given path. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/index.mdx b/content/packer/content/docs/templates/hcl_templates/functions/index.mdx new file mode 100644 index 0000000000..9a074eeec0 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/index.mdx @@ -0,0 +1,25 @@ +--- +page_title: Built-in functions reference overview +description: |- + Functions built into HCL language can transform and combine values. Learn about using built-in functions in your Packer templates. +--- + +# Built-in functions overview + +This topic provides an overview of the built-in HashiCorp configuration language (HCL) functions you can use in your Packer templates. + +## Functions syntax + +You can use functions in Packer templates to transform and combine values. The +syntax for function calls is a function name followed by comma-separated +arguments in parentheses: + +```hcl +max(5, 12, 9) +``` + +For information on invoking functions in string literals, refer to the +[string literals reference](/packer/docs/templates/hcl_templates/expressions#string-literals) section. + +HCL does not support user-defined functions. You can only call +the functions built into the language. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/ipnet/cidrhost.mdx b/content/packer/content/docs/templates/hcl_templates/functions/ipnet/cidrhost.mdx new file mode 100644 index 0000000000..0682f1b2e4 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/ipnet/cidrhost.mdx @@ -0,0 +1,49 @@ +--- +page_title: cidrhost function reference +description: |- + The `cidrhost` function calculates a full host IP address in a given + IP network address prefix. +--- + +# `cidrhost` Function + +`cidrhost` calculates a full host IP address for a given host number within +a given IP network address prefix. + +```hcl +cidrhost(prefix, hostnum) +``` + +`prefix` must be given in CIDR notation, as defined in +[RFC 4632 section 3.1](https://tools.ietf.org/html/rfc4632#section-3.1). + +`hostnum` is a whole number that can be represented as a binary integer with +no more than the number of digits remaining in the address after the given +prefix. For more details on how this function interprets CIDR prefixes and +populates host numbers, see the worked example for +[`cidrsubnet`](/packer/docs/templates/hcl_templates/functions/ipnet/cidrsubnet). + +Conventionally host number zero is used to represent the address of the +network itself and the host number that would fill all the host bits with +binary 1 represents the network's broadcast address. These numbers should +generally not be used to identify individual hosts except in unusual +situations, such as point-to-point links. + +This function accepts both IPv6 and IPv4 prefixes, and the result always uses +the same addressing scheme as the given prefix. + +## Examples + +```shell-session +> cidrhost("10.12.127.0/20", 16) +10.12.112.16 +> cidrhost("10.12.127.0/20", 268) +10.12.113.12 +> cidrhost("fd00:fd12:3456:7890:00a2::/72", 34) +fd00:fd12:3456:7890::22 +``` + +## Related Functions + +- [`cidrsubnet`](/packer/docs/templates/hcl_templates/functions/ipnet/cidrsubnet) calculates a subnet address under a given + network address prefix. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/ipnet/cidrnetmask.mdx b/content/packer/content/docs/templates/hcl_templates/functions/ipnet/cidrnetmask.mdx new file mode 100644 index 0000000000..04869b9825 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/ipnet/cidrnetmask.mdx @@ -0,0 +1,31 @@ +--- +page_title: cidrnetmask function reference +description: |- + The `cidrnetmask` function converts an IPv4 address prefix given in CIDR + notation into a subnet mask address. +--- + +# `cidrnetmask` Function + +`cidrnetmask` converts an IPv4 address prefix given in CIDR notation into +a subnet mask address. + +```hcl +cidrnetmask(prefix) +``` + +`prefix` must be given in IPv4 CIDR notation, as defined in +[RFC 4632 section 3.1](https://tools.ietf.org/html/rfc4632#section-3.1). + +The result is a subnet address formatted in the conventional dotted-decimal +IPv4 address syntax, as expected by some software. + +CIDR notation is the only valid notation for IPv6 addresses, so `cidrnetmask` +produces an error if given an IPv6 address. + +## Examples + +```shell-session +> cidrnetmask("172.16.0.0/12") +255.240.0.0 +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/ipnet/cidrsubnet.mdx b/content/packer/content/docs/templates/hcl_templates/functions/ipnet/cidrsubnet.mdx new file mode 100644 index 0000000000..055bfb11c3 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/ipnet/cidrsubnet.mdx @@ -0,0 +1,166 @@ +--- +page_title: cidrsubnet function reference +description: |- + The `cidrsubnet` function calculates a subnet address within a given IP network + address prefix. +--- + +# `cidrsubnet` Function + +`cidrsubnet` calculates a subnet address within given IP network address prefix. + +```hcl +cidrsubnet(prefix, newbits, netnum) +``` + +`prefix` must be given in CIDR notation, as defined in +[RFC 4632 section 3.1](https://tools.ietf.org/html/rfc4632#section-3.1). + +`newbits` is the number of additional bits with which to extend the prefix. +For example, if given a prefix ending in `/16` and a `newbits` value of +`4`, the resulting subnet address will have length `/20`. + +`netnum` is a whole number that can be represented as a binary integer with +no more than `newbits` binary digits, which will be used to populate the +additional bits added to the prefix. + +This function accepts both IPv6 and IPv4 prefixes, and the result always uses +the same addressing scheme as the given prefix. + +Unlike the related function [`cidrsubnets`](/packer/docs/templates/hcl_templates/functions/ipnet/cidrsubnets), `cidrsubnet` +allows you to give a specific network number to use. `cidrsubnets` can allocate +multiple network addresses at once, but numbers them automatically starting +with zero. + +## Examples + +```shell-session +> cidrsubnet("172.16.0.0/12", 4, 2) +172.18.0.0/16 +> cidrsubnet("10.1.2.0/24", 4, 15) +10.1.2.240/28 +> cidrsubnet("fd00:fd12:3456:7890::/56", 16, 162) +fd00:fd12:3456:7800:a200::/72 +``` + +## Netmasks and Subnets + +Using `cidrsubnet` requires familiarity with some network addressing concepts. + +The most important idea is that an IP address (whether IPv4 or IPv6) is +fundamentally constructed from binary digits, even though we conventionally +represent it as either four decimal octets (for IPv4) or a sequence of 16-bit +hexadecimal numbers (for IPv6). + +Taking our example above of `cidrsubnet("10.1.2.0/24", 4, 15)`, the function +will first convert the given IP address string into an equivalent binary +representation: + +```text + 10 . 1 . 2 . 0 +00001010 00000001 00000010 | 00000000 + network | host +``` + +The `/24` at the end of the prefix string specifies that the first 24 +bits -- or, the first three octets -- of the address identify the network +while the remaining bits (32 - 24 = 8 bits in this case) identify hosts +within the network. + +The CLI tool [`ipcalc`](https://gitlab.com/ipcalc/ipcalc) is useful for +visualizing CIDR prefixes as binary numbers. We can confirm the conversion +above by providing the same prefix string to `ipcalc`: + +```shell-session +$ ipcalc 10.1.2.0/24 +Address: 10.1.2.0 00001010.00000001.00000010. 00000000 +Netmask: 255.255.255.0 = 24 11111111.11111111.11111111. 00000000 +Wildcard: 0.0.0.255 00000000.00000000.00000000. 11111111 +=> +Network: 10.1.2.0/24 00001010.00000001.00000010. 00000000 +HostMin: 10.1.2.1 00001010.00000001.00000010. 00000001 +HostMax: 10.1.2.254 00001010.00000001.00000010. 11111110 +Broadcast: 10.1.2.255 00001010.00000001.00000010. 11111111 +Hosts/Net: 254 Class A, Private Internet +``` + +This gives us some additional information but also confirms (using a slightly +different notation) the conversion from decimal to binary and shows the range +of possible host addresses in this network. + +While [`cidrhost`](/packer/docs/templates/hcl_templates/functions/ipnet/cidrhost) allows calculating single host IP addresses, +`cidrsubnet` on the other hand creates a new network prefix _within_ the given +network prefix. In other words, it creates a subnet. + +When we call `cidrsubnet` we also pass two additional arguments: `newbits` and +`netnum`. `newbits` decides how much longer the resulting prefix will be in +bits; in our example here we specified `4`, which means that the resulting +subnet will have a prefix length of 24 + 4 = 28 bits. We can imagine these +bits breaking down as follows: + +```text + 10 . 1 . 2 . ? 0 +00001010 00000001 00000010 | XXXX | 0000 + parent network | netnum | host +``` + +Four of the eight bits that were originally the "host number" are now being +repurposed as the subnet number. The network prefix no longer falls on an +exact octet boundary, so in effect we are now splitting the last decimal number +in the IP address into two parts, using half of it to represent the subnet +number and the other half to represent the host number. + +The `netnum` argument then decides what number value to encode into those +four new subnet bits. In our current example we passed `15`, which is +represented in binary as `1111`, allowing us to fill in the `XXXX` segment +in the above: + +```text + 10 . 1 . 2 . 15 0 +00001010 00000001 00000010 | 1111 | 0000 + parent network | netnum | host +``` + +To convert this back into normal decimal notation we need to recombine the +two portions of the final octet. Converting `11110000` from binary to decimal +gives 240, which can then be combined with our new prefix length of 28 to +produce the result `10.1.2.240/28`. Again we can pass this prefix string to +`ipcalc` to visualize it: + +```shell-session +$ ipcalc 10.1.2.240/28 +Address: 10.1.2.240 00001010.00000001.00000010.1111 0000 +Netmask: 255.255.255.240 = 28 11111111.11111111.11111111.1111 0000 +Wildcard: 0.0.0.15 00000000.00000000.00000000.0000 1111 +=> +Network: 10.1.2.240/28 00001010.00000001.00000010.1111 0000 +HostMin: 10.1.2.241 00001010.00000001.00000010.1111 0001 +HostMax: 10.1.2.254 00001010.00000001.00000010.1111 1110 +Broadcast: 10.1.2.255 00001010.00000001.00000010.1111 1111 +Hosts/Net: 14 Class A, Private Internet +``` + +The new subnet has four bits available for host numbering, which means +that there are 14 host addresses available for assignment once we subtract +the network's own address and the broadcast address. You can thus use +[`cidrhost`](/packer/docs/templates/hcl_templates/functions/ipnet/cidrhost) function to calculate those host addresses by +providing it a value between 1 and 14: + +```shell-session +> cidrhost("10.1.2.240/28", 1) +10.1.2.241 +> cidrhost("10.1.2.240/28", 14) +10.1.2.254 +``` + +For more information on CIDR notation and subnetting, see +[Classless Inter-domain Routing](https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing). + +## Related Functions + +- [`cidrhost`](/packer/docs/templates/hcl_templates/functions/ipnet/cidrhost) calculates the IP address for a single host + within a given network address prefix. +- [`cidrnetmask`](/packer/docs/templates/hcl_templates/functions/ipnet/cidrnetmask) converts an IPv4 network prefix in CIDR + notation into netmask notation. +- [`cidrsubnets`](/packer/docs/templates/hcl_templates/functions/ipnet/cidrsubnets) can allocate multiple consecutive + addresses under a prefix at once, numbering them automatically. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/ipnet/cidrsubnets.mdx b/content/packer/content/docs/templates/hcl_templates/functions/ipnet/cidrsubnets.mdx new file mode 100644 index 0000000000..71035f4ee9 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/ipnet/cidrsubnets.mdx @@ -0,0 +1,93 @@ +--- +page_title: cidrsubnets function reference +description: |- + The `cidrsubnets` function calculates a sequence of consecutive IP address + ranges in a CIDR prefix. +--- + +# `cidrsubnets` Function + +`cidrsubnets` calculates a sequence of consecutive IP address ranges within +a particular CIDR prefix. + +```hcl +cidrsubnets(prefix, newbits...) +``` + +`prefix` must be given in CIDR notation, as defined in +[RFC 4632 section 3.1](https://tools.ietf.org/html/rfc4632#section-3.1). + +The remaining arguments, indicated as `newbits` above, each specify the number +of additional network prefix bits for one returned address range. The return +value is therefore a list with one element per `newbits` argument, each +a string containing an address range in CIDR notation. + +For more information on IP addressing concepts, see the documentation for the +related function [`cidrsubnet`](/packer/docs/templates/hcl_templates/functions/ipnet/cidrsubnet). `cidrsubnet` calculates +a single subnet address within a prefix while allowing you to specify its +subnet number, while `cidrsubnets` can calculate many at once, potentially of +different sizes, and assigns subnet numbers automatically. + +When using this function to partition an address space as part of a network +address plan, you must not change any of the existing arguments once network +addresses have been assigned to real infrastructure, or else later address +assignments will be invalidated. However, you _can_ append new arguments to +existing calls safely, as long as there is sufficient address space available. + +This function accepts both IPv6 and IPv4 prefixes, and the result always uses +the same addressing scheme as the given prefix. + +## Examples + +```shell-session +> cidrsubnets("10.1.0.0/16", 4, 4, 8, 4) +[ + "10.1.0.0/20", + "10.1.16.0/20", + "10.1.32.0/24", + "10.1.48.0/20", +] + +> cidrsubnets("fd00:fd12:3456:7890::/56", 16, 16, 16, 32) +[ + "fd00:fd12:3456:7800::/72", + "fd00:fd12:3456:7800:100::/72", + "fd00:fd12:3456:7800:200::/72", + "fd00:fd12:3456:7800:300::/88", +] +``` + +You can use nested `cidrsubnets` calls with +[`for` expressions](/packer/docs/templates/hcl_templates/expressions#for-expressions) +to concisely allocate groups of network address blocks: + +```shell-session +> [for cidr_block in cidrsubnets("10.0.0.0/8", 8, 8, 8, 8) : cidrsubnets(cidr_block, 4, 4)] +[ + [ + "10.0.0.0/20", + "10.0.16.0/20", + ], + [ + "10.1.0.0/20", + "10.1.16.0/20", + ], + [ + "10.2.0.0/20", + "10.2.16.0/20", + ], + [ + "10.3.0.0/20", + "10.3.16.0/20", + ], +] +``` + +## Related Functions + +- [`cidrhost`](/packer/docs/templates/hcl_templates/functions/ipnet/cidrhost) calculates the IP address for a single host + within a given network address prefix. +- [`cidrnetmask`](/packer/docs/templates/hcl_templates/functions/ipnet/cidrnetmask) converts an IPv4 network prefix in CIDR + notation into netmask notation. +- [`cidrsubnet`](/packer/docs/templates/hcl_templates/functions/ipnet/cidrsubnet) calculates a single subnet address, allowing + you to specify its network number. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/numeric/abs.mdx b/content/packer/content/docs/templates/hcl_templates/functions/numeric/abs.mdx new file mode 100644 index 0000000000..f3b806e7c7 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/numeric/abs.mdx @@ -0,0 +1,21 @@ +--- +page_title: abs function reference +description: The `abs` function returns the absolute value of the given number. +--- + +# `abs` Function + +`abs` returns the absolute value of the given number. In other words, if the +number is zero or positive then it is returned as-is, but if it is negative +then it is multiplied by -1 to make it positive before returning it. + +## Examples + +```shell-session +> abs(23) +23 +> abs(0) +0 +> abs(-12.4) +12.4 +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/numeric/ceil.mdx b/content/packer/content/docs/templates/hcl_templates/functions/numeric/ceil.mdx new file mode 100644 index 0000000000..8034b0f4db --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/numeric/ceil.mdx @@ -0,0 +1,25 @@ +--- +page_title: ceil function reference +description: |- + The `ceil` function returns the closest whole number greater than or equal to + the given value. +--- + +# `ceil` Function + +`ceil` returns the closest whole number that is greater than or equal to the +given value, which may be a fraction. + +## Examples + +```shell-session +> ceil(5) +5 +> ceil(5.1) +6 +``` + +## Related Functions + +- [`floor`](/packer/docs/templates/hcl_templates/functions/numeric/floor), which rounds to the nearest whole number _less than_ + or equal. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/numeric/floor.mdx b/content/packer/content/docs/templates/hcl_templates/functions/numeric/floor.mdx new file mode 100644 index 0000000000..574e6d61bf --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/numeric/floor.mdx @@ -0,0 +1,25 @@ +--- +page_title: floor function reference +description: |- + The `floor` function returns the closest whole number less than or equal to + the given value. +--- + +# `floor` Function + +`floor` returns the closest whole number that is less than or equal to the +given value, which may be a fraction. + +## Examples + +```shell-session +> floor(5) +5 +> floor(4.9) +4 +``` + +## Related Functions + +- [`ceil`](/packer/docs/templates/hcl_templates/functions/numeric/ceil), which rounds to the nearest whole number _greater than_ + or equal. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/numeric/log.mdx b/content/packer/content/docs/templates/hcl_templates/functions/numeric/log.mdx new file mode 100644 index 0000000000..591b50a441 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/numeric/log.mdx @@ -0,0 +1,33 @@ +--- +page_title: log function reference +description: The `log` function returns the logarithm of a given number in a given base. +--- + +# `log` Function + +`log` returns the logarithm of a given number in a given base. + +```hcl +log(number, base) +``` + +## Examples + +```shell-session +> log(50, 10) +1.6989700043360185 +> log(16, 2) +4 +``` + +`log` and `ceil` can be used together to find the minimum number of binary +digits required to represent a given number of distinct values: + +```shell-session +> ceil(log(15, 2)) +4 +> ceil(log(16, 2)) +4 +> ceil(log(17, 2)) +5 +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/numeric/max.mdx b/content/packer/content/docs/templates/hcl_templates/functions/numeric/max.mdx new file mode 100644 index 0000000000..f35dc050a4 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/numeric/max.mdx @@ -0,0 +1,27 @@ +--- +page_title: max function reference +description: The `max` function returns the largest number from the given set of one or more numbers. +--- + +# `max` Function + +`max` takes one or more numbers and returns the greatest number from the set. + +## Examples + +```shell-session +> max(12, 54, 3) +54 +``` + +If the numbers are in a list or set value, use `...` to expand the collection +to individual arguments: + +```shell-session +> max([12, 54, 3]...) +54 +``` + +## Related Functions + +- [`min`](/packer/docs/templates/hcl_templates/functions/numeric/min), which returns the _smallest_ number from a set. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/numeric/min.mdx b/content/packer/content/docs/templates/hcl_templates/functions/numeric/min.mdx new file mode 100644 index 0000000000..4915ddd0d2 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/numeric/min.mdx @@ -0,0 +1,27 @@ +--- +page_title: min function reference +description: The `min` function returns the smallest number in a set of one or more numbers. +--- + +# `min` Function + +`min` takes one or more numbers and returns the smallest number from the set. + +## Examples + +```shell-session +> min(12, 54, 3) +3 +``` + +If the numbers are in a list or set value, use `...` to expand the collection +to individual arguments: + +```shell-session +> min([12, 54, 3]...) +3 +``` + +## Related Functions + +- [`max`](/packer/docs/templates/hcl_templates/functions/numeric/max), which returns the _greatest_ number from a set. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/numeric/parseint.mdx b/content/packer/content/docs/templates/hcl_templates/functions/numeric/parseint.mdx new file mode 100644 index 0000000000..4665b2919f --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/numeric/parseint.mdx @@ -0,0 +1,50 @@ +--- +page_title: parseint function reference +description: >- + The `parseint` function parses the given string as a representation of an + integer. +--- + +# `parseint` Function + +`parseint` parses the given string as a representation of an integer in +the specified base and returns the resulting number. The base must be between 2 +and 62 inclusive. + +All bases use the arabic numerals 0 through 9 first. Bases between 11 and 36 +inclusive use case-insensitive latin letters to represent higher unit values. +Bases 37 and higher use lowercase latin letters and then uppercase latin +letters. + +If the given string contains any non-digit characters or digit characters that +are too large for the given base then `parseint` will produce an error. + +## Examples + +```shell-session +> parseint("100", 10) +100 + +> parseint("FF", 16) +255 + +> parseint("-10", 16) +-16 + +> parseint("1011111011101111", 2) +48879 + +> parseint("aA", 62) +656 + +> parseint("12", 2) + +Error: Invalid function argument + +Invalid value for "number" parameter: cannot parse "12" as a base 2 integer. +``` + +## Related Functions + +- [`format`](/packer/docs/templates/hcl_templates/functions/string/format) can format numbers and other values into strings, + with optional zero padding, alignment, etc. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/numeric/pow.mdx b/content/packer/content/docs/templates/hcl_templates/functions/numeric/pow.mdx new file mode 100644 index 0000000000..ca90688f6b --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/numeric/pow.mdx @@ -0,0 +1,17 @@ +--- +page_title: pow function reference +description: The `pow` function raises a number to a power. +--- + +# `pow` Function + +`pow` calculates an exponent, by raising its first argument to the power of the second argument. + +## Examples + +```shell-session +> pow(3, 2) +9 +> pow(4, 0) +1 +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/numeric/signum.mdx b/content/packer/content/docs/templates/hcl_templates/functions/numeric/signum.mdx new file mode 100644 index 0000000000..d10002a387 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/numeric/signum.mdx @@ -0,0 +1,20 @@ +--- +page_title: signum function reference +description: The `signum` function determines the sign of a number. +--- + +# `signum` Function + +`signum` determines the sign of a number, returning a number between -1 and +1 to represent the sign. + +## Examples + +```shell-session +> signum(-13) +-1 +> signum(0) +0 +> signum(344) +1 +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/numeric/sum.mdx b/content/packer/content/docs/templates/hcl_templates/functions/numeric/sum.mdx new file mode 100644 index 0000000000..91ae0d9a0b --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/numeric/sum.mdx @@ -0,0 +1,17 @@ +--- +page_title: sum - Functions - Configuration Language +description: The sum function takes a list or set of numbers and returns the sum of those numbers. +--- + +# `sum` Function + +`sum` takes a list or set of numbers and returns the sum of those numbers. + +`sum` fails when given an empty list or set. + +## Examples + +``` +> sum([10, 13, 6, 4.5]) +33.5 +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/chomp.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/chomp.mdx new file mode 100644 index 0000000000..dd311a204e --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/chomp.mdx @@ -0,0 +1,27 @@ +--- +page_title: chomp function reference +description: The `chomp` function removes newline characters at the end of a string. +--- + +# `chomp` Function + +`chomp` removes newline characters at the end of a string. + +This can be useful if, for example, the string was read from a file that has +a newline character at the end. + +## Examples + +```shell-session +> chomp("hello\n") +hello +> chomp("hello\r\n") +hello +> chomp("hello\n\n") +hello +``` + +## Related Functions + +- [`trimspace`](/packer/docs/templates/hcl_templates/functions/string/trimspace), which removes all types of whitespace from + both the start and the end of a string. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/endswith.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/endswith.mdx new file mode 100644 index 0000000000..3cfc87d211 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/endswith.mdx @@ -0,0 +1,27 @@ +--- +page_title: endswith - Functions - Configuration Language +description: |- + The endswith function takes two values: a string to check and a suffix string. It returns true if the first string ends with that exact suffix. +--- + +# `endswith` Function + +`endswith` takes two values: a string to check and a suffix string. The function returns true if the first string ends with that exact suffix. + +```hcl +endswith(string, suffix) +``` + +## Examples + +``` +> endswith("hello world", "world") +true +> endswith("hello world", "hello") +false +``` + +## Related Functions + +- [`startswith`](/packer/docs/templates/hcl_templates/functions/string/startswith) takes two values: a string to check + and a prefix string. The function returns true if the string begins with that exact prefix. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/format.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/format.mdx new file mode 100644 index 0000000000..3bfd9ee3a8 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/format.mdx @@ -0,0 +1,118 @@ +--- +page_title: format function reference +description: |- + The `format` function applies a specification to format values and produce a new string. +--- + +# `format` Function + +`format` produces a string by formatting a number of other values according +to a specification string. It is similar to the `printf` function in C, and +other similar functions in other programming languages. + +```hcl +format(spec, values...) +``` + +## Examples + +```shell-session +> format("Hello, %s!", "Ander") +Hello, Ander! +> format("There are %d lights", 4) +There are 4 lights +``` + +Simple format verbs like `%s` and `%d` behave similarly to template +interpolation syntax, which is often more readable: + +```shell-session +> format("Hello, %s!", var.name) +Hello, Valentina! +> "Hello, ${var.name}!" +Hello, Valentina! +``` + +The `format` function is therefore more useful when you use more complex format +specifications, as described in the following section. + +## Specification Syntax + +The specification is a string that includes formatting verbs that are introduced +with the `%` character. The function call must then have one additional argument +for each verb sequence in the specification. The verbs are matched with +consecutive arguments and formatted as directed, as long as each given argument +is convertible to the type required by the format verb. + +The specification may contain the following verbs: + +| Verb | Result | +| ----- | ----------------------------------------------------------------------------------------- | +| `%%` | Literal percent sign, consuming no value. | +| `%v` | Default formatting based on the value type, as described below. | +| `%#v` | JSON serialization of the value, as with `jsonencode`. | +| `%t` | Convert to boolean and produce `true` or `false`. | +| `%b` | Convert to integer number and produce binary representation. | +| `%d` | Convert to integer number and produce decimal representation. | +| `%o` | Convert to integer number and produce octal representation. | +| `%x` | Convert to integer number and produce hexadecimal representation with lowercase letters. | +| `%X` | Like `%x`, but use uppercase letters. | +| `%e` | Convert to number and produce scientific notation, like `-1.234456e+78`. | +| `%E` | Like `%e`, but use an uppercase `E` to introduce the exponent. | +| `%f` | Convert to number and produce decimal fraction notation with no exponent, like `123.456`. | +| `%g` | Like `%e` for large exponents or like `%f` otherwise. | +| `%G` | Like `%E` for large exponents or like `%f` otherwise. | +| `%s` | Convert to string and insert the string's characters. | +| `%q` | Convert to string and produce a JSON quoted string representation. | + +When `%v` is used, one of the following format verbs is chosen based on the value type: + +| Type | Verb | +| --------- | ----- | +| `string` | `%s` | +| `number` | `%g` | +| `bool` | `%t` | +| any other | `%#v` | + +Null values produce the string `null` if formatted with `%v` or `%#v`, and +cause an error for other verbs. + +A width modifier can be included with an optional decimal number immediately +preceding the verb letter, to specify how many characters will be used to +represent the value. Precision can be specified after the (optional) width +with a period (`.`) followed by a decimal number. If width or precision are +omitted then default values are selected based on the given value. For example: + +| Sequence | Result | +| -------- | ---------------------------- | +| `%f` | Default width and precision. | +| `%9f` | Width 9, default precision. | +| `%.2f` | Default width, precision 2. | +| `%9.2f` | Width 9, precision 2. | + +The following additional symbols can be used immediately after the `%` symbol +to set additional flags: + +| Symbol | Result | +| ------ | -------------------------------------------------------------- | +| space | Leave a space where the sign would be if a number is positive. | +| `+` | Show the sign of a number even if it is positive. | +| `-` | Pad the width with spaces on the left rather than the right. | +| `0` | Pad the width with leading zeros rather than spaces. | + +By default, `%` sequences consume successive arguments starting with the first. +Introducing a `[n]` sequence immediately before the verb letter, where `n` is a +decimal integer, explicitly chooses a particular value argument by its +one-based index. Subsequent calls without an explicit index will then proceed +with `n`+1, `n`+2, etc. + +The function produces an error if the format string requests an impossible +conversion or access more arguments than are given. An error is produced also +for an unsupported format verb. + +## Related Functions + +- [`formatdate`](/packer/docs/templates/hcl_templates/functions/datetime/formatdate) is a specialized formatting function for + human-readable timestamps. +- [`formatlist`](/packer/docs/templates/hcl_templates/functions/string/formatlist) uses the same specification syntax to + produce a list of strings. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/formatlist.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/formatlist.mdx new file mode 100644 index 0000000000..edf3284eab --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/formatlist.mdx @@ -0,0 +1,48 @@ +--- +page_title: formatlist function reference +description: |- + The `formatlist` function applies a specification to format values and produce a list of strings. +--- + +# `formatlist` Function + +`formatlist` produces a list of strings by formatting a number of other +values according to a specification string. + +```hcl +formatlist(spec, values...) +``` + +The specification string uses +[the same syntax as `format`](/packer/docs/templates/hcl_templates/functions/string/format#specification-syntax). + +The given values can be a mixture of list and non-list arguments. Any given +lists must be the same length, which decides the length of the resulting list. + +The list arguments are iterated together in order by index, while the non-list +arguments are used repeatedly for each iteration. The format string is evaluated +once per element of the list arguments. + +## Examples + +```shell-session +> formatlist("Hello, %s!", ["Valentina", "Ander", "Olivia", "Sam"]) +[ + "Hello, Valentina!", + "Hello, Ander!", + "Hello, Olivia!", + "Hello, Sam!", +] +> formatlist("%s, %s!", "Salutations", ["Valentina", "Ander", "Olivia", "Sam"]) +[ + "Salutations, Valentina!", + "Salutations, Ander!", + "Salutations, Olivia!", + "Salutations, Sam!", +] +``` + +## Related Functions + +- [`format`](/packer/docs/templates/hcl_templates/functions/string/format) defines the specification syntax used by this + function and produces a single string as its result. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/indent.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/indent.mdx new file mode 100644 index 0000000000..d91428b7a4 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/indent.mdx @@ -0,0 +1,31 @@ +--- +page_title: indent function reference +description: |- + The `indent` function adds spaces to the beginnings of all but the + first line of a given multi-line string. +--- + +# `indent` Function + +`indent` adds a given number of spaces to the beginnings of all but the first +line in a given multi-line string. + +```hcl +indent(num_spaces, string) +``` + +## Examples + +This function is useful for inserting a multi-line string into an +already-indented context in another string: + +```shell-session +> " items: %{indent(2, "[\n foo,\n bar,\n]\n")}" + items: [ + foo, + bar, + ] +``` + +The first line of the string is not indented so that, as above, it can be +placed after an introduction sequence that has already begun the line. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/join.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/join.mdx new file mode 100644 index 0000000000..149c874df5 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/join.mdx @@ -0,0 +1,29 @@ +--- +page_title: join function reference +description: |- + The `join` function produces a string by concatenating the elements of a list + with a given delimiter. +--- + +# `join` Function + +`join` produces a string by concatenating together all elements of a given +list of strings with the given delimiter. + +```hcl +join(separator, list) +``` + +## Examples + +```shell-session +> join(", ", ["foo", "bar", "baz"]) +foo, bar, baz +> join(", ", ["foo"]) +foo +``` + +## Related Functions + +- [`split`](/packer/docs/templates/hcl_templates/functions/string/split) performs the opposite operation: producing a list + by separating a single string using a given delimiter. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/lower.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/lower.mdx new file mode 100644 index 0000000000..8815d01b34 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/lower.mdx @@ -0,0 +1,26 @@ +--- +page_title: lower function reference +description: >- + The `lower` function converts all uppercase letters in the given string to + lowercase. +--- + +# `lower` Function + +`lower` converts all cased letters in the given string to lowercase. + +## Examples + +```shell-session +> lower("HELLO") +hello +> lower("АЛЛО!") +алло! +``` + +This function uses Unicode's definition of letters and of upper- and lowercase. + +## Related Functions + +- [`upper`](/packer/docs/templates/hcl_templates/functions/string/upper) converts letters in a string to _uppercase_. +- [`title`](/packer/docs/templates/hcl_templates/functions/string/title) converts the first letter of each word in a string to uppercase. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/regex.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/regex.mdx new file mode 100644 index 0000000000..316149f424 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/regex.mdx @@ -0,0 +1,155 @@ +--- +page_title: regex function reference +description: |- + The `regex` function applies a regular expression to a string and returns the + matching substrings. +--- + +# `regex` Function + +`regex` applies a +[regular expression](https://en.wikipedia.org/wiki/Regular_expression) +to a string and returns the matching substrings. + +```hcl +regex(pattern, string) +``` + +The return type of `regex` depends on the capture groups, if any, in the +pattern: + +- If the pattern has no capture groups at all, the result is a single string + covering the substring matched by the pattern as a whole. +- If the pattern has one or more _unnamed_ capture groups, the result is a + list of the captured substrings in the same order as the definition of + the capture groups. +- If the pattern has one or more _named_ capture groups, the result is a + map of the captured substrings, using the capture group names as map keys. + +It's not valid to mix both named and unnamed capture groups in the same pattern. + +If the given pattern does not match at all, the `regex` raises an error. To +_test_ whether a given pattern matches a string, use +[`regexall`](/packer/docs/templates/hcl_templates/functions/string/regexall) and test that the result has length greater than +zero. + +The pattern is a string containing a mixture of literal characters and special +matching operators as described in the following table. Note that when giving a +regular expression pattern as a literal quoted string in the Packer +language, the quoted string itself already uses backslash `\` as an escape +character for the string, so any backslashes intended to be recognized as part +of the pattern must be escaped as `\\`. + +| Sequence | Matches | +| --------------------- | -------------------------------------------------------------------------------- | +| `.` | Any character except newline | +| `[xyz]` | Any character listed between the brackets (`x`, `y`, and `z` in this example) | +| `[a-z]` | Any character between `a` and `z`, inclusive | +| `[^xyz]` | The opposite of `[xyz]` | +| `\d` | ASCII digits (0 through 9, inclusive) | +| `\D` | Anything except ASCII digits | +| `\s` | ASCII spaces (space, tab, newline, carriage return, form feed) | +| `\S` | Anything except ASCII spaces | +| `\w` | The same as `[0-9A-Za-z_]` | +| `\W` | Anything except the characters matched by `\w` | +| `[[:alnum:]]` | The same as `[0-9A-Za-z]` | +| `[[:alpha:]]` | The same as `[A-Za-z]` | +| `[[:ascii:]]` | Any ASCII character | +| `[[:blank:]]` | ASCII tab or space | +| `[[:cntrl:]]` | ASCII/Unicode control characters | +| `[[:digit:]]` | The same as `[0-9]` | +| `[[:graph:]]` | All "graphical" (printable) ASCII characters | +| `[[:lower:]]` | The same as `[a-z]` | +| `[[:print:]]` | The same as `[[:graph:]]` | +| `[[:punct:]]` | The same as `` [!-/:-@[-`{-~] `` | +| `[[:space:]]` | The same as `[\t\n\v\f\r ]` | +| `[[:upper:]]` | The same as `[A-Z]` | +| `[[:word:]]` | The same as `\w` | +| `[[:xdigit:]]` | The same as `[0-9A-Fa-f]` | +| `\pN` | Unicode character class by using single-letter class names ("N" in this example) | +| `\p{Greek}` | Unicode character class by unicode name ("Greek" in this example) | +| `\PN` | The opposite of `\pN` | +| `\P{Greek}` | The opposite of `\p{Greek}` | +| `xy` | `x` followed immediately by `y` | +| x|y | either `x` or `y`, preferring `x` | +| `x*` | zero or more `x`, preferring more | +| `x*?` | zero or more `x`, preferring fewer | +| `x+` | one or more `x`, preferring more | +| `x+?` | one or more `x`, preferring fewer | +| `x?` | zero or one `x`, preferring one | +| `x??` | zero or one `x`, preferring zero | +| `x{n,m}` | between `n` and `m` repetitions of `x`, preferring more | +| `x{n,m}?` | between `n` and `m` repetitions of `x`, preferring fewer | +| `x{n,}` | at least `n` repetitions of `x`, preferring more | +| `x{n,}?` | at least `n` repetitions of `x`, preferring fewer | +| `x{n}` | exactly `n` repetitions of `x` | +| `(x)` | unnamed capture group for sub-pattern `x` | +| `(?Px)` | named capture group, named `name`, for sub-pattern `x` | +| `(?:x)` | non-capturing sub-pattern `x` | +| `\*` | Literal `*` for any punctuation character `*` | +| `\Q...\E` | Literal `...` for any text `...` as long as it does not include literally `\E` | + +In addition to the above matching operators that consume the characters they +match, there are some additional operators that _only_ match, but consume +no characters. These are "zero-width" matching operators: + +| Sequence | Matches | +| -------- | ------------------------------------------------------------------------------------------------ | +| `^` | At the beginning of the given string | +| `$` | At the end of the given string | +| `\A` | At the beginning of the given string | +| `\z` | At the end of the given string | +| `\b` | At an ASCII word boundary (transition between `\w` and either `\W`, `\A` or `\z`, or vice-versa) | +| `\B` | Not at an ASCII word boundary | + +Packer uses the +[RE2](https://github.com/google/re2/wiki/Syntax) regular expression language. +This engine does not support all of the features found in some other regular +expression engines; in particular, it does not support backreferences. + +## Matching Flags + +Some of the matching behaviors described above can be modified by setting +matching flags, activated using either the `(?flags)` operator (to activate +within the current sub-pattern) or the `(?flags:x)` operator (to match `x` with +the modified flags). Each flag is a single letter, and multiple flags can be +set at once by listing multiple letters in the `flags` position. +The available flags are listed in the table below: + +| Flag | Meaning | +| ---- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `i` | Case insensitive: a literal letter in the pattern matches both lowercase and uppercase versions of that letter | +| `m` | The `^` and `$` operators also match the beginning and end of lines within the string, marked by newline characters; behavior of `\A` and `\z` is unchanged | +| `s` | The `.` operator also matches newline | +| `U` | The meaning of presence or absense `?` after a repetition operator is inverted. For example, `x*` is interpreted like `x*?` and vice-versa. | + +## Examples + +``` +> regex("[a-z]+", "53453453.345345aaabbbccc23454") +aaabbbccc + +> regex("(\\d\\d\\d\\d)-(\\d\\d)-(\\d\\d)", "2019-02-01") +[ + "2019", + "02", + "01", +] + +> regex("[a-z]+", "53453453.34534523454") + +Error: Error in function call + +Call to function "regex" failed: pattern did not match any part of the given +string. +``` + +## Related Functions + +- [`regexall`](/packer/docs/templates/hcl_templates/functions/string/regexall) searches for potentially multiple matches of a given pattern in a string. +- [`replace`](/packer/docs/templates/hcl_templates/functions/string/replace) replaces a substring of a string with another string, optionally matching using the same regular expression syntax as `regex`. + +If Packer already has a more specialized function to parse the syntax you +are trying to match, prefer to use that function instead. Regular expressions +can be hard to read and can obscure your intent, making a configuration harder +to read and understand. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/regex_replace.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/regex_replace.mdx new file mode 100644 index 0000000000..f5e02ec70b --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/regex_replace.mdx @@ -0,0 +1,42 @@ +--- +page_title: regex_replace function reference +description: >- + The `regex_replace` function matches and replaces all occurrences of a substring with a replacement string. +--- + +# `regex_replace` Function + +`regex_replace` searches a given string for another given substring, and +replaces each occurrence with a given replacement string. The substring +argument can be a valid regular expression or a string. + +```hcl +regex_replace(string, substring, replacement) +``` + +`substring` should not be wrapped in forward slashes, it is always treated as a +regular expression. The `replacement` string can incorporate captured strings +from the input by using an `$n` or `${n}` sequence, where `n` is the index or +name of a capture group. + +## Examples + +```shell-session +> regex_replace("hello world", "world", "everybody") +hello everybody + + +> regex_replace("hello world", "w.*d", "everybody") +hello everybody + +> regex_replace("-ab-axxb-", "a(x*)b", "$1W") +--- + +> regex_replace("-ab-axxb-", "a(x*)b", "${1}W") +-W-xxW- +``` + +## Related Functions + +- [`replace`](/packer/docs/templates/hcl_templates/functions/string/replace) searches a given string for another given + substring, and replaces all occurrences with a given replacement string. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/regexall.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/regexall.mdx new file mode 100644 index 0000000000..0a1f92a853 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/regexall.mdx @@ -0,0 +1,56 @@ +--- +page_title: regexall function reference +description: |- + The `regexall` function applies a regular expression to a string and returns a list of all matches. +--- + +# `regexall` Function + +`regexall` applies a +[regular expression](https://en.wikipedia.org/wiki/Regular_expression) +to a string and returns a list of all matches. + +```hcl +regexall(pattern, string) +``` + +`regexall` is a variant of [`regex`](/packer/docs/templates/hcl_templates/functions/string/regex) and uses the same pattern +syntax. For any given input to `regex`, `regexall` returns a list of whatever +type `regex` would've returned, with one element per match. That is: + +- If the pattern has no capture groups at all, the result is a list of + strings. +- If the pattern has one or more _unnamed_ capture groups, the result is a + list of lists. +- If the pattern has one or more _named_ capture groups, the result is a + list of maps. + +`regexall` can also be used to test whether a particular string matches a +given pattern, by testing whether the length of the resulting list of matches +is greater than zero. + +## Examples + +``` +> regexall("[a-z]+", "1234abcd5678efgh9") +[ + "abcd", + "efgh", +] + +> length(regexall("[a-z]+", "1234abcd5678efgh9")) +2 + +> length(regexall("[a-z]+", "123456789")) > 0 +false +``` + +## Related Functions + +- [`regex`](/packer/docs/templates/hcl_templates/functions/string/regex) searches for a single match of a given pattern, and + returns an error if no match is found. + +If Packer already has a more specialized function to parse the syntax you +are trying to match, prefer to use that function instead. Regular expressions +can be hard to read and can obscure your intent, making a configuration harder +to read and understand. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/replace.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/replace.mdx new file mode 100644 index 0000000000..e430827c7b --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/replace.mdx @@ -0,0 +1,29 @@ +--- +page_title: replace function reference +description: |- + The `replace` function replaces all occurrences of substring with a given string. +--- + +# `replace` Function + +`replace` searches a given string for another given substring, and replaces +each occurrence with a given replacement string. + +```hcl +replace(string, substring, replacement) +``` + +## Examples + +```shell-session +> replace("1 + 2 + 3", "+", "-") +1 - 2 - 3 + +> replace("hello world", "world", "everybody") +hello everybody +``` + +## Related Functions + +- [`regex_replace`](/packer/docs/templates/hcl_templates/functions/string/regex_replace) searches a given string for another given substring, + and replaces each occurrence with a given replacement string. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/split.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/split.mdx new file mode 100644 index 0000000000..8ee3d57880 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/split.mdx @@ -0,0 +1,38 @@ +--- +page_title: split function reference +description: |- + The `split` function divides the given string according to the specified separator to produce a list. +--- + +# `split` Function + +`split` produces a list by dividing a given string at all occurrences of a +given separator. + +```hcl +split(separator, string) +``` + +## Examples + +```shell-session +> split(",", "foo,bar,baz") +[ + "foo", + "bar", + "baz", +] +> split(",", "foo") +[ + "foo", +] +> split(",", "") +[ + "", +] +``` + +## Related Functions + +- [`join`](/packer/docs/templates/hcl_templates/functions/string/join) performs the opposite operation: producing a string + joining together a list of strings with a given separator. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/startswith.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/startswith.mdx new file mode 100644 index 0000000000..2faa5391a6 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/startswith.mdx @@ -0,0 +1,27 @@ +--- +page_title: startsswith - Functions - Configuration Language +description: |- + The startswith function takes two values: a string to check and a prefix string. It returns true if the string begins with that exact prefix. +--- + +# `startswith` Function + +`startswith` takes two values: a string to check and a prefix string. The function returns true if the string begins with that exact prefix. + +```hcl +startswith(string, prefix) +``` + +## Examples + +``` +> startswith("hello world", "hello") +true +> startswith("hello world", "world") +false +``` + +## Related Functions + +- [`endswith`](/packer/docs/templates/hcl_templates/functions/string/endswith) takes two values: a string to check + and a suffix string. The function returns true if the first string ends with that exact suffix. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/strcontains.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/strcontains.mdx new file mode 100644 index 0000000000..fec3f45224 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/strcontains.mdx @@ -0,0 +1,25 @@ +--- +page_title: strcontains function reference +description: |- + The `strcontains` function searches for a substring within a string. +--- + +# `strcontains` Function + +The `strcontains` function searches for a substring within a string. + +```hcl +strcontains(string, substr) +``` + +## Examples + +``` +> strcontains("hello world", "wor") +true +``` + +``` +> strcontains("hello world", "wod") +false +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/strrev.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/strrev.mdx new file mode 100644 index 0000000000..0b1c08b445 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/strrev.mdx @@ -0,0 +1,26 @@ +--- +page_title: strrev function reference +description: The `strrev` function reverses a string. +--- + +# `strrev` Function + +`strrev` reverses the characters in a string. +Note that the characters are treated as _Unicode characters_ (in technical terms, Unicode [grapheme cluster boundaries](https://unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries) are respected). + +```hcl +strrev(string) +``` + +## Examples + +```shell-session +> strrev("hello") +olleh +> strrev("a ☃") +☃ a +``` + +## Related Functions + +- [`reverse`](/packer/docs/templates/hcl_templates/functions/collection/reverse) reverses a sequence. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/substr.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/substr.mdx new file mode 100644 index 0000000000..d186768b5d --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/substr.mdx @@ -0,0 +1,29 @@ +--- +page_title: substr function reference +description: |- + The `substr` function extracts a substring from a given string using an offset and + length. +--- + +# `substr` Function + +`substr` extracts a substring from a given string by offset and length. + +```hcl +substr(string, offset, length) +``` + +## Examples + +```shell-session +> substr("hello world", 1, 4) +ello +``` + +The offset and length are both counted in _unicode characters_ rather than +bytes: + +```shell-session +> substr("🤔🤷", 0, 1) +🤔 +``` diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/title.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/title.mdx new file mode 100644 index 0000000000..37b6ce2e0d --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/title.mdx @@ -0,0 +1,24 @@ +--- +page_title: title function reference +description: |- + The `title` function converts the first letter of each word in a given string + to uppercase. +--- + +# `title` Function + +`title` converts the first letter of each word in the given string to uppercase. + +## Examples + +```shell-session +> title("hello world") +Hello World +``` + +This function uses Unicode's definition of letters and of upper- and lowercase. + +## Related Functions + +- [`upper`](/packer/docs/templates/hcl_templates/functions/string/upper) converts _all_ letters in a string to uppercase. +- [`lower`](/packer/docs/templates/hcl_templates/functions/string/lower) converts all letters in a string to lowercase. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/trim.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/trim.mdx new file mode 100644 index 0000000000..bbd378953e --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/trim.mdx @@ -0,0 +1,25 @@ +--- +page_title: trim function reference +description: |- + The `trim` function removes the specified characters from the start and end of + a given string. +--- + +# `trim` Function + +`trim` removes the specified characters from the start and end of the given +string. + +## Examples + +```shell-session +> trim("?!hello?!", "!?") +hello +``` + +## Related Functions + +- [`trimprefix`](/packer/docs/templates/hcl_templates/functions/string/trimprefix) removes a word from the start of a string. +- [`trimsuffix`](/packer/docs/templates/hcl_templates/functions/string/trimsuffix) removes a word from the end of a string. +- [`trimspace`](/packer/docs/templates/hcl_templates/functions/string/trimspace) removes all types of whitespace from + both the start and the end of a string. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/trimprefix.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/trimprefix.mdx new file mode 100644 index 0000000000..29ddb7df79 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/trimprefix.mdx @@ -0,0 +1,24 @@ +--- +page_title: trimprefix function reference +description: |- + The `trimprefix` function removes the specified prefix from the start of a + given string. +--- + +# `trimprefix` Function + +`trimprefix` removes the specified prefix from the start of the given string. + +## Examples + +```shell-session +> trimprefix("helloworld", "hello") +world +``` + +## Related Functions + +- [`trim`](/packer/docs/templates/hcl_templates/functions/string/trim) removes characters at the start and end of a string. +- [`trimsuffix`](/packer/docs/templates/hcl_templates/functions/string/trimsuffix) removes a word from the end of a string. +- [`trimspace`](/packer/docs/templates/hcl_templates/functions/string/trimspace) removes all types of whitespace from + both the start and the end of a string. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/trimspace.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/trimspace.mdx new file mode 100644 index 0000000000..22308139db --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/trimspace.mdx @@ -0,0 +1,27 @@ +--- +page_title: trimspace function reference +description: |- + The `trimspace` function removes space characters from the start and end of + a given string. +--- + +# `trimspace` Function + +`trimspace` removes any space characters from the start and end of the given +string. + +This function follows the Unicode definition of "space", which includes +regular spaces, tabs, newline characters, and various other space-like +characters. + +## Examples + +```shell-session +> trimspace(" hello\n\n") +hello +``` + +## Related Functions + +- [`chomp`](/packer/docs/templates/hcl_templates/functions/string/chomp) removes just line ending characters from the _end_ of + a string. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/trimsuffix.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/trimsuffix.mdx new file mode 100644 index 0000000000..9c8ec3945d --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/trimsuffix.mdx @@ -0,0 +1,24 @@ +--- +page_title: trimsuffix function reference +description: |- + The `trimsuffix` function removes the specified suffix from the end of a + given string. +--- + +# `trimsuffix` Function + +`trimsuffix` removes the specified suffix from the end of the given string. + +## Examples + +```shell-session +> trimsuffix("helloworld", "world") +hello +``` + +## Related Functions + +- [`trim`](/packer/docs/templates/hcl_templates/functions/string/trim) removes characters at the start and end of a string. +- [`trimprefix`](/packer/docs/templates/hcl_templates/functions/string/trimprefix) removes a word from the start of a string. +- [`trimspace`](/packer/docs/templates/hcl_templates/functions/string/trimspace) removes all types of whitespace from + both the start and the end of a string. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/string/upper.mdx b/content/packer/content/docs/templates/hcl_templates/functions/string/upper.mdx new file mode 100644 index 0000000000..62d83c5152 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/string/upper.mdx @@ -0,0 +1,26 @@ +--- +page_title: upper function reference +description: >- + The `upper` function converts all lowercase letters in the given string to + uppercase. +--- + +# `upper` Function + +`upper` converts all cased letters in the given string to uppercase. + +## Examples + +```shell-session +> upper("hello") +HELLO +> upper("алло!") +АЛЛО! +``` + +This function uses Unicode's definition of letters and of upper- and lowercase. + +## Related Functions + +- [`lower`](/packer/docs/templates/hcl_templates/functions/string/lower) converts letters in a string to _lowercase_. +- [`title`](/packer/docs/templates/hcl_templates/functions/string/title) converts the first letter of each word in a string to uppercase. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/uuid/uuidv4.mdx b/content/packer/content/docs/templates/hcl_templates/functions/uuid/uuidv4.mdx new file mode 100644 index 0000000000..d05fe59de6 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/uuid/uuidv4.mdx @@ -0,0 +1,27 @@ +--- +page_title: uuidv4 function reference +description: The `uuidv4` function generates a unique string ID according to RFC 4122 version 4. +--- + +# `uuidv4` Function + +`uuidv4` generates a unique identifier string. + +The ID is a generated and formatted as required by [RFC 4122 section +4.4](https://tools.ietf.org/html/rfc4122#section-4.4), producing a Version 4 +UUID. The result is a UUID generated only from pseudo-random numbers. + +This function produces a new value each time it is called, and so using it +directly in resource arguments will result in spurious diffs. We do not +recommend using the `uuidv4` function in resource configurations. + +## Examples + +```shell-session +> uuidv4() +9fc99a70-7cd5-482d-bb2b-03af016e4e94 +``` + +## Related Functions + +- [`uuidv5`](/packer/docs/templates/hcl_templates/functions/uuid/uuidv5), which generates name-based UUIDs. diff --git a/content/packer/content/docs/templates/hcl_templates/functions/uuid/uuidv5.mdx b/content/packer/content/docs/templates/hcl_templates/functions/uuid/uuidv5.mdx new file mode 100644 index 0000000000..b99653a311 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/functions/uuid/uuidv5.mdx @@ -0,0 +1,77 @@ +--- +page_title: uuidv5 function reference +description: |- + The `uuidv5` function generates a unique name-based ID for the value according to RFC 4122 version 5. +--- + +# `uuidv5` Function + +`uuidv5` generates a _name-based_ UUID, as described in [RFC 4122 section +4.3](https://tools.ietf.org/html/rfc4122#section-4.3), also known as a "version +5" UUID. + +```hcl +uuidv5(namespace, name) +``` + +Unlike the pseudo-random UUIDs generated by [`uuidv4`](/packer/docs/templates/hcl_templates/functions/uuid/uuidv4), +name-based UUIDs derive from namespace and an name, producing the same UUID +value every time if the namespace and name are unchanged. + +Name-based UUID namespaces are themselves UUIDs, but for readability this +function accepts some keywords as aliases for the namespaces that were assigned +by RFC 4122: + +| Keyword | Namespace ID | Name format | +| -------- | -------------------------------------- | ---------------------------------------------------------------------------- | +| `"dns"` | `6ba7b810-9dad-11d1-80b4-00c04fd430c8` | A fully-qualified DNS domain name. | +| `"url"` | `6ba7b811-9dad-11d1-80b4-00c04fd430c8` | Any valid URL as defined in [RFC 3986](https://tools.ietf.org/html/rfc3986). | +| `"oid"` | `6ba7b812-9dad-11d1-80b4-00c04fd430c8` | An [ISO/IEC object identifier](https://oidref.com/) | +| `"x500"` | `6ba7b814-9dad-11d1-80b4-00c04fd430c8` | [X.500 Distinguished Name](https://tools.ietf.org/html/rfc1779) | + +To use any other namespace not included in the above table, pass its assigned +namespace ID directly in the first argument in the usual UUID string format. + +## Examples + +Use the namespace keywords where possible, to make the intent more obvious to +a future reader: + +```shell-session +> uuidv5("dns", "developer.hashicorp.com") +a5008fae-b28c-5ba5-96cd-82b4c53552d6 + +> uuidv5("url", "https://developer.hashicorp.com/") +9db6f67c-dd95-5ea0-aa5b-e70e5c5f7cf5 + +> uuidv5("oid", "1.3.6.1.4") +af9d40a5-7a36-5c07-b23a-851cd99fbfa5 + +> uuidv5("x500", "CN=Example,C=GB") +84e09961-4aa4-57f8-95b7-03edb1073253 +``` + +The namespace keywords treated as equivalent to their corresponding namespace +UUIDs, and in some special cases it may be more appropriate to use the +UUID form: + +```shell-session +> uuidv5("6ba7b810-9dad-11d1-80b4-00c04fd430c8", "developer.hashicorp.com") +a5008fae-b28c-5ba5-96cd-82b4c53552d6 +``` + +If you wish to use a namespace defined outside of RFC 4122, using the namespace +UUID is required because no corresponding keyword is available: + +```shell-session +> uuidv5("743ac3c0-3bf7-4a5b-9e6c-59360447c757", "LIBS:diskfont.library") +ede1a974-df7e-5f17-84b9-76208818b2c8 +``` + +When using raw UUID namespaces, consider including a comment alongside the +expression that indicates which namespace this represents in a +human-significant manner, such as by reference to the standard that defined it. + +## Related Functions + +- [`uuidv4`](/packer/docs/templates/hcl_templates/functions/uuid/uuidv4), which generates pseudorandom UUIDs. diff --git a/content/packer/content/docs/templates/hcl_templates/index.mdx b/content/packer/content/docs/templates/hcl_templates/index.mdx new file mode 100644 index 0000000000..0bffa7622c --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/index.mdx @@ -0,0 +1,72 @@ +--- +page_title: HCL templates overview +description: |- + HCL templates are configuration files that describe infrastructure and set variables using HashiCorp configuration language (HCL). Learn about HCL templates for Packer. +--- + +# HCL templates overview + +This topic provides overview information about HashiCorp configuration language (HCL) templates for Packer. + +## Introduction + +Packer reads and applies configurations defined in HCL template files. HCL templates provide concise descriptions of the required steps to get to a build file. You can add arguments, blocks, and expressions to your HCL templates to define your build as code. Refer to [Introduction to Packer HCL2](/packer/guides/hcl) for additional information. + +## Builds + +The main purpose of the HCL language is defining builds and sources. All other +language features exist only to make the definition of builds more flexible and +convenient. + +`packer build` takes one argument. When a directory is passed, all files in the +folder with a name ending with `.pkr.hcl` or `.pkr.json` will be parsed using +the HCL2 format. When a file ending with `.pkr.hcl` or `.pkr.json` is passed it +will be parsed using the HCL2 schema. For every other case; the _JSON only_ old +packer schema will be used. + +## Arguments, blocks, and expressions + +The syntax of the HCL language consists of only a few basic elements: + +```hcl +source "amazon-ebs" "main" { + ami_name = "main-ami" +} + + "" "" { + # Block body + = # Argument +} +``` + +- _Blocks_ are containers for other content and usually represent the + configuration of some kind of object, like a source. Blocks have a + _block type,_ can have zero or more _labels,_ and have a _body_ that contains + any number of arguments and nested blocks. Most of Packer's features are + controlled by top-level blocks in a configuration file. +- _Arguments_ assign a value to a name. They appear within blocks. +- _Expressions_ represent a value, either literally or by referencing and + combining other values. They appear as values for arguments, or within other + expressions. + +For full details about Packer's syntax, see: + +- [Configuration Syntax](/packer/docs/templates/hcl_templates/syntax) +- [Expressions](/packer/docs/templates/hcl_templates/expressions) + +## Code organization + +The HCL language uses configuration files that are named with the `.pkr.hcl` +file extension. There is also [a JSON-based variant of the +language](/packer/docs/templates/hcl_templates/syntax-json) that is named with the `.pkr.json` file +extension. + +Configuration files must always use UTF-8 encoding, and by convention are +usually maintained with Unix-style line endings (LF) rather than Windows-style +line endings (CRLF), though both are accepted. + +## Configuration ordering + +The ordering of root blocks is not significant. The order of `provisioner` or +`post-processor` blocks within a `build` is the only major feature where block +order matters. diff --git a/content/packer/content/docs/templates/hcl_templates/locals.mdx b/content/packer/content/docs/templates/hcl_templates/locals.mdx new file mode 100644 index 0000000000..794088022a --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/locals.mdx @@ -0,0 +1,124 @@ +--- +page_title: Local variables +description: >- + The `local` and `locals` blocks define local variables that assign a name to an expression. Learn how to configure local variables. +--- + +# Local variables + +This topic provides reference information about the `local` and `locals` blocks, which define local variables that you can use in your Packer templates. To learn about input variables, refer to +[Input variables](/packer/docs/templates/hcl_templates/variables). + +## Introduction + +Local variables assign a name to an expression, which you can use multiple +times within a folder. The expression is evaluated at run time, and can +reference input variables, other local variables, data sources, and HCL +functions. You cannot overwrite local variables. + +Refer to [Input Variables and local variables](/packer/guides/hcl/variables) for additional information about variables in Packer. + +## Examples + +Local variables are defined in a `local` or `locals` block: + +```hcl +# Using the local block allows you to mark locals as sensitive, which will +# filter their values from logs. +local "mylocal" { + expression = "${var.secret_api_key}" + sensitive = true +} + +# Using the locals block is more compact and efficient for declaring many locals +# Ids for multiple sets of EC2 instances, merged together +locals { + instance_ids = "${concat(aws_instance.blue.*.id, aws_instance.green.*.id)}" +} + +# A computed default name prefix +locals { + default_name_prefix = "${var.project_name}-web" + name_prefix = "${var.name_prefix != "" ? var.name_prefix : local.default_name_prefix}" +} + +# Local variables can be referenced using the "local." prefix. +source "virtualbox-iso" "example" { + output = "${local.name_prefix}-files" + # ... +} +``` + +Named local maps can be merged with local maps to implement common or default +values: + +```hcl +# Define the common tags for all resources +locals { + common_tags = { + Component = "awesome-app" + Environment = "production" + } +} + +# Create a resource that blends the common tags with instance-specific tags. +source "amazon-ebs" "server" { + source_ami = "ami-123456" + instance_type = "t2.micro" + + tags = "${merge( + local.common_tags, + { + "Name" = "awesome-app-server", + "Role" = "server" + } + )}" + # ... +} +``` + +## Single `local` block + +The `local` block defines exactly one local variable within a folder. The block +label is the name of the local, and the "expression" is the expression that +should be evaluated to create the local. Using this block, you can optionally +supply a "sensitive" boolean to mark the variable as sensitive and filter it +from logs. + +```hcl +local "mylocal" { + expression = "${var.secret_api_key}" + sensitive = true +} +``` + +This block is also very useful for defining complex locals. Packer might take some time to expand and evaluate `locals` +with complex expressions dependent on other locals. The `locals` block is read as a map. Maps are not sorted, and therefore +the evaluation time is not deterministic. + +To avoid that, singular `local` blocks should be used instead. These will be +evaluated in the order they are defined, and the evaluation order and time will always be the same. + +## `locals` block + +The `locals` block defines one or more local variables within a folder. + +The names given for the items in the `locals` block must be unique throughout a +folder. The given value can be any expression that is valid within the current +folder. + +The expression of a local value can refer to other locals, but reference cycles +are not allowed. That is, a local cannot refer to itself or to a variable that +refers (directly or indirectly) back to it. + +It's recommended to group together logically-related local variables into a single +block, particularly if they depend on each other. This will help the reader +understand the relationships between variables. Conversely, prefer to define +_unrelated_ local variables in _separate_ blocks, and consider annotating each +block with a comment describing any context common to all of the enclosed +locals. + +## Known limitations +`@include 'datasources/local-dependency-limitation.mdx'` + + diff --git a/content/packer/content/docs/templates/hcl_templates/onlyexcept.mdx b/content/packer/content/docs/templates/hcl_templates/onlyexcept.mdx new file mode 100644 index 0000000000..3ca8832653 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/onlyexcept.mdx @@ -0,0 +1,81 @@ +--- +page_title: only and except keywords reference +description: >- + The `only` and `except` keywords are filters for Packer builds. Learn how to use `only` and `except` to selectively run provisioners and post-processors. +--- + +# `only` and `except` keywords + +The `only` and `except` keywords are filters that let you selectively run provisioners and post-processors in your Packer build. You can use the keywords in your Packer templates and as flags on the Packer CLI. + +## Command line flags + +`@include 'commands/except.mdx'` + +`@include 'commands/only.mdx'` + +## Templates + +You can add the `only` and `except` keywords to run or skip provisioners and +post-processors for a specific source: + +```hcl +source "amazon-ebs" "first-example" { +} + +source "amazon-ebs" "second-example" { +} + +source "amazon-ebs" "third-example" { +} + +build { + name = "my_build" + sources = [ + "source.amazon-ebs.first-example", + ] + source "source.amazon-ebs.second-example" { + // setting the name field allows you to rename the source only for this + // build section. To match this builder, you need to use + // second-example-local-name, not second-example + name = "second-example-local-name" + } + + provisioner "shell-local" { + only = ["amazon-ebs.first-example"] + inline = ["echo I will only run for the first example source"] + } + + provisioner "shell-local" { + except = ["amazon-ebs.second-example-local-name"] + inline = ["echo I will never run for the second example source"] + } +} + +build { + sources = [ + "source.amazon-ebs.third-example", + ] +} + +# this file will result in Packer creating three builds named: +# my_build.amazon-ebs.first-example +# my_build.amazon-ebs.second-example +# amazon-ebs.third-example +``` + +Note that CLI arguments can be used with a glob operator, using the previous +configuration: + +- `packer build -only 'my_build.*' dir`: will only run the builds in blocks + named `my_build`. + +- `packer build -only '*.amazon-ebs.*' dir`: will only run the builds with a + source of type `amazon-ebs`. + +- `packer build -only '*.second-example-local-name' dir`: will only run that + specifically named build. + +-> Note: In the cli `only` and `except` will match against **build names** (for +example:`my_build.amazon-ebs.first-example`) but in a provisioner they will +match on the **source name** (for example:`amazon-ebs.third-example`). diff --git a/content/packer/content/docs/templates/hcl_templates/path-variables.mdx b/content/packer/content/docs/templates/hcl_templates/path-variables.mdx new file mode 100644 index 0000000000..420ff3af3d --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/path-variables.mdx @@ -0,0 +1,47 @@ +--- +page_title: Path variables reference +description: |- + Path variables are special variables that enable you to provide directory information. Learn how to use path variables in Packer templates. +--- + +# Path variables + +This topic provides reference information about path variables for Packer templates written in HCL. + +## Description + +Use the following variables to build paths: + +- `path.cwd`: the directory from where Packer was started. + +- `path.root`: the directory of the input HCL file or the input folder. + +## Examples + +```HCL +locals { + settings_file = "${path.cwd}/settings.txt" + scripts_folder = "${path.root}/scripts" + root = path.root +} +``` + +## Path separators + +`@include 'path/separator-note.mdx'` + +## Related functions + +- [`abspath`](/packer/docs/templates/hcl_templates/functions/file/abspath) takes a string containing + a filesystem path and converts it to an absolute path. + +- [`basename`](/packer/docs/templates/hcl_templates/functions/file/basename) returns _only_ the last + portion of a filesystem path, discarding the portion that would be returned + by `dirname`. + +- [`fileset`](/packer/docs/templates/hcl_templates/functions/file/fileset) enumerates a set of + regular file names given a path and pattern. + +- [`dirname`](/packer/docs/templates/hcl_templates/functions/file/dirname) returns all of the + segments of a filesystem path _except_ the last, discarding the portion that + would be returned by `basename`. diff --git a/content/packer/content/docs/templates/hcl_templates/syntax-json.mdx b/content/packer/content/docs/templates/hcl_templates/syntax-json.mdx new file mode 100644 index 0000000000..276f78433c --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/syntax-json.mdx @@ -0,0 +1,303 @@ +--- +page_title: JSON syntax reference +description: |- + A JSON configuration expresses HCL Packer templates as JSON-compatible syntax. Learn about the JSON syntax for creating HCL Packer templates. +--- + +# JSON syntax reference + +This topic provides reference information about the JSON syntax you can use to express HCL Packer templates as JSON. For information about the HCL syntax, refer to [HCL syntax reference](/packer/docs/templates/hcl_templates/syntax). + +## Introduction + +Packer supports templates written in JSON, which is useful when generating portions of a configuration programmatically. This is because you can use existing JSON libraries to prepare the generated +configuration files. + +This syntax is not to be confused with the pre-version-1.5 "legacy" Packer +templates, which were exclusively JSON and follow a different format. + +The JSON syntax is defined in terms of the native syntax. Everything that can +be expressed in native syntax can also be expressed in JSON syntax, but some +constructs are more complex to represent in JSON due to limitations of the +JSON grammar. + +Packer expects native syntax for files named with a `.pkr.hcl` suffix, and JSON +syntax for files named with a `.pkr.json` suffix. If you leave out the `.pkr` +portion of suffix, Packer will try to read your json file as a legacy Packer +template. + +The low-level JSON syntax, just as with the native syntax, is defined in terms +of a specification called _HCL_. It is not necessary to know all of the details +of HCL syntax or its JSON mapping in order to use Packer, and so this page +summarizes the most important differences between native and JSON syntax. If +you are interested, you can find a full definition of HCL's JSON syntax in [its +specification](https://github.com/hashicorp/hcl/blob/hcl2/hclsyntax/spec.md). + +## JSON File Structure + +At the root of any JSON-based Packer configuration is a JSON object. The +properties of this object correspond to the top-level block types of the +Packer language. For example: + +```json +{ + "variables": { + "example": "value" + } +} +``` + +Each top-level object property must match the name of one of the expected +top-level block types. Block types that expect labels, such as `variable` shown +above, are represented by one nested object value for each level of label. +`source` blocks expect two labels, so two levels of nesting are required: + +```json +{ + "source": { + "amazon-ebs": { + "example": { + "instance_type": "t2.micro", + "ami_name": "ami-abc123" + } + } + } +} +``` + +After any nested objects representing the labels, finally one more nested +object represents the body of the block itself. In the above example the +`instance_type` and `ami_name` arguments for `source "amazon-ebs" "example"` +are specified. + +Taken together, the above two configuration files are equivalent to the +following blocks in the native syntax: + +```hcl +variables { + example = "value" +} + +source "amazon-ebs" "example" { + instance_type = "t2.micro" + ami_name = "ami-abc123" +} +``` + +Within each top-level block type the rules for mapping to JSON are slightly +different (see [block-type-specific exceptions](#block-type-specific-exceptions) below), but the following general rules apply in most cases: + +- The JSON object representing the block body contains properties that + correspond either to argument names or to nested block type names. + +- Where a property corresponds to an argument that accepts + [arbitrary expressions](/packer/docs/templates/hcl_templates/expressions) in the native syntax, the + property value is mapped to an expression as described under + [_Expression Mapping_](#expression-mapping) below. For arguments that + do _not_ accept arbitrary expressions, the interpretation of the property + value depends on the argument, as described in the + [block-type-specific exceptions](#block-type-specific-exceptions) + given later in this page. + +- Where a property name corresponds to an expected nested block type name, + the value is interpreted as described under + [_Nested Block Mapping_](#nested-block-mapping) below, unless otherwise + stated in [the block-type-specific exceptions](#block-type-specific-exceptions) + given later in this page. + +## Expression Mapping + +Since JSON grammar is not able to represent all of the Packer language +[expression syntax](/packer/docs/templates/hcl_templates/expressions), JSON values interpreted as expressions +are mapped as follows: + +| JSON | Packer Language Interpretation | +| ------- | -------------------------------------------------------------------------------------------------------------------------------- | +| Boolean | A literal `bool` value. | +| Number | A literal `number` value. | +| String | Parsed as a [string template](/packer/docs/templates/hcl_templates/expressions#string-templates) and then evaluated as described below. | +| Object | Each property value is mapped per this table, producing an `object(...)` value with suitable attribute types. | +| Array | Each element is mapped per this table, producing a `tuple(...)` value with suitable element types. | +| Null | A literal `null`. | + +When a JSON string is encountered in a location where arbitrary expressions are +expected, its value is first parsed as a [string template](/packer/docs/templates/hcl_templates/expressions#string-templates) +and then it is evaluated to produce the final result. + +If the given template consists _only_ of a single interpolation sequence, +the result of its expression is taken directly, without first converting it +to a string. This allows non-string expressions to be used within the +JSON syntax. + +## Nested Block Mapping + +When a JSON object property is named after a nested block type, the value +of this property represents one or more blocks of that type. The value of +the property must be either a JSON object or a JSON array. + +The simplest situation is representing only a single block of the given type +when that type expects no labels, as with the `tags` nested block used +within `source` blocks: + +```json +{ + "source": { + "amazon-ebs": { + "example": { + "tags": { + "key": "value" + } + } + } + } +} +``` + +The above is equivalent to the following native syntax configuration: + +```hcl +source "amazon-ebs" "example" { + tags = { + key = "value" + } +} +``` + +When the nested block type requires one or more labels, or when multiple +blocks of the same type can be given, the mapping gets a little more +complicated. For example, the `provisioner` nested block type used +within `build` blocks expects a label giving the provisioner to use, +and the ordering of provisioner blocks is significant to decide the order +of operations. + +The following native syntax example shows a `build` block with a number +of provisioners of different types: + +```hcl +build { + # (source configuration omitted for brevity) + + provisioner "shell-local" { + inline = ["echo 'Hello World' >example.txt"] + } + provisioner "file" { + source = "example.txt" + destination = "/tmp/example.txt" + } + provisioner "shell" { + inline = [ + "sudo install-something -f /tmp/example.txt", + ] + } +} +``` + +In order to preserve the order of these blocks, you must use a JSON array +as the direct value of the property representing this block type, as in +this JSON equivalent of the above: + +```json +{ + "build": { + "//": "(source configuration omitted for brevity)", + + "provisioner": [ + { + "shell-local": { + "inline": ["echo 'Hello World' >example.txt"] + } + }, + { + "file": { + "source": "example.txt", + "destination": "/tmp/example.txt" + } + }, + { + "shell": { + "inline": ["sudo install-something -f /tmp/example.txt"] + } + } + ] + } +} +``` + +Each element of the `provisioner` array is an object with a single property +whose name represents the label for each `provisioner` block. For block types +that expect multiple labels, this pattern of alternating array and object +nesting can be used for each additional level. + +If a nested block type requires labels but the order does _not_ matter, you +may omit the array and provide just a single object whose property names +correspond to unique block labels. This is allowed as a shorthand for the above +for simple cases, but the alternating array and object approach is the most +general. We recommend using the most general form if systematically converting +from native syntax to JSON, to ensure that the meaning of the configuration is +preserved exactly. + +### Comment Properties + +Although we do not recommend hand-editing of JSON syntax configuration files +-- this format is primarily intended for programmatic generation and consumption -- +a limited form of _comments_ are allowed inside JSON objects that represent +block bodies using a special property name: + +```json +{ + "source": { + "amazon-ebs": { + "example": { + "//": "This instance runs the scheduled tasks for backup", + + "instance_type": "t2.micro", + "ami_name": "ami-abc123" + } + } + } +} +``` + +In any object that represents a block body, properties named `"//"` are +ignored by Packer entirely. This exception does _not_ apply to objects +that are being [interpreted as expressions](#expression-mapping), where this +would be interpreted as an object type attribute named `"//"`. + +This special property name can also be used at the root of a JSON-based +configuration file. This can be useful to note which program created the file. + +```json +{ + "//": "This file is generated by generate-outputs.py. DO NOT HAND-EDIT!" +} +``` + +## Block-type-specific Exceptions + +[inpage-block]: #block-type-specific-exceptions + +Certain arguments within specific block types are processed in a special way, +and so their mapping to the JSON syntax does not follow the general rules +described above. The following sub-sections describe the special mapping rules +that apply to each top-level block type. + +### `variable` blocks + +All arguments inside `variable` blocks have non-standard mappings to JSON: + +- `type`: a string containing a type expression, like `"string"` or `"list(string)"`. +- `default`: a literal JSON value that can be converted to the given type. + Strings within this value are taken literally and _not_ interpreted as + string templates. +- `description`: a literal JSON string, _not_ interpreted as a template. + +```json +{ + "variable": { + "example": { + "type": "string", + "default": "hello" + } + } +} +``` diff --git a/content/packer/content/docs/templates/hcl_templates/syntax.mdx b/content/packer/content/docs/templates/hcl_templates/syntax.mdx new file mode 100644 index 0000000000..3cea5da646 --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/syntax.mdx @@ -0,0 +1,110 @@ +--- +page_title: HCL syntax reference +description: |- + HCL syntax combines declarative structure with expressions into Packer templates. Learn how to use HCL syntax to define infrastructure as code. +--- + +# HCL syntax reference + +This topic provides reference information about the native HashiCorp configuration language (HCL) syntax you can use to create Packer templates. For information about HCL constructs in JSON, refer to the [JSON syntax reference](/packer/docs/templates/hcl_templates/syntax-json). + +## Introduction + +Learning the HCL syntax helps you understand the building blocks that Packer template constructs +are built from. HCL is a rich language designed to be easy for humans to read and write. +Otherwhich is also used by configuration languages in other applications, and in +particular other HashiCorp products. It is not necessary to know all of the +details of HCL in order to use Packer, and so this page summarizes the most +important details. If you are interested, you can find a full definition of HCL +syntax in [the HCL native syntax +specification](https://github.com/hashicorp/hcl/blob/hcl2/hclsyntax/spec.md). + +## Arguments and Blocks + +HCL syntax is built around two key syntax constructs: +arguments and blocks. + +### Arguments + +An _argument_ assigns a value to a particular name: + +```hcl +image_id = "abc123" +``` + +The identifier before the equals sign is the _argument name_, and the expression +after the equals sign is the argument's value. + +The context where the argument appears determines what value types are valid +(for example, each source type has a schema that defines the types of its +arguments), but many arguments accept arbitrary +[expressions](/packer/docs/templates/hcl_templates/expressions), which allow the value to +either be specified literally or generated from other values programmatically. + +### Blocks + +A _block_ is a container for other content: + +```hcl +source "amazon-ebs" "example" { + ami_name = "abc123" + + tags = { + # ... + } +} +``` + +A block has a _type_ (`source` in this example). Each block type defines +how many _labels_ must follow the type keyword. The `source` block type +expects two labels, which are `amazon-ebs` and `example` in the example above. +A particular block type may have any number of required labels, or it may +require none as with the nested `tags` block type. + +After the block type keyword and any labels, the block _body_ is delimited +by the `{` and `}` characters. Within the block body, further arguments +and blocks may be nested, creating a hierarchy of blocks and their associated +arguments. + +HCL uses a limited number of _top-level block types,_ which +are blocks that can appear outside of any other block in a configuration file. +Most of Packer's features (including resources, input variables, output +values, data sources, etc.) are implemented as top-level blocks. + +## Identifiers + +Argument names, block type names, and the names of most Packer-specific +constructs like resources, input variables, etc. are all _identifiers_. + +Identifiers can contain letters, digits, underscores (`_`), and hyphens (`-`). +The first character of an identifier must not be a digit, to avoid ambiguity +with literal numbers. + +For complete identifier rules, Packer implements +[the Unicode identifier syntax](http://unicode.org/reports/tr31/), extended to +include the ASCII hyphen character `-`. + +## Comments + +HCL supports three different syntaxes for comments: + +- `#` begins a single-line comment, ending at the end of the line. +- `//` also begins a single-line comment, as an alternative to `#`. +- `/*` and `*/` are start and end delimiters for a comment that might span + over multiple lines. + +The `#` single-line comment style is the default comment style and should be +used in most cases. Automatic configuration formatting tools may automatically +transform `//` comments into `#` comments, since the double-slash style is +not idiomatic. + +## Character Encoding and Line Endings + +Packer configuration files must always be UTF-8 encoded. While the +delimiters of the language are all ASCII characters, Packer accepts +non-ASCII characters in identifiers, comments, and string values. + +Packer accepts configuration files with either Unix-style line endings +(LF only) or Windows-style line endings (CR then LF), but the idiomatic style +is to use the Unix convention, and so automatic configuration formatting tools +may automatically transform CRLF endings to LF. diff --git a/content/packer/content/docs/templates/hcl_templates/variables.mdx b/content/packer/content/docs/templates/hcl_templates/variables.mdx new file mode 100644 index 0000000000..169d9ed81f --- /dev/null +++ b/content/packer/content/docs/templates/hcl_templates/variables.mdx @@ -0,0 +1,322 @@ +--- +page_title: Input variables reference +description: |- + Input variables are parameters for Packer modules. + Learn how to use the input variable syntax for Packer templates written in HCL. +--- + +# Input Variables + +This topic provides reference information about input variables in Packer templates. Refer to [Local Variables](/packer/docs/templates/hcl_templates/locals) for information about using local variables in Packer. + +## Introduction + +Input variables, sometimes referred to as variables or Packer variables, are the parameters for a Packer build. Input variables let you customize aspects of a build without altering the build's own source code. Some input variables have default values that you can override using command line options, environment variables, or variable definitions files. You cannot change the value of an input variable after the initial override. + +Refer to [Input Variables and local variables](/packer/guides/hcl/variables) for additional information. + +## Declaring an Input Variable + +Each input variable accepted by a build must be declared using a `variable` +block : + +```hcl +variable "image_id" { + type = string +} + +variable "availability_zone_names" { + type = list(string) + default = ["us-west-1a"] +} + +variable "docker_ports" { + type = list(object({ + internal = number + external = number + protocol = string + })) + default = [ + { + internal = 8300 + external = 8300 + protocol = "tcp" + } + ] +} +``` + +Or a less precise variables block: + +```hcl +variables { + foo = "value" + my_secret = "foo" +} +``` + +The label after the `variable` keyword or a label of a `variables` block is a +name for the variable, which must be unique among all variables in the same +build. This name is used to assign a value to the variable from outside and to +reference the variable's value from within the build. + + +## Arguments + +Packer defines the following arguments for variable declarations: + +* [`default`][inpage-default] - A default value which then makes the variable optional. +* [`type`][inpage-type] - This argument specifies what value types are accepted for the variable. +* [`description`][inpage-description] - This specifies the input variable's documentation. +* [`validation`][inpage-validation] - A block to define validation rules, usually in addition to type constraints. +* [`sensitive`][inpage-sensitive] - This causes string-values from that variable to be obfuscated from Packer's output. + + + +### Default values + +[inpage-default]: #default-values + +The variable declaration can also include a `default` argument. If present, +the variable is considered to be _optional_ and the default value will be used +if no value is set when running Packer. The `default` +argument requires a literal value and cannot reference other objects in the +configuration. + + +### Type Constraints +[inpage-type]: #type-constraints + +The `type` argument in a `variable` block allows you to restrict the [type of +value](/packer/docs/templates/hcl_templates/expressions#types-and-values) that will be accepted as the value +for a variable. If no type constraint is set then a value of any type is +accepted. + +While type constraints are optional, we recommend specifying them; they serve +as easy reminders for users of the build, and allow Packer to return a helpful +error message if the wrong type is used. + +Type constraints are created from a mixture of type keywords and type +constructors. The supported type keywords are: + +- `string` +- `number` +- `bool` + +The type constructors allow you to specify complex types such as collections: + +- `list()` +- `set()` +- `map()` +- `object({ = , ... })` +- `tuple([, ...])` + +The keyword `any` may be used to indicate that any type is acceptable. For more +information on the meaning and behavior of these different types, as well as +detailed information about automatic conversion of complex types, see [Type +Constraints](/terraform/docs/configuration/types). + +If both the `type` and `default` arguments are specified, the given default +value must be convertible to the specified type. + +If only `default` is specified, the type of the default value will be used. + +When the `type` and `default` are both _not_ specified and you try to set a +variable [from env vars](#environment-variables) or [from the command +line](#variables-on-the-command-line), the variable will always be interpreted +as a string. + +### Input Variable Documentation + +[inpage-description]: #input-variable-documentation + +Because the input variables of a build are part of its user interface, you can +briefly describe the purpose of each variable using the optional `description` +argument: + +```hcl +variable "image_id" { + type = string + description = "The ID of the machine image (AMI) to use for the server." +} +``` + +The description should concisely explain the purpose of the variable and what +kind of value is expected. This description string might be included in +documentation about the build, and so it should be written from the perspective +of the user of the build rather than its maintainer. For commentary for build +maintainers, use comments. + +`@include 'from-1.5/variables/custom-validation.mdx'` + +`@include 'from-1.5/variables/sensitive.mdx'` + + +## Using Input Variable Values + +Within the build that declared a variable, its value can be accessed from +within [expressions](/packer/docs/templates/hcl_templates/expressions) as `var.`, where `` +matches the label given in the declaration block: + +```hcl +source "googlecompute" "debian" { + zone = var.gcp_zone + tags = var.gcp_debian_tags +} +``` + +The value assigned to a variable can be accessed only from expressions within +the folder where it was declared. + + +`@include 'from-1.5/variables/assignment.mdx'` + +The following sections describe these options in more detail. + +### Variables on the Command Line + +To specify individual variables on the command line, use the `-var` option when +running the `packer build` command: + +```shell-session +$ packer build -var="image_id=ami-abc123" +$ packer build -var='image_id_list=["ami-abc123","ami-def456"]' +$ packer build -var='image_id_map={"us-east-1":"ami-abc123","us-east-2":"ami-def456"}' +``` + +The `-var` option can be used any number of times in a single command. + +If you plan to assign variables via the command line, we strongly recommend that +you at least set a default type instead of using empty blocks; this helps the +HCL parser understand what is being set. Otherwise, the interpreter will assume +that any variable set on the command line is a string. + +### Standard Variable Definitions Files + +To set lots of variables, it is more convenient to specify their values in a +_variable definitions file_ with a filename ending in either `.pkrvars.hcl` or +`.pkrvars.json` and then specify that file on the command line with +`-var-file`: + +```shell-session +$ packer build -var-file="testing.pkrvars.hcl" +``` + +A variable definitions file uses the same basic syntax as Packer language +files, but consists only of variable name and its assigned values: + +```hcl +image_id = "ami-abc123" +availability_zone_names = [ + "us-east-1a", + "us-west-1c", +] +``` + +~> **Important**: Unlike legacy JSON templates the input variables within a variable definitions file must be declared +via a variables block within a standard HCL2 template file `*.pkr.hcl` before it can be assigned a value. +Failure to do so will result in an unknown variable error during Packer's runtime. + +### Auto-loaded Variable Definitions Files + +Packer also has the ability automatically load one or more variable definitions files if they +are present: + +- Any files with names ending in `.auto.pkrvars.hcl` or `.auto.pkrvars.json`. + +Files whose names end with `.json` are parsed as JSON objects instead of HCL, +with the root object properties corresponding to variable names: + +```json +{ + "image_id": "ami-abc123", + "availability_zone_names": ["us-west-1a", "us-west-1c"] +} +``` + +~> **Important**: Unlike legacy JSON templates the input variables within a variable definitions file must be declared +via a variables block within a standard HCL2 template file `*.pkr.hcl` before it can be assigned a value. +Failure to do so will result in an unknown variable error during Packer's runtime. + + +### Environment Variables + +As a fallback for the other ways of defining variables, Packer searches the +environment of its own process for environment variables named `PKR_VAR_` +followed by the name of a declared variable. + +This can be useful when running Packer in automation, or when running a +sequence of Packer commands in succession with the same variables. For example, +at a `bash` prompt on a Unix system: + +```shell-session +$ export PKR_VAR_image_id=ami-abc123 +$ packer build gcp/debian/ +... +``` + +On operating systems where environment variable names are case-sensitive, +Packer matches the variable name exactly as given in configuration, and so the +required environment variable name will usually have a mix of upper and lower +case letters as in the above example. + +### Complex-typed Values + +When variable values are provided in a variable definitions file, Packer's +[usual syntax](/packer/docs/templates/hcl_templates/expressions) can be used to assign +complex-typed values, like lists and maps. + +Some special rules apply to the `-var` command line option and to environment +variables. For convenience, Packer defaults to interpreting `-var` and +environment variable values as literal strings, which do not need to be quoted: + +```shell-session +$ export PKR_VAR_image_id=ami-abc123 +``` + +However, if a build variable uses a [type constraint](#type-constraints) to +require a complex value (list, set, map, object, or tuple), Packer will instead +attempt to parse its value using the same syntax used within variable +definitions files, which requires careful attention to the string escaping +rules in your shell: + +```shell-session +$ export PKR_VAR_availability_zone_names='["us-west-1b","us-west-1d"]' +``` + +For readability, and to avoid the need to worry about shell escaping, we +recommend always setting complex variable values via variable definitions +files. + +### Variable Definition Precedence + +The above mechanisms for setting variables can be used together in any +combination. + +Packer loads variables in the following order, with later sources taking +precedence over earlier ones: + +- Environment variables (lowest priority) +- Any `*.auto.pkrvars.hcl` or `*.auto.pkrvars.json` files, processed in lexical + order of their filenames. +- Any `-var` and `-var-file` options on the command line, in the order they are + provided. (highest priority) + +If the same variable is assigned multiple values using different mechanisms, +Packer uses the _last_ value it finds, overriding any previous values. Note +that the same variable cannot be assigned multiple values within a single source. + +~> **Important:** Variables with map and object values behave the same way as +other variables: the last value found overrides the previous values. + +`@include 'from-1.5/variables/must-be-set.mdx'` + +### Setting an unknown variable will not always fail: + +| Usage | packer validate | any other packer command | +| :----------------------------: | :---------------------: | :-----------------------: | +| `bar=yz` in .pkrvars.hcl file. | error, "bar undeclared" | warning, "bar undeclared" | +| `var.bar` in .pkr.hcl file | error, "bar undeclared" | error, "bar undeclared" | +| `-var bar=yz` argument | error, "bar undeclared" | error, "bar undeclared" | +| `export PKR_VAR_bar=yz` | - | - | + diff --git a/content/packer/content/docs/templates/index.mdx b/content/packer/content/docs/templates/index.mdx new file mode 100644 index 0000000000..1eda888445 --- /dev/null +++ b/content/packer/content/docs/templates/index.mdx @@ -0,0 +1,26 @@ +--- +description: | + Packer templates are configuration files that determine Packer behavior. Learn about the Packer template configuration language. +page_title: Packer templates reference overview +--- + +# Packer templates reference overview + +This topic provides overview information about Packer templates, which are configuration files that determine Packer behavior. + +## Introduction + +A Packer template contains a series of declarations and commands for Packer to follow. Templates define the following instructions for Packer: + +- Which plugins, such as builders, provisioners, and post-processors to use +- How to configure the plugins +- In what order to run them + +The template contains flexible variable injection tools, as well as built-in +functions to help you customize your builds. + +## Template language + +Packer supports templates written in [HCL2](/packer/docs/templates/hcl_templates) or [JSON](/packer/docs/templates/legacy_json_templates). Since v1.7.0, HCL2 is the official format for Packer configurations. + +HCL2 is also the configuration language used in configurations for other HashiCorp products, such as Terraform. HCL is more flexible, modular, and concise than the JSON template format. Refer to [Upgrade Packer JSON template to HCL2](/packer/tutorials/configuration-language/hcl2-upgrade) for a tutorial on upgrading JSON templates to HCL templates. diff --git a/content/packer/content/docs/templates/json_to_hcl.mdx b/content/packer/content/docs/templates/json_to_hcl.mdx new file mode 100644 index 0000000000..f6ff565312 --- /dev/null +++ b/content/packer/content/docs/templates/json_to_hcl.mdx @@ -0,0 +1,161 @@ +--- +description: | + Learn about the differences between JSON and HCL2 templates and how converting templates to HCL affects plugins. +page_title: Differences between HCL2 and JSON templates +--- + +# Differences between HCL2 and JSON templates + +This document explains the differences between JSON to HCL2 templates, including how converting JSON templates to HCL2 affects builder, provisioner, and post-processor behavior. + +## Introduction + +Packer uses JSON or HCL2 templates to orchestrate builds for one or more artifacts. + +In [legacy JSON templates](/packer/docs/templates/legacy_json_templates), declare a series of builders, provisioners, and post-processors to build images. + +In [HCL2 templates](/packer/docs/templates/hcl_templates), you can specify builders through sources and collect them into build blocks. + +You can use the `packer hcl2_upgrade` command when converting your existing JSON templates to HCL2. Refer to the [hcl2_upgrade](/packer/docs/commands/hcl2_upgrade) page for more information on its usage. + + +## Builders + +Builders are components that are specialized for a platform. +Their role is to set the stage for the steps that will let you build the final image from your configuration template. + +### JSON + +In JSON, the `builders` attribute declares how a builder will be invoked to build an image. + +```json +{ + "builders": [ + { + "type": "null", + "name": "test", + "communicator": "none" + } + ] +} +``` + +### HCL2 + +In HCL2, builders are declared through a `source` block. Sources on their own are not enough, and must be invoked in a `build` block. +The `build` block serves as a container for all the steps (i.e. `source`, `provisioners` and `post-processors`) to go from a blank image to a finished one. + +```hcl +source "null" "test" { + communicator = "none" +} + +build { + sources = ["null.test"] +} +``` + +## Provisioners + +Provisioners are components that modify the state of the machine image you are building. +They can be used for installing packages, writing data to the remote file system, or executing remote commands. +They can be defined in both JSON or HCL templates. + +### JSON + +In a JSON template, provisioners are declared at the top-level of the template as an array of sequentially +invoked components to provision the image you are building. They apply to all the builders defined in the template. + +```json +{ + "builders": [ + { + "type": "null", + "name": "test", + "communicator": "none" + } + ], + "provisioners": [ + { + "type": "shell-local", + "inline": ["echo test"] + } + ] +} +``` + +### HCL2 + +In HCL2, provisioners are declared in the `build` block, and apply only to the sources invoked as +part of this build block. +This lets you declare multiple build blocks, each with its own set of provisioners, which will +result in multiple different builds executed in parallel. + +```hcl +source "null" "test" { + communicator = "none" +} + +build { + sources = ["null.test"] + + provisioner "shell-local" { + inline = ["echo test"] + } +} +``` + +## Post-processors + +Post-processors are components that are invoked after the image was produced. Their +role is to consume an artifact produced by a builder, or another post-processor, and +build a new artifact from that. + +### JSON + +As with provisioners, in JSON templates, post-processors are sequentially invoked after a +build finished provisioning, and are declared in the template under "post-processors". + +```json +{ + "builders": [ + { + "type": "null", + "name": "test", + "communicator": "none" + } + ], + "provisioners": [ + { + "type": "shell-local", + "inline": ["echo test"] + } + ], + "post-processors": [ + { + "type": "manifest" + } + ] +} +``` + +### HCL2 + +In HCL2, much like provisioners, they are also part of the `build` block, and similar rules +apply as with provisioners. + +```hcl +source "null" "test" { + communicator = "none" +} + +build { + sources = ["null.test"] + + provisioner "shell-local" { + inline = ["echo test"] + } + + post-processor "manifest" {} +} +``` diff --git a/content/packer/content/docs/templates/legacy_json_templates/builders.mdx b/content/packer/content/docs/templates/legacy_json_templates/builders.mdx new file mode 100644 index 0000000000..4996d367e3 --- /dev/null +++ b/content/packer/content/docs/templates/legacy_json_templates/builders.mdx @@ -0,0 +1,79 @@ +--- +description: > + The `builders` block contains builders that create machines and generate images. Learn how to configure the `builders` block in JSON templates for Packer. +page_title: builders reference for JSON templates +--- + +# `builders` reference for JSON templates + +This topic provides reference information about the `builders` block in JSON templates. + +`@include 'from-1.5/legacy-json-warning.mdx'` + +## Introduction + +The `builders` block contains an array of all the builders +that Packer should use to generate machine images for the template. + +Builders create machines and generate images from them +for various platforms. For example, there are separate builders for EC2, +VMware, VirtualBox, etc. Packer comes with many builders by default, and can +also be extended to add new builders. + +This documentation page will cover how to configure a builder in a template. +The specific configuration options available for each builder, however, must be +referenced from the documentation for that specific builder. + +Within a template, a section of builder definitions looks like this: + +```json +{ + "builders": [ + // ... one or more builder definitions here + ] +} +``` + +## Builder Definition + +A single builder definition maps to exactly one +[build](/packer/docs/terminology#builds). A builder definition is a +JSON object that requires at least a `type` key. The `type` is the name of the +builder that will be used to create a machine image for the build. + +In addition to the `type`, other keys configure the builder itself. For +example, the AWS builder requires an `access_key`, `secret_key`, and some other +settings. These are placed directly within the builder definition. + +An example builder definition is shown below, in this case configuring the AWS +builder: + +```json +{ + "type": "amazon-ebs", + "access_key": "...", + "secret_key": "..." +} +``` + +## Named Builds + +Each build in Packer has a name. By default, the name is just the name of the +builder being used. In general, this is good enough. Names only serve as an +indicator in the output of what is happening. If you want, however, you can +specify a custom name using the `name` key within the builder definition. + +This is particularly useful if you have multiple builds defined that use the +same underlying builder. In this case, you must specify a name for at least one +of them since the names must be unique. + +## Communicators + +Every build is associated with a single +[communicator](/packer/docs/templates/legacy_json_templates/communicator). Communicators are used to +establish a connection for provisioning a remote machine (such as an AWS +instance or local virtual machine). + +All the examples for the various builders show some communicator (usually SSH), +but the communicators are highly customizable so we recommend reading the +[communicator documentation](/packer/docs/templates/legacy_json_templates/communicator). diff --git a/content/packer/content/docs/templates/legacy_json_templates/communicator.mdx b/content/packer/content/docs/templates/legacy_json_templates/communicator.mdx new file mode 100644 index 0000000000..59d4073e0f --- /dev/null +++ b/content/packer/content/docs/templates/legacy_json_templates/communicator.mdx @@ -0,0 +1,52 @@ +--- +description: | + Packer uses communicators to upload files, execute scripts, and perform actions on the machine it creates. Learn how to configure communicators in JSON templates. +page_title: Communicators reference for JSON templates +--- + +# Communicators reference for JSON templates + +This topic describes how to use communicators in Packer templates written in JSON format. + +`@include 'from-1.5/legacy-json-warning.mdx'` + +## Description + +Packer uses communicators to upload files, execute scripts, and perform other actions on the machine being created. Configure communicators in the +[`builder`](/packer/docs/templates/legacy_json_templates/builders) section. + +All communicators have the following options: + +@include 'packer-plugin-sdk/communicator/Config-not-required.mdx' + +## Getting Ready to Use the Communicator + +Depending on your builder, your communicator may not have all it needs in order +to work "out of the box". + +If you are building from a cloud image (for example, building on Amazon), there +is a good chance that your cloud provider has already preconfigured SSH on the +image for you, meaning that all you have to do is configure the communicator in +the Packer template. + +However, if you are building from a brand-new and unconfigured operating system +image, you will almost always have to perform some extra work to configure SSH +on the guest machine. For most operating system distributions, this work will +be performed by a [boot_command](/packer/plugins/builders/vmware/iso#boot-configuration) that references a file which +provides answers to the normally-interactive questions you get asked when +installing an operating system. The name of this file varies by operating +system; some common examples are the "preseed" file required by Debian, the +"kickstart" file required by CentOS or the "answer file", also known as the +Autounattend.xml file, required by Windows. For simplicity's sake, we'll refer +to this file as the "preseed" file in the rest of the documentation. + +If you are unfamiliar with how to use a preseed file for automatic +bootstrapping of an image, please either take a look at our quick guides to +image bootstrapping, or research automatic configuration for your specific +guest operating system. Knowing how to automatically initalize your operating +system is critical for being able to successfully use Packer. + +## Communicator-Specific Options + +For more details on how to use each communicator, visit the +[communicators](/packer/docs/communicators) page. diff --git a/content/packer/content/docs/templates/legacy_json_templates/engine.mdx b/content/packer/content/docs/templates/legacy_json_templates/engine.mdx new file mode 100644 index 0000000000..90c31813a8 --- /dev/null +++ b/content/packer/content/docs/templates/legacy_json_templates/engine.mdx @@ -0,0 +1,384 @@ +--- +description: | + The JSON template engine processes strings in JSON templates. Learn how to use JSON template engine functions and variables. +page_title: JSON template engine reference +--- + +# JSON template engine reference + +This topic describes the Packer engine that processes JSON templates. + +`@include 'from-1.5/legacy-json-warning.mdx'` + +## Description + +All strings within templates are processed by a common Packer templating +engine. The engine uses variables and functions to modify the value of a +configuration parameter at runtime. + +The syntax of templates uses the following conventions: + +- Anything template related happens within double-braces: `{{ }}`. +- Functions are specified directly within the braces, such as + `{{timestamp}}`. +- Template variables are prefixed with a period and capitalized, such as + `{{.Variable}}`. + +## Functions + +Functions perform operations on and within strings, for example the +`{{timestamp}}` function can be used in any string to generate the current +timestamp. This is useful for configurations that require unique keys, such as +AMI names. By setting the AMI name to something like `My Packer AMI {{timestamp}}`, the AMI name will be unique down to the second. If you need +greater than one second granularity, you should use `{{uuid}}`, for example +when you have multiple builders in the same template. + +Here is a full list of the available functions for reference. + +- `build_name` - The name of the build being run. +- `build_type` - The type of the builder being used currently. +- `clean_resource_name` - Image names can only contain certain characters and + have a maximum length, eg 63 on GCE & 80 on Azure. `clean_resource_name` + will convert upper cases to lower cases and replace illegal characters with + a "-" character. Example: + + `"mybuild-{{isotime | clean_resource_name}}"` will become `mybuild-2017-10-18t02-06-30z`. + + Note: Valid Azure image names must match the regex + `^[^_\\W][\\w-._)]{0,79}$` + + Note: Valid GCE image names must match the regex + `(?:[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?)` + + This engine does not guarantee that the final image name will match the + regex; it will not truncate your name if it exceeds the maximum number of + allowed characters, and it will not validate that the beginning and end of + the engine's output are valid. For example, `"image_name": {{isotime | clean_resource_name}}"` will cause your build to fail because the image + name will start with a number, which is why in the above example we prepend + the isotime with "mybuild". + Exact behavior of `clean_resource_name` will depend on which builder it is + being applied to; refer to build-specific docs below for more detail on how + each function will behave. + +- `env` - Returns environment variables. See example in [using home + variable](/packer/docs/templates/legacy_json_templates/user-variables#using-home-variable) +- `build` - This engine will allow you to access, from provisioners and post-processors, special variables that + provide connection information and basic instance state information. + Usage example: + + ```json + { + "type": "shell-local", + "environment_vars": ["TESTVAR={{ build `PackerRunUUID`}}"], + "inline": ["echo $TESTVAR"] + } + ``` + + Valid variables to request are: + + - **ID**: Represents the VM being provisioned. For example, in Amazon it is the instance ID; in DigitalOcean, + it is the Droplet ID; in VMware, it is the VM name. + + - **Host**, **Port**, **User** and **Password**: The host, port, user, and password that Packer uses to access the machine. + Useful for using the shell local provisioner to run Ansible or Inspec against the provisioned instance. + + - **ConnType**: Type of communicator being used. For example, for SSH communicator this will be "ssh". + + - **PackerRunUUID**: Current build's unique ID. Can be used to specify build artifacts. + An example of that, is when multiple builds runs at the same time producing the same artifact. + It's possible to differentiate these artifacts by naming them with the builds' unique IDs. + + - **PackerHTTPIP**, **PackerHTTPPort**, and **PackerHTTPAddr**: HTTP IP, port, and address of the file server Packer creates to serve items in the "http" dir to the VM. The HTTP address is displayed in the format `IP:PORT`. + + - **SSHPublicKey** and **SSHPrivateKey**: The public and private key that Packer uses to connect to the instance. + These are unique to the SSH communicator and are unset when using other communicators. + **SSHPublicKey** and **SSHPrivateKey** can have escape sequences and special characters so their output should be single quoted to avoid surprises. For example: + + ```json + { ... "provisioners": [{ + "type": "shell", + "inline": [ "echo '{{ build `SSHPrivateKey`}}' > /tmp/packer-session.pem" ] + }] + } + ``` + + For backwards compatibility, `WinRMPassword` is also available through this + engine, though it is no different than using the more general `Password`. + + This function is only for use within specific options inside of + _provisioners_ -- these options will be listed as being template engines + in the provisioner documentation. + + For builder-specific builder variables, please also refer to the builder docs: + + - Amazon EC2: [chroot](/packer/plugins/builders/amazon/chroot#build-shared-information-variables), + [EBS Volume](/packer/plugins/builders/amazon/ebsvolume#build-shared-information-variables), + [EBS](/packer/plugins/builders/amazon/ebs#build-shared-information-variables), + [EBS Surrogate](/packer/plugins/builders/amazon/ebssurrogate#build-shared-information-variables), + [Instance](/packer/plugins/builders/amazon/instance#build-shared-information-variables). + + This engine is in beta; please report any issues or requests on the Packer + issue tracker on GitHub. + +- `isotime [FORMAT]` - UTC time, which can be + [formatted](https://pkg.go.dev/time#example_Time_Format). See more + examples below in [the `isotime` format + reference](/packer/docs/templates/legacy_json_templates/engine#isotime-function-format-reference). + `strftime FORMAT` - UTC time, formated using the ISO C standard format + `FORMAT`. See + [jehiah/go-strftime](https://github.com/jehiah/go-strftime) for a list + of available format specifiers. + + Please note that if you are using a large number of builders, + provisioners or post-processors, using the isotime engine directly in the + plugin configuration may cause the timestamp to be slightly diffferent for + each plugin. This is because the timestamp is generated when each plugin is + launched rather than in the initial Packer process. In order to avoid this + and make sure the timestamp is consistent across all plugins, set it as a user + variable and then access the user variable within your plugins. + +- `lower` - Lowercases the string. +- `packer_version` - Returns Packer version. +- `pwd` - The working directory while executing Packer. +- `replace` - ( old, new string, n int, s ) Replace returns a copy of the + string s with the first n non-overlapping instances of old replaced by new. +- `replace_all` - ( old, new string, s ) ReplaceAll returns a copy of the + string s with all non-overlapping instances of old replaced by new. +- `split` - Split an input string using separator and return the requested + substring. +- `template_dir` - The directory to the template for the build. +- `timestamp` - The Unix timestamp in UTC when the Packer process was + launched. Please note that if you are using a large number of builders, + provisioners or post-processors, the timestamp may be slightly + different for each one because it is from when the plugin is + launched not the initial Packer process. In order to avoid this and make + the timestamp consistent across all plugins, set it as a user variable + and then access the user variable within your plugins. +- `uuid` - Returns a random UUID. +- `upper` - Uppercases the string. +- `user` - Specifies a user variable. + +#### Specific to Amazon builders: + +- `clean_resource_name` - AMI names + can only contain certain characters. This function will replace illegal + characters with a '-" character. Example usage since ":" is not a legal AMI + name is: `{{isotime | clean_resource_name}}`. + +#### Specific to Google Compute builders: + +- `clean_resource_name` - GCE + image names can only contain certain characters and the maximum length is + + 63. This function will convert upper cases to lower cases and replace + illegal characters with a "-" character. Example: + + `"mybuild-{{isotime | clean_resource_name}}"` will become + `mybuild-2017-10-18t02-06-30z`. + + Note: Valid GCE image names must match the regex + `(?:[a-z](?:[-a-z0-9]{0,61}[a-z0-9])?)` + + This engine does not guarantee that the final image name will match the + regex; it will not truncate your name if it exceeds 63 characters, and it + will not validate that the beginning and end of the engine's output are + valid. For example, `"image_name": {{isotime | clean_resource_name}}"` will + cause your build to fail because the image name will start with a number, + which is why in the above example we prepend the isotime with "mybuild". + +#### Specific to Azure builders: + +- `clean_resource_name` - Azure + managed image names can only contain certain characters and the maximum + length is 80. This function will replace illegal characters with a "-" + character. Example: + + `"mybuild-{{isotime | clean_resource_name}}"` will become + `mybuild-2017-10-18t02-06-30z`. + + Note: Valid Azure image names must match the regex + `^[^_\\W][\\w-._)]{0,79}$` + + This engine does not guarantee that the final image name will match the + regex; it will not truncate your name if it exceeds 80 characters, and it + will not validate that the beginning and end of the engine's output are + valid. It will truncate invalid characters from the end of the name when + converting illegal characters. For example, `"managed_image_name: "My-Name::"` will be converted to `"managed_image_name: "My-Name"` + +## Template variables + +Template variables are special variables automatically set by Packer at build +time. Some builders, provisioners and other components have template variables +that are available only for that component. Template variables are recognizable +because they're prefixed by a period, such as `{{ .Name }}`. For example, when +using the [`shell`](/packer/plugins/builders/vmware/iso) builder template variables +are available to customize the +[`execute_command`](/packer/docs/provisioners/shell#execute_command) parameter +used to determine how Packer will run the shell command. + +```json +{ + "provisioners": [ + { + "type": "shell", + "execute_command": "{{.Vars}} sudo -E -S bash '{{.Path}}'", + "scripts": ["scripts/bootstrap.sh"] + } + ] +} +``` + +The `{{ .Vars }}` and `{{ .Path }}` template variables will be replaced with +the list of the environment variables and the path to the script to be executed +respectively. + +-> **Note:** In addition to template variables, you can specify your own +user variables. See the [user variable](/packer/docs/templates/legacy_json_templates/user-variables) +documentation for more information on user variables. + +## `isotime` Function Format Reference + +The `isotime` template engine uses Go to generate timestamps. If you're +unfamiliar with Go, then the way you format the timestamp is going to +feel a bit unusual compared to how you may be used to formatting +datetime strings. + +Full docs and examples for the Go time formatting function can be found +[here](https://pkg.go.dev/time#example_Time_Format) + +However, the formatting basics are worth describing here. From the [Go docs](https://pkg.go.dev/time#pkg-constants): + +> These are predefined layouts for use in Time.Format and time.Parse. The +> reference time used in the layouts is the specific time: +> +> Mon Jan 2 15:04:05 MST 2006 +> +> which is Unix time 1136239445. Since MST is GMT-0700, the reference time +> can be thought of as +> +> 01/02 03:04:05PM '06 -0700 +> +> To define your own format, write down what the reference time would look like +> formatted your way; see the values of constants like ANSIC, StampMicro or +> Kitchen for examples. The model is to demonstrate what the reference time +> looks like so that the Format and Parse methods can apply the same +> transformation to a general time value. + +So what does that look like in a Packer template function? Here's an example +of how you'd declare a variable using the isotime function. + +```json +"variables": { + "myvar": "packer-{{isotime `2006-01-02 03:04:05`}}" +} +``` + +You can try and modify the following examples in a packer template or in +`packer console` to get an idea of how to set different timestamps: + +| Input | Output | +| ------------------------------------------------ | ---------------------------- | +| `` "packer-{{isotime `2006-01-02`}}" `` | "packer-2021-05-17" | +| `` "packer-{{isotime `Jan-_2-15:04:05.000`}}" `` | "packer-May-17-23:40:16.786" | +| `` "packer-{{isotime `3:04PM`}}" `` | "packer-11:40PM" | +| `"{{ isotime }}"` | "June 7, 7:22:43pm 2014" | +| `` "{{isotime `2006-01-02`}}" `` | "2014-06-07" | +| `` "{{isotime `Mon 1504`}}" `` | "Sat 1922" | +| `` "{{isotime `02-Jan-06 03\_04\_05`}}" `` | "07-Jun-2014 07_22_43" | +| `` "{{isotime `Hour15Year200603`}}" `` | "Hour19Year201407" | + +Formatting for the function `isotime` uses the magic reference date **Mon Jan 2 +15:04:05 -0700 MST 2006**, which breaks down to the following: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Day of WeekMonthDateHourMinuteSecondYearTimezone
Numeric-010203 (15)040506-0700
TextualMonday (Mon)January (Jan)-----MST
+ +_The values in parentheses are the abbreviated, or 24-hour clock values_ + +Note that "-0700" is always formatted into "+0000" because `isotime` is always +UTC time. + +## `split` Function Format Reference + +The function `split` takes an input string, a seperator string, and a numeric +component value and returns the requested substring. + +Please note that you cannot use the `split` function on user variables, because +we can't nest the functions currently. This function is intended to be used on +builder variables like build_name. If you need a split user variable, the best +way to do it is to create a separate variable. + +Here are some examples using the above options: + +```liquid +build_name = foo-bar-provider + +{{split build_name "-" 0}} = foo +{{split "fixed-string" "-" 1}} = string +``` + +Please note that double quote characters need escaping inside of templates (in +this case, on the `fixed-string` value): + +```json +{ + "post-processors": [ + [ + { + "type": "vagrant", + "compression_level": 9, + "keep_input_artifact": false, + "vagrantfile_template": "tpl/{{split build_name \"-\" 1}}.rb", + "output": "output/{{build_name}}.box", + "only": ["org-name-provider"] + } + ] + ] +} +``` + +## `replace` Function Format Reference + +Here are some examples using the `replace` options: + +```liquid +build_name = foo-bar-provider + +{{ replace_all "-" "/" build_name }} = foo/bar/provider +{{ build_name | replace "-" "/" 1 }} = foo/bar-provider +``` \ No newline at end of file diff --git a/content/packer/content/docs/templates/legacy_json_templates/index.mdx b/content/packer/content/docs/templates/legacy_json_templates/index.mdx new file mode 100644 index 0000000000..3c82475371 --- /dev/null +++ b/content/packer/content/docs/templates/legacy_json_templates/index.mdx @@ -0,0 +1,178 @@ +--- +description: > + JSON templates are files that configure the Packer components that create one or more machine images. Learn how to configure JSON templates. +page_title: JSON templates configuration reference +--- + +# JSON templates configuration reference + +This topic provides reference information for creating JSON templates for Packer. + +`@include 'from-1.5/legacy-json-warning.mdx'` + +## Introduction + +You can create Packer templates as JSON files that configure the various components +of Packer in order to create one or more machine images. Templates are portable, +static, and readable and writable by both humans and computers. Use the Packer CLI to run builds described in the template to produce any resulting machine images. + +## JSON Template Structure + +The template is a JSON object that has a set of keys configuring various +components of Packer. The available keys within a template are listed below. +Along with each key, it is noted whether it is required or not. + +- `builders` (_required_) is an array of one or more objects that defines the + builders that will be used to create machine images for this template, and + configures each of those builders. For more information on how to define + and configure a builder, read the sub-section on [configuring builders in + templates](/packer/docs/templates/legacy_json_templates/builders). + +- `description` (optional) is a string providing a description of what the + template does. This output is used only in the [inspect + command](/packer/docs/commands/inspect). + +- `min_packer_version` (optional) is a string that has a minimum Packer + version that is required to parse the template. This can be used to ensure + that proper versions of Packer are used with the template. A max version + can't be specified because Packer retains backwards compatibility with + `packer fix`. + +- `post-processors` (optional) is an array of one or more objects that + defines the various post-processing steps to take with the built images. If + not specified, then no post-processing will be done. For more information + on what post-processors do and how they're defined, read the sub-section on + [configuring post-processors in + templates](/packer/docs/templates/legacy_json_templates/post-processors). + +- `provisioners` (optional) is an array of one or more objects that defines + the provisioners that will be used to install and configure software for + the machines created by each of the builders. If it is not specified, then + no provisioners will be run. For more information on how to define and + configure a provisioner, read the sub-section on [configuring provisioners + in templates](/packer/docs/templates/legacy_json_templates/provisioners). + +- `variables` (optional) is an object of one or more key/value strings that + defines user variables contained in the template. If it is not specified, + then no variables are defined. For more information on how to define and + use user variables, read the sub-section on [user variables in + templates](/packer/docs/templates/legacy_json_templates/user-variables). + +## Comments + +JSON doesn't support comments and Packer reports unknown keys as validation +errors. If you'd like to comment your template, you can prefix a _root level_ +key with an underscore. Example: + +```json +{ + "_comment": "This is a comment", + "builders": [{}] +} +``` + +**Important:** Only _root level_ keys can be underscore prefixed. Keys within +builders, provisioners, etc. will still result in validation errors. + +-> **Note:** Packer supports HCL2 from version 1.6.0. The HashiCorp +Configuration Language does support comments anywhere in template files. +If comments are important to you, consider upgrading your +JSON template to HCL2 using the `packer hcl2_upgrade` command. + +One workaround if you are not ready to upgrade to HCL is to use jq to strip +unsupported comments from a Packer template before you run `packer build`. + +For example, here is a file named `commented_template.json`: + +```json +{ + "_comment": ["this is", "a multi-line", "comment"], + "builders": [ + { + "_comment": "this is a comment inside a builder", + "type": "null", + "communicator": "none" + } + ], + "_comment": "this is a root level comment", + "provisioners": [ + { + "_comment": "this is a different comment", + "type": "shell", + "_comment": "this is yet another comment", + "inline": ["echo hellooooo"] + } + ] +} +``` + +If you use the following jq command: + +```shell-session +$ jq 'walk(if type == "object" then del(._comment) else . end)' commented_template.json > uncommented_template.json +``` + +The tool will produce a new file containing: + +```json +{ + "builders": [ + { + "type": "null", + "communicator": "none" + } + ], + "provisioners": [ + { + "type": "shell", + "inline": ["echo hellooooo"] + } + ] +} +``` + +Once you've got your uncommented file, you can call `packer build` on it like +you normally would. + +If your install of jq does not have the walk function and you get an error like + +```text +jq: error: walk/1 is not defined at , +``` + +You can create a file `~/.jq` and add the [walk function](https://github.com/stedolan/jq/blob/ad9fc9f559e78a764aac20f669f23cdd020cd943/src/builtin.jq#L255-L262) to it by hand. + +## Example Template + +Below is an example of a basic template that could be invoked with +`packer build`. It would create an instance in AWS, and once running copy a +script to it and run that script using SSH. + +-> **Note:** This example requires an account with Amazon Web Services. +There are a number of parameters which need to be provided for a functional +build to take place. See the [Amazon builder](/packer/plugins/builders/amazon) +documentation for more information. + +```json +{ + "builders": [ + { + "type": "amazon-ebs", + "access_key": "...", + "secret_key": "...", + "region": "us-east-1", + "source_ami": "ami-fce3c696", + "instance_type": "t2.micro", + "ssh_username": "ubuntu", + "ami_name": "packer {{timestamp}}" + } + ], + + "provisioners": [ + { + "type": "shell", + "script": "setup_things.sh" + } + ] +} +``` diff --git a/content/packer/content/docs/templates/legacy_json_templates/post-processors.mdx b/content/packer/content/docs/templates/legacy_json_templates/post-processors.mdx new file mode 100644 index 0000000000..42645e4a44 --- /dev/null +++ b/content/packer/content/docs/templates/legacy_json_templates/post-processors.mdx @@ -0,0 +1,172 @@ +--- +description: | + The `post-processor` block defines actions to perform on images after Packer builds them. Learn how to configure the `post-processor` block in JSON templates. +page_title: post-processors reference for JSON templates +--- + +# `post-processors` reference for JSON templates + +This documentation describes how to configure a `post-processor` block in JSON +templates. Refer to the documentation for each post-processor type for their specific configuration options. + +`@include 'from-1.5/legacy-json-warning.mdx'` + +## Description + +The `post-processor` block defines additional actions to perform on the images built by the builders. For example, you can configure the `post-processor` to compress files and upload artifacts. + +Post-processors are _optional_. If no post-processors are defined within a +template, then no post-processing will be done to the image. The resulting +artifact of a build is just the image outputted by the builder. + + +Within a template, a section of post-processor definitions looks like this: + +```json +{ + "post-processors": [ + // ... one or more post-processor definitions here + ] +} +``` + +For each post-processor definition, Packer will take the result of each of the +defined builders and send it through the post-processors. This means that if +you have one post-processor defined and two builders defined in a template, the +post-processor will run twice (once for each builder), by default. There are +ways, which will be covered later, to control what builders post-processors +apply to, if you wish. It is also possible to prevent a post-processor from +running. + +## Post-Processor Definition + +Within the `post-processors` array in a template, there are three ways to +define a post-processor. There are _simple_ definitions, _detailed_ +definitions, and _sequence_ definitions. Another way to think about this is +that the "simple" and "detailed" definitions are shortcuts for the "sequence" +definition. + +A **simple definition** is just a string; the name of the post-processor. An +example is shown below. Simple definitions are used when no additional +configuration is needed for the post-processor. + +```json +{ + "post-processors": ["compress"] +} +``` + +A **detailed definition** is a JSON object. It is very similar to a builder or +provisioner definition. It contains a `type` field to denote the type of the +post-processor, but may also contain additional configuration for the +post-processor. A detailed definition is used when additional configuration is +needed beyond simply the type for the post-processor. An example is shown +below. + +```json +{ + "post-processors": [ + { + "type": "compress", + "format": "tar.gz" + } + ] +} +``` + +A **sequence definition** is a JSON array comprised of other **simple** or +**detailed** definitions. The post-processors defined in the array are run in +order, with the artifact of each feeding into the next, and any intermediary +artifacts being discarded. A sequence definition may not contain another +sequence definition. Sequence definitions are used to chain together multiple +post-processors. An example is shown below, where the artifact of a build is +compressed then uploaded, but the compressed result is not kept. + +It is very important that any post processors that need to be run in order, be +sequenced! + +```json +{ + "post-processors": [ + ["compress", { "type": "upload", "endpoint": "http://example.com" }] + ] +} +``` + +As you may be able to imagine, the **simple** and **detailed** definitions are +simply shortcuts for a **sequence** definition of only one element. + +## Input Artifacts + +When using post-processors, the input artifact (coming from a builder or +another post-processor) is discarded by default after the post-processor runs. +This is because generally, you don't want the intermediary artifacts on the way +to the final artifact created. + +In some cases, however, you may want to keep the intermediary artifacts. You +can tell Packer to keep these artifacts by setting the `keep_input_artifact` +configuration to `true`. An example is shown below: + +```json +{ + "post-processors": [ + { + "type": "compress", + "keep_input_artifact": true + } + ] +} +``` + +This setting will only keep the input artifact to _that specific_ +post-processor. If you're specifying a sequence of post-processors, then all +intermediaries are discarded by default except for the input artifacts to +post-processors that explicitly state to keep the input artifact. + +-> **Note:** The intuitive reader may be wondering what happens if multiple +post-processors are specified (not in a sequence). Does Packer require the +configuration to keep the input artifact on all the post-processors? The answer +is no, of course not. Packer is smart enough to figure out that at least one +post-processor requested that the input be kept, so it will keep it around. + +## Run on Specific Builds + +You can use the `only` or `except` fields to run a post-processor only with +specific builds. These two fields do what you expect: `only` will only run the +post-processor on the specified builds and `except` will run the post-processor +on anything other than the specified builds. A sequence of post-processors will +execute until a skipped post-processor. + +An example of `only` being used is shown below, but the usage of `except` is +effectively the same. `only` and `except` can only be specified on "detailed" +fields. If you have a sequence of post-processors to run, `only` and `except` +will affect that post-processor and stop the sequence. + +The `-except` option can specifically skip a named post processor. The `-only` +option _ignores_ post-processors. + +```json +([ + { + "name": "vbox", + "type": "vagrant", + "only": ["virtualbox-iso"] + }, + { + "type": "compress" + } +], +[ + "compress", + { + "type": "upload", + "endpoint": "http://example.com" + } +]) +``` + +The values within `only` or `except` are _build names_, not builder types. +Name is a required block label in HCL, but in legacy JSON, build names default +to the types of their builders (e.g. `docker` or `amazon-ebs` or +`virtualbox-iso`), unless a specific `name` attribute is specified within the +configuration. diff --git a/content/packer/content/docs/templates/legacy_json_templates/provisioners.mdx b/content/packer/content/docs/templates/legacy_json_templates/provisioners.mdx new file mode 100644 index 0000000000..4374e52880 --- /dev/null +++ b/content/packer/content/docs/templates/legacy_json_templates/provisioners.mdx @@ -0,0 +1,212 @@ +--- +description: | + The `provisioners` block contains provisioners that Packer uses to install and configure software on machines. Learn how to configure provisioners in JSON templates. +page_title: provisioners reference for JSON templates +--- + +# `provisioners` reference for JSON templates + +This topic provides reference information about the `provisioners` block in JSON templates for Packer. Refer to the documentation for specific provisioner types to learn about their configuration options. + +`@include 'from-1.5/legacy-json-warning.mdx'` + +## Description + +The `provisioners` block contains provisioners that Packer uses to install and configure software within running machines prior to turning them into machine images. + +Provisioners are optional. When you omit the `provisioners` block, Packer installs only the default software within the resulting machine images. + +Use the following syntax to add the `provisioners` block to your JSON template: + +```json +{ + "provisioners": [ + // ... one or more provisioner definitions here + ] +} +``` + +For each of the definitions, Packer will run the provisioner for each of the +configured builds. The provisioners will be run in the order they are defined +within the template. + +## Provisioner Definition + +A provisioner definition is a JSON object that must contain at least the `type` +key. This key specifies the name of the provisioner to use. Additional keys +within the object are used to configure the provisioner, with the exception of +a handful of special keys, covered later. + +As an example, the "shell" provisioner requires a key such as `script` which +specifies a path to a shell script to execute within the machines being +created. + +An example provisioner definition is shown below, configuring the shell +provisioner to run a local script within the machines: + +```json +{ + "type": "shell", + "script": "script.sh" +} +``` + +## Run on Specific Builds + +You can use the `only` or `except` configurations to run a provisioner only +with specific builds. These two configurations do what you expect: `only` will +only run the provisioner on the specified builds and `except` will run the +provisioner on anything other than the specified builds. + +An example of `only` being used is shown below, but the usage of `except` is +effectively the same: + +```json +{ + "type": "shell", + "script": "script.sh", + "only": ["virtualbox-iso"] +} +``` + +The values within `only` or `except` are _build names_, not builder types. If +you recall, build names by default are just their builder type, but if you +specify a custom `name` parameter, then you should use that as the value +instead of the type. +Values within `except` could also be a _post-processor_ name. + +## On Error Provisioner + +You can optionally create a single specialized provisioner field called an +`error-cleanup-provisioner`. This provisioner will not run unless the normal +provisioning run fails. If the normal provisioning run does fail, this special +error provisioner will run _before the instance is shut down_. This allows you +to make last minute changes and clean up behaviors that Packer may not be able +to clean up on its own. + +For examples, users may use this provisioner to make sure that the instance is +properly unsubscribed from any services that it connected to during the build +run. + +Toy usage example for the error cleanup script: + +```json +{ + "builders": [ + { + "type": "null", + "communicator": "none" + } + ], + "provisioners": [ + { + "type": "shell-local", + "inline": ["exit 2"] + } + ], + "error-cleanup-provisioner": { + "type": "shell-local", + "inline": ["echo 'rubber ducky'> ducky.txt"] + } +} +``` + +## Build-Specific Overrides + +While the goal of Packer is to produce identical machine images, it sometimes +requires periods of time where the machines are different before they +eventually converge to be identical. In these cases, different configurations +for provisioners may be necessary depending on the build. This can be done +using build-specific overrides. + +An example of where this might be necessary is when building both an EC2 AMI +and a VMware machine. The source EC2 AMI may setup a user with administrative +privileges by default, whereas the VMware machine doesn't have these +privileges. In this case, the shell script may need to be executed differently. +Of course, the goal is that hopefully the shell script converges these two +images to be identical. However, they may initially need to be run differently. + +This example is shown below: + +```json +{ + "type": "shell", + "script": "script.sh", + "override": { + "vmware-iso": { + "execute_command": "echo 'password' | sudo -S bash {{.Path}}" + } + } +} +``` + +As you can see, the `override` key is used. The value of this key is another +JSON object where the key is the name of a [builder +definition](/packer/docs/templates/legacy_json_templates/builders). The value of this is in turn +another JSON object. This JSON object simply contains the provisioner +configuration as normal. This configuration is merged into the default +provisioner configuration. + +## Pausing Before Running + +With certain provisioners it is sometimes desirable to pause for some period of +time before running it. Specifically, in cases where a provisioner reboots the +machine, you may want to wait for some period of time before starting the next +provisioner. + +Every provisioner definition in a Packer template can take a special +configuration `pause_before` that is the amount of time to pause before running +that provisioner. By default, there is no pause. An example is shown below: + +```json +{ + "type": "shell", + "script": "script.sh", + "pause_before": "10s" +} +``` + +For the above provisioner, Packer will wait 10 seconds before uploading and +executing the shell script. + +## Retry on error + +With certain provisioners it is sometimes desirable to retry when it fails. +Specifically, in cases where the provisioner depends on external processes that are not done yet. + +Every provisioner definition in a Packer template can take a special +configuration `max_retries` that is the maximum number of times a provisioner will retry on error. +By default, there `max_retries` is zero and there is no retry on error. An example is shown below: + +```json +{ + "type": "shell", + "script": "script.sh", + "max_retries": 5 +} +``` + +For the above provisioner, Packer will retry maximum five times until stops failing. +If after five retries the provisioner still fails, then the complete build will fail. + +## Timeout + +Sometimes a command can take much more time than expected + +Every provisioner definition in a Packer template can take a special +configuration `timeout` that is the amount of time to wait before +considering that the provisioner failed. By default, there is no timeout. An +example is shown below: + +```json +{ + "type": "shell", + "script": "script.sh", + "timeout": "5m" +} +``` + +For the above provisioner, Packer will cancel the script if it takes more than +5 minutes. + +Timeout has no effect in debug mode. diff --git a/content/packer/content/docs/templates/legacy_json_templates/user-variables.mdx b/content/packer/content/docs/templates/legacy_json_templates/user-variables.mdx new file mode 100644 index 0000000000..a9afbcc6a9 --- /dev/null +++ b/content/packer/content/docs/templates/legacy_json_templates/user-variables.mdx @@ -0,0 +1,421 @@ +--- +description: | + The `variables` block contains parameterized values for your templates. Learn how to configure `variables` to improve a template's security, portability, and shareability. +page_title: variables reference for JSON templates +--- + +# `variables` reference for JSON templates + +This topic describes how to use user variables in Packer templates written in JSON. + +`@include 'from-1.5/legacy-json-warning.mdx'` + +## Description + +The `variables` block lets you further configure a template with variables +from the command-line, environment variables, Vault, or files. This lets you +parameterize your templates so that you can keep secret tokens, +environment-specific data, and other types of information out of your +templates. This maximizes the portability of the template. + +Using user variables expects you to know how [configuration +templates](/packer/docs/templates/legacy_json_templates/engine) work. If you don't know how +configuration templates work yet, please read that page first. + +## Usage + +In order to set a user variable, you must define it either within the +`variables` section within your template, or using the command-line `-var` or +`-var-file` flags. + +Even if you want a user variable to default to an empty string, it is best to +explicitly define it. This explicitness helps reduce the time it takes for +newcomers to understand what can be modified using variables in your template. + +The `variables` section is a key/value mapping of the user variable name to a +default value. A default value can be the empty string. An example is shown +below: + +```json +{ + "variables": { + "aws_access_key": "", + "aws_secret_key": "" + }, + + "builders": [ + { + "type": "amazon-ebs", + "access_key": "{{user `aws_access_key`}}", + "secret_key": "{{user `aws_secret_key`}}" + // ... + } + ] +} +``` + +In the above example, the template defines two user variables: `aws_access_key` +and `aws_secret_key`. They default to empty values. Later, the variables are +used within the builder we defined in order to configure the actual keys for +the Amazon builder. + +If the default value is `null`, then the user variable will be _required_. This +means that the user must specify a value for this variable or template +validation will fail. + +User variables are used by calling the `{{user}}` function in the form of +`{{user 'variable'}}`. This function can be used in _any value_ but `type` +within the template: in builders, provisioners, _anywhere outside the `variables` section_. +User variables are available globally within the rest of the template. + +## Environment variables + +Environment variables can be used within your template using user variables. +The `env` function is available _only_ within the default value of a user +variable, allowing you to default a user variable to an environment variable. +An example is shown below: + +```json +{ + "variables": { + "my_secret": "{{env `MY_SECRET`}}" + } +} +``` + +This will default "my_secret" to be the value of the "MY_SECRET" environment +variable (or an empty string if it does not exist). + +-> **Why can't I use environment variables elsewhere?** User variables are +the single source of configurable input to a template. We felt that having +environment variables used _anywhere_ in a template would confuse the user +about the possible inputs to a template. By allowing environment variables only +within default values for user variables, user variables remain as the single +source of input to a template that a user can easily discover using +`packer inspect`. + +-> **Why can't I use `~` for home variable?** `~` is an special variable +that is evaluated by shell during a variable expansion. As Packer doesn't run +inside a shell, it won't expand `~`. + +## Consul keys + +Consul keys can be used within your template using the `consul_key` function. +This function is available _only_ within the default value of a user variable, +for reasons similar to environment variables above. + +```json +{ + "variables": { + "soft_versions": "{{ consul_key `my_image/softs_versions/next` }}" + } +} +``` + +This will default `soft_versions` to the value of the key +`my_image/softs_versions/next` in consul. + +The configuration for consul (address, tokens, ...) must be specified as +environment variables, as specified in the +[Documentation](/consul/docs/commands#environment-variables). + +## Vault variables + +Packer can read secrets read from HashiCorp Vault and add them to your template as user variables. You can only use the `vault` function +in the default value of a user variable. This allows you to default a user variable to a Vault secret. Refer to the [Vault documentation](https://developer.hashicorp.com/vault/docs/secrets) for additional information about integrating with Vault secrets engines. + +An example of using a v2 kv engine: + +If you store a value in vault using `vault kv put secret/hello foo=world`, you +can access it using the following template engine: + +```json +{ + "variables": { + "my_secret": "{{ vault `/secret/data/hello` `foo`}}" + } +} +``` + +which will assign "my_secret": "world" + +An example of using a v1 kv engine: + +If you store a value in vault using: + + vault secrets enable -version=1 -path=secrets kv + vault kv put secrets/hello foo=world + +You can access it using the following template engine: + + { + "variables": { + "VAULT_SECRETY_SECRET": "{{ vault `secrets/hello` `foo`}}" + } + } + +This example accesses the Vault path `secret/data/foo` and returns the value +stored at the key `bar`, storing it as "my_secret". + +In order for this to work, you must set the environment variables `VAULT_TOKEN` +and `VAULT_ADDR` to valid values. + +The api tool we use allows for more custom configuration of the Vault client via +environment variables. + +The full list of available environment variables is: + +```text +"VAULT_ADDR" +"VAULT_AGENT_ADDR" +"VAULT_CACERT" +"VAULT_CAPATH" +"VAULT_CLIENT_CERT" +"VAULT_CLIENT_KEY" +"VAULT_CLIENT_TIMEOUT" +"VAULT_SKIP_VERIFY" +"VAULT_NAMESPACE" +"VAULT_TLS_SERVER_NAME" +"VAULT_WRAP_TTL" +"VAULT_MAX_RETRIES" +"VAULT_TOKEN" +"VAULT_MFA" +"VAULT_RATE_LIMIT" +``` + +and detailed documentation for usage of each of those variables can be found +[here](/vault/docs/commands#environment-variables). + +## AWS Secrets Manager variables + +Secrets can be read from [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/) +and used within your template as user variables. The `aws_secretsmanager` function is +available _only_ within the default value of a user variable, allowing you to default +a user variable to an AWS Secrets Manager secret. + +### Plaintext secrets + +```json +{ + "variables": { + "password": "{{ aws_secretsmanager `globalpassword` }}" + } +} +``` + +In the example above it is assumed that the secret `globalpassword` is not +stored as a key pair but as a single non-JSON string value. Which the +`aws_secretsmanager` function will return as a raw string. + +### Single key secrets + +```json +{ + "variables": { + "password": "{{ aws_secretsmanager `sample/app/password` }}" + } +} +``` + +In the example above it is assumed that only one key is stored in +`sample/app/password` if there are multiple keys stored in it then you need +to indicate the specific key you want to fetch as shown below. + +### Multiple key secrets + +```json +{ + "variables": { + "db_password": "{{ aws_secretsmanager `sample/app/passwords` `db` }}", + "api_key": "{{ aws_secretsmanager `sample/app/passwords` `api_key` }}" + } +} +``` + +In order to use this function you have to configure valid AWS credentials using +one of the following methods: + +- [Environment Variables](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-envvars.html) +- [CLI Configuration Files](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html) +- [Container Credentials](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html) +- [Instance Profile Credentials](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html) + +## Use array values + +Some templates call for array values. You can use template variables for these, +too. For example, the `amazon-ebs` builder has a configuration parameter called +`ami_regions`, which takes an array of regions that it will copy the AMI to. +You can parameterize this by using a variable that is a list of regions, joined +by a `,`. For example: + +```json +{ + "variables": { + "destination_regions": "us-west-1,us-west-2" + }, + "builders": [ + { + "ami_name": "packer-qs-{{timestamp}}", + "instance_type": "t2.micro", + "region": "us-east-1", + "source_ami_filter": { + "filters": { + "name": "*ubuntu-xenial-16.04-amd64-server-*", + "root-device-type": "ebs", + "virtualization-type": "hvm" + }, + "most_recent": true, + "owners": ["099720109477"] + }, + "ami_regions": "{{user `destination_regions`}}", + "ssh_username": "ubuntu", + "type": "amazon-ebs" + } + ] +} +``` + +## Set variables + +Now that we covered how to define and use user variables within a template, the +next important point is how to actually set these variables. Packer exposes two +methods for setting user variables: from the command line or from a file. + +### CLI + +To set user variables from the command line, the `-var` flag is used as a +parameter to `packer build` (and some other commands). Continuing our example +above, we could build our template using the command below. The command is +split across multiple lines for readability, but can of course be a single +line. + +```shell-session +$ packer build \ + -var 'aws_access_key=foo' \ + -var 'aws_secret_key=bar' \ + template.json +``` + +As you can see, the `-var` flag can be specified multiple times in order to set +multiple variables. Also, variables set later on the command-line override any +earlier set variable of the same name. + +**warning** If you are calling Packer from cmd.exe, you should double-quote +your variables rather than single-quoting them. For example: + +`packer build -var "aws_secret_key=foo" template.json` + +### From file + +Variables can also be set from an external JSON file. The `-var-file` flag +reads a file containing a key/value mapping of variables to values and sets +those variables. An example JSON file may look like this: + +```json +{ + "aws_access_key": "foo", + "aws_secret_key": "bar" +} +``` + +It is a single JSON object where the keys are variables and the values are the +variable values. Assuming this file is in `variables.json`, we can build our +template using the following command: + +```text +On Linux : +$ packer build -var-file=variables.json template.json +On Windows : +packer build -var-file variables.json template.json +``` + +The `-var-file` flag can be specified multiple times and variables from +multiple files will be read and applied. As you'd expect, variables read from +files specified later override a variable set earlier. + +Combining the `-var` and `-var-file` flags together also works how you'd +expect. Variables set later in the command override variables set earlier. So, +for example, in the following command with the above `variables.json` file: + +```shell-session +$ packer build \ + -var 'aws_access_key=bar' \ + -var-file=variables.json \ + -var 'aws_secret_key=baz' \ + template.json +``` + +Results in the following variables: + +| Variable | Value | +| -------------- | ----- | +| aws_access_key | foo | +| aws_secret_key | baz | + +## Sensitive variables + +If you use the environment to set a variable that is sensitive, you probably +don't want that variable printed to the Packer logs. You can make sure that +sensitive variables won't get printed to the logs by adding them to the +"sensitive-variables" list within the Packer template: + +```json +{ + "variables": { + "my_secret": "{{env `MY_SECRET`}}", + "not_a_secret": "plaintext", + "foo": "bar" + }, + + "sensitive-variables": ["my_secret", "foo"], + ... +} +``` + +The above snippet of code will function exactly the same as if you did not set +"sensitive-variables", except that the Packer UI and logs will replace all +instances of "bar" and of whatever the value of "my_secret" is with +``. This allows you to be confident that you are not printing +secrets in plaintext to our logs by accident. + +## Recipes + +### Make a provisioner step conditional on the value of a variable + +There is no specific syntax in Packer templates for making a provisioner step +conditional, depending on the value of a variable. However, you may be able to +do this by referencing the variable within a command that you execute. For +example, here is how to make a `shell-local` provisioner only run if the +`do_nexpose_scan` variable is non-empty. + +```json +{ + "type": "shell-local", + "command": "if [ ! -z \"{{user `do_nexpose_scan`}}\" ]; then python -u trigger_nexpose_scan.py; fi" +} +``` + +### Using HOME Variable + +In order to use `$HOME` variable, you can create a `home` variable in Packer: + +```json +{ + "variables": { + "home": "{{env `HOME`}}" + } +} +``` + +And this will be available to be used in the rest of the template, i.e.: + +```json +{ + "builders": [ + { + "type": "google", + "account_file": "{{ user `home` }}/.secrets/gcp-{{ user `env` }}.json" + } + ] +} +``` \ No newline at end of file diff --git a/content/packer/content/docs/terminology.mdx b/content/packer/content/docs/terminology.mdx new file mode 100644 index 0000000000..7cd4497c95 --- /dev/null +++ b/content/packer/content/docs/terminology.mdx @@ -0,0 +1,47 @@ +--- +description: > + Packer and the Packer documentation uses a vocabulary of terms that have specific meaning. Learn about Packer terminology in this glossary. +page_title: Packer terminology +--- + +# Packer terminology + +This page documents terminology we use throughout this documentation that have a specific meaning in Packer. These terms may not be immediately obvious if you have not used Packer before. + +- _Artifacts_ are the results of a single build, and are usually a set of IDs + or files to represent a machine image. Every builder produces a single + artifact. As an example, in the case of the Amazon EC2 builder, the + artifact is a set of AMI IDs (one per region). For the VMware builder, the + artifact is a directory of files comprising the created virtual machine. + +- _Builds_ are a single task that eventually produces an artifact for a single + platform. Multiple builds run in parallel. Example usage in a sentence: + "The Packer build produced an AMI to run our web application." Or: "Packer + is running the builds now for VMware, AWS, and VirtualBox." + +- _Builders_ are components of Packer that are able to create a machine image + for a single platform. Builders read in some configuration and use that to + run and generate a machine image. A builder is invoked as part of a build + in order to create the actual resulting artifacts. Example builders include + VirtualBox, VMware, and Amazon EC2. + +- _Commands_ are sub-commands for the `packer` program that perform a job. + An example command is "build", which is invoked as `packer build`. Packer + ships with a set of commands out of the box in order to define its + command-line interface. + +- _Data sources_ are components of Packer that fetch data from outside Packer + and make it available to use within the template. + Example of data sources include Amazon AMI, and Amazon Secrets Manager. + +- _Post-processors_ are components of Packer that take the result of a + builder or another post-processor and process that to create a new + artifact. Examples of post-processors are compress to compress artifacts, + upload to upload artifacts, etc. + +- _Provisioners_ are components of Packer that install and configure software + within a running machine prior to that machine being turned into a static + artifact. They perform the major work of making the artifact contain useful + software. Example provisioners include shell scripts, Chef, Puppet, etc. + +- _Templates_ are files that define one or more builds by configuring various Packer components. Packer reads a template and uses the information to create multiple machine images in parallel. You can write templates in HCL or JSON files format. diff --git a/content/packer/content/docs/upgrade/v1_11.mdx b/content/packer/content/docs/upgrade/v1_11.mdx new file mode 100644 index 0000000000..77270c4ee3 --- /dev/null +++ b/content/packer/content/docs/upgrade/v1_11.mdx @@ -0,0 +1,75 @@ +--- +page_title: Upgrade to v1.11 +description: Learn how to upgrade your Packer installation to v1.11 +--- + +# Upgrade to v1.11 + +This topic describes upgrade considerations for Packer v1.11. Refer to the [v1.11 release notes](/packer/docs/release-notes/v1_11) for details about changes in v1.11. + +## Overview + +You can use the upgrade command for the package manager you installed Packer with or download and replace the existing binary to upgrade Packer. Refer to [Install Packer](/packer/install) for instructions + +## Upgrade from 1.10 + +Take the following actions as a result of changes between v1.10 and v1.11. + +### Review templates + +Packer v1.11 supports existing v1.10 templates. No action is required. + +### Review plugin management processes + +Because we have made the following changes to how Packer manages plugins, you should review your plugin management processes and make changes accordingly: + +* Single-component plugins are no longer supported. Update your dependencies to multi-component plugins. For example, update `packer-builder-hashicups-coffee` to `packer-plugin-hashicups`. + +* Packer requires all plugins to be installed in the plugins directory. Plugins must follow the hierarchy that matches the expected source. Plugins must follow the `packer-plugin-` naming convention. Plugins must be accompanied by a `SHA256SUM` file. If you manage plugin installations outside of Packer's plugin directory, modify your installation processes to meet these requirements. Refer to [Install Plugins](/packer/docs/plugins/install) for additional information. + +* Packer requires semantic versioning for plugins installed from Packer HCL2 templates. Refer to [Requirements](/packer/docs/plugins/install#requiremets) in the Packer plugin installation documentation for additional information. + +* You can use `-dev` pre-release versions of plugins in your builds using the `packer plugins install --path` command. Refer to [Use a plugin under development](/packer/docs/plugins/install#use-a-plugin-under-development). + +* You can use the `packer plugins install --path` command to install plugins from non-Github hosts. This command installs the plugin in the Packer plugin directory and creates the file hierarchy that matches the supplied source. As result, Packer is able to discover non-Github sources specified in the `required_plugins` blocks. This enables you to add version constraints on those plugins if you so choose. + Although you can add non-GitHub sources to `required_plugins` for validation purposes, remote installation is still limited to Github-sourced plugins. Refer to [Define plugin source](/packer/docs/templates/hcl_templates/blocks/packer#define-plugin-source) in the `required_plugins` reference documentation for additional information. + +## Troubleshooting + +This section provides guidance on potential issues you may experience after upgrading. + +### Missing plugins error + +Packer may report that you one or more plugins are missing after the upgrade. +This may occur if you had used one of the following plugin installation methods that are longer supported: + +* Installed the plugin to the directory Packer is invoked from. +* Installed the plugin alongside the Packer binary. +* Installed the plugin under the root of the `PACKER_PLUGIN_PATH` directory. + +For any missing components, you may need to reinstall their respective plugins as described in [Installing Plugins](/packer/docs/plugins/install). + +### Use `required_plugins` message + +Packer may print a message directing you to specify plugins in the `required_plugins` block in your template and install them using the `packer init` command. +This is the preferred way to manage the plugins, but it is not required. You can still install plugins manually using the `packer plugins install` command. Refer to [Installing Plugins](/packer/docs/plugins/install) + +### Blocked remote access to plugin + +If you cannot remotely install a plugin for any reason, such as a firewall configuration or GitHub is blocking access, you can use the local installation methods. Refer to [Installing Plugins](/packer/docs/plugins/install) + +### Install plugins from legacy JSON template + +The `required_plugins` block is only available within HCL templates. Legacy JSON users must use `packer plugins install` + +## Installing plugins outside of namespace directories + +Packer no longer supports installing plugins that are named `packer-plugin-` under a single root directory. +You can still use Packer CLI commands to install plugins and customise which root directory manages those plugins by setting the `PACKER_PLUGIN_PATH` environment variable so that it points to the custom directory. +Once set, Packer automatically creates the subdirectories that match the sources of the plugins under that directory. + +## Reference + +Refer to the following topics for additional information about using the Packer CLI to install plugins: +* [`packer init`](https://developer.hashicorp.com/packer/docs/commands/init) +* [`packer plugins install`](https://developer.hashicorp.com/packer/docs/commands/plugins/install). \ No newline at end of file diff --git a/content/packer/content/docs/upgrade/v1_12.mdx b/content/packer/content/docs/upgrade/v1_12.mdx new file mode 100644 index 0000000000..fc19fe8804 --- /dev/null +++ b/content/packer/content/docs/upgrade/v1_12.mdx @@ -0,0 +1,47 @@ +--- +page_title: Upgrade to v1.12 +description: Learn how to upgrade your Packer installation to v1.12 +--- + +# Upgrade to v1.12 + +The topic describes upgrade considerations for Packer v1.12. Refer to the [v1.12 release notes](/packer/docs/release-notes/v1_12) for details about changes in v1.12. + +## Overview + +You can use the upgrade command for the package manager you installed Packer with or download and replace the existing binary to upgrade Packer. Refer to [Install Packer](https://developer.hashicorp.com/packer/install) for instructions. + +## Upgrade from 1.11 + +Packer 1.12's notable changes can be summarised by the following points: + +1. Protocol changes for Packer/plugin communication +2. Introduction of a Directed Acyclic Graph (DAG) for Data Sources and Locals + +**Note**: These are not breaking changes, and your templates should continue working as-is. + +## Protocol changes + +When initialising, Packer is now able to pick between two different protocols for communication with plugins: gob (legacy), and protobuf/msgpack. + +The protobuf/msgpack protocol will become the standard approach after this release, but in order to avoid breaking retro-compatibility, Packer only chooses this protocol if the following conditions are true: + +1. All the plugin candidates for a build support the new protocol +2. Packer supports the protocol + +This is an internal Packer change and should be completely transparent to you. +However, if you start seeing errors with Packer/plugin communication like serialization errors, please [report it to us](https://github.com/hashicorp/packer/issues) so we can fix it in a later version. +You can set the `PACKER_FORCE_GOB` environment variable to force Packer and plugins to use the legacy serialization format for communication. + +Example: `export PACKER_FORCE_GOB=1` will force using gob for serialization. + +## Data Source and Locals DAG support + +In order to allow more flexibility in how you define data sources and locals, we introduced a graph-based approach to scheduling the evaluation of those components. +With this change, you can use a local variable in a data source, and vice-versa. You are not bound to a strict evaluation order. + +If your templates worked with older versions of Packer, this change should be transparent to you. +However, if you are experiencing errors with your data sources or locals, typically errors due to unkown values, or other dependency problems, please [report it to us](https://github.com/hashicorp/packer/issues) so we can fix it in a later version. +You can use the `--use-sequential-evaluation` command-line option can be used to revert back to the evaluation scheduler that Packer used prior to v1.12.0. + +Example: `packer build --use-sequential-evaluation