Implement custom configuration and SSH key path flexibility (#137)

* Add config merging with custom overrides, SSH key path flexibility, and switch from /etc to /target

- Implement default config loading from azure-init.conf with optional overrides via 10-azure-init-sshd.
- SSH now uses sshd -G or falls back to ~/.ssh/authorized_keys if query mode is disabled.
- Changed config path from /etc/azure-init to target/azure-init to avoid using sudo during testing.
- Requesting feedback on how to handle the /target vs /etc issue for production.
- Larger test cases needed once config and SSH key handling are finalized.

* Refactor configuration handling for azure-init

- Introduced Config struct to centralize configuration management and simplify passing configuration data within Provision.
- Default configuration values are now defined directly within config.rs, eliminating the need for an externally installed config file in /etc.
- Removed installation of /etc/azure-init/azure-init.toml from build.rs to reflect new default handling.
- Added support for --config CLI parameter in main.rs to allow users to specify either a configuration file or directory:
    - When a file is specified, loads configuration directly from that file.
    - When a directory is specified, loads the base azure-init.toml, followed by all .toml files in .d in alphabetical order.
- Integrated hostname_provisioner, user_provisioner, and password_provisioner enums into config.rs, allowing provisioners to be set directly within Config and ensuring that these settings can be controlled or overridden through the Config struct.
- Revised SSH provisioning to respect the Config struct and load defaults or user-specified configuration paths dynamically.
- Applied other requested changes for consistency, including error handling improvements, simplified default trait derivations, and ensuring centralized configuration for all settings.
- Updated documentation with configuration hierarchy and merging rules to reflect the new --config option and configuration handling behavior.

* Resolving problems with ssh.rs and addressing other general PR comments:
- Added strict tracing and error logging and fail-fast behavior for missing/invalid sshd -G output and authorized_keys_path,rather than falling back to a default.
- Updated the  config structs for Imds and Wireserver to use floats and cleaned up unused config files / config.rs Structs.
- Updated markdown documentation to describe azure-init’s behavior on invalid configurations.

* Refactor SSH provisioning to use updated configuration and query logic

- Updated Ssh struct to use query_sshd_config boolean instead of fallback_sshd_default and query_mode, essentially combining those two into the same field.
- Modified provision_ssh function to align with new config:
  - If query_sshd_config is true and sshd -G succeeds, dynamically sets authorized keys path.
  - If sshd -G fails, logs a warning but uses authorized_keys_path to proceed with provisioning.
  - If query_sshd_config is false, uses authorized_keys_path as configured.
- Removed unused configure_password_authentication field from the Ssh struct.
- Adjusted unit tests to reflect new query_sshd_config logic and ensure accurate path setup in all scenarios.
- Added documentation for updated Ssh struct and provision_ssh logic.

* PR comments, as well as broken logic for merge_toml for debugging help in preserving merged values

* Resolving broken merge logic to merge at the field level rather than at the function level.

- Consolidated merge and merge_toml into a unified merge_toml function for cleaner and more consistent merging logic.
- Refactored load_from_directory to work with raw TOML values, ensuring progressive merging while preserving values.
- Updated test suite to reflect changes in the merging and loading logic.
- Added missing rustdoc comment to fn provisioning() in mod.rs

* Add --config argument to CLI Struct and tests for file and directory inputs

- Added --config argument to the CLI struct for specifying configuration files or directories.
- Created MockCli to replicate CLI behavior for testing.
- Added test_custom_config_via_cli to verify --config with a file input.
- Added test_directory_config_via_cli to verify --config with a directory input.

* Removing extra whitespace and dependencies, simplifying configuration.md document

* Refactor HostnameProvisioner and switch to ok_or

- Consolidated HostnameProvisioner and Provisioner enums into a single enum in config.rs.
- Replaced ok_or_else with ok_or for static error handling in provision() to simplify and optimize the code.

* Refactor config load/merge logic to use Figment and other PR comments

- Replaced custom configuration load and merge logic with Figment to simplify and standardize handling of config sources.
- Resolved other minor PR comments about whitespace, spelling and unused dependencies.
- Updated unit tests to validate Figment-based configuration management.

* Unify UserProvisioner and PasswordProvisioner logic to exist entirely in config.rs, just like HostnameProvisioner. Removing strum dependency and defining those config structs as non-exhaustive

* Fix configuration loading logic to ensure proper merging order: apply defaults, load azure-init.toml if present, merge files from azure-init.toml.d in lexicographical order, and apply CLI overrides. Added helper function to avoid duplicate logic.

* PR comments resolved about load() using absolute paths instead of relative paths, refactoring config and unit tests to use load_from wrapper function, and other minor changes to configuration.md and merge_toml_directory().

* Amending rustdoc commment for load() function.

Author: peytonr18

doc/configuration.md

@@ -0,0 +1,174 @@
+# Custom Configuration Design for azure-init
+
+## Objective
+
+The azure-init custom configuration architecture enables dynamic and flexible management of various settings for virtual machines provisioned with the light-weight agent. Customizable settings include SSH, IMDS, provisioning media, azure proxy agent, wireserver, and telemetry.
+
+## Design
+
+The system supports default configurations with user-specified overrides via configuration files or CLI parameters.
+
+### Key Features
+
+- **Config Validation**: Validates user-provided settings against supported options, rejecting invalid values during deserialization.
+- **Built-in Defaults**: The system defines defaults directly in the code, eliminating the need for a separate default config file.
+- **Merging of Configurations**: Combines configuration sources hierarchically, applying defaults first, followed by file-based overrides, and finally CLI parameters for the highest precedence.
+
+## Config File Structure
+
+**Format**: TOML
+
+- Users can override default settings by supplying a single configuration file or multiple `.toml` files in a directory.
+
+### CLI Parameters
+
+Example: `--config /etc/azure-init/`
+
+## Configuration Hierarchy for `azure-init`
+
+Configuration can be set via a single file, a directory containing multiple files, or the default values defined in the code.
+
+### Configuration Loading Order
+
+#### 1. Defaults in Code
+
+The configuration process starts with the built-in defaults specified in `Config::default()`.
+
+#### 2. Base File and Directory Loading Logic
+
+- After applying default values, `azure-init` checks for a base `azure-init.toml` file. If it exists, it is loaded as the base configuration.
+- If an `azure-init.toml.d` directory exists, its `.toml` files are loaded and merged in lexicographical order.
+- If neither the `azure-init.toml` nor the directory exists, the configuration remains as defined by the built-in defaults.
+
+     ```text
+     /etc/azure-init/
+     ├── azure-init.toml            # Base configuration
+     └── azure-init.toml.d/
+         ├── 01-network.toml        # Additional network configuration
+         ├── 02-ssh.toml            # Additional SSH configuration
+         └── 99-overrides.toml      # Final overrides
+     ```
+
+- Each `.toml` file is merged into the configuration in the sorted order. If two files define the same configuration field, the value from the file processed last will take precedence. For example, in the order above, the final value(s) would come from `99-overrides.toml`.
+
+#### 3. CLI Override (`--config`)
+
+- The `--config` flag specifies a configuration path that can point to either a single file or a directory.
+  - **File:** If a file is specified, it is merged as the final layer, overriding all prior configurations.
+  - **Directory:** If a directory is specified, `.toml` files within it are loaded and merged in, following the same rules specified in the Directory Loading Logic section.
+  - **Example:** `azure-init --config /path/to/custom-config.toml`
+
+### Example: Directory with Multiple .toml Files
+
+**Command:**
+
+```sh
+azure-init --config /path/to/custom-config-directory
+```
+
+**Directory Structure:**
+
+```bash
+/path/to/custom-config-directory/
+├── azure-init.toml                # Base configuration
+└── azure-init.toml.d/
+    ├── 01-network.toml            # Network configuration
+    ├── 02-ssh.toml                # SSH configuration
+    └── 99-overrides.toml          # Overrides
+```
+
+**Order of Merging:**
+
+1. Applies defaults from `Config::default()` as defined in `config.rs`.
+2. Loads `azure-init.toml` as the base configuration, if present.
+3. Merges configuration `.toml` files found in `azure-init.toml.d` in lexicographical order. The last file in the sorted order takes precedence.
+   - `01-network.toml`
+   - `02-ssh.toml`
+   - `99-overrides.toml`
+4. Applies any CLI overrides, either from a file or a directory.
+
+## Validation and Deserialization Process
+
+Azure Init uses strict validation on configuration fields to ensure they match expected types and values. If a configuration includes an unsupported value or incorrect type, deserialization will fail.
+
+### Error Handling During Deserialization
+
+- When a configuration file is loaded, its contents are parsed and converted from `.toml` into structured data. If a field in the file contains an invalid value (e.g., `query_sshd_config` is set to `"not_a_boolean"` instead of `true` or `false`), the parsing process will fail with a deserialization error due to the mismatched type.
+
+### Propagation of Deserialization Errors
+
+- When deserialization fails, an error is logged to indicate that the configuration file could not be parsed correctly. This error propagates through the application, causing the provisioning process to fail. The application will not proceed with provisioning if the configuration is invalid.
+
+### Example of an Unsupported Value
+
+Here’s an example configuration with an invalid value for `query_sshd_config`. This field expects a boolean (`true` or `false`), but in this case, an unsupported string value `"not_a_boolean"` is provided.
+
+```toml
+# Invalid value for query_sshd_config (not a boolean)
+[ssh]
+query_sshd_config = "not_a_boolean" # This will cause a validation failure
+```
+
+## Sample of Valid Configuration File
+
+```toml
+[ssh]
+authorized_keys_path = ".ssh/authorized_keys"
+query_sshd_config = true
+
+[hostname_provisioners]
+backends = ["hostnamectl"]
+
+[user_provisioners]
+backends = ["useradd"]
+
+[password_provisioners]
+backends = ["passwd"]
+
+[imds]
+connection_timeout_secs = 2.0
+read_timeout_secs = 60
+total_retry_timeout_secs = 300
+
+[provisioning_media]
+enable = true
+
+[azure_proxy_agent]
+enable = true
+
+[wireserver]
+connection_timeout_secs = 2.0
+read_timeout_secs = 60
+total_retry_timeout_secs = 1200
+
+[telemetry]
+kvp_diagnostics = true
+```
+
+## Behavior of `azure-init` on Invalid Configuration
+
+`azure-init` handles configuration issues by logging errors and either using default values or halting functionality, depending on the severity of the issue. Here’s how it responds to different types of problems:
+
+### 1. Invalid Configuration
+
+- If a configuration file contains syntax errors (e.g., malformed TOML) or unsupported values for fields (e.g., invalid enums),  `azure-init` logs the error and terminates. The provisioning process does not proceed when configuration parsing fails.
+
+### 2. Missing or Invalid SSH Configuration
+
+- `query_sshd_config = true`:
+  - `azure-init` attempts to dynamically query the authorized keys path using the `sshd -G` command.
+  - If `sshd -G` succeeds: The dynamically queried path is used for the authorized keys.
+  - If `sshd -G` fails: The failure is logged, but azure-init continues using the fallback path specified in authorized_keys_path (default: `.ssh/authorized_keys`).
+- `query_sshd_config = false`:
+  - `azure-init` skips querying `sshd -G` entirely
+  - The value in `authorized_keys_path` is used directly, without any dynamic path detection.
+
+### 3. Handling of Provisioners in `azure-init`
+
+The `azure-init` configuration allows for custom settings of hostnames, user creation, and password setup through the use of provisioners. If `backends` are specified but do not contain a valid provisioner, `azure-init` logs an error  and halts provisioning.
+
+## Package Considerations
+
+To ensure smooth operation and compatibility across distributions, `azure-init`, should be packaged with a consistent configuration setup.
+
+- Distributions packaging `azure-init` are expected to maintain the base configuration file at `/etc/azure-init/azure-init.toml`, with necessary overrides applied from a `.d` subdirectory.

libazureinit/Cargo.toml

@@ -19,18 +19,20 @@ serde_json = "1.0.96"
 nix = {version = "0.29.0", features = ["fs", "user"]}
 block-utils = "0.11.1"
 tracing = "0.1.40"
-strum = { version = "0.26.3", features = ["derive"] }
 fstab = "0.4.0"
+toml = "0.5"
 regex = "1"
 lazy_static = "1.4"
 tempfile = "3.3.0"
+figment = { version = "0.10", features = ["toml"] }
 
 [dev-dependencies]
 tracing-test = { version = "0.2", features = ["no-env-filter"] }
 tempfile = "3"
 tokio = { version = "1", features = ["full"] }
 tokio-util = "0.7.11"
 whoami = "1"
+anyhow = "1.0.81"
 
 [lib]
 name = "libazureinit"