Skip to content

Commit 04333ba

Browse files
authored
Add ML job state resource (#1337)
* Add ML job state resource implementation * PR feedback * Apply timeout to the entire operation
1 parent ff7bdc9 commit 04333ba

File tree

23 files changed

+1112
-17
lines changed

23 files changed

+1112
-17
lines changed

.env

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ ELASTICSEARCH_CONTAINER_NAME=terraform-elasticstack-es
33
ELASTICSEARCH_PASSWORD=password
44
ELASTICSEARCH_PORT=9200
55
ELASTICSEARCH_URL=http://localhost:${ELASTICSEARCH_PORT}
6-
ELASTICSEARCH_JAVA_OPTS="-Xms128m -Xmx2g"
6+
ELASTICSEARCH_JAVA_OPTS="-Xms128m -Xmx1g"
77
KIBANA_CONTAINER_NAME=terraform-elasticstack-kb
88
KIBANA_SETTINGS_CONTAINER_NAME=terraform-elasticstack-kb-settings
99
FLEET_SETTINGS_CONTAINER_NAME=terraform-elasticstack-fleet-settings
Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
---
2+
page_title: "elasticstack_elasticsearch_ml_job_state Resource - terraform-provider-elasticstack"
3+
subcategory: ""
4+
description: |-
5+
ML Job State Resource
6+
Manages the state of an Elasticsearch Machine Learning (ML) job, allowing you to open or close ML jobs.
7+
This resource uses the following Elasticsearch APIs:
8+
Open ML Job API https://www.elastic.co/guide/en/elasticsearch/reference/current/ml-open-job.htmlClose ML Job API https://www.elastic.co/guide/en/elasticsearch/reference/current/ml-close-job.htmlGet ML Job Stats API https://www.elastic.co/guide/en/elasticsearch/reference/current/ml-get-job-stats.html
9+
Important Notes
10+
This resource manages the state of an existing ML job, not the job configuration itself.The ML job must already exist before using this resource.Opening a job allows it to receive and process data.Closing a job stops data processing and frees up resources.Jobs can be opened and closed multiple times throughout their lifecycle.
11+
---
12+
13+
# elasticstack_elasticsearch_ml_job_state (Resource)
14+
15+
# ML Job State Resource
16+
17+
Manages the state of an Elasticsearch Machine Learning (ML) job, allowing you to open or close ML jobs.
18+
19+
This resource uses the following Elasticsearch APIs:
20+
- [Open ML Job API](https://www.elastic.co/guide/en/elasticsearch/reference/current/ml-open-job.html)
21+
- [Close ML Job API](https://www.elastic.co/guide/en/elasticsearch/reference/current/ml-close-job.html)
22+
- [Get ML Job Stats API](https://www.elastic.co/guide/en/elasticsearch/reference/current/ml-get-job-stats.html)
23+
24+
## Important Notes
25+
26+
- This resource manages the **state** of an existing ML job, not the job configuration itself.
27+
- The ML job must already exist before using this resource.
28+
- Opening a job allows it to receive and process data.
29+
- Closing a job stops data processing and frees up resources.
30+
- Jobs can be opened and closed multiple times throughout their lifecycle.
31+
32+
## Example Usage
33+
34+
```terraform
35+
provider "elasticstack" {
36+
elasticsearch {}
37+
}
38+
39+
# First create an ML anomaly detection job
40+
resource "elasticstack_elasticsearch_ml_anomaly_detector" "example" {
41+
job_id = "example-ml-job"
42+
description = "Example anomaly detection job"
43+
44+
analysis_config = {
45+
bucket_span = "15m"
46+
detectors = [
47+
{
48+
function = "count"
49+
detector_description = "Count detector"
50+
}
51+
]
52+
}
53+
54+
data_description = {
55+
time_field = "@timestamp"
56+
time_format = "epoch_ms"
57+
}
58+
}
59+
60+
# Manage the state of the ML job - open it
61+
resource "elasticstack_elasticsearch_ml_job_state" "example" {
62+
job_id = elasticstack_elasticsearch_ml_anomaly_detector.example.job_id
63+
state = "opened"
64+
65+
# Optional settings
66+
force = false
67+
job_timeout = "30s"
68+
69+
# Timeouts for asynchronous operations
70+
timeouts {
71+
create = "5m"
72+
update = "5m"
73+
}
74+
75+
depends_on = [elasticstack_elasticsearch_ml_anomaly_detector.example]
76+
}
77+
78+
# Example with different configuration options
79+
resource "elasticstack_elasticsearch_ml_job_state" "example_with_options" {
80+
job_id = elasticstack_elasticsearch_ml_anomaly_detector.example.job_id
81+
state = "closed"
82+
83+
# Use force close for quicker shutdown
84+
force = true
85+
86+
# Custom timeout
87+
job_timeout = "2m"
88+
89+
# Custom timeouts for asynchronous operations
90+
timeouts {
91+
create = "10m"
92+
update = "3m"
93+
}
94+
95+
depends_on = [elasticstack_elasticsearch_ml_anomaly_detector.example]
96+
}
97+
```
98+
99+
<!-- schema generated by tfplugindocs -->
100+
## Schema
101+
102+
### Required
103+
104+
- `job_id` (String) Identifier for the anomaly detection job.
105+
- `state` (String) The desired state for the ML job. Valid values are `opened` and `closed`.
106+
107+
### Optional
108+
109+
- `elasticsearch_connection` (Block List, Deprecated) Elasticsearch connection configuration block. (see [below for nested schema](#nestedblock--elasticsearch_connection))
110+
- `force` (Boolean) When closing a job, use to forcefully close it. This method is quicker but can miss important clean up tasks.
111+
- `job_timeout` (String) Timeout for the operation. Examples: `30s`, `5m`, `1h`. Default is `30s`.
112+
- `timeouts` (Attributes) (see [below for nested schema](#nestedatt--timeouts))
113+
114+
### Read-Only
115+
116+
- `id` (String) Internal identifier of the resource
117+
118+
<a id="nestedblock--elasticsearch_connection"></a>
119+
### Nested Schema for `elasticsearch_connection`
120+
121+
Optional:
122+
123+
- `api_key` (String, Sensitive) API Key to use for authentication to Elasticsearch
124+
- `bearer_token` (String, Sensitive) Bearer Token to use for authentication to Elasticsearch
125+
- `ca_data` (String) PEM-encoded custom Certificate Authority certificate
126+
- `ca_file` (String) Path to a custom Certificate Authority certificate
127+
- `cert_data` (String) PEM encoded certificate for client auth
128+
- `cert_file` (String) Path to a file containing the PEM encoded certificate for client auth
129+
- `endpoints` (List of String, Sensitive) A list of endpoints where the terraform provider will point to, this must include the http(s) schema and port number.
130+
- `es_client_authentication` (String, Sensitive) ES Client Authentication field to be used with the JWT token
131+
- `headers` (Map of String, Sensitive) A list of headers to be sent with each request to Elasticsearch.
132+
- `insecure` (Boolean) Disable TLS certificate validation
133+
- `key_data` (String, Sensitive) PEM encoded private key for client auth
134+
- `key_file` (String) Path to a file containing the PEM encoded private key for client auth
135+
- `password` (String, Sensitive) Password to use for API authentication to Elasticsearch.
136+
- `username` (String) Username to use for API authentication to Elasticsearch.
137+
138+
139+
<a id="nestedatt--timeouts"></a>
140+
### Nested Schema for `timeouts`
141+
142+
Optional:
143+
144+
- `create` (String) A string that can be [parsed as a duration](https://pkg.go.dev/time#ParseDuration) consisting of numbers and unit suffixes, such as "30s" or "2h45m". Valid time units are "s" (seconds), "m" (minutes), "h" (hours).
145+
- `update` (String) A string that can be [parsed as a duration](https://pkg.go.dev/time#ParseDuration) consisting of numbers and unit suffixes, such as "30s" or "2h45m". Valid time units are "s" (seconds), "m" (minutes), "h" (hours).
Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
provider "elasticstack" {
2+
elasticsearch {}
3+
}
4+
5+
# First create an ML anomaly detection job
6+
resource "elasticstack_elasticsearch_ml_anomaly_detector" "example" {
7+
job_id = "example-ml-job"
8+
description = "Example anomaly detection job"
9+
10+
analysis_config = {
11+
bucket_span = "15m"
12+
detectors = [
13+
{
14+
function = "count"
15+
detector_description = "Count detector"
16+
}
17+
]
18+
}
19+
20+
data_description = {
21+
time_field = "@timestamp"
22+
time_format = "epoch_ms"
23+
}
24+
}
25+
26+
# Manage the state of the ML job - open it
27+
resource "elasticstack_elasticsearch_ml_job_state" "example" {
28+
job_id = elasticstack_elasticsearch_ml_anomaly_detector.example.job_id
29+
state = "opened"
30+
31+
# Optional settings
32+
force = false
33+
job_timeout = "30s"
34+
35+
# Timeouts for asynchronous operations
36+
timeouts {
37+
create = "5m"
38+
update = "5m"
39+
}
40+
41+
depends_on = [elasticstack_elasticsearch_ml_anomaly_detector.example]
42+
}
43+
44+
# Example with different configuration options
45+
resource "elasticstack_elasticsearch_ml_job_state" "example_with_options" {
46+
job_id = elasticstack_elasticsearch_ml_anomaly_detector.example.job_id
47+
state = "closed"
48+
49+
# Use force close for quicker shutdown
50+
force = true
51+
52+
# Custom timeout
53+
job_timeout = "2m"
54+
55+
# Custom timeouts for asynchronous operations
56+
timeouts {
57+
create = "10m"
58+
update = "3m"
59+
}
60+
61+
depends_on = [elasticstack_elasticsearch_ml_anomaly_detector.example]
62+
}

go.mod

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@ require (
1313
github.com/hashicorp/go-version v1.7.0
1414
github.com/hashicorp/terraform-plugin-framework v1.16.1
1515
github.com/hashicorp/terraform-plugin-framework-jsontypes v0.2.0
16+
github.com/hashicorp/terraform-plugin-framework-timeouts v0.6.0
1617
github.com/hashicorp/terraform-plugin-framework-validators v0.19.0
1718
github.com/hashicorp/terraform-plugin-go v0.29.0
1819
github.com/hashicorp/terraform-plugin-log v0.9.0

go.sum

Lines changed: 3 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -379,8 +379,6 @@ github.com/erikgeiser/coninput v0.0.0-20211004153227-1c3628e74d0f/go.mod h1:vw97
379379
github.com/evanphx/json-patch/v5 v5.9.11 h1:/8HVnzMq13/3x9TPvjG08wUGqBTmZBsCWzjTM0wiaDU=
380380
github.com/evanphx/json-patch/v5 v5.9.11/go.mod h1:3j+LviiESTElxA4p3EMKAB9HXj3/XEtnUf6OZxqIQTM=
381381
github.com/fatih/color v1.13.0/go.mod h1:kLAiJbzzSOZDVNGyDpeOxJ47H46qBXwg5ILebYFFOfk=
382-
github.com/fatih/color v1.17.0 h1:GlRw1BRJxkpqUCBKzKOw098ed57fEsKeNjpTe3cSjK4=
383-
github.com/fatih/color v1.17.0/go.mod h1:YZ7TlrGPkiz6ku9fK3TLD/pl3CpsiFyu8N92HLgmosI=
384382
github.com/fatih/color v1.18.0 h1:S8gINlzdQ840/4pfAwic/ZE0djQEH3wM94VfqLTZcOM=
385383
github.com/fatih/color v1.18.0/go.mod h1:4FelSpRwEGDpQ12mAdzqdOukCy4u8WUtOY6lkT/6HfU=
386384
github.com/fatih/set v0.2.1 h1:nn2CaJyknWE/6txyUDGwysr3G5QC6xWB/PtVjPBbeaA=
@@ -617,6 +615,8 @@ github.com/hashicorp/terraform-plugin-framework v1.16.1 h1:1+zwFm3MEqd/0K3YBB2v9
617615
github.com/hashicorp/terraform-plugin-framework v1.16.1/go.mod h1:0xFOxLy5lRzDTayc4dzK/FakIgBhNf/lC4499R9cV4Y=
618616
github.com/hashicorp/terraform-plugin-framework-jsontypes v0.2.0 h1:SJXL5FfJJm17554Kpt9jFXngdM6fXbnUnZ6iT2IeiYA=
619617
github.com/hashicorp/terraform-plugin-framework-jsontypes v0.2.0/go.mod h1:p0phD0IYhsu9bR4+6OetVvvH59I6LwjXGnTVEr8ox6E=
618+
github.com/hashicorp/terraform-plugin-framework-timeouts v0.6.0 h1:Vv16e7EW4nT9668IV0RhdpEmnLl0im7BZx6J+QMlUkg=
619+
github.com/hashicorp/terraform-plugin-framework-timeouts v0.6.0/go.mod h1:rpHo9hZLn4vEkvNL5xsSdLRdaDZKSinuc0xL+BdOpVA=
620620
github.com/hashicorp/terraform-plugin-framework-validators v0.19.0 h1:Zz3iGgzxe/1XBkooZCewS0nJAaCFPFPHdNJd8FgE4Ow=
621621
github.com/hashicorp/terraform-plugin-framework-validators v0.19.0/go.mod h1:GBKTNGbGVJohU03dZ7U8wHqc2zYnMUawgCN+gC0itLc=
622622
github.com/hashicorp/terraform-plugin-go v0.29.0 h1:1nXKl/nSpaYIUBU1IG/EsDOX0vv+9JxAltQyDMpq5mU=
@@ -1377,9 +1377,8 @@ gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
13771377
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
13781378
gotest.tools/gotestsum v1.13.0 h1:+Lh454O9mu9AMG1APV4o0y7oDYKyik/3kBOiCqiEpRo=
13791379
gotest.tools/gotestsum v1.13.0/go.mod h1:7f0NS5hFb0dWr4NtcsAsF0y1kzjEFfAil0HiBQJE03Q=
1380-
gotest.tools/v3 v3.0.3 h1:4AuOwCGf4lLR9u3YOe2awrHygurzhO/HeQ6laiA6Sx0=
1381-
gotest.tools/v3 v3.0.3/go.mod h1:Z7Lb0S5l+klDB31fvDQX8ss/FlKDxtlFlw3Oa8Ymbl8=
13821380
gotest.tools/v3 v3.5.2 h1:7koQfIKdy+I8UTetycgUqXWSDwpgv193Ka+qRsmBY8Q=
1381+
gotest.tools/v3 v3.5.2/go.mod h1:LtdLGcnqToBH83WByAAi/wiwSFCArdFIUV/xxN4pcjA=
13831382
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
13841383
honnef.co/go/tools v0.0.0-20190523083050-ea95bdfd59fc/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
13851384
honnef.co/go/tools v0.0.1-2019.2.3/go.mod h1:a3bituU0lyd329TUQxRnasdCoJDkEUEAqEt0JzvZhAg=

0 commit comments

Comments
 (0)