You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Dec 14, 2022. It is now read-only.
Since new data is continuously stored in Elasticsearch in various indexes, these must of course be removed after a certain period of time.
681
-
The solution uses the Elasticsearch [ILM](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html) for this purpose, which defines different lifecycle stages per index. The so-called ILM policies are automatically configured by the solution using [configuration files](apibuilder4elastic/elasticsearch_config) and can be reviewed in Kibana.
681
+
Since version 2.0.0, the solution uses the Elasticsearch [ILM](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html) feature for this purpose, which defines different lifecycle stages per index. The so-called ILM policies are automatically configured by the solution using [configuration files](apibuilder4elastic/elasticsearch_config) and can be reviewed in Kibana.
682
682
The indices pass through stages such as Hot, Warm, Cold which can be used to deploy different performance hardware per stage. This means that traffic details from two weeks ago no longer have to be stored on high-performance machines.
683
683
684
684
The configuration is defined here per data type (e.g. Summary, Details, Audit, ...). The following table gives an overview.
685
685
686
-
687
686
| Data-Type | Description | Hot (Size/Days) | Warm | Cold | Delete | Total |
688
687
| :--- |:--- | :--- | :--- | :--- | :--- | :--- |
689
688
|**Traffic-Summary**| Main index for traffic-monitor overview and primary dashboard | 30GB / 7 days | 15 days | 30 days | 10 days | 62 days |
@@ -692,7 +691,7 @@ The configuration is defined here per data type (e.g. Summary, Details, Audit, .
692
691
|**Traffic-Trace**| Trace-Messages belonging to an API-Request shown in Traffic-Monitor | 30GB / 7 days | 7 days | 10 days | 5 days | 29 days |
693
692
|**General-Trace**| General trace messages, like Start- & Stop-Messages | 30GB / 7 days | 7 days | 10 days | 5 days | 29 days |
694
693
|**Gateway-Monitoring**| System status information (CPU, HDD, etc.) from Event-Files | 30GB / 15 days | 15 days | 15 days | 15 days | 60 days |
695
-
|**Domain-Audit**| Domain Audit-Information as configured in Admin-Node-Manager |30GB / 270 days | 270 days| 720 days| 15 days | >3 years|
694
+
|**Domain-Audit**| Domain Audit-Information as configured in Admin-Node-Manager |10GB / 270 days | 270 days| 720 days| 15 days | >3 years|
696
695
697
696
Please note:
698
697
:point_right: It's optional to use different hardware per stage
@@ -714,7 +713,7 @@ The following recommendations are based on our tests and is splitted by the desi
714
713
#### 7 Days rentention period
715
714
716
715
Please note the following:
717
-
The Standard Index Lifecycle Policy defines that an index can grow to 50 GB and rolls into a new after 30 days. For __7 days__, the indexes should be rolled after __2 days__ instead of 30 days.
716
+
The relevant indices for the messaurment below are the Traffic-Summary & Traffic-Details indices that are driving the API-Gateway Traffic-Monitor. The Lifecycle Policy for these indices defines that an index grows to 30 GB or rolls into a new after 7 days (whatevery comes first).
718
717
The recommendation contains only one ElasticSearch node, which provides no data redundancy if this node fails. If you need data redundancy, another ElasticSearch node must be added. After adding another node the data is automatically distributed between them.
@@ -731,7 +730,6 @@ The recommendation contains only one ElasticSearch node, which provides no data
731
730
#### 14 Days rentention period
732
731
733
732
Please note the following:
734
-
The Standard Index Lifecycle Policy defines that an index can grow to 50 GB and rolls into a new after 30 days. For __14 days__, the indexes should be rolled after __4 days__ instead of 30 days.
735
733
The recommendation contains only one ElasticSearch node up to a volume of max. 25 million transactions. This means no data redundancy if this node fails. If you need data redundancy, another ElasticSearch node must be added. After adding another node the data is automatically distributed between them.
@@ -748,7 +746,6 @@ The recommendation contains only one ElasticSearch node up to a volume of max. 2
748
746
749
747
#### 30 Days rentention period
750
748
751
-
The Standard Index Lifecycle Policy is sufficient for 30 days data retention.
752
749
The recommendation contains only one ElasticSearch node up to a volume of max. 10 million transactions. This means no data redundancy if this node fails. If you need data redundancy, another ElasticSearch node must be added. After adding another node the data is automatically distributed between them.
0 commit comments