Updated ILM min_age - as they depend on the Roll-Overdate

#99 - The ILM-Stage min-age depends on the Rollover from Hot to Warm, hence all other stages min_age must add a delay be on that.

Author: Chris Wiechmann

CHANGELOG.md

@@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 ### Fixed
 - TransactionData not shown, if TransactionElement legs are not in a row [#97](https://github.com/Axway-API-Management-Plus/apigateway-openlogging-elk/issues/97)
 - Already existing ServiceName and Operation (Policy-Based) is removed during API-Lookup, when native API cannot be looked up [#98](https://github.com/Axway-API-Management-Plus/apigateway-openlogging-elk/issues/98)
+- Data is deleted too early - Data is not visible anymore after ap. 7 days [#99](https://github.com/Axway-API-Management-Plus/apigateway-openlogging-elk/issues/99)
 
 ### Added
 - Added a new parameter: `DISABLE_CUSTOM_PROPERTIES` to disable custom properties support [#95](https://github.com/Axway-API-Management-Plus/apigateway-openlogging-elk/issues/95)

README.md

@@ -779,12 +779,12 @@ The configuration is defined here per data type (e.g. Summary, Details, Audit, .
 
 | Data-Type              | Description                                                            | Hot (Size/Days) | Warm    | Cold    | Delete  | Total   |
 | :---                   |:---                                                                    | :---            | :---    | :---    | :---    | :---    |
-| **Traffic-Summary**    | Main index for traffic-monitor overview and primary dashboard          | 30GB / 7 days   | 4 days  | 3 days  | 0 days  | 14 days |
-| **Traffic-Details**    | Details in Traffic-Monitor for Policy, Headers and Payload reference   | 30GB / 7 days   | 4 days  | 3 days  | 0 days  | 14 days |
-| **Traffic-Trace**      | Trace-Messages belonging to an API-Request shown in Traffic-Monitor    | 30GB / 7 days   | 4 days  | 3 days  |  0 days | 14 days |
-| **General-Trace**      | General trace messages, like Start- & Stop-Messages                    | 30GB / 7 days   | 4 days  | 3 days  | 0 days  | 14 days |
+| **Traffic-Summary**    | Main index for traffic-monitor overview and primary dashboard          | 30GB / 7 days   | 5 days  | 3 days  | 0 days  | 15 days |
+| **Traffic-Details**    | Details in Traffic-Monitor for Policy, Headers and Payload reference   | 30GB / 7 days   | 5 days  | 3 days  | 0 days  | 15 days |
+| **Traffic-Trace**      | Trace-Messages belonging to an API-Request shown in Traffic-Monitor    | 30GB / 7 days   | 5 days  | 3 days  | 0 days  | 15 days |
+| **General-Trace**      | General trace messages, like Start- & Stop-Messages                    | 30GB / 7 days   | 5 days  | 3 days  | 0 days  | 15 days |
 | **Gateway-Monitoring** | System status information (CPU, HDD, etc.) from Event-Files            | 30GB / 60 days  | 30 days | 15 days | 0 days  | 105 days|
-| **Domain-Audit**       | Domain Audit-Information as configured in Admin-Node-Manager           | 10GB / 270 days | 270 days| 720 days| 15 days | >3 years|
+| **Domain-Audit**       | Domain Audit-Information as configured in Admin-Node-Manager           | 10GB / 270 days | 270 days| 720 days| 30 days | >3 years|
 
 Please note:  
 :point_right: It's optional to use different hardware per stage  

apibuilder4elastic/elasticsearch_config/apigw-domainaudit/ilm_policy.json

@@ -39,7 +39,7 @@
         }
       },
       "delete": {
-        "min_age": "15d",
+        "min_age": "750d",
         "actions": {
           "delete": {
             "delete_searchable_snapshot": true

apibuilder4elastic/elasticsearch_config/apigw-monitoring/ilm_policy.json

@@ -31,15 +31,15 @@
         }
       },
       "cold": {
-        "min_age": "30d",
+        "min_age": "90d",
         "actions": {
           "set_priority": {
             "priority": 0
           }
         }
       },
       "delete": {
-        "min_age": "15d",
+        "min_age": "105d",
         "actions": {
           "delete": {
             "delete_searchable_snapshot": true

apibuilder4elastic/elasticsearch_config/trace-messages/ilm_policy.json

@@ -31,15 +31,15 @@
         }
       },
       "cold": {
-        "min_age": "4d",
+        "min_age": "12d",
         "actions": {
           "set_priority": {
             "priority": 20
           }
         }
       },
       "delete": {
-        "min_age": "3d",
+        "min_age": "15d",
         "actions": {
           "delete": {
             "delete_searchable_snapshot": true

apibuilder4elastic/elasticsearch_config/traffic-details/ilm_policy.json

@@ -31,15 +31,15 @@
         }
       },
       "cold": {
-        "min_age": "4d",
+        "min_age": "12d",
         "actions": {
           "set_priority": {
             "priority": 20
           }
         }
       },
       "delete": {
-        "min_age": "3d",
+        "min_age": "15d",
         "actions": {
           "delete": {
             "delete_searchable_snapshot": true

apibuilder4elastic/elasticsearch_config/traffic-summary/ilm_policy.json

@@ -31,15 +31,15 @@
         }
       },
       "cold": {
-        "min_age": "4d",
+        "min_age": "12d",
         "actions": {
           "set_priority": {
             "priority": 20
           }
         }
       },
       "delete": {
-        "min_age": "3d",
+        "min_age": "15d",
         "actions": {
           "delete": {
             "delete_searchable_snapshot": true