You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Dec 14, 2022. It is now read-only.
This step is optional, but required to monitor your Filebeat instances as part of the stack monitoring. To obtain the Cluster UUID run the following in your browser:
534
+
`https://elasticsearch1:9200/` (if you have already activated authentication you can use the elastic user here)
Before a document is send to Elasticsearch, additional information for the processed API is requested by Logstash from the API-Manager through an API lookup. This lookup is handled by the API-Builder and performed against the configured API-Manager.
@@ -710,6 +726,42 @@ The monitoring users are used to send metric information to Elasticsearch to ena
_If you are using an existing Elasticsearch cluster, you have to provide the required CA to the solution to allow certificate validation._
732
+
733
+
The project is shipped with sample long running certificates/keys that should help you to get started with the solution. For a production environment these certificates and keys should be replaced with custom certificates, as the sample certificates & key are public available on GitHub.
734
+
735
+
After you have created the corresponding certificates and keys based on your CA, you must save them in the folder: `certificates`.
736
+
Afterwards these certificates must be configured in the `.env` file.
The API-Builder project for providing access to Elasticsearch data has no access restrictions right now. To ensure only API-Gateway Manager users (topology administrators with proper RBAC role) or other users with appropriate access rights can query the log data, one can expose this API via API-Manager and add security here.
754
+
755
+
To import the API Builder application REST-API into your API-Manager, you can access the Swagger/OpenAPI definition here (replace docker-host and port appropriately for the container that is hosting the API-Builder project):
It is important that the solution is monitored appropriately and by default Internal-Stack Monitoring is used for this purpose, which monitors the Elasticsearch cluster, Kibana, Logstash and Filebeat.
763
+
You can alternatively use Metricbeat and you can find more guidance in this section on platform monitoring.
764
+
713
765
### Enable Metricbeat
714
766
715
767
In the default configuration, the solution uses the so-called self-monitoring. This means that components such as Logstash, Kibana, Filebeat, etc. independently send monitoring information (metrics) to Elasticsearch. However, this approach is not recommended by Elastic and is deprecated.
@@ -764,57 +816,20 @@ You can enable Application Performance Monitoring (APM) to monitor APIBuilder4El
This step is optional, but required to monitor your Filebeat instances as part of the stack monitoring. To obtain the Cluster UUID run the following in your browser:
773
-
`https://elasticsearch1:9200/` (if you have already activated authentication you can use the elastic user here)
_If you are using an existing Elasticsearch cluster, you have to provide the required CA to the solution to allow certificate validation._
789
-
790
-
The project is shipped with sample long running certificates/keys that should help you to get started with the solution. For a production environment these certificates and keys should be replaced with custom certificates, as the sample certificates & key are public available on GitHub.
821
+
### Disk-Usage monitoring
791
822
792
-
After you have created the corresponding certificates and keys based on your CA, you must save them in the folder: `certificates`.
793
-
Afterwards these certificates must be configured in the `.env` file.
You can find more information about the individual certificates in the `.env` file.
823
+
It is important that you monitor the disk usage of the Elasticsearch cluster and get alarmed accordingly.
824
+
Elasticsearch also independently monitors disk usage against preconfigured thresholds and closes write operations when the high disk watermark index is exceeded. This means that no more new data can be written.
825
+
To avoid this condition, your alerts should already warn below the Elasticsearch thresholds. The thresholds for Elasticsearch:
The API-Builder project for providing access to Elasticsearch data has no access restrictions right now. To ensure only API-Gateway Manager users (topology administrators with proper RBAC role) or other users with appropriate access rights can query the log data, one can expose this API via API-Manager and add security here.
811
-
812
-
To import the API Builder application REST-API into your API-Manager, you can access the Swagger/OpenAPI definition here (replace docker-host and port appropriately for the container that is hosting the API-Builder project):
So your alerts should report a critical alert before 90%. For more information, please read here: [Disk-based shard allocation settings](https://www.elastic.co/guide/en/elasticsearch/reference/7.16/modules-cluster.html#disk-based-shard-allocation)
816
831
817
-
###Lifecycle Management
832
+
## Lifecycle Management
818
833
819
834
Since new data is continuously stored in Elasticsearch in various indexes, these must of course be removed after a certain period of time.
820
835
Since version 2.0.0, the solution uses the Elasticsearch [ILM](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html) feature for this purpose, which defines different lifecycle stages per index. The so-called ILM policies are automatically configured by the solution using [configuration files](apibuilder4elastic/elasticsearch_config) and can be reviewed in Kibana.
0 commit comments