Skip to content
This repository was archived by the owner on Apr 28, 2025. It is now read-only.

Commit acc0a4f

Browse files
authored
Merge pull request #330 from grafana/series-limit-playbook
Add a simple playbook for ingester series limit alert.
2 parents b320e0f + 2c174a2 commit acc0a4f

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

cortex-mixin/docs/playbooks.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,9 @@ If nothing obvious from the above, check for increased load:
2626

2727
### CortexIngesterReachingSeriesLimit
2828

29-
_TODO: this playbook has not been written yet._
29+
First check the writes resources dashboard and scaling dashboard. The usual target is 1.5M active series per ingester and a max of 2.5M.
30+
31+
Scaling up the ingesters will help, but it won't resolve the alert immediately, as series are active until the next TSDB Head compaction (every 2h or so). You may also want to temporarily increase the per ingester series limit (this is a runtime reloadable config option) until that next Head compaction occurs.
3032

3133
### CortexIngesterReachingTenantsLimit
3234

0 commit comments

Comments
 (0)