Skip to content

Conversation

@vmercierfr
Copy link
Collaborator

Objective

Add alert for RDS instance with full disk space

Why

RDS instance with full disk space requires immediate action to recover

How

  • Move instructions to increase RDS storage to shortcodes to be reused
  • Add RDSFullDiskSpace runbook
  • Add RDSFullDiskSpace alert

Release plan

Release new minor version

@vmercierfr vmercierfr requested review from dcupif and qfritz September 1, 2025 14:41
@vmercierfr vmercierfr added the enhancement New feature or request label Sep 1, 2025
@qfritz
Copy link
Contributor

qfritz commented Sep 1, 2025

Why is this one needed in addition to RDSDiskSpaceLimit and RDSDiskSpacePrediction?

@vmercierfr
Copy link
Collaborator Author

These alerts are quite similar, but relevant because :

  • RDSDiskSpaceLimit is a warning severity, and may be disabled
  • RDSDiskSpacePrediction is critical but on write intensive workload, prediction could be received after storage full or severity could be overrided to warning by convention (service is not broken)
  • RDSFullDiskSpace is critical provides a straight forward remediation instructions as there is not other option than scale. Just by looking at titleFullDiskSpace, on call engineers will have better understanding of situation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants