Commit 45d12a0
committed
fix: never suppress alerts during a real outage-share spike
_should_suppress_alert only checked prev_overall != curr_overall before
folding upstream-flap deltas into the noise-suppression path. If overall
has already been parked on one bucket (e.g. DEGRADED) for an extended
period and a new outage manifests as more 5xx-flavored probes without
moving that bucket, the alert -- including to the LOCOSP #health mirror --
could get suppressed as "just more flap."
Wire the existing outage-share signal (same fraction --watch already uses
for outage-aware backoff) into the suppression decision: if outage_share
>= outage_threshold, never suppress, regardless of classification. New
params default to a no-op (0.0 >= 1.0 is always false) so existing 3-arg
callers and the pre-existing test suite are unaffected.1 parent ffa8bac commit 45d12a0
2 files changed
Lines changed: 64 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
195 | 195 | | |
196 | 196 | | |
197 | 197 | | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
198 | 249 | | |
199 | 250 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
909 | 909 | | |
910 | 910 | | |
911 | 911 | | |
912 | | - | |
| 912 | + | |
| 913 | + | |
| 914 | + | |
913 | 915 | | |
914 | 916 | | |
915 | 917 | | |
916 | 918 | | |
917 | 919 | | |
918 | 920 | | |
| 921 | + | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
919 | 925 | | |
920 | 926 | | |
| 927 | + | |
| 928 | + | |
| 929 | + | |
| 930 | + | |
921 | 931 | | |
922 | 932 | | |
923 | 933 | | |
| |||
2094 | 2104 | | |
2095 | 2105 | | |
2096 | 2106 | | |
2097 | | - | |
| 2107 | + | |
| 2108 | + | |
2098 | 2109 | | |
2099 | 2110 | | |
2100 | 2111 | | |
| |||
0 commit comments