Add daily operations report with hardened log-noise filtering
Brings the previously untracked daily-status-report.sh and send-operations-report-mail.sh into the repo, plus a refactor of the log-noise pipeline: - New helper services/posture-check/lib/normalize-noise-patterns.sh strips comments, empty lines and trailing whitespace from log-noise.patterns before grep -f sees it. A stray empty line in the pattern file would otherwise have made grep -Eaif match every hit and silently wipe the log highlights. - log-noise.patterns is now documented per-pattern (Why / Re-check). The Vaultwarden pattern is split: token/session noise stays as noise; DNS/Connect/Resolve/reqwest/hyper errors are removed from the noise set so real network signals stay visible. - collect_log_highlights now reports a per-container and per-pattern noise breakdown (Top N) and an escalation flag when any pattern exceeds NOISE_ESCALATION_THRESHOLD (default 500). The flag is fed into derive_report_status and the management summary. - New shell tests under services/posture-check/tests/ verify the normalize helper handles comments, empty lines, whitespace-only lines, and that unknown error lines remain in the attention set. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,27 @@
|
||||
#!/usr/bin/env bash
|
||||
# normalize-noise-patterns.sh
|
||||
#
|
||||
# Read a log-noise.patterns file and emit a normalized stream of patterns
|
||||
# that is safe to feed into `grep -Eaif`.
|
||||
#
|
||||
# Behaviour:
|
||||
# - Lines starting with `#` (after optional leading whitespace) are dropped.
|
||||
# - Empty / whitespace-only lines are dropped.
|
||||
# - Leading and trailing whitespace is trimmed from each pattern.
|
||||
# - Patterns that become empty after trimming are dropped.
|
||||
#
|
||||
# Why this exists:
|
||||
# A single empty / whitespace-only line in the input file would make
|
||||
# `grep -Eaif` match every input line, silently wiping the entire log
|
||||
# highlights signal. Always pipe patterns through this normalizer first.
|
||||
#
|
||||
# Usage:
|
||||
# normalize-noise-patterns.sh <file>
|
||||
# cat patterns | normalize-noise-patterns.sh
|
||||
set -euo pipefail
|
||||
|
||||
src="${1:-/dev/stdin}"
|
||||
|
||||
grep -Ev '^[[:space:]]*(#|$)' "$src" \
|
||||
| sed -E 's/^[[:space:]]+//; s/[[:space:]]+$//' \
|
||||
| grep -v '^$' || true
|
||||
Reference in New Issue
Block a user