Alerts

Taranac watches itself and the boxes it runs on, and turns “something is wrong” into a small, stable list of alerts you can work through. An alert is a condition — disk filling up, RADIUS unreachable, an identity being brute-forced — not a stream of events. It opens (fires) when the condition appears and resolves when it clears, so a problem that lasts ten minutes is one row, not six hundred log lines.

This page covers where alerts come from, how severity is assigned, how operators acknowledge and clear them in the admin UI, and how to tune email notifications so you hear about the things that matter without being buried.

The Alerts page: severity-sorted firing conditions with acknowledge and resolve actions.

alerts · condition lifecycle

disk:/var ×7 critical

1. firing · disk > 80% warning
2. recurs · same fingerprint count++
3. acknowledged · still firing repeats off
4. escalates · disk > 90% critical · ack cleared
5. resolved · auto, on recovery all clear

Alert — a single condition with a lifecycle. It carries a severity, a title and message, the originating source, an occurrence count, and timestamps for first-seen, last-seen and (once cleared) resolved.

Status — firing / resolved. This is the machine-driven axis. The system opens an alert when it detects the condition and resolves it automatically when the condition goes away. You normally do not set this by hand.

Acknowledged. This is the operator axis, completely separate from status. Acknowledging an alert means “I have seen this and I am on it.” It does not close the alert — the condition is still firing — but it silences repeat notifications so you stop getting reminded about something you are already handling.

Source. The subsystem that raised the alert: monitoring (host resources and service health), tacacs / radius / nac (AAA anomalies bridged from the protocol daemons), the config validators (tacacs_config, radius_config, nac_config), pki, email (the alert engine’s own delivery self-check), and — when running clustered — cluster_license (edition / node-count mismatch against the licence) and cluster_sync (HA health: replica lag, a node down, orphaned replication slots, the config-apply readiness gate, and NAC session-accounting writes dropped during a failover). The cluster_* sources only appear on an HA deployment.

Fingerprint. A stable internal identity for the condition (for example disk:/var or nac:brute_force:peap:jdoe). It is what keeps a recurring problem collapsed into one alert: while a condition keeps recurring, Taranac bumps the existing alert’s occurrence count rather than opening duplicates. Once an alert resolves, a fresh occurrence later opens a new one — so you get clean history, not endless churn.

Node. On an HA cluster every alert also records the node that observed and opened it, stamped once on open and kept stable across recurrences. For per-node conditions — daemon health, host resources, config validation, MFA reachability, the readiness gate — the fingerprint is additionally scoped to that node, so a healthy peer can never resolve another node’s firing alert (each node keeps its own row). Cluster-wide conditions (the cluster_license and leader-only cluster_sync reconciles) stay a single alert per cluster, stable across failover. On a standalone install there is one node and this dimension is inert.

Severity is assigned by the producer, not by the operator. The four levels follow a deliberate rubric:

Severity	Meaning	Examples
info	Nothing broken, informational only (rare).	—
warning	Not broken yet — a deadline or trend approaching.	Disk at 80%; brute-force detected; unknown device seen.
error	Something has failed, but core AAA still works.	Config validation failed; email delivery failing.
critical	Active authentication outage or security condition.	RADIUS / TACACS+ unreachable.

Two boundary rules make this easy to reason about:

• warning vs error — “about to happen” versus “already happened.”
• error vs critical — is authentication itself impacted, or just a peripheral function?

If a condition gets worse while already firing (for example disk crosses from warning into critical), the alert escalates: its severity is raised, any acknowledgement is cleared, and it becomes eligible to notify again — so an escalation always gets your attention even if you had silenced the milder version.

The evaluator reads live host metrics every cycle and compares them against configurable percentage thresholds:

Condition	Fingerprint	Default warning	Default critical
Disk usage (per mount)	disk:<mount>	80%	90%
CPU usage	cpu:load	90%	disabled
Memory usage	memory:usage	85%	95%
Swap usage	swap:usage	disabled	disabled

A threshold breach must persist for alerts.sustained_minutes (default 2) before the alert opens — this debounce stops a momentary CPU spike from paging you. Recovery clears the alert immediately, no debounce. Setting a level’s threshold to 0 disables it.

Service reachability is checked with no debounce — an AAA outage should surface at once:

Condition	Fingerprint	Severity
TACACS+ unreachable	service:tacacs	critical
RADIUS unreachable	service:radius	critical

When you change AAA configuration, the config jobs re-render and validate it. A failed validation opens an error alert (tacacs_config:validation, radius_config:validation, nac_config:validation); a subsequent successful reload resolves it. This is your safety net against pushing a config the daemons cannot load.

Email is the alert engine’s own notification channel, which makes a broken SMTP a blind spot — alert mail would silently never arrive. Taranac watches its own outgoing-mail audit trail and opens an error alert (email:delivery) after alerts.email_failure_threshold consecutive send failures since the last success (default 3). It resolves on the next successful send. Set the threshold to 0 to turn the self-check off.

The 802.1X / NAC path raises two kinds of alert, bridged from the NAC daemon into the stateful alert list:

• Brute-force detection. Repeated failed credential authentications for one identity (PEAP / EAP-TTLS) within a window raise a warning alert. Detection is keyed by the inner username being guessed, counted once per authentication attempt (an EAP attempt is many packets — Taranac counts the logical outcome, not each round-trip). Defaults: enabled, threshold 10 failures, window 15 minutes (nac.bruteforce_*).
• Unknown device. A NAC request from a switch (NAS) that is not in the device inventory raises a throttled warning alert per NAS, pointing you at adding the device or reviewing broad-admit configuration.

TACACS+ and RADIUS raise the same brute_force and unknown_device alerts from their own daemons (sources tacacs / radius); each protocol’s condition is tracked independently.

On an HA deployment two further sources surface cluster health, so operational problems that only exist when clustered do not stay silent:

• cluster_license — the running node count or edition does not match the installed licence (fingerprint cluster:max_nodes). One firing alert per cluster; it auto-resolves when the roster fits the licence again.
• cluster_sync — a family of HA-health conditions sharing one source: a replica whose replay lag exceeds the operator threshold, an active roster node Patroni reports as not running, a phantom streamer whose name matches no roster node, an orphaned replication slot the reaper declined to auto-drop, the per-node config-apply readiness gate freezing a lagging node’s daemon reloads, and NAC session-accounting writes dropped by a node during a prolonged failover (fingerprint cluster:nac_session_drop:<node>). The leader-only reconciles are cluster-wide single alerts; the readiness gate and dropped-write conditions are per-node.

These sources are absent on a standalone install. See High availability for the cluster architecture behind them.

Notifications are sent on transitions and on repeats — never on every evaluation cycle. Taranac mails a configured recipient list when:

• an alert fires (newly opened, or just escalated) and its severity is at or above alerts.notify_min_severity;
• an alert is still firing, unacknowledged, and its per-severity repeat interval has elapsed (a reminder);
• an alert resolves after having been notified (the “all clear”), if alerts.notify_on_resolve is on.

Acknowledging an alert stops the repeat reminders for it — that is the primary noise-control lever. Repeat cadence is set per severity, so critical conditions nag often and warnings rarely.

Delivery is best-effort: a mail failure is logged, never breaks evaluation, and (via the email self-check above) surfaces as its own alert.

The Alerts page lists conditions most-severe-first, then most recent. From there:

• Filter by status (firing / resolved), severity, source, acknowledgement, or a text search across title, message, source and fingerprint. On an HA cluster the list also shows a Node column and a node filter, so you can see and narrow to which node raised each alert.
• Acknowledge a single alert (or Acknowledge all firing ones) to silence its repeat emails while you work it. The alert stays firing; the system records who acknowledged and when.
• Resolve an alert manually when you have decided it is handled. If the underlying condition is still bad, the next detection simply opens a fresh alert — so manual resolve is safe, not a way to hide a live problem.

Access is governed by RBAC under the alerts section: view to see the list and stats, acknowledge to acknowledge, and manage to manually resolve. The built-in operator role has all three; auditor is view-only. Alerts cannot be created through the UI or API — only the background evaluator produces them.

A disk is filling up. /var crosses 80%, persists past the debounce, and a warning opens (disk:/var). You get a firing email. It keeps climbing past 90%, the alert escalates to critical, your earlier acknowledgement is cleared, and you are notified again. You free space; on the next cycle the metric drops and the alert auto-resolves with an “all clear” mail.

RADIUS goes down. The health check fails and a critical alert fires immediately (no debounce), since authentication is impacted. You acknowledge to stop the repeats while you investigate, fix the daemon, and the alert resolves on the next reachable check.

A password-spray against a user. Failed PEAP logins for jdoe cross the NAC threshold inside the window; a warning fires (nac:brute_force:peap:jdoe). The device is not banned — the alert tells you to investigate and, if warranted, block the endpoint manually. When the spray stops, the alert ages out and auto-resolves.

A switch you forgot to onboard. A NAS not in inventory hits the NAC path; a throttled unknown_device warning fires for that NAS IP. You add it under Network devices, and the alert clears once the requests stop.

Bad config push. A TACACS+ profile change fails validation; an error opens (tacacs_config:validation). You correct the profile, the reload validates cleanly, and the alert resolves.

Goal	Use
Stop being reminded about something you are already handling	Acknowledge the alert (keeps it firing, silences repeats)
Mark a condition handled and clear it from the active list	Resolve it manually (a live condition just reopens)
Hear about more / fewer alerts by email	alerts.notify_min_severity (e.g. error to mute warnings)
Reduce reminder frequency	alerts.repeat_*_minutes (set 0 to disable repeats for a severity)
Stop nuisance resource alerts on a busy box	Raise the relevant *_warning_percent / *_critical_percent, or set 0 to disable a level
Tame momentary spikes	Increase alerts.sustained_minutes (the debounce)
Turn the whole engine off	alerts.enabled = false

All alert settings live under alerts.* and are edited on the system settings page. Resource thresholds are percentages where 0 disables that level; repeat intervals are in minutes where 0 disables repeats for that severity.

Key	Default	Purpose
alerts.enabled	true	Master switch for evaluation, lifecycle and notifications
alerts.eval_interval_seconds	60	How often conditions are evaluated
alerts.sustained_minutes	2	Debounce — how long a resource breach must persist before opening
alerts.resolved_retention_days	30	Days to keep resolved alerts (0 = keep forever)
alerts.event_ttl_minutes	60	Idle period after which event-type alerts (brute-force, unknown device) auto-resolve
alerts.disk_warning_percent / _critical_percent	80 / 90	Disk usage thresholds
alerts.cpu_warning_percent / _critical_percent	90 / 0	CPU usage thresholds
alerts.mem_warning_percent / _critical_percent	85 / 95	Memory usage thresholds
alerts.swap_warning_percent / _critical_percent	0 / 0	Swap usage thresholds (off by default)
alerts.email_failure_threshold	3	Email-delivery self-check trip point (0 disables)
alerts.notify_email_enabled	true	Email notification channel on/off
alerts.notify_recipients	(empty)	Comma-separated recipient addresses
alerts.notify_min_severity	warning	Minimum severity that triggers an email
alerts.notify_on_resolve	true	Send the “all clear” on resolve
alerts.repeat_critical_minutes	60	Reminder cadence for unacknowledged firing critical
alerts.repeat_error_minutes	360	Reminder cadence for unacknowledged firing error
alerts.repeat_warning_minutes	1440	Reminder cadence for unacknowledged firing warning

NAC brute-force detection is tuned separately under nac.*:

Key	Default	Purpose
nac.bruteforce_enabled	true	Enable NAC (802.1X) brute-force detection — alert-only, never bans
nac.bruteforce_threshold	10	Failed credential auths per identity within the window before alerting
nac.bruteforce_window_minutes	15	Detection window per identity

• Audit & syslog — the event trail behind delivery self-checks and operator actions
• Authentication logs — the per-request log that brute-force counters draw from
• Network devices — onboard a NAS to clear unknown-device alerts
• Endpoints & profiling — the block lifecycle that NAC enforcement actually uses
• Settings reference — SMTP configuration for the email channel
• Taranac on GitHub