Skip to content

01-azure-monitor.md

Latest commit

 

History

History
299 lines (202 loc) · 9.75 KB

01-azure-monitor.md

File metadata and controls

299 lines (202 loc) · 9.75 KB
Raw

Azure Monitor Foundations: Metrics, Logs, Alerts, and Insights

Azure Monitor is the core observability platform in Azure. It collects, stores, and correlates telemetry so administrators can detect issues, investigate root cause, and automate response.

Overview

In AZ-104, Azure Monitor is a foundational topic because it connects several operational capabilities together:

  • platform metrics
  • activity and resource logs
  • Log Analytics investigation
  • alerts and Action Groups
  • backup, health, and service visibility

The goal is not just to “collect data,” but to route the right signals to the right place and make them useful during real incidents.

What You Will Learn

  • The major Azure Monitor signal types and how they differ
  • How telemetry is routed with diagnostic settings and DCRs
  • How Log Analytics fits into investigation workflows
  • How alerts and insights depend on correct collection design
  • Common exam traps and operational mistakes to avoid

Acronyms and Terms (Do Not Assume)

  • KPI = Key Performance Indicator
  • SLO = Service Level Objective
  • SLA = Service Level Agreement
  • DCR = Data Collection Rule
  • AMA = Azure Monitor Agent
  • KQL = Kusto Query Language
  • MTTR = Mean Time to Recovery

These terms are used in monitoring architecture reviews and incident response.

Azure Monitor Mental Model

[Azure resource / VM / service]
      |
      +--> [Platform metrics] -------> [Metric alerts]
      |
      +--> [Activity Log] -----------> [Activity log alerts]
      |
      +--> [Resource logs]
                 |
                 v
        [Diagnostic settings / DCRs]
            |        |        |
            v        v        v
   [Log Analytics] [Storage] [Event Hub]
            |
            v
   [KQL queries / Workbooks / Log alerts]

Core Signal Types

1. Platform metrics

  • numeric time-series data
  • optimized for fast threshold alerting
  • common examples: CPU percentage, transaction count, latency, availability

Best when you need near-real-time operational monitoring.

2. Activity Log

  • subscription-level control-plane events
  • records operations such as create, update, delete, policy action, and service health events
  • useful for governance, auditing, and change tracking

3. Resource logs

  • service-specific logs from resource providers
  • usually need diagnostic settings to send them to destinations
  • useful for detailed behavior, access analysis, and troubleshooting

4. Guest OS telemetry

  • VM-level telemetry collected using Azure Monitor Agent (AMA) and Data Collection Rules (DCRs)
  • used for guest performance counters, events, and other machine-level insights

Not all telemetry is collected or routed automatically. Configuration matters.

Diagnostic Settings vs DCRs

This distinction is a common AZ-104 point.

Feature Used for Typical scenario
Diagnostic settings Platform resource logs and export routing Storage account logs, Key Vault logs, activity exports
Data Collection Rules (DCRs) Guest/agent-based data collection via AMA VM performance counters, Windows events, syslog

Simple memory aid

  • PaaS/platform logs → often diagnostic settings
  • VM guest data → typically AMA + DCR

Common Data Destinations

Azure Monitor data is often sent to one or more of these targets:

  • Log Analytics workspace for KQL investigation and correlation
  • Storage account for longer-term archival patterns
  • Event Hub for streaming or integration with external tools

Operational guidance

  1. define a baseline collection pattern for important resource types
  2. centralize investigation data in a manageable workspace strategy
  3. align retention and cost controls with business and compliance needs

Azure Monitor Insights and Operational Use

Azure Monitor also powers higher-level experiences such as:

  • VM Insights
  • Workbooks
  • Application and infrastructure dashboards
  • Alerting and incident response

These experiences depend on correct data collection. If the data path is missing, the insight view may look empty or incomplete.

Alert Integration and Response

Azure Monitor supports several alert types:

  • metric alerts for numeric thresholds
  • log alerts for KQL-based conditions
  • activity log alerts for control-plane change events

Those alerts are then connected to Action Groups for notifications or automation.

Design principle:

An alert without clear ownership or an action path is only noise.

Operating Model: From Signal to Actionable Incident

Professional monitoring programs define a full signal lifecycle:

  1. Collect: metrics, logs, and activity events with clear purpose.
  2. Detect: alert rules identify conditions tied to service risk.
  3. Route: Action Groups send incidents to the right owner.
  4. Respond: runbooks and on-call procedures guide remediation.
  5. Improve: post-incident tuning improves alert quality and MTTR.

This closes the loop from telemetry to operations outcomes.

Monitoring Coverage Strategy

For each critical workload, define minimum coverage in four categories:

  1. Golden signals: latency, traffic, errors, saturation.
  2. Control-plane change visibility: who changed what and when.
  3. Dependency visibility: database, network, identity, and external service health.
  4. Recovery signals: backup status, replication health, and failover readiness.

Without dependency and recovery signals, teams often miss root cause during incidents.

Example Operational Workflow

When investigating an incident:

  1. check metrics for health degradation or saturation
  2. review Activity Log for recent changes, policy actions, or service events
  3. query resource logs in Log Analytics for detailed behavior
  4. correlate timing across these signals before concluding root cause

This avoids jumping to conclusions based on a single data source.

CLI Reference

List metric definitions for a resource

az monitor metrics list-definitions \
  --resource <resource-id> \
  -o table

List diagnostic settings on a resource

az monitor diagnostic-settings list \
  --resource <resource-id> \
  -o jsonc

Show a Log Analytics workspace

az monitor log-analytics workspace show \
  --resource-group <rg> \
  --workspace-name <workspace-name> \
  -o table

Best Practices

  1. Collect only the telemetry that supports an operational purpose.
  2. Route important logs to Log Analytics for centralized investigation.
  3. Standardize diagnostic settings and DCR use across environments.
  4. Align retention with cost and compliance expectations.
  5. Build alerting around signals that operators can actually act on.

Common Pitfalls

  • Assuming all required logs are collected by default.
  • Confusing diagnostic settings with DCR-based guest collection.
  • Treating metrics as a full replacement for detailed logs.
  • Building alert rules with no clear action routing.
  • Ignoring ingestion delays in some log-based workflows.
  • Generating too many low-value alerts and training teams to ignore them.

Key Takeaways

  • Azure Monitor combines metrics, logs, alerts, and investigation tools.
  • Metrics, Activity Log, resource logs, and guest telemetry are different signal types with different purposes.
  • Good observability depends on correct routing, not just turning monitoring “on.”
  • Operational maturity comes from being able to correlate signals, not only view them separately.

Advanced: Monitoring Architecture and Signal Quality

Telemetry Design Principles

  • Collect only actionable signals tied to operational decisions
  • Standardize dimensions/tags for consistent correlation
  • Balance verbosity with cost and retention goals

Control Plane and Workload Correlation

  • Correlate platform metrics, logs, and activity events
  • Build investigative workflows from alert to root cause
  • Ensure critical services emit both health and dependency signals

Governance and Retention

  • Define retention and archive strategy by data class
  • Restrict access to sensitive logs with least privilege
  • Review monitoring coverage as architecture evolves

Extended Troubleshooting Matrix (Azure Monitor)

Symptom Likely cause Validation step Fix
Missing logs from resource Diagnostic settings/DCR not configured correctly Check data collection configuration Correct destination and collection rules
Alert triggered but no useful context Poor signal design or sparse dimensions Review query and alert payload Enrich telemetry and alert context
Monitoring cost spikes Excessive ingestion or verbose logging Analyze ingestion by table/resource Reduce noise and tune collection scope
Investigation takes too long No correlation path between signals Test incident workflow end-to-end Add correlated dashboards and query templates

Production Readiness Checklist (Azure Monitor)

  • Monitoring objectives mapped to service SLOs
  • Data collection rules/diagnostic settings standardized
  • Alert-to-incident workflow tested and documented
  • Retention and cost controls implemented
  • Access controls for log data validated
  • Coverage review cadence established

Further Reading