Monitoring Hybrid VPN Connectivity Between Azure and AWS

Hybrid cloud architectures connecting Azure and AWS through site-to-site VPN tunnels are increasingly common. What is far less common is proper monitoring of those tunnels. Most teams configure the VPN, verify it works once, and then discover at 2 AM on a Saturday that the tunnel has been down for hours — taking critical cross-cloud workloads with it.

I have seen this pattern repeatedly across enterprise engagements. The VPN itself is well-architected (redundant tunnels, proper BGP configuration), but the monitoring is an afterthought. This guide covers how to build comprehensive VPN monitoring that catches problems before your users do.

The Monitoring Gap in Hybrid Networks

Native cloud monitoring tools — CloudWatch for AWS and Azure Monitor — provide basic tunnel status metrics. But they have blind spots:

• CloudWatch reports TunnelState (UP/DOWN) and TunnelDataIn/TunnelDataOut, but gives you no visibility into latency, packet loss, or application-layer connectivity.
• Azure Monitor provides TunnelAverageBandwidth and TunnelEgressBytes, but the granularity is often insufficient for SLA reporting.
• Neither tool monitors the actual application connectivity through the tunnel. A tunnel can be “UP” at the network layer while an application behind it is unreachable due to routing issues or security group misconfigurations.

You need monitoring at three layers: tunnel state, network quality (latency and packet loss), and application reachability.

Architecture: The Monitoring Stack

The architecture I recommend combines native tools with open-source components:

[AWS VPN Gateway] <---> [Azure VPN Gateway]
       |                        |
  CloudWatch                Azure Monitor
       |                        |
       v                        v
[Prometheus + Blackbox Exporter]  <--- scrapes both sides
       |
       v
[Grafana Dashboards + AlertManager]
       |
       v
[PagerDuty / Slack / Email]

Additionally, a custom Python monitor runs in both clouds to perform active probing and publish custom metrics.

Native Tools: What They Give You

AWS: CloudWatch VPN Metrics

AWS publishes these metrics automatically for VPN connections:

Set up CloudWatch Alarms for TunnelState < 1 with a 1-minute evaluation period. This is your baseline — it catches complete tunnel failures.

AWS: VPC Reachability Analyzer

The VPC Reachability Analyzer is underused but powerful for troubleshooting. It performs path analysis between a source and destination, evaluating every route table, security group, NACL, and gateway in the path. Use it to validate that traffic CAN flow before blaming the VPN itself.

aws ec2 create-network-insights-path \
  --source eni-0abc123 \
  --destination 10.1.0.50 \
  --protocol TCP \
  --destination-port 443

Azure: Network Watcher

Azure Network Watcher provides:

• Connection Monitor: continuous probing between VMs across the tunnel.
• IP Flow Verify: checks if traffic is allowed or denied by NSG rules.
• Next Hop: validates routing is correct for cross-cloud traffic.

Configure a Connection Monitor test group that probes from an Azure VM to target IPs in AWS every 30 seconds. This gives you round-trip latency and success rate from the Azure side.

Prometheus + Blackbox Exporter

The Blackbox Exporter is the core of active monitoring. Deploy it on EC2 instances in AWS and on VMs in Azure. It performs:

• ICMP probes: basic ping to measure latency and packet loss.
• TCP probes: verifies port reachability (databases, APIs).
• HTTP probes: validates application-layer responses through the tunnel.

Blackbox Exporter Configuration

modules:
  icmp_cross_cloud:
    prober: icmp
    timeout: 5s
    icmp:
      preferred_ip_protocol: ip4
  tcp_database:
    prober: tcp
    timeout: 5s
    tcp:
      preferred_ip_protocol: ip4
  http_api:
    prober: http
    timeout: 10s
    http:
      method: GET
      preferred_ip_protocol: ip4
      valid_http_versions: ["HTTP/1.1", "HTTP/2.0"]
      valid_status_codes: [200, 201, 204]

Prometheus Scrape Configuration

scrape_configs:
  - job_name: 'vpn-cross-cloud-icmp'
    metrics_path: /probe
    params:
      module: [icmp_cross_cloud]
    static_configs:
      - targets:
          - 10.1.0.50    # Azure SQL Server
          - 10.1.0.100   # Azure App Server
          - 10.2.0.50    # AWS RDS endpoint
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - source_labels: [__param_target]
        target_label: instance
      - target_label: __address__
        replacement: blackbox-exporter:9115

  - job_name: 'vpn-cross-cloud-tcp'
    metrics_path: /probe
    params:
      module: [tcp_database]
    static_configs:
      - targets:
          - 10.1.0.50:1433   # Azure SQL on port 1433
          - 10.2.0.50:5432   # AWS RDS PostgreSQL

Custom Python VPN Monitor

For scenarios requiring more sophisticated testing — such as validating SQL connectivity or publishing custom CloudWatch metrics — I use a custom Python monitor:

import socket
import time
import statistics
import boto3
import pyodbc

class VPNMonitor:
    def __init__(self, targets, cloudwatch_namespace='Custom/VPN'):
        self.targets = targets
        self.cw_client = boto3.client('cloudwatch')
        self.namespace = cloudwatch_namespace

    def test_tcp_port(self, host, port, timeout=5):
        """Test TCP connectivity and measure latency."""
        start = time.time()
        try:
            sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            sock.settimeout(timeout)
            result = sock.connect_ex((host, port))
            latency = (time.time() - start) * 1000  # ms
            sock.close()
            return {
                'reachable': result == 0,
                'latency_ms': latency if result == 0 else None
            }
        except socket.error:
            return {'reachable': False, 'latency_ms': None}

    def test_sql_connection(self, connection_string, query="SELECT 1"):
        """Test actual SQL connectivity through the tunnel."""
        start = time.time()
        try:
            conn = pyodbc.connect(connection_string, timeout=10)
            cursor = conn.cursor()
            cursor.execute(query)
            cursor.fetchone()
            latency = (time.time() - start) * 1000
            conn.close()
            return {'connected': True, 'latency_ms': latency}
        except Exception as e:
            return {'connected': False, 'latency_ms': None, 'error': str(e)}

    def publish_metrics(self, target_name, metrics):
        """Publish custom metrics to CloudWatch."""
        metric_data = []
        if metrics.get('latency_ms') is not None:
            metric_data.append({
                'MetricName': 'CrossCloudLatency',
                'Value': metrics['latency_ms'],
                'Unit': 'Milliseconds',
                'Dimensions': [
                    {'Name': 'Target', 'Value': target_name},
                    {'Name': 'Direction', 'Value': 'AWS-to-Azure'}
                ]
            })
        metric_data.append({
            'MetricName': 'CrossCloudReachability',
            'Value': 1.0 if metrics.get('reachable') or metrics.get('connected') else 0.0,
            'Unit': 'None',
            'Dimensions': [
                {'Name': 'Target', 'Value': target_name}
            ]
        })
        self.cw_client.put_metric_data(
            Namespace=self.namespace,
            MetricData=metric_data
        )

    def run_checks(self):
        """Execute all monitoring checks."""
        for target in self.targets:
            if target['type'] == 'tcp':
                result = self.test_tcp_port(target['host'], target['port'])
            elif target['type'] == 'sql':
                result = self.test_sql_connection(target['connection_string'])
            self.publish_metrics(target['name'], result)

Deploy this as a Lambda function (with VPC access) running every minute, or as a containerized service on ECS/EKS.

Key Metrics and Alert Thresholds

Based on experience with production hybrid environments, here are the metrics that matter and their alert thresholds:

Connectivity Metrics

Latency Metrics

Throughput and Packet Loss

Grafana Dashboard Design

The monitoring dashboard should answer three questions at a glance:

1. Are the tunnels up? (Green/Red status panel per tunnel)
2. Is performance normal? (Latency time series with P50/P95/P99 overlay)
3. Are applications reachable? (Heatmap of target reachability over time)

Key panels:

• Tunnel Status: Stat panel showing current state of each tunnel (Active/Active for redundant tunnels).
• Cross-Cloud Latency: Time series graph with percentile bands.
• Reachability Matrix: Table showing every target with its current status, last check time, and 24h availability percentage.
• Packet Loss: Time series with threshold markers.
• Bandwidth Utilization: Gauge panel showing current vs. maximum VPN throughput (AWS VPN supports 1.25 Gbps per tunnel).

Terraform: Infrastructure as Code

Here is a sample Terraform configuration for the CloudWatch alarm and SNS notification:

resource "aws_cloudwatch_metric_alarm" "vpn_tunnel_down" {
  alarm_name          = "vpn-tunnel-azure-down"
  comparison_operator = "LessThanThreshold"
  evaluation_periods  = 2
  metric_name         = "TunnelState"
  namespace           = "AWS/VPN"
  period              = 60
  statistic           = "Maximum"
  threshold           = 1
  alarm_description   = "VPN tunnel to Azure is DOWN"
  
  dimensions = {
    VpnId    = aws_vpn_connection.azure.id
    TunnelIpAddress = aws_vpn_connection.azure.tunnel1_address
  }

  alarm_actions = [aws_sns_topic.vpn_alerts.arn]
  ok_actions    = [aws_sns_topic.vpn_alerts.arn]
}

resource "aws_cloudwatch_metric_alarm" "vpn_latency_high" {
  alarm_name          = "vpn-cross-cloud-latency-high"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = 3
  metric_name         = "CrossCloudLatency"
  namespace           = "Custom/VPN"
  period              = 60
  statistic           = "p95"
  threshold           = 100
  alarm_description   = "Cross-cloud P95 latency exceeds 100ms"

  dimensions = {
    Direction = "AWS-to-Azure"
  }

  alarm_actions = [aws_sns_topic.vpn_alerts.arn]
}

Implementation Roadmap

I recommend a phased approach:

Phase 1: Foundation (Week 1-2)

• Deploy CloudWatch Alarms for tunnel state on AWS.
• Configure Azure Connection Monitor for basic probing.
• Set up SNS/PagerDuty integration for immediate alerting on tunnel down events.
• Deliverable: You know within 60 seconds when a tunnel goes down.

Phase 2: Active Monitoring (Week 3-4)

• Deploy Blackbox Exporter in both clouds.
• Configure Prometheus scraping for ICMP and TCP targets.
• Build Grafana dashboards with latency and reachability panels.
• Deploy custom Python monitor for SQL/application-layer testing.
• Deliverable: Full visibility into latency, packet loss, and application reachability.

Phase 3: Intelligence (Week 5-6)

• Establish baselines for all metrics (2-week observation period).
• Configure dynamic thresholds based on time-of-day patterns.
• Build runbooks for common failure scenarios.
• Implement automated remediation for known-recoverable issues (e.g., tunnel flap recovery).
• Deliverable: Proactive alerting that catches degradation before outage.

Commercial Alternatives

If building a custom stack is not feasible, consider these commercial solutions:

In my experience, the open-source stack (Prometheus + Blackbox + Grafana) provides 90% of the value at 10% of the cost for most hybrid VPN monitoring use cases. Commercial tools add value primarily when you need internet path visibility or have dozens of VPN connections to monitor.

Conclusion

VPN monitoring in hybrid environments is not optional — it is a reliability requirement. The key principles:

1. Monitor at multiple layers. Tunnel state alone is insufficient. You need network quality metrics AND application-layer reachability.
2. Active probing is essential. Passive metrics from CloudWatch and Azure Monitor tell you about the tunnel. Active probes tell you about the user experience.
3. Establish baselines before setting alerts. Every environment has different “normal” latency. Alert on deviation from YOUR baseline, not arbitrary thresholds.
4. Automate the response. A 3 AM alert with no runbook is worse than no alert at all. Document what to do for every alert before you enable it.

Start with Phase 1 today — it takes less than an hour and covers the critical failure scenario. Then build toward full observability over the following weeks.