Notification Hub - Multi-Channel Delivery

Unified notification delivery service supporting SMS, Voice, Email, Slack, Teams, Push, and WebSocket channels with reliable delivery guarantees.

Overview

The Notification Hub provides a single interface for sending notifications across multiple channels. It handles provider integration, template rendering, delivery tracking, and retry logic automatically.

Supported Channels

Channel	Provider	Cost	Best For
SMS	Twilio	~$0.0079/segment	Urgent alerts, OTPs
Voice	Twilio	~$0.013/min	Critical escalations
Email	SendGrid	~$0.00025/email	Detailed notifications
Slack	Webhook	Free	Team channels
Teams	Webhook	Free	Enterprise orgs
Push	FCM/APNs	Free	Mobile apps
WebSocket	Internal	Free	Real-time dashboards

Quick Start

Send a Simple Notification

POST /api/v1/notifications/send
Content-Type: application/json

{
  "channel": "sms",
  "to": "+1234567890",
  "template": "alert_notification",
  "data": {
    "title": "High CPU Alert",
    "service": "api-gateway",
    "severity": "P2"
  }
}

Send Multi-Channel Notification

POST /api/v1/notifications/send
Content-Type: application/json

{
  "channels": ["sms", "email", "slack"],
  "recipients": {
    "sms": "+1234567890",
    "email": "oncall@company.com",
    "slack": "#incidents"
  },
  "template": "incident_created",
  "data": {
    "incident_id": "inc_001",
    "title": "API Gateway Outage",
    "severity": "P1"
  }
}

Channel Configuration

SMS (Twilio)

Configuration:

notifications:
  sms:
    provider: twilio
    account_sid: ${TWILIO_ACCOUNT_SID}
    auth_token: ${TWILIO_AUTH_TOKEN}
    from_number: "+1555123456"
    messaging_service_sid: "MS..." # Optional, for high volume

Features:

Automatic message splitting (1600 char limit)
Delivery status tracking
International support (150+ countries)
MMS support for images

Example:

{
  "channel": "sms",
  "to": "+1234567890",
  "message": "🚨 P1 Alert: API Gateway down. Acknowledge at https://cockpit.olympuscloud.ai/alert/001"
}

Voice (Twilio)

Configuration:

notifications:
  voice:
    provider: twilio
    account_sid: ${TWILIO_ACCOUNT_SID}
    auth_token: ${TWILIO_AUTH_TOKEN}
    from_number: "+1555123456"
    tts_voice: "Polly.Matthew"
    fallback_channel: "sms" # If no answer

Features:

Text-to-speech with multiple voices
DTMF input for acknowledgment (Press 1 to acknowledge)
Automatic SMS fallback if no answer
Call recording (optional)

Example:

{
  "channel": "voice",
  "to": "+1234567890",
  "twiml": "<Response><Say voice='Polly.Matthew'>Alert! API Gateway is experiencing high error rates. Press 1 to acknowledge.</Say><Gather numDigits='1' action='/voice/ack'/></Response>"
}

Email (SendGrid)

Configuration:

notifications:
  email:
    provider: sendgrid
    api_key: ${SENDGRID_API_KEY}
    from_email: "alerts@olympuscloud.ai"
    from_name: "Olympus Alerting"

Features:

Rich HTML templates
Plain text fallback
Click tracking
Unsubscribe handling
Attachments support

Example:

{
  "channel": "email",
  "to": "oncall@company.com",
  "subject": "[P1] API Gateway Outage - Action Required",
  "template": "incident_email",
  "data": {
    "incident_id": "inc_001",
    "title": "API Gateway Outage",
    "description": "High error rate detected...",
    "dashboard_url": "https://grafana.olympuscloud.ai/..."
  }
}

Slack

Configuration:

notifications:
  slack:
    default_webhook: ${SLACK_WEBHOOK_URL}
    channels:
      incidents: "https://hooks.slack.com/services/..."
      alerts: "https://hooks.slack.com/services/..."

Features:

Block Kit support for rich formatting
Interactive buttons (Acknowledge, Resolve)
Thread replies for updates
Channel-specific routing

Example:

{
  "channel": "slack",
  "to": "#incidents",
  "blocks": [
    {
      "type": "header",
      "text": {"type": "plain_text", "text": "🚨 P1 Incident Created"}
    },
    {
      "type": "section",
      "text": {"type": "mrkdwn", "text": "*API Gateway Outage*\nHigh error rate affecting all services"}
    },
    {
      "type": "actions",
      "elements": [
        {"type": "button", "text": {"type": "plain_text", "text": "Acknowledge"}, "action_id": "ack"},
        {"type": "button", "text": {"type": "plain_text", "text": "View Dashboard"}, "url": "https://..."}
      ]
    }
  ]
}

Microsoft Teams

Configuration:

notifications:
  teams:
    webhooks:
      incidents: "https://outlook.office.com/webhook/..."
      alerts: "https://outlook.office.com/webhook/..."

Features:

Adaptive Cards for rich formatting
Action buttons
@mentions support

Push Notifications

Configuration:

notifications:
  push:
    fcm:
      project_id: "olympus-cloud"
      credentials_path: "/secrets/firebase-adminsdk.json"
    apns:
      key_id: ${APNS_KEY_ID}
      team_id: ${APNS_TEAM_ID}
      bundle_id: "ai.olympuscloud.workforce"

Features:

iOS and Android support
Silent push for background updates
Badge count management
Deep linking

WebSocket (Real-Time)

Configuration:

notifications:
  websocket:
    enabled: true
    endpoint: "wss://cockpit.olympuscloud.ai/ws"

Features:

Real-time delivery to Cockpit dashboard
No external dependencies
Automatic reconnection
Message queuing if disconnected

Templates

Template System

Templates use Handlebars-style syntax for variable substitution:

# templates/alert_notification.yaml
name: alert_notification
channels:
  sms:
    body: "🚨 {{severity}} Alert: {{title}} - {{service}}. Ack: {{ack_url}}"

  email:
    subject: "[{{severity}}] {{title}}"
    html: |
      <h1>{{title}}</h1>
      <p><strong>Service:</strong> {{service}}</p>
      <p><strong>Severity:</strong> {{severity}}</p>
      <a href="{{ack_url}}">Acknowledge Alert</a>

  slack:
    blocks:
      - type: header
        text:
          type: plain_text
          text: "🚨 {{severity}} Alert"
      - type: section
        text:
          type: mrkdwn
          text: "*{{title}}*\nService: {{service}}"

Built-in Templates

Template	Description	Channels
`alert_created`	New alert notification	All
`alert_acknowledged`	Alert acknowledged	SMS, Slack
`alert_resolved`	Alert resolved	All
`incident_created`	New incident	All
`incident_update`	Incident status change	Email, Slack
`oncall_handoff`	On-call rotation change	SMS, Email
`schedule_reminder`	Shift reminder	SMS, Push

Custom Templates

POST /api/v1/notifications/templates
Content-Type: application/json

{
  "name": "daily_report",
  "channels": {
    "email": {
      "subject": "Daily Ops Report - {{date}}",
      "html": "<h1>Daily Report</h1>..."
    }
  }
}

Delivery Tracking

Delivery Status

Status	Description
`pending`	Queued for delivery
`sent`	Sent to provider
`delivered`	Confirmed delivered
`failed`	Delivery failed
`bounced`	Email bounced
`clicked`	Link clicked (email)
`opened`	Email opened

Get Delivery Status

GET /api/v1/notifications/{notification_id}/status

Response:

{
  "notification_id": "notif_001",
  "channels": {
    "sms": {
      "status": "delivered",
      "delivered_at": "2026-01-24T21:00:05Z",
      "provider_id": "SM123..."
    },
    "email": {
      "status": "opened",
      "delivered_at": "2026-01-24T21:00:03Z",
      "opened_at": "2026-01-24T21:02:15Z"
    },
    "slack": {
      "status": "delivered",
      "delivered_at": "2026-01-24T21:00:01Z"
    }
  }
}

Webhooks for Status Updates

POST /api/v1/notifications/webhooks
Content-Type: application/json

{
  "url": "https://your-app.com/notifications/status",
  "events": ["delivered", "failed", "bounced", "clicked"]
}

Retry Logic

Automatic Retry Strategy

Attempt	Delay	Notes
1	Immediate	First attempt
2	1 second	Quick retry
3	10 seconds	Short delay
4	1 minute	Medium delay
5	5 minutes	Final attempt

Retry Configuration

notifications:
  retry:
    max_attempts: 5
    backoff_type: exponential
    initial_delay: 1s
    max_delay: 5m
    retryable_errors:
      - timeout
      - rate_limit
      - temporary_failure

Manual Retry

POST /api/v1/notifications/{notification_id}/retry

User Preferences

Set User Notification Preferences

PUT /api/v1/users/{user_id}/notification-preferences
Content-Type: application/json

{
  "channels": {
    "sms": {
      "enabled": true,
      "phone": "+1234567890"
    },
    "email": {
      "enabled": true,
      "address": "user@company.com"
    },
    "push": {
      "enabled": true,
      "devices": ["device_token_1"]
    }
  },
  "quiet_hours": {
    "enabled": true,
    "start": "22:00",
    "end": "07:00",
    "timezone": "America/New_York",
    "bypass_for": ["P1"]
  },
  "digest": {
    "enabled": true,
    "frequency": "daily",
    "time": "09:00"
  }
}

Get Preferences

GET /api/v1/users/{user_id}/notification-preferences

Rate Limiting

Provider Limits

Provider	Limit	Window
Twilio SMS	400/second	Per account
Twilio Voice	100/second	Per account
SendGrid	600/second	Per API key
Slack	1/second	Per webhook

Internal Rate Limiting

notifications:
  rate_limits:
    per_user:
      sms: 10/hour
      voice: 5/hour
      email: 50/hour
    per_channel:
      slack: 100/minute

Analytics

Get Notification Metrics

GET /api/v1/notifications/analytics

Response:

{
  "period": "last_24_hours",
  "total_sent": 1524,
  "by_channel": {
    "sms": {"sent": 456, "delivered": 450, "failed": 6},
    "email": {"sent": 823, "delivered": 810, "bounced": 13},
    "slack": {"sent": 245, "delivered": 245, "failed": 0}
  },
  "delivery_rate": 0.987,
  "avg_delivery_time_ms": 342,
  "costs": {
    "sms": 3.60,
    "voice": 1.56,
    "email": 0.21,
    "total": 5.37
  }
}

Best Practices

Channel Selection

Scenario	Recommended Channels
P1 Alert	Voice + SMS + Slack
P2 Alert	SMS + Slack
P3/P4 Alert	Slack + Email
Schedule Change	Push + Email
Daily Report	Email
Real-time Dashboard	WebSocket

Cost Optimization

tip

SMS and voice are the most expensive notification channels. Use free channels (Slack, Email, Push, WebSocket) for non-urgent alerts, and reserve SMS/Voice for P1/P2 incidents where immediate acknowledgment is required.

Use Slack/Email first - Free channels for non-urgent notifications
Batch where possible - Combine multiple alerts into one notification
Set quiet hours - Prevent unnecessary night-time SMS
Use templates - Optimize message length to reduce SMS segments

Reliability

Enable fallbacks - Configure SMS as voice fallback
Monitor delivery rates - Alert on delivery failures
Verify contact info - Validate phone/email on entry
Test regularly - Send test notifications to verify channels

Alerting API - Alert management
Notifications API - API reference
Webhooks API - Webhook configuration
AIOps Guide - Intelligent alerting

Overview​

Supported Channels​

Quick Start​

Send a Simple Notification​

Send Multi-Channel Notification​

Channel Configuration​

SMS (Twilio)​

Voice (Twilio)​

Email (SendGrid)​

Slack​

Microsoft Teams​

Push Notifications​

WebSocket (Real-Time)​

Templates​

Template System​

Built-in Templates​

Custom Templates​

Delivery Tracking​

Delivery Status​

Get Delivery Status​

Webhooks for Status Updates​

Retry Logic​

Automatic Retry Strategy​

Retry Configuration​

Manual Retry​

User Preferences​

Set User Notification Preferences​

Get Preferences​

Rate Limiting​

Provider Limits​

Internal Rate Limiting​

Analytics​

Get Notification Metrics​

Best Practices​

Channel Selection​

Cost Optimization​

Reliability​

Related Documentation​

Overview

Supported Channels

Quick Start

Send a Simple Notification

Send Multi-Channel Notification

Channel Configuration

SMS (Twilio)

Voice (Twilio)

Email (SendGrid)

Slack

Microsoft Teams

Push Notifications

WebSocket (Real-Time)

Templates

Template System

Built-in Templates

Custom Templates

Delivery Tracking

Delivery Status

Get Delivery Status

Webhooks for Status Updates

Retry Logic

Automatic Retry Strategy

Retry Configuration

Manual Retry

User Preferences

Set User Notification Preferences

Get Preferences

Rate Limiting

Provider Limits

Internal Rate Limiting

Analytics

Get Notification Metrics

Best Practices

Channel Selection

Cost Optimization

Reliability

Related Documentation