Webhook Notifications for Merlin

Overview

The Merlin addon emits webhook events for every stage of the royalty data import lifecycle. By subscribing to these events, you can:

Monitor import batch progress in real-time
Trigger downstream processes when files are approved or completed
Get notified of failures for immediate action
Build custom integrations with external systems
Maintain an audit trail of all import activities

All Merlin events are sent via HTTP POST to your configured webhook endpoint with standardized JSON payloads.

Quick Start

1. Configure Your Webhook Endpoint

First, set up a webhook endpoint that accepts POST requests:

// Example Node.js/Express webhook handler
app.post('/webhooks/merlin', (req, res) => {
  const event = req.body;

  // Verify signature (recommended)
  const signature = req.headers['x-royalti-signature'];
  if (!verifySignature(req.body, signature)) {
    return res.status(401).json({ error: 'Invalid signature' });
  }

  // Log the event
  console.log(`Received ${event.event} for tenant ${event.tenant.id}`);

  // Process the event based on type
  switch (event.data.event.type) {
    case 'MERLIN_FILES_DISCOVERED':
      handleFilesDiscovered(event);
      break;
    case 'MERLIN_IMPORT_APPROVED':
      handleImportApproved(event);
      break;
    case 'MERLIN_IMPORT_COMPLETED':
      handleImportCompleted(event);
      break;
    case 'MERLIN_IMPORT_FAILED':
      handleImportFailed(event);
      break;
  }

  // Return success quickly
  res.json({ received: true });
});

Use the webhook configuration API to subscribe to Merlin events:

Users can subscribe to Merlin webhooks via the dashboard or programmatically using the webhook configuration endpoints. By default, Merlin events are not enabled - users must explicitly enable them.

# Enable webhooks for all Merlin events
curl -X PUT https://api.royalti.io/webhook/config \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "config": {
      "url": "https://your-domain.com/webhooks/merlin",
      "enabledEvents": [
        "MERLIN_FILES_DISCOVERED",
        "MERLIN_DOWNLOAD_COMPLETE",
        "MERLIN_IMPORT_APPROVED",
        "MERLIN_IMPORT_COMPLETED",
        "MERLIN_IMPORT_FAILED"
      ],
      "enableSignatureValidation": true,
      "secretToken": "whsec_your_secret_token",
      "isActive": true,
      "retryAttempts": 3,
      "timeoutMs": 30000
    }
  }'

Event Types

merlin.files.discovered

Emitted when the FTP scan discovers new royalty files from Merlin. Importance: Medium Automatic Retries: Yes (3 attempts)

{
  "id": "whd_550e8400-e29b-41d4-a716-446655440000",
  "event": "MERLIN_FILES_DISCOVERED",
  "timestamp": "2025-12-07T10:30:00.000Z",
  "version": "1.0",
  "tenant": {
    "id": 456,
    "name": "My Music Label"
  },
  "source": {
    "service": "royalti-api",
    "environment": "production",
    "traceId": "trace-123"
  },
  "data": {
    "event": {
      "id": "evt_550e8400",
      "type": "MERLIN_FILES_DISCOVERED",
      "category": "MERLIN",
      "timestamp": "2025-12-07T10:30:00.000Z",
      "importance": "medium"
    },
    "actor": {
      "id": "system",
      "type": "system"
    },
    "resource": {
      "type": "merlinBatch",
      "id": "batch-550e8400",
      "url": "/merlin/import/batch/batch-550e8400",
      "displayName": "Merlin Import Batch"
    },
    "attributes": {
      "batchId": "batch-550e8400",
      "filesCount": 5,
      "ftpHost": "ftp.merlin.com",
      "salePeriods": ["202501", "202502"]
    }
  },
  "delivery": {
    "attempt": 1,
    "maxAttempts": 3
  }
}

Use Cases:

Notify users that new files are available for review
Trigger automated scanning of FTP for specific DSPs
Log import batch initiation events

merlin.download.complete

Emitted when a file finishes downloading from FTP. Importance: Low Automatic Retries: Yes (3 attempts)

{
  "id": "whd_550e8400-e29b-41d4-a716-446655440001",
  "event": "MERLIN_DOWNLOAD_COMPLETE",
  "timestamp": "2025-12-07T10:31:00.000Z",
  "data": {
    "event": {
      "type": "MERLIN_DOWNLOAD_COMPLETE",
      "importance": "low"
    },
    "resource": {
      "type": "merlinPendingImport",
      "id": "pending-123",
      "displayName": "spotify_202501.csv"
    },
    "attributes": {
      "batchId": "batch-550e8400",
      "pendingImportId": "pending-123",
      "fileName": "spotify_202501.csv",
      "fileSize": 1024000
    }
  }
}

Use Cases:

Track download progress for large batches
Update UI with real-time file download status
Monitor for slow or stalled downloads

merlin.import.approved

Emitted when a pending import is approved (manually or automatically). Importance: Medium Automatic Retries: Yes (3 attempts)

{
  "id": "whd_550e8400-e29b-41d4-a716-446655440002",
  "event": "MERLIN_IMPORT_APPROVED",
  "timestamp": "2025-12-07T10:32:00.000Z",
  "data": {
    "event": {
      "type": "MERLIN_IMPORT_APPROVED",
      "importance": "medium"
    },
    "context": {
      "actorType": "system",
      "actorId": null
    },
    "resource": {
      "type": "merlinPendingImport",
      "id": "pending-123",
      "displayName": "spotify_202501.csv"
    },
    "attributes": {
      "pendingImportId": "pending-123",
      "fileName": "spotify_202501.csv",
      "isAutoApproved": true,
      "approvedBy": null
    }
  }
}

Use Cases:

Notify managers of auto-approved imports
Trigger downstream accounting processes
Update external systems with approved metadata

merlin.import.completed

Emitted when an approved file finishes processing and is converted to royalty data. Importance: High Automatic Retries: Yes (3 attempts)

{
  "id": "whd_550e8400-e29b-41d4-a716-446655440003",
  "event": "MERLIN_IMPORT_COMPLETED",
  "timestamp": "2025-12-07T10:35:00.000Z",
  "data": {
    "event": {
      "type": "MERLIN_IMPORT_COMPLETED",
      "importance": "high"
    },
    "resource": {
      "type": "merlinPendingImport",
      "id": "pending-123",
      "displayName": "spotify_202501.csv"
    },
    "attributes": {
      "batchId": "batch-550e8400",
      "pendingImportId": "pending-123",
      "fileId": "file-999",
      "fileName": "spotify_202501.csv",
      "royaltySourceName": "Spotify",
      "salePeriod": "202501"
    }
  }
}

Use Cases:

Trigger payment calculations
Notify artists of new royalty data
Generate reports or reconciliation
Update data warehouse or BI systems
Send notifications to team members

merlin.import.failed

Emitted when file processing fails due to errors. Importance: Critical Automatic Retries: Yes (3 attempts)

{
  "id": "whd_550e8400-e29b-41d4-a716-446655440004",
  "event": "MERLIN_IMPORT_FAILED",
  "timestamp": "2025-12-07T10:33:00.000Z",
  "data": {
    "event": {
      "type": "MERLIN_IMPORT_FAILED",
      "importance": "critical"
    },
    "resource": {
      "type": "merlinPendingImport",
      "id": "pending-123",
      "displayName": "spotify_202501.csv"
    },
    "attributes": {
      "batchId": "batch-550e8400",
      "pendingImportId": "pending-123",
      "fileName": "spotify_202501.csv",
      "errorMessage": "Invalid CSV format - missing required columns",
      "errorCode": "INVALID_FORMAT"
    }
  }
}

Use Cases:

Alert operators of failed imports
Create tickets for manual review
Log errors for debugging
Implement retry logic with user notification
Track failure patterns for FTP issues

Webhook Payload Structure

All Merlin webhook events follow this standardized structure:

POST /your/webhook/endpoint HTTP/1.1
Content-Type: application/json
X-Royalti-Signature: sha256=abcd1234...
X-Royalti-Webhook-Id: whd_550e8400...
X-Royalti-Event-Type: MERLIN_IMPORT_COMPLETED

{
  "id": "whd_550e8400-e29b-41d4-a716-446655440000",  // Unique delivery ID
  "event": "MERLIN_IMPORT_COMPLETED",                  // Event type
  "timestamp": "2025-12-07T10:35:00.000Z",            // Event time (ISO 8601)
  "version": "1.0",                                    // API version
  "tenant": {                                           // Your tenant
    "id": 456,
    "name": "My Music Label"
  },
  "source": {                                           // Event source
    "service": "royalti-api",
    "environment": "production",
    "traceId": "trace-123"
  },
  "data": {
    "event": {                                          // Event metadata
      "id": "evt_550e8400",
      "type": "MERLIN_IMPORT_COMPLETED",
      "category": "MERLIN",
      "timestamp": "2025-12-07T10:35:00.000Z",
      "importance": "high"
    },
    "actor": {                                          // Who triggered the event
      "id": "user-123",
      "type": "system"  // or "user", "integration"
    },
    "resource": {                                       // Primary affected resource
      "type": "merlinPendingImport",
      "id": "pending-123",
      "url": "/merlin/pending/pending-123",
      "displayName": "spotify_202501.csv"
    },
    "attributes": {                                     // Event-specific data
      "batchId": "batch-550e8400",
      "pendingImportId": "pending-123",
      "fileId": "file-999",
      "fileName": "spotify_202501.csv",
      "royaltySourceName": "Spotify",
      "salePeriod": "202501"
    },
    "previous": null                                    // Previous state (updates only)
  },
  "delivery": {
    "attempt": 1,                                       // Attempt number (1-3)
    "maxAttempts": 3                                    // Total retry attempts
  }
}

Security

Signature Validation

All Merlin webhook events can include HMAC-SHA256 signatures for verification. Enable this in your webhook configuration:

const crypto = require('crypto');

function verifySignature(requestBody, signature, secret) {
  // Signature is computed as: HMAC-SHA256(request_body, secret)
  const computed = crypto
    .createHmac('sha256', secret)
    .update(JSON.stringify(requestBody))
    .digest('hex');

  // Use constant-time comparison to prevent timing attacks
  return crypto.timingSafeEqual(
    Buffer.from(signature),
    Buffer.from(computed)
  );
}

// Verify on incoming webhook
app.post('/webhooks/merlin', (req, res) => {
  const signature = req.headers['x-royalti-signature'];
  const secret = process.env.MERLIN_WEBHOOK_SECRET;

  if (!verifySignature(req.body, signature, secret)) {
    return res.status(401).json({ error: 'Invalid signature' });
  }

  // Process webhook...
});

HTTPS Only

Always use HTTPS endpoints for production webhooks. HTTP endpoints are rejected.

Request Headers

Every webhook request includes these headers:

X-Royalti-Signature - HMAC-SHA256 signature (if enabled)
X-Royalti-Webhook-Id - Unique delivery ID
X-Royalti-Event-Type - Event type
X-Royalti-Tenant-Id - Your tenant ID
Content-Type: application/json

Best Practices

1. Return Quickly

Return a 2xx status code as soon as possible. Process events asynchronously:

app.post('/webhooks/merlin', async (req, res) => {
  const event = req.body;

  // Return immediately
  res.json({ received: true });

  // Process asynchronously
  processEvent(event).catch(error => {
    logger.error('Failed to process webhook', { error, eventId: event.id });
  });
});

2. Implement Idempotency

Webhooks may be delivered multiple times. Implement idempotent handlers:

async function handleImportCompleted(event) {
  const { fileId, batchId } = event.data.attributes;

  // Check if already processed using event ID + delivery ID
  const idempotencyKey = `${event.id}-${event.data.event.id}`;
  const cached = await redis.get(`webhook:${idempotencyKey}`);

  if (cached) {
    logger.info('Webhook already processed', { idempotencyKey });
    return;
  }

  // Process the event
  await updateRoyaltyData(fileId, batchId);

  // Mark as processed
  await redis.setex(`webhook:${idempotencyKey}`, 86400, 'processed');
}

3. Log All Events

Maintain an audit trail of all webhook deliveries:

async function logWebhookEvent(event) {
  await WebhookLog.create({
    tenantId: event.tenant.id,
    eventType: event.data.event.type,
    eventId: event.data.event.id,
    deliveryId: event.id,
    payload: event,
    receivedAt: new Date(),
  });
}

4. Monitor Webhook Health

Track delivery success rates and error patterns:

async function updateWebhookMetrics(event, success, error = null) {
  const metrics = {
    eventType: event.data.event.type,
    success,
    timestamp: new Date(),
    error: error?.message,
  };

  await WebhookMetrics.create(metrics);

  // Alert if error rate exceeds threshold
  const recentMetrics = await WebhookMetrics.findAll({
    where: { timestamp: { [Op.gte]: new Date(Date.now() - 3600000) } },
  });

  const errorRate = recentMetrics.filter(m => !m.success).length / recentMetrics.length;
  if (errorRate > 0.1) {
    await sendAlert(`Webhook error rate high: ${(errorRate * 100).toFixed(2)}%`);
  }
}

Retry Logic

Failed webhook deliveries are automatically retried with exponential backoff:

Attempt 1: Immediate
Attempt 2: 1 second delay
Attempt 3: 2 seconds delay

After 3 failed attempts, the event is logged but not retried. Check webhook health in your dashboard. Status Code Guidelines:

2xx - Success (delivery accepted)
3xx - Temporary redirect (retried)
4xx - Client error (not retried)
5xx - Server error (retried)

Example: Complete Implementation

const express = require('express');
const crypto = require('crypto');
const app = express();

app.use(express.json());

// Webhook handler with full implementation
app.post('/webhooks/merlin', async (req, res) => {
  const event = req.body;
  const secret = process.env.MERLIN_WEBHOOK_SECRET;

  try {
    // 1. Verify signature
    const signature = req.headers['x-royalti-signature'];
    if (secret && !verifySignature(JSON.stringify(event), signature, secret)) {
      logger.warn('Invalid webhook signature', { eventId: event.id });
      return res.status(401).json({ error: 'Invalid signature' });
    }

    // 2. Return immediately
    res.json({ received: true });

    // 3. Log the event
    await logWebhookEvent(event);

    // 4. Check for duplicates
    const cached = await redis.get(`webhook:${event.id}`);
    if (cached) {
      logger.info('Duplicate webhook', { eventId: event.id });
      return;
    }

    // 5. Process based on event type
    const { type } = event.data.event;
    switch (type) {
      case 'MERLIN_FILES_DISCOVERED':
        await handleFilesDiscovered(event);
        break;
      case 'MERLIN_IMPORT_APPROVED':
        await handleImportApproved(event);
        break;
      case 'MERLIN_IMPORT_COMPLETED':
        await handleImportCompleted(event);
        break;
      case 'MERLIN_IMPORT_FAILED':
        await handleImportFailed(event);
        break;
    }

    // 6. Mark as processed
    await redis.setex(`webhook:${event.id}`, 86400, 'processed');

  } catch (error) {
    logger.error('Webhook processing failed', { error, eventId: event.id });
    // Don't throw - already returned 2xx to caller
  }
});

async function handleImportCompleted(event) {
  const { fileId, batchId, royaltySourceName, salePeriod } = event.data.attributes;

  // Update system with new royalty data
  await RoyaltyData.create({
    fileId,
    batchId,
    source: royaltySourceName,
    period: salePeriod,
    importedAt: new Date(),
  });

  // Trigger downstream processes
  await triggerPaymentCalculation(event.tenant.id, salePeriod);

  // Notify team
  await sendNotification({
    title: 'Royalty Import Complete',
    message: `${royaltySourceName} data for ${salePeriod} imported successfully`,
    recipients: event.tenant.id,
  });
}

function verifySignature(body, signature, secret) {
  const computed = crypto
    .createHmac('sha256', secret)
    .update(body)
    .digest('hex');

  return crypto.timingSafeEqual(
    Buffer.from(signature),
    Buffer.from(computed)
  );
}

app.listen(3000, () => console.log('Webhook server running on :3000'));

Troubleshooting

Events Not Being Received

Check webhook is enabled:

curl https://api.royalti.io/webhook/config \
  -H "Authorization: Bearer YOUR_TOKEN"

Verify event types are subscribed:
- Ensure enabledEvents includes the Merlin events you want
- Default: all Merlin events are disabled
Check endpoint is publicly accessible:
- Merlin can only send webhooks to HTTPS endpoints
- Test with: curl -X POST https://your-endpoint.com/webhooks/merlin
Review webhook delivery logs:
- Check dashboard for failed deliveries
- Look for error messages in WebhookDeliveryLog

Signature Validation Failing

Verify secret token:
- Check stored secret in webhook config
- Ensure it matches what you’re using to verify
Check request body encoding:
- Use raw request body (not parsed JSON)
- Ensure consistent whitespace/formatting
Timing issues:
- Use constant-time comparison: crypto.timingSafeEqual()

High Error Rates

Check endpoint availability:
- Ensure your server is running and accessible
- Check firewall/security group rules
Review response codes:
- Return 2xx for success
- Don’t timeout requests (timeout is 30 seconds)
Monitor webhook health:
- Use the metrics API to track success rates
- Set up alerts for high error rates

API Reference

See the Merlin Configuration and Merlin API documentation for complete details on webhook setup and usage.

​Overview

​Quick Start

​1. Configure Your Webhook Endpoint

​2. Subscribe to Merlin Events

​Event Types

​merlin.files.discovered

​merlin.download.complete

​merlin.import.approved

​merlin.import.completed

​merlin.import.failed

​Webhook Payload Structure

​Security

​Signature Validation

​HTTPS Only

​Request Headers

​Best Practices

​1. Return Quickly

​2. Implement Idempotency

​3. Log All Events

​4. Monitor Webhook Health

​Retry Logic

​Example: Complete Implementation

​Troubleshooting

​Events Not Being Received

​Signature Validation Failing

​High Error Rates

​API Reference

Overview

Quick Start

1. Configure Your Webhook Endpoint

2. Subscribe to Merlin Events

Event Types

merlin.files.discovered

merlin.download.complete

merlin.import.approved

merlin.import.completed

merlin.import.failed

Webhook Payload Structure

Security

Signature Validation

HTTPS Only

Request Headers

Best Practices

1. Return Quickly

2. Implement Idempotency

3. Log All Events

4. Monitor Webhook Health

Retry Logic

Example: Complete Implementation

Troubleshooting

Events Not Being Received

Signature Validation Failing

High Error Rates

API Reference