Skip to main content

Overview

The Merlin addon enables automated discovery, download, and processing of royalty data files from FTP servers. It provides intelligent file recognition, duplicate detection, and a comprehensive approval workflow with confidence scoring.

Key Features

  • Encrypted credential storage using PostgreSQL pgcrypto with AES-256
  • Scheduled and manual imports with configurable cron patterns, timezone support, and blackout windows
  • Intelligent file discovery with DSP folder mapping and filename pattern recognition
  • Duplicate detection via path and content hash to prevent re-importing files
  • Confidence-based auto-approval for high-confidence matches with detailed scoring breakdown
  • Multi-step approval workflow with single, grouped, and bulk operations
  • File preview capability to review contents before approval
  • AI-powered recommendations based on historical patterns and success rates
  • Complete audit trail with event notifications and metrics tracking
  • Advanced analytics with success rates, processing times, and trend analysis

Prerequisites

The Merlin addon requires an active subscription with addon access enabled. Contact support to enable the addon for your workspace.

Environment Variables

Set the following environment variables for encryption:
Never commit encryption keys to version control. Use environment secrets or a secure key management service like Google Cloud Secret Manager.

Quick Start

Step 1: Configure FTP Credentials

Node.js
Credentials are automatically encrypted using PostgreSQL pgcrypto before storage. Plaintext credentials are never stored in the database.

Step 2: Test Connection

Node.js

Step 3: Configure Import Settings

Node.js

Step 4: Trigger First Import

Node.js

Understanding Confidence Scoring

The Merlin addon uses a sophisticated 7-factor confidence scoring algorithm to determine file authenticity and correct period detection. Each factor contributes to an overall score from 0.0 (no confidence) to 1.0 (complete confidence).

Scoring Factors

Confidence scores are calculated when files are discovered and updated throughout the import process as more data becomes available.

Interpretation Guide

1

Low Confidence (0.0 - 0.3)

  • File doesn’t match expected patterns
  • Ambiguous period detection
  • Unusual file structure
  • Action: Requires manual review and explicit confirmation
2

Medium Confidence (0.3 - 0.75)

  • Some factors align with expectations
  • Period detection is clear but source less certain, or vice versa
  • Action: Shows in approval queue, user should preview before approving
3

High Confidence (0.75 - 0.9)

  • Most factors align with known patterns
  • Clear period and source detection
  • Matches historical successful patterns
  • Action: Can be approved with confidence, suitable for auto-approval policies
4

Very High Confidence (0.9 - 1.0)

  • Nearly all factors align perfectly
  • Exact match to known patterns
  • Multiple confirming factors
  • Action: Excellent candidate for automatic approval

Viewing Detailed Scores

Node.js

Advanced Approval Patterns

The Merlin addon supports three distinct approval workflows for different use cases. Choose the pattern that best fits your operational process.

Pattern 1: Single File Approval

Best for careful, deliberate reviews with full control over each file.
Node.js
When to use: High-value files, unfamiliar sources, policy/regulatory compliance reviews, training/onboarding scenarios.

Pattern 2: Grouped Approval by DSP + Period

Best for batch processing when you trust the source and period.
Node.js
When to use: Regular DSP uploads, established revenue sources, high-volume import scenarios, end-of-month batch processing.

Pattern 3: Bulk Approval by Period Only

Best for multi-DSP months or rapid processing.
Node.js
When to use: Month-end closing procedures, all sources received for a period, rapid deployment environments.

Decision Tree

File Preview Before Approval

Preview file contents to validate structure and detect issues before committing to import.
Node.js
What to look for in preview:
  • Are all expected columns present?
  • Do data values look reasonable?
  • Are there data quality warnings?
  • Does the sample data match the expected format for this DSP?

Auto-Approval & Recommendations

Streamline approval workflows with intelligent recommendations and automatic approval for high-confidence files.

Get Auto-Approval Candidates

Node.js

Get AI-Powered Recommendations

Node.js

Enable Auto-Approval for High Confidence Files

Node.js
Auto-approval bypasses manual review. Only enable for established sources with consistently high confidence scores. Monitor auto-approved files in your import history to catch any issues quickly.
Find and filter pending imports with sophisticated query capabilities.

Filter by Multiple Criteria

Node.js

Available Filters

Advanced Grouping for Batch Operations

Node.js

Enhanced Scheduled Imports

Configure sophisticated import schedules with timezone support and blackout windows.

Simple Daily Schedule

Node.js

Weekly Schedule

Node.js

Monthly Schedule

Node.js

Custom Cron Expression

Node.js

Blackout Windows

Prevent imports during certain periods (maintenance, cutoff times, etc.):
Node.js
Use blackout windows to avoid system maintenance windows, data cutoff times, or high-traffic periods in your accounting system.

Import History & Analytics

Track completed imports and analyze operational metrics.

View Import History

Node.js

Get Import Statistics

Node.js

Get Detailed Metrics

Node.js

Error Handling & Troubleshooting

Common Issues

1

Connection Failed

Cause: FTP credentials incorrect or server unreachable
2

Files Not Discovered

Cause: Incorrect path or no files in expected location
3

Low Confidence Scores

Cause: File structure differs from expected patterns
  • Check file naming against DSP conventions
  • Verify CSV/Excel column headers match expected names
  • Ensure period information is clearly embedded
  • Look for warnings in file preview
Solution: Use file preview to diagnose, or manually confirm accounting period in single-file approval
4

Duplicate File Detection

Cause: File already imported based on content hash
Solution: Use file preview to confirm it’s actually different content, then contact support if needed

Handling Import Failures

Node.js

Best Practices

1

1. Start with Manual Reviews

Begin with single-file approvals to understand patterns before enabling auto-approval.
2

2. Monitor Initial Confidence Scores

Track confidence scores for 2-3 weeks to establish baseline and identify problematic sources.
3

3. Use File Preview Regularly

Preview files to validate structure, especially from new DSP sources.
4

4. Set Appropriate Confidence Thresholds

Don’t set auto-approval threshold too low. Start at 0.90 and lower gradually as you verify accuracy.
5

5. Establish Blackout Windows

Add blackout windows during accounting period cutoffs or system maintenance.
6

6. Review Metrics Regularly

Check import success rates and average confidence monthly to catch trends early.
7

7. Handle Rejections Properly

When rejecting files, include detailed reason notes for audit trail and future pattern learning.

API Reference

For complete API documentation, see the Merlin Addon API Reference.

Key Endpoints

  • Configuration: /merlin/config, /merlin/import/config, /merlin/features
  • Connection: /merlin/test-connection, /merlin/connection-status
  • Import Operations: /merlin/import/trigger
  • Pending Management: /merlin/pending, /merlin/pending/{id}, /merlin/pending/grouped
  • Approval Workflows: /merlin/pending/{id}/approve, /merlin/pending/bulk-approve
  • Intelligence: /merlin/pending/recommendations, /merlin/pending/auto-approvable
  • History: /merlin/history, /merlin/stats, /merlin/metrics

Support & Questions

For issues, feature requests, or questions:
1

Check Documentation

Review this guide and API reference for common patterns
2

Review Import History

Check /merlin/history and /merlin/metrics for operational insights
3

Test Connection

Use /merlin/test-connection to verify FTP credentials and data availability
4

Contact Support

Reach out to support@royalti.io with your tenant ID and specific error messages