Overview
The Merlin addon enables automated discovery, download, and processing of royalty data files from FTP servers. It provides intelligent file recognition, duplicate detection, and a comprehensive approval workflow with confidence scoring.Key Features
- Encrypted credential storage using PostgreSQL pgcrypto with AES-256
- Scheduled and manual imports with configurable cron patterns, timezone support, and blackout windows
- Intelligent file discovery with DSP folder mapping and filename pattern recognition
- Duplicate detection via path and content hash to prevent re-importing files
- Confidence-based auto-approval for high-confidence matches with detailed scoring breakdown
- Multi-step approval workflow with single, grouped, and bulk operations
- File preview capability to review contents before approval
- AI-powered recommendations based on historical patterns and success rates
- Complete audit trail with event notifications and metrics tracking
- Advanced analytics with success rates, processing times, and trend analysis
Prerequisites
The Merlin addon requires an active subscription with addon access enabled. Contact support to enable the addon for your workspace.
Environment Variables
Set the following environment variables for encryption:Quick Start
Step 1: Configure FTP Credentials
Node.js
Credentials are automatically encrypted using PostgreSQL pgcrypto before storage. Plaintext credentials are never stored in the database.
Step 2: Test Connection
Node.js
Step 3: Configure Import Settings
Node.js
Step 4: Trigger First Import
Node.js
Understanding Confidence Scoring
The Merlin addon uses a sophisticated 7-factor confidence scoring algorithm to determine file authenticity and correct period detection. Each factor contributes to an overall score from 0.0 (no confidence) to 1.0 (complete confidence).Scoring Factors
Confidence scores are calculated when files are discovered and updated throughout the import process as more data becomes available.
Interpretation Guide
1
Low Confidence (0.0 - 0.3)
- File doesn’t match expected patterns
- Ambiguous period detection
- Unusual file structure
- Action: Requires manual review and explicit confirmation
2
Medium Confidence (0.3 - 0.75)
- Some factors align with expectations
- Period detection is clear but source less certain, or vice versa
- Action: Shows in approval queue, user should preview before approving
3
High Confidence (0.75 - 0.9)
- Most factors align with known patterns
- Clear period and source detection
- Matches historical successful patterns
- Action: Can be approved with confidence, suitable for auto-approval policies
4
Very High Confidence (0.9 - 1.0)
- Nearly all factors align perfectly
- Exact match to known patterns
- Multiple confirming factors
- Action: Excellent candidate for automatic approval
Viewing Detailed Scores
Node.js
Advanced Approval Patterns
The Merlin addon supports three distinct approval workflows for different use cases. Choose the pattern that best fits your operational process.Pattern 1: Single File Approval
Best for careful, deliberate reviews with full control over each file.Node.js
Pattern 2: Grouped Approval by DSP + Period
Best for batch processing when you trust the source and period.Node.js
Pattern 3: Bulk Approval by Period Only
Best for multi-DSP months or rapid processing.Node.js
Decision Tree
File Preview Before Approval
Preview file contents to validate structure and detect issues before committing to import.Node.js
- Are all expected columns present?
- Do data values look reasonable?
- Are there data quality warnings?
- Does the sample data match the expected format for this DSP?
Auto-Approval & Recommendations
Streamline approval workflows with intelligent recommendations and automatic approval for high-confidence files.Get Auto-Approval Candidates
Node.js
Get AI-Powered Recommendations
Node.js
Enable Auto-Approval for High Confidence Files
Node.js
Advanced Filtering & Search
Find and filter pending imports with sophisticated query capabilities.Filter by Multiple Criteria
Node.js
Available Filters
Advanced Grouping for Batch Operations
Node.js
Enhanced Scheduled Imports
Configure sophisticated import schedules with timezone support and blackout windows.Simple Daily Schedule
Node.js
Weekly Schedule
Node.js
Monthly Schedule
Node.js
Custom Cron Expression
Node.js
Blackout Windows
Prevent imports during certain periods (maintenance, cutoff times, etc.):Node.js
Import History & Analytics
Track completed imports and analyze operational metrics.View Import History
Node.js
Get Import Statistics
Node.js
Get Detailed Metrics
Node.js
Error Handling & Troubleshooting
Common Issues
1
Connection Failed
Cause: FTP credentials incorrect or server unreachable
2
Files Not Discovered
Cause: Incorrect path or no files in expected location
3
Low Confidence Scores
Cause: File structure differs from expected patterns
- Check file naming against DSP conventions
- Verify CSV/Excel column headers match expected names
- Ensure period information is clearly embedded
- Look for warnings in file preview
4
Duplicate File Detection
Cause: File already imported based on content hashSolution: Use file preview to confirm it’s actually different content, then contact support if needed
Handling Import Failures
Node.js
Best Practices
1
1. Start with Manual Reviews
Begin with single-file approvals to understand patterns before enabling auto-approval.
2
2. Monitor Initial Confidence Scores
Track confidence scores for 2-3 weeks to establish baseline and identify problematic sources.
3
3. Use File Preview Regularly
Preview files to validate structure, especially from new DSP sources.
4
4. Set Appropriate Confidence Thresholds
Don’t set auto-approval threshold too low. Start at 0.90 and lower gradually as you verify accuracy.
5
5. Establish Blackout Windows
Add blackout windows during accounting period cutoffs or system maintenance.
6
6. Review Metrics Regularly
Check import success rates and average confidence monthly to catch trends early.
7
7. Handle Rejections Properly
When rejecting files, include detailed reason notes for audit trail and future pattern learning.
API Reference
For complete API documentation, see the Merlin Addon API Reference.Key Endpoints
- Configuration:
/merlin/config,/merlin/import/config,/merlin/features - Connection:
/merlin/test-connection,/merlin/connection-status - Import Operations:
/merlin/import/trigger - Pending Management:
/merlin/pending,/merlin/pending/{id},/merlin/pending/grouped - Approval Workflows:
/merlin/pending/{id}/approve,/merlin/pending/bulk-approve - Intelligence:
/merlin/pending/recommendations,/merlin/pending/auto-approvable - History:
/merlin/history,/merlin/stats,/merlin/metrics
Support & Questions
For issues, feature requests, or questions:1
Check Documentation
Review this guide and API reference for common patterns
2
Review Import History
Check
/merlin/history and /merlin/metrics for operational insights3
Test Connection
Use
/merlin/test-connection to verify FTP credentials and data availability4
Contact Support
Reach out to support@royalti.io with your tenant ID and specific error messages