Cloud DB Migration with Search and Replication

Scenario

Use this workflow when modernizing an on-premises database to a cloud-native architecture that requires full-text search and cross-region disaster recovery. The pipeline stages backups in OSS, ingests them into RDS as the primary transactional layer, syncs data to Elasticsearch for low-latency analytics, and establishes a replicated RDS instance for high availability.

Integration steps

1. Stage Backup in OSS: Upload your on-prem dump using the OSS CLI: ossutil cp /path/to/backup.sql.gz oss://<bucket-name>/db-staging/ --storage-class Standard.
2. Import to RDS: Trigger the native RDS import via API: aliyun rds ImportDataFromOSS --DBInstanceId <rds-id> --FileName db-staging/backup.sql.gz --Engine MySQL --OSSBucket <bucket-name>. Poll DescribeImportDataFromOSSStatus until Status=Success.
3. Sync RDS to Elasticsearch: Initialize a DTS sync job: aliyun dts CreateSynchronizationJob --JobName rds-to-es-sync --SourceEndpointEngineName MySQL --DestEndpointEngineName Elasticsearch --SynchronizationDirection Forward --TableMapping '{"source":"db.","dest":"es-index-"}'.
4. Provision Cross-Region Replica: Create a standby RDS instance in the DR region: aliyun rds CreateDBInstance --Engine MySQL --EngineVersion 8.0 --RegionId <target-region> --VPCId <vpc-id> --DBInstanceClass rds.mysql.s2.large.
5. Enable Cross-Region Replication: Link primary and replica via DTS incremental sync: aliyun dts CreateSynchronizationJob --JobName rds-dr-sync --SourceRegion <src> --DestRegion <target> --SourceEndpointInstanceID <primary-id> --DestEndpointInstanceID <replica-id> --SynchronizationMode Incremental.
6. Validate & Route Traffic: Verify parity with SELECT COUNT(*) on both RDS instances and _cat/indices?v on ES. Update application configs to route search queries to the ES endpoint and read-heavy workloads to the replica.

Architecture

Data flows unidirectionally from on-premises to OSS as a durable staging layer. RDS acts as the primary ingestion and transactional engine. From RDS, data bifurcates: one stream replicates incrementally to Elasticsearch for full-text indexing, while a second stream asynchronously syncs to a cross-region RDS instance for disaster recovery and read scaling.

Prerequisites

• RAM roles with AliyunOSSFullAccess, AliyunRDSFullAccess, and AliyunDTSFullAccess attached.
• Pre-provisioned VPCs with Express Connect or CEN peering for cross-region sync.
• On-prem backup in .sql or .sql.gz format, validated against target RDS engine version.
• Elasticsearch cluster with pre-defined index templates matching RDS schema.

Common pitfalls

• Character Set Corruption: Importing UTF-8 dumps into latin1 RDS instances breaks multibyte data. Always enforce --default-character-set=utf8mb4 during import.
• OSS File Size Limits: RDS native import rejects files >5GB. Pre-split large dumps with split -b 4G backup.sql before uploading.
• ES Mapping Conflicts: Auto-syncing RDS DATETIME or JSON columns triggers strict mapping errors. Define explicit ES index mappings before starting DTS.
• Replication Lag Spikes: Cross-region sync latency increases during bulk writes. Monitor DtsLag metrics and delay application cutover until Lag < 5s.

Typical questions

• migrate on-prem DB to cloud with search and replication
• full database migration to RDS with ES and DR
• upload backup to OSS import to RDS sync ES and replicate
• 从本地数据库迁移到云端并配置搜索和容灾
• 数据库迁移到RDS后添加搜索和多地域复制
• complete cloud migration with search and HA
• migrate DB to RDS via OSS then replicate across regions
• 端到端数据库迁移到云端含搜索和高可用

Q: How do I migrate an on-premises database to the cloud with search and replication? A: You can execute this workflow by uploading on-premises database backups to OSS, importing them into RDS, and syncing the data to Elasticsearch for full-text search. You then use RDS data migration to replicate the database across regions or instance types to ensure high availability and disaster recovery.