# rds-transmission

Part of **RDS**

# ApsaraDB RDS Data Synchronization and Integration Troubleshooting Guide

## Problem Index

| Problem | Symptom | Severity | Solution Summary |
|--------|--------|---------|------------------|
| Client Partition Empty in DTS Change Tracking | Error message: `client partition is empty` in DTS logs | Low | Understand this is expected behavior during active client failover; no action needed |
| DTS Change Tracking Fails to Deliver Data | No data received by subscriber despite active tracking task | Medium | Verify only one client is actively consuming from the same change tracking ID |
| Replication Lag During Synchronization | High latency between source and target databases | High | Check network connectivity, resource utilization, and DTS task configuration |

## Problem Details

### Problem 1: Client Partition Empty Error in DTS Change Tracking Logs

**Symptoms**
- Error message: `client partition is empty`
- Behavior: A DTS change tracking client receives no data and logs this message
- Context: Occurs when multiple clients attempt to consume from the same change tracking ID simultaneously

**Root Cause**
- DTS change tracking is designed to support disaster recovery with active-passive client architecture.
- Only one client can actively receive data at a time for a given change tracking ID.
- When a second client connects while the first is active, it is placed in standby mode and its partition appears "empty" by design.

**Solution**
1. Confirm that only one client application is actively consuming data from the change tracking instance at any time.
2. If implementing high availability, ensure your failover mechanism properly deactivates the primary client before activating the standby.
3. Do not treat this log message as an error—it is informational and indicates correct disaster recovery behavior.

**Verification**
- After ensuring single active consumption, the active client should receive continuous data streams.
- The standby client will continue to log `client partition is empty`, which is expected and requires no intervention.

### Problem 2: DTS Change Tracking Client Receives No Data

**Symptoms**
- No data is delivered to the subscriber application
- DTS task status shows "Running" but consumer lag increases
- Logs may show repeated connection attempts or idle states

**Root Cause**
- Multiple clients are configured with the same change tracking ID, causing contention
- Network restrictions block the client from connecting to DTS endpoints
- Incorrect consumer group or authentication credentials

**Solution**
1. Validate that your application uses a unique change tracking ID per active consumer.
2. Ensure outbound access from your client environment to DTS service endpoints is allowed (check security groups and VPC routing).
3. Verify the AccessKey ID and AccessKey secret used by the client have the required permissions for DTS operations (`AliyunDTSFullAccess` or equivalent).

```bash
# Example: Test connectivity to DTS endpoint (replace with actual endpoint)
telnet dts.<region>.aliyuncs.com 80
```

4. Review client SDK initialization code to confirm correct parameters:

```python
from aliyunsdkdts.request.v20200101 import DescribeSubscriptionInstanceStatusRequest

# Ensure correct InstanceId and RegionId are used
request = DescribeSubscriptionInstanceStatusRequest.DescribeSubscriptionInstanceStatusRequest()
request.set_SubscriptionInstanceId("dtstxxxxxx")
request.set_accept_format('json')
```

**Verification**
- Use the DTS console to check "Consumer Group Status" under the change tracking task.
- Expected behavior: One consumer shows "Active", others show "Standby".
- Monitor data delivery metrics in CloudMonitor for the DTS task.

### Problem 3: High Replication Lag During Data Synchronization

**Symptoms**
- Target database lags significantly behind the source
- DTS task shows "Delay" metric increasing over time
- Write-heavy workloads on source instance

**Root Cause**
- Insufficient resources on source or target database (CPU, IOPS)
- Network latency between regions or across cloud boundaries
- Large transactions or DDL operations blocking replication stream

**Solution**
1. Scale up the RDS instance class if CPU or I/O usage exceeds 70%.
2. For cross-region synchronization, consider using same-region targets or enabling bandwidth acceleration.
3. Avoid long-running transactions on the source during initial sync or heavy load periods.
4. In the DTS console, adjust the "Throughput" setting for the synchronization task:

   - Navigate to **DTS Console > Synchronization Tasks > Task Details > Performance Tuning**
   - Increase "Maximum Concurrent Threads" (default: 4) up to 16 based on target capacity

5. Enable "Parallel Reprocessing" for tables with primary keys to improve apply speed.

**Verification**
- Monitor the "Replication Delay" metric in the DTS task dashboard.
- Expected outcome: Delay stabilizes below 5 seconds under normal load.
- Use `SHOW SLAVE STATUS` (for MySQL-compatible targets) to verify `Seconds_Behind_Master`.

## FAQ

**Q: Is the "client partition is empty" message an error that needs fixing?**  
A: No. This is expected behavior in DTS change tracking. It indicates that another client is actively consuming data from the same change tracking ID, which is part of the designed disaster recovery mechanism. Standby clients will show this message until they become active.

**Q: How many clients can consume from a single DTS change tracking ID?**  
A: Multiple clients can connect, but only one can be active at a time. Additional clients remain in standby mode and will not receive data until the active client disconnects or fails, triggering failover.

**Q: What permissions are required to consume DTS change tracking data?**  
A: The RAM user must have permissions to call DTS APIs, typically granted via the `AliyunDTSFullAccess` policy or a custom policy allowing `dts:DescribeSubscriptionInstances`, `dts:StartSubscriptionInstance`, and related actions.

**Q: How can I monitor data synchronization health?**  
A: Use the DTS console to view task status, delay metrics, and error logs. Additionally, integrate with CloudMonitor to set alerts on replication lag, error counts, or throughput drops.

**Q: Can I use DTS change tracking across different Alibaba Cloud accounts?**  
A: Yes, but you must grant cross-account permissions using RAM roles and ensure network connectivity between the accounts’ VPCs (via Cloud Enterprise Network or VPC peering).