# rds-logs

Part of **RDS**

<!-- intent-backlink:auto -->

> 💡 **Path Selection**: This skill is one implementation path for [Monitor and analyze database performance metrics](../../intent/rds-monitor-performance/SKILL.md). If you're unsure which path to take, check the routing skill first.

# ApsaraDB RDS Log Management

## Capabilities Overview

| Sub-capability | Calling Mode | Description |
|----------------|--------------|-------------|
| Manage SQL Audit Logs | Synchronous | Enable/disable SQL Explorer, modify retention periods, and query SQL audit and related logs. |
| Query System Logs | Synchronous | Query error logs and slow query logs for troubleshooting and performance analysis. |
| Query SQL Audit Log Files | Synchronous | Retrieve and manage SQL audit log files for database auditing purposes. |

## API Calling Patterns

### Authentication
Use **AccessKey-based signature authentication** as the primary method for calling RDS Log Management APIs.

- Include standard Alibaba Cloud common request parameters: `AccessKeyId`, `Signature`, `SignatureMethod`, `Timestamp`, `Version`, etc.
- Environment variable: `ALIBABA_CLOUD_ACCESS_KEY_ID` and `ALIBABA_CLOUD_ACCESS_KEY_SECRET`
- While some examples show `Authorization: Bearer <token>`, the official and recommended method is signature-based authentication using AccessKey pairs. Use the [Alibaba Cloud SDKs](https://github.com/aliyun) which handle signing automatically.

### Service Endpoint (Endpoint)
All RDS Log Management APIs use a single global endpoint:

- Base URL: `https://rds.aliyuncs.com/`
- This endpoint serves all regions; region-specific endpoints are not required for these APIs.
- Common regions where RDS instances are deployed: `cn-hangzhou`, `cn-shanghai`, `cn-beijing`

### Synchronous API Pattern
All log management operations follow a **synchronous** request-response pattern:

1. Send an HTTP GET or POST request to `https://rds.aliyuncs.com/` with:
   - Required action parameter (e.g., `Action=DescribeSQLLogRecords`)
   - Instance identifier (`DBInstanceId`)
   - Time range and filtering parameters (as applicable)
   - Common request parameters (for authentication)
2. The server processes the request immediately and returns a JSON or XML response.
3. Parse the response to extract log data, metadata, or operation status.
4. Handle pagination using `PageNumber` and `PageSize` for large result sets.

No polling or async task tracking is needed — responses are returned directly.

## Parameter Reference

### Manage SQL Audit Logs

| Parameter | Type | Required | Default | Constraints | Description |
|----------|------|----------|---------|-------------|-------------|
| Action | String | Yes | — | — | The operation to perform. Valid values: `ModifySQLCollectorPolicy`, `ModifySQLCollectorRetention`, `DescribeSQLCollectorRetention` |
| DBInstanceId | String | Yes | — | — | The ID of the RDS instance |
| SQLCollectorStatus | String | Yes (for ModifySQLCollectorPolicy) | — | One of: `Enable`, `Disabled` | Specifies whether to enable or disable SQL Explorer |
| ConfigValue | String | Yes (for ModifySQLCollectorRetention) | — | One of: `30`, `180`, `365`, `1095`, `1825` | Log retention period in days |
| ResourceGroupId | String | No | — | — | The ID of the resource group |

### Query System Logs

| Parameter | Type | Required | Default | Constraints | Description |
|----------|------|----------|---------|-------------|-------------|
| Action | String | Yes | — | — | Operation name: `DescribeErrorLogs` or `DescribeSlowLogRecords` |
| DBInstanceId | String | Yes | — | — | The ID of the RDS instance |
| StartTime | String | Yes | — | ISO 8601 format (`yyyy-MM-ddTHH:mmZ`), UTC | Start of query time range |
| EndTime | String | Yes | — | ISO 8601 format, UTC, > StartTime, < 31 days apart | End of query time range |
| PageSize | Integer | No | `30` | Range: `30`–`100` | Number of entries per page |
| PageNumber | Integer | No | `1` | Positive integer | Page number to return |
| DBName | String | No | — | — | Filter by database name (slow logs only) |
| SQLHASH | String | No | — | — | Unique ID of SQL statement (slow logs only) |

### Query SQL Audit Logs & Files

| Parameter | Type | Required | Default | Constraints | Description |
|----------|------|----------|---------|-------------|-------------|
| Action | String | Yes | — | — | `DescribeSQLLogRecords` or `DescribeSQLLogFiles` |
| DBInstanceId | String | Yes | — | — | The ID of the RDS instance |
| StartTime | String | Yes | — | ISO 8601 (`yyyy-MM-ddTHH:mm:ssZ`), UTC, within last 30 days | Start time for audit log query |
| EndTime | String | Yes | — | ISO 8601, UTC, > StartTime, < 30 days apart | End time for audit log query |
| Form | String | No | `Stream` | One of: `File`, `Stream` | Return logs inline (`Stream`) or generate downloadable file (`File`) |
| QueryKeywords | String | No | — | Up to 10 keywords, space-separated, logical AND | Keywords to filter SQL text |
| Database | String | No | — | — | Filter by database name |
| User | String | No | — | — | Filter by username |
| FileName | String | No | — | — | Name of specific audit log file to query |
| ClientToken | String | No | — | ASCII only, ≤64 chars | Ensures request idempotence |

## Code Examples

### Enable SQL Explorer - Bash - All Regions

```bash
curl -v "https://rds.aliyuncs.com/?Action=ModifySQLCollectorPolicy&DBInstanceId=rm-uf6wjk5xxxxxxx&SQLCollectorStatus=Enable&<Common request parameters>"
```

### Query SQL Audit Logs - Python - All Regions

```python
import requests

url = 'https://rds.aliyuncs.com/'
params = {
    'Action': 'DescribeSQLLogRecords',
    'DBInstanceId': 'rm-uf6wjk5xxxxxxx',
    'StartTime': '2021-03-13T16:00:00Z',
    'EndTime': '2021-03-18T16:00:00Z',
    # Add signed common request parameters (AccessKeyId, Signature, etc.)
}

response = requests.get(url, params=params)
print(response.json())
```

### Query Error Logs - Bash - All Regions

```bash
curl -v "https://rds.aliyuncs.com/?Action=DescribeErrorLogs&DBInstanceId=rm-uf6wjk5xxxxxxx&StartTime=2021-05-01T20:10Z&EndTime=2021-05-30T20:10Z&<Common request parameters>"
```

### Query Slow Query Logs - Bash - All Regions

```bash
curl -v "https://rds.aliyuncs.com/?Action=DescribeSlowLogRecords&DBInstanceId=rm-uf6wjk5*******&StartTime=2020-06-17T16:00Z&EndTime=2020-06-18T16:00Z&<Common request parameters>"
```

### Modify SQL Log Retention Period - Bash - All Regions

```bash
curl -v "https://rds.aliyuncs.com/?Action=ModifySQLCollectorRetention&DBInstanceId=rm-uf6wjk5xxxxxxx&ConfigValue=365&ResourceGroupId=rg-acfmyxxxx&<Common request parameters>"
```

### Query Audit Log Files - Bash - All Regions

```bash
curl -v "https://rds.aliyuncs.com/?Action=DescribeSQLLogFiles&DBInstanceId=rm-uf6wjk5xxxxxx&<Common request parameters>"
```

## Response Format

```json
{
  "PageNumber": 1,
  "TotalRecordCounts": 1,
  "ItemsCounts": 1,
  "SQLItems": {
    "SQLItem": [
      {
        "DBName": "test",
        "AccountName": "accounttest",
        "HostAddress": "192.168.0.121",
        "SQLText": "update test.zxb set id=0 limit 1",
        "TotalExecutionTimes": 12,
        "ReturnRowCounts": 34,
        "ExecuteTime": "2011-06-11T15:00:23Z"
      }
    ]
  },
  "RequestId": "08A3B71B-FE08-4B03-974F-CC7EA6DB1828"
}
```

**Key Fields**:
- `RequestId` — Unique identifier for the API request (useful for support)
- `PageNumber`, `TotalRecordCounts`, `ItemsCounts` — Pagination metadata
- `SQLItems.SQLItem[].SQLText` — The actual SQL statement executed
- `SQLItems.SQLItem[].ExecuteTime` — Timestamp of execution (UTC)
- `SQLItems.SQLItem[].HostAddress` — Client IP address that executed the query
- `SQLItems.SQLItem[].DBName` — Database name targeted by the query
- `items[].LogDownloadURL` (in DescribeSQLLogFiles) — Pre-signed URL to download audit log file

## Error Handling

| Error Code | Description | Recommended Action |
|------------|-------------|-------------------|
| 400 | Invalid request parameters. Check that all required parameters are provided and correctly formatted. | Validate parameter names, formats (especially time in ISO 8601 UTC), and value constraints |
| 400 | The time span between the start time and the end time exceeds 31 days. | Reduce the time range to less than 31 days |
| 400 | This operation is not supported when the SQL Explorer feature is disabled. | Enable SQL Explorer first using `ModifySQLCollectorPolicy` |
| 403 | Access denied. Ensure the API key has sufficient permissions to access the specified RDS instance. | Verify RAM user permissions or use owner's AccessKey |
| 404 | Instance not found. Verify that the DBInstanceId is correct and the instance exists. | Double-check instance ID and region |
| 429 | Too Many Requests. The request rate exceeds the limit of 1,000 calls per minute per account. | Implement rate limiting or exponential backoff |
| 500 | Internal server error. Retry the request after a short delay. | Retry with exponential backoff; contact support if persistent |

### Rate Limits & Retry
- General QPS limit: **100 queries per second** per account for most log APIs
- `DescribeSQLLogRecords` has a stricter limit: **1,000 calls per minute** per account (includes RAM users)
- If you receive a `429` error, wait and retry using exponential backoff (e.g., 1s, 2s, 4s, 8s)
- The `Retry-After` header is not used; rely on client-side backoff logic

## Requirements

- Use official [Alibaba Cloud SDKs](https://github.com/aliyun) (e.g., `aliyun-python-sdk-rds`) for automatic request signing
- Set environment variables:
  ```bash
  export ALIBABA_CLOUD_ACCESS_KEY_ID=your_access_key_id
  export ALIBABA_CLOUD_ACCESS_KEY_SECRET=your_access_key_secret
  ```
- For direct HTTP calls, implement [Alibaba Cloud signature version 1.0](https://help.aliyun.com/document_detail/25491.html)

## FAQ

Q: How do I enable SQL audit logging for my RDS instance?
A: Call `ModifySQLCollectorPolicy` with `SQLCollectorStatus=Enable`. Ensure your instance runs MySQL, SQL Server 2008 R2+, or PostgreSQL.

Q: Why am I getting "Access Denied" when querying logs?
A: Your AccessKey or RAM user lacks the `rds:Describe*Log*` permissions. Attach the `AliyunRDSReadOnlyAccess` or custom policy with log read permissions.

Q: Can I query logs older than 30 days?
A: No. SQL audit logs via `DescribeSQLLogRecords` are only available for the last 30 days. However, if you set a longer retention period (up to 1825 days) and export logs to files using `Form=File`, those files remain accessible via `DescribeSQLLogFiles`.

Q: What’s the difference between `DescribeSQLLogRecords` and `DescribeSQLLogFiles`?
A: `DescribeSQLLogRecords` returns log entries directly in the API response (suitable for recent analysis). `DescribeSQLLogFiles` lists metadata and download URLs for audit log files previously generated by setting `Form=File` in `DescribeSQLLogRecords`.

Q: Are there free API calls for log queries?
A: Some operations offer free tiers (e.g., 100–1000 free requests/month), but this varies by API. Check the pricing section for details. Note that enabling SQL Explorer may incur separate storage fees.

## Pricing & Billing

### Billing Model
All log management APIs use **per-request billing**. Each API call counts as one request, regardless of the number of log entries returned.

### Price Reference

| Tier | Input Price | Output Price | Other Fees |
|------|-------------|--------------|-----------|
| default | ¥0.001/request | ¥0.001/request | — |
| standard | ¥0.001/request | ¥0.001/request | — |

### Free Tier
- `DescribeSQLLogReportList`: 1000 free requests per month
- `DescribeErrorLogs`: 1000 free calls per month
- `DescribeSQLLogFiles`: 100 free requests per month
- Other APIs: typically 100 free requests per month

### Usage Limits
- Max QPS: 100 per account (most APIs)
- `DescribeSQLLogRecords`: 1,000 calls per minute per account
- Time range limits: ≤30 days for audit logs, ≤31 days for error/slow logs

### Billing Notes
- Charges apply per API call, even if no logs are returned
- Using `Form=File` in `DescribeSQLLogRecords` still counts as one request; downloading the file from OSS is billed separately under OSS pricing
- Log storage costs (for retained SQL Explorer logs) are billed separately from API calls