# es-analytics

Part of **ES**

# Elasticsearch Monitoring and Troubleshooting

## Capabilities Overview

| Sub-capability | Calling Mode | Description |
|----------------|--------------|-------------|
| Query Statistical Reports | Synchronous | Queries statistical reports such as application, drop-down suggestion, top search hint, A/B test, and data quality reports in Elasticsearch. |
| Query Log Statistics | Synchronous | Queries log statistics such as application error logs, hotword rankings, and slow query logs. |
| Manage Slow Query Service | Synchronous | Enables and disables the slow query optimization service for an application group. |
| Analyze Slow Queries | Synchronous | Starts slow query analysis, lists optimization categories, and retrieves detailed slow query information. |
| List Quota Review Tasks | Synchronous | Lists quota approval tickets for a specified OpenSearch application, including resource change details and approval status. |

## API Calling Patterns

### Authentication
The primary authentication method is **Bearer Token**.

- Include the header: `Authorization: Bearer <your_api_key>`
- Store your API key in the environment variable: `DASHSCOPE_API_KEY` (or `ALIYUN_API_KEY` in some examples)
- All API requests must include this header; alternative methods are not documented.

### Service Endpoint
APIs use region-specific endpoints with the pattern:

```text
https://opensearch.{region}.aliyuncs.com/v4/openapi/...
```

Common regions:
- `cn-hangzhou` (China)
- `cn-shanghai` (International)

Some APIs also support a global endpoint:
- `https://api.aliyun.com/v4/openapi/...`
- `https://api.alibabacloud.com/v4/openapi/...`

Use the region-specific endpoint when managing application groups tied to a specific region.

### Synchronous API Pattern
All functions in this domain use **synchronous** calling:

1. Send an HTTP request (GET or POST) to the appropriate endpoint
2. Include required parameters in the URL path, query string, or request body
3. Receive a complete JSON response immediately
4. Parse the `result` field for data and `requestId` for tracing

No polling or streaming is required. Each call returns final results in one response.

## Parameter Reference

### Query Statistical Reports

| Parameter | Type | Required | Default | Constraints | Description |
|----------|------|--------|--------|------------|------------|
| appGroupIdentity | string | Yes | — | — | The name of the application. |
| moduleName | string | Yes | — | one of: app, app-query, abtest, suggest, hot, hint, data-quality, es | The name of the module. |
| startTime | integer | No | — | — | The start timestamp, in seconds. |
| endTime | integer | No | current timestamp | — | The end timestamp, in seconds. |
| pageNumber | integer | No | 1 | range 1 to max | The page number. |
| pageSize | integer | No | 10 | range 1 to 100 | The number of entries per page. |
| columns | string | No | — | comma-separated list of field names | Fields to query (e.g., `pv,uv`). |
| query | string | No | — | format: key1:value1,key2:value2 | Query conditions (keys: experimentSerialNumber, sceneTag, bizType, modelId). |

### Query Log Statistics

| Parameter | Type | Required | Default | Constraints | Description |
|----------|------|--------|--------|------------|------------|
| appGroupIdentity | string | Yes | — | — | The application name. |
| moduleName | string | Yes | — | one of: hot, error, slow-log | The module name. |
| startTime | integer | No | current day 00:00:00 | UNIX timestamp | Start of time range. |
| stopTime | integer | No | current day 24:00:00 | UNIX timestamp | End of time range. |
| pageNumber | integer | No | 1 | range 1 to max | Page number. |
| pageSize | integer | No | 10 | max 100 | Entries per page. |
| query | string | No | — | format: 'field:value' | Query clause content. |
| sortBy | string | No | — | format: '-field' (desc), 'field' (asc) | Sort clause. |
| distinct | boolean | No | false | true or false | Use distinct clause. |
| columns | string | No | — | comma-separated field names | Fields to query (e.g., `wordsTopPv`). |

### Manage & Analyze Slow Queries

| Parameter | Type | Required | Default | Constraints | Description |
|----------|------|--------|--------|------------|------------|
| appGroupIdentity | string | Yes | — | — | The name of the application. |
| start | integer | Yes (for list/category) | — | — | Start UNIX timestamp (seconds). |
| end | integer | Yes (for list/category) | — | — | End UNIX timestamp (seconds). |
| sort | string | No | max | one of: max, avg, sum, queryProportion | Sorting method for categories. |
| categoryIndex | string | Yes (for queries) | — | — | ID of the optimization suggestion. |
| size | integer | No | 10 | — | Number of queries to return. |

### List Quota Review Tasks

| Parameter | Type | Required | Default | Constraints | Description |
|----------|------|--------|--------|------------|------------|
| appGroupIdentity | string | Yes | — | — | The name of the application group. |
| pageNumber | integer | No | 1 | — | Page number. |
| pageSize | integer | No | 10 | — | Entries per page. |

## Code Examples

### Query Statistical Reports - curl - all

```bash
curl -X GET 'https://api.aliyun.com/v4/openapi/app-groups/{appGroupIdentity}/statistic-report/{moduleName}' \
  -H 'Authorization: Bearer $DASHSCOPE_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{"startTime": 1582214400, "endTime": 1582646399, "pageNumber": 1, "pageSize": 10, "columns": "pv,uv", "query": "bizType:test,sceneTag:myTag"}'
```

### Query Log Statistics - Python - all

```python
import requests

url = "https://api.alibabacloud.com/v4/openapi/app-groups/{appGroupIdentity}/statistic-logs/{moduleName}"
headers = {
    "Authorization": "Bearer $API_KEY",
    "Content-Type": "application/json"
}
params = {
    "startTime": 1582214400,
    "stopTime": 1682222400,
    "pageNumber": 1,
    "pageSize": 10
}

response = requests.get(url, headers=headers, params=params)
print(response.json())
```

### Start Slow Query Analysis - bash - all

```bash
curl -X POST https://openapi.aliyun.com/v4/openapi/app-groups/kevintest2/optimizers/slow-query/actions/run \
-H "Authorization: Bearer $ALIYUN_API_KEY" \
-H "Content-Type: application/json" \
-d "{}"
```

### Disable Slow Query Optimization - Python - china

```python
import requests

url = "https://opensearch.cn-hangzhou.aliyuncs.com/v4/openapi/app-groups/kevintest2/optimizers/slow-query/actions/disable"
headers = {
    "Authorization": "Bearer $DASHSCOPE_API_KEY",
    "Content-Type": "application/json"
}
response = requests.post(url, headers=headers)
print(response.json())
```

### List Slow Query Categories - Python - china

```python
import requests

url = "https://opensearch.cn-hangzhou.aliyuncs.com/v4/openapi/app-groups/kevintest2/optimizers/slow-query/categories"
params = {
    "start": 1589986800,
    "end": 1589990340,
    "sort": "max"
}
headers = {
    "Authorization": "Bearer $DASHSCOPE_API_KEY"
}

response = requests.get(url, params=params, headers=headers)
print(response.json())
```

### List Quota Review Tasks - Python - all

```python
import requests

url = "https://api.aliyun.com/v4/openapi/app-groups/demo/quota-review-tasks"
headers = {
    "Authorization": "Bearer $DASHSCOPE_API_KEY",
    "Content-Type": "application/json"
}

response = requests.get(url, headers=headers)
print(response.json())
```

## Response Format

```json
{
  "totalCount": 43,
  "requestId": "F65C8BB2-C14F-5983-888B-41C4E082D3BC",
  "result": [
    {}
  ]
}
```

**Key Fields**:
- `requestId` — Unique identifier for the API request (useful for support)
- `totalCount` — Total number of items matching the query (for paginated responses)
- `result` — Array or object containing the actual data payload
- `result.analyzeStatus` — Status of slow query analysis (e.g., `PENDING`, `COMPLETED`)
- `result.appQuery` — The actual slow query string
- `result[].approved` — Whether a quota review task was approved

## Error Handling

| Error Code | Description | Recommended Action |
|-----------|-------------|-------------------|
| 400 | Bad Request – Invalid request parameters or malformed request. | Validate all required parameters and ensure values match constraints (e.g., correct moduleName enum). |
| 401 | Unauthorized – Missing or invalid authentication credentials. | Verify that `DASHSCOPE_API_KEY` is set and valid; check token expiration. |
| 403 | Forbidden – Insufficient permissions to access the specified AppGroup. | Ensure your account has `opensearch:ListSlowQueryQueries` or equivalent permissions. |
| 404 | Not Found – The requested AppGroup, module, or resource does not exist. | Confirm the `appGroupIdentity` exists and is spelled correctly. |
| 500 | Internal Server Error – An unexpected error occurred on the server side. | Retry after a short delay; contact support if persistent. |
| 503 | Service Unavailable – The service is temporarily unavailable due to overload or maintenance. | Implement exponential backoff and retry later. |

### Rate Limits & Retry
- Most APIs enforce **100 QPS per user or app group**
- Some actions (e.g., enabling slow query) are limited to **10 QPS**
- Failed requests still count toward quota
- Recommended retry strategy: exponential backoff with jitter (e.g., 1s, 2s, 4s, 8s)
- No `Retry-After` header is documented; use fixed backoff intervals

## Environment Requirements

- Set your API key: `export DASHSCOPE_API_KEY=your_api_key_here`
- For Python examples: `pip install requests`
- No specific Python/Java version requirements documented
- SDK usage is optional; raw HTTP calls are fully supported

## FAQ

Q: How do I enable slow query analysis for my application?
A: First call `EnableSlowQuery` to activate the service, then use `StartSlowQueryAnalyzer` to trigger immediate analysis. You can then list categories and queries using the respective APIs.

Q: Why am I getting a 403 error even with a valid API key?
A: Your account may lack permissions for the specific `appGroupIdentity`. Ensure your RAM user or role has the necessary OpenSearch permissions (e.g., `opensearch:ListStatisticReports`).

Q: What time format should I use for startTime and endTime?
A: Use UNIX timestamps in **seconds** (not milliseconds). For example, `1582214400` represents 2020-02-21 00:00:00 UTC.

Q: Are failed API calls billed?
A: Yes. According to billing notes, **all requests are billed**, including those that return 4xx or 5xx errors.

Q: Can I use the same API key across regions?
A: Yes. The Bearer token (`DASHSCOPE_API_KEY`) is global, but you must use the correct regional endpoint for your application group.

## Pricing & Billing

### Billing Model
All APIs use **per-request** billing. Each successful or failed call counts as one billable request.

### Price Reference

| Tier | Input Price | Output Price |
|------|------------|-------------|
| default | 0.001 / | 0.001 / |
| standard | 0.0001 / | 0.0001 / |

### Free Tier
- Most APIs: **1000 free requests per month**
- Enable/Disable slow query: **100 free calls per month**

### Usage Limits
- General: **100 QPS per user or app group**
- Enable/Disable actions: **10 QPS**
- Slow query analysis: **max 1-hour time range per request**
- Category listing: **max 7-day time range per request**

### Billing Notes
- Billing occurs per API call regardless of response size or success
- Free tier resets monthly
- Failed requests consume both quota and billing units