# opensearch-index

Part of **OPENSEARCH**

# OpenSearch Index and Data Management Console Guide

## Operations Overview

| Operation | Console Navigation Path | Prerequisites | Description |
|----------|------------------------|---------------|-------------|
| Create Index Table | Configuration Center > Index Schema | A data source must be available; at least one primary key field defined; one or more fields selected for search results | Define and create a new index table schema with fields, indexes, and versioning |
| Configure Subdocument Index | Configuration Center > Index Schema → Advanced Configurations → O&M Center > O&M Management | A data source associated with the index table; administrator privileges for Administrator Mode | Set up indexing for subdocuments using JSON schema and processor chains |
| Modify Index Schema | O&M Management > Change Index Table | Existing index table; permissions to modify and reindex | Update an existing index schema through a three-stage workflow: edit, configure, reindex |
| Modify Index Table | O&M Management > Modify Index Table | Existing index table; access to O&M page; publish and rebuild permissions | Make configuration changes to an index table and trigger full reindexing |
| Restore Data from Index | O&M Center > O&M Management > Restore Data from Index | API data source with completed full indexing; all fields configured as attribute or search result fields | Recover data from a previous index version during full indexing |
| Configure OBJECT Field Type | Instances > Configure Index Schema | Running OpenSearch instance; appropriate console permissions | Enable indexing of nested JSON data using the OBJECT field type |
| Commit Documents Batch | Applications > [Application Name] > Data Management | Configured application and table; valid credentials; SDK environment | Stage and commit batch document operations (ADD/UPDATE/DELETE) via SDK buffer |
| Pause Specific Cluster Operations | O&M Center > O&M Management | Active OpenSearch instance; console access | Temporarily halt specific cluster operations using pause switches |

## Step-by-Step Instructions

### Create Index Table

**Navigation**: Console > Configuration Center > Index Schema

**Prerequisites**:
- A data source must be available before adding an index table
- At least one primary key field must be defined
- One or more fields must be selected for display in search results

1. Click **Add Index Table**  
   - Element: **Add Index Table** (button) — top-right corner of the Index Schema page

2. Set the Index Table name, select a Data Source, and set the Data Shards  
   - Element: **Index Table** (text_input) — main content area

3. Configure fields including field type, compression, and multi-value settings  
   - Element: **Fields** (text_input) — main content area  
   - Notes: For multi-value fields, only single-character separators are supported; full-width characters are not allowed.

4. Configure index settings such as index type and compression  
   - Element: **Index** (text_input) — main content area  
   - Notes: Primary key indexes do not support compression.

5. Click **Save Version**, enter a remark (optional), and click **Publish**  
   - Element: **Publish** (button) — dialog box  
   - Notes: After publishing, the new version is marked as 'Not in Use'.

6. Go to Operation Center > Operations Management > Update Configuration and execute **Push Configuration** and **Trigger Index Rebuild**  
   - Element: **Push Configuration** (button) — Update Configuration page  
   - Notes: This step is required to apply the new index table to the cluster.

7. Monitor the progress of the full data build in Operation Center > Change History > **Data Source Change**  
   - Element: **Data Source Change** (tab) — Change History section

8. Edit an existing index table by finding the version with 'Editing' status and clicking **Edit**  
   - Element: **Edit** (button) — version list  
   - Notes: Only one editing version can exist at a time.

9. After making changes, click **Save Version** and then **Publish** with a remark  
   - Element: **Publish** (button) — dialog box  
   - Notes: The system generates a new version with 'Not in Use' status.

10. Apply the updated version by executing **Push Configuration** and **Trigger Index Rebuild**  
    - Element: **Trigger Index Rebuild** (button) — Update Configuration page

11. Delete an index table version that has 'Not in Use' status  
    - Element: **Delete** (button) — version list  
    - Notes: You cannot delete a version with 'In Use' status directly.

12. To delete an index table with an 'In Use' version, first unsubscribe it in Deployment Management, then delete it from Index Schema  
    - Element: **Unsubscribe** (button) — Deployment Management  
    - Notes: Failure to delete the index table after unsubscribing may affect the online cluster.

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Index Table | text_input | Yes | — | The name of the index table. Cannot be modified after creation. |
| Data Source | dropdown | Yes | — | Select the data source for the index table. Required before creating an index table. |
| Data Shards | number_input | Yes | — | Number of data shards for the index table. Must ensure sufficient data nodes based on replicas. |
| Field Name | text_input | Yes | — | Name of the field in the index table. |
| Field Type | dropdown | Yes | INT8, UINT8, INT16, UINT16, INTEGER, UINT32, INT64, UINT64, FLOAT, DOUBLE, STRING, TEXT | Specifies the data type of the field. TEXT fields require an analyzer. |
| Multi-value | checkbox | No | True, False | Enables multiple values for a field. Multi-value attributes do not support updates. |
| Multi-value Field Separator | text_input | No | — | Separator used for multi-value fields. Only single-character separators are supported. Default: ^] |
| Compression | dropdown | No | None, file_compressor | Enables compression for attribute or field content. Default is disabled. |
| Index Type | dropdown | Yes | PRIMARYKEY64, STRING, TEXT | Type of index to create. Primary key indexes must be unique and cannot be compressed. |
| Remark | text_input | No | — | Optional description for the version. Required when publishing. |

### Configure Subdocument Index

**Navigation**: Console > Configuration Center > Index Schema → Configuration Center > Advanced Configurations → O&M Center > O&M Management

**Prerequisites**:
- A data source must be associated with each index table
- The instance must have administrator privileges to configure in Administrator Mode

1. Navigate to the **Index Schema** page  
   - Element: **Configuration Center > Index Schema** (link) — left-side navigation pane

2. Set the **Mode** parameter to Administrator Mode and configure index settings  
   - Element: **Mode** (dropdown) — main content area  
   - Notes: Specify Index Table, Data Source, and Data Shards parameters

3. Enter subdocument code in **Developer Mode**  
   - Element: **Developer Mode** (tab) — main content area  
   - Notes: Use the provided JSON sample code for sub_schema

4. Click **Publish** to save the configuration  
   - Element: **Publish** (button) — bottom of the form

5. Go to **Advanced Configurations** and view data_tables directory  
   - Element: **Configuration Center > Advanced Configurations** (link) — left-side navigation pane

6. Click **View** in the Actions column to access the data_tables directory  
   - Element: **View** (link) — Actions column

7. Click **Modify** to edit the xxx_table.json file  
   - Element: **Modify** (button) — top of the file editor

8. Add subdocument processor chain configuration in the file  
   - Element: **processor_chain_config** (text_input) — JSON editor  
   - Notes: Replace ${table_name} and ${clusters} with actual values from the Index Schema page

9. Click **Publish** on the Data Processing Configuration tab  
   - Element: **Publish** (button) — top of the tab

10. Go to **O&M Center > O&M Management** and update configurations  
    - Element: **O&M Center > O&M Management** (link) — left-side navigation pane

11. Select the new data processing configuration version and trigger reindexing  
    - Element: **Push Configurations and Trigger Reindexing** (checkbox) — Instance Configuration Update panel

12. View reindexing progress on the **Data Source Change Tab**  
    - Element: **Data Source Change Tab** (tab) — Change History page

13. Modify the qrs.json file for query configuration  
    - Element: **Query Configurations tab** (tab) — Advanced Configurations page

14. Click **Modify** to edit the qrs.json file  
    - Element: **Modify** (button) — Actions column

15. Add subdocument configurations to the **extend_processors** parameter  
    - Element: **extend_processors** (text_input) — JSON editor  
    - Notes: Include the sample JSON code with SubDocIdName, searcher_schema_path, etc.

16. Click **Publish** on the Query Configurations tab  
    - Element: **Publish** (button) — top of the tab

17. Update online configurations using the new query version  
    - Element: **Update Configurations** (button) — O&M Management page

18. Select the new query configuration version and click **OK**  
    - Element: **OK** (button) — Instance Configuration Update panel

19. View the update progress on the **Cluster Changes tab**  
    - Element: **Cluster Changes tab** (tab) — Change History page

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Mode | dropdown | Yes | Administrator Mode, Developer Mode | Specifies the configuration mode for the index schema |
| Index Table | text_input | Yes | — | The name of the index table to be created |
| Data Source | text_input | Yes | — | The data source that the index table corresponds to |
| Data Shards | number_input | Yes | — | Number of shards for the index table |
| Trigger Reindexing | checkbox | No | Push Configurations and Trigger Reindexing | Enables reindexing after configuration update |
| Configuration Type | dropdown | Yes | Online Configurations, Offline Configurations | Type of configuration to apply |
| Query Configuration Version | dropdown | Yes | — | Selects the version of query configuration to deploy |

### Modify Index Schema

**Navigation**: Console > O&M Management > Change Index Table

**Prerequisites**:
- An OpenSearch Retrieval Engine Edition application with at least one index table
- Permissions required to modify index configurations and trigger reindexing

1. Click on the **Change Index Table** button  
   - Element: **Change Index Table** (button) — O&M Management page

2. Find the target index table and click **View/Modify** in the Actions column  
   - Element: **View/Modify** (button) — Actions column

3. Make changes in the index_config_edit version, then save and publish the version  
   - Element: **Save Version** (button) — Edit schema section  
   - Notes: After publishing, a new version like index_config_v2 is generated. Version numbers increment with each publish.

4. Click **Next** to proceed to the next step  
   - Element: **Next** (button) — Bottom of the page

5. Configure parameters such as Data Source Name, Index Table, Index Table Version, Dictionary Configuration Version, and Destination Cluster  
   - Element: **Next** (button) — Bottom of the page

6. Select a data source type and choose the appropriate option based on the scenario  
   - Element: **MaxCompute Data Source** (radio) — Data Source Type selection  
   - Notes: For API Data Source, options include 'Empty Data' or 'Restore Data from Index'.

7. Click **Next** to start reindexing  
   - Element: **Next** (button) — Bottom of the page

8. Monitor progress on the **Data Source Changes** tab of the **Change History** page  
   - Element: **Change History** (link) — Top navigation  
   - Notes: The index table becomes queryable immediately after reindexing completes.

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Data Source Name | text_input | Yes | — | The name of the data source to use for reindexing |
| Index Table | text_input | Yes | — | The index table to update |
| Index Table Version | text_input | Yes | — | The target version of the index table |
| Dictionary Configuration Version | text_input | Yes | — | The version of the dictionary configuration to apply |
| Destination Cluster | text_input | Yes | — | The cluster where the reindexed data will be written |
| Data Source Type | dropdown | Yes | MaxCompute Data Source, API Data Source | Specifies the type of data source for reindexing |
| Data Source | dropdown | Yes | Empty Data, Restore Data from Index | Specifies whether to start from empty data or restore from an existing index |

### Modify Index Table

**Navigation**: Console > Elasticsearch > O&M Management > Modify Index Table

**Prerequisites**:
- An existing index table to modify
- Access to the O&M management page
- Permissions to publish new versions and rebuild indexes

1. Click **Modify Index Table** on the O&M management page  
   - Element: **Modify Index Table** (button) — main content area

2. Find the target index table and click **View/Modify** in the Actions column  
   - Element: **View/Modify** (link) — Actions column

3. After editing the schema, click **Save Version and Publish**  
   - Element: **Save Version and Publish** (button) — bottom of form  
   - Notes: This creates a new version like index_config_v4 with incremented ordinal number

4. Click **Next** to proceed to update configuration  
   - Element: **Next** (button) — bottom of page

5. Configure update settings: select data source, index table, new version, dictionary version, and destination cluster  
   - Element: **Next** (button) — bottom of page

6. Select the corresponding data source and click **Next**  
   - Element: **Next** (button) — bottom of page  
   - Notes: The behavior varies based on data source type: MaxCompute or API Push with different data origin options

7. Wait for the index rebuild process to complete and view progress under **Data Source Changes**  
   - Element: **Data Source Changes** (tab) — change history section  
   - Notes: Page automatically redirects after rebuild starts

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Data Source Type | dropdown | Yes | MaxCompute Data Source, API Push Data Source | Specifies the source system from which data is ingested |
| Data Origin | dropdown | No | Empty Data, Recover data from index | Determines whether to start with empty data or restore from existing index |
| Timestamp | text_input | Yes | — | The timestamp used for data ingestion; must not be later than the push time from API data source |

### Restore Data from Index

**Navigation**: Instance Management > O&M Center > O&M Management > Restore Data from Index

**Prerequisites**:
- An API data source must have completed full indexing at least once
- All fields must be configured as attribute fields or search result fields

1. Navigate to O&M Center > O&M Management and click **Restore Data from Index**  
   - Element: **Restore Data from Index** (button) — left navigation panel

2. Select the target data source from the list in the dialog box  
   - Element: **radio button for the target data source** (radio) — main content area  
   - Notes: Only API data sources are displayed. You can restore data from only one data source at a time.

3. Click **OK** to submit the request  
   - Element: **OK** (button) — bottom of dialog box  
   - Notes: If conditions are not met, fields not marked as attribute or search result fields will be cleared.

4. Monitor the restore progress on the **Data Source Changes** tab in Change History  
   - Element: **Data Source Changes** (tab) — top navigation of O&M Center  
   - Notes: The Recover FSM in Index process includes stages like init, create snapshot, generate data, check gen_data status, prepare bs build data, bs build check, update finished time, and kill build task. A green checkmark appears after each stage completes.

5. After completion, perform other operations requiring full indexing with **Restore Data from Index** selected  
   - Element: **Details** (link) — in the progress table  
   - Notes: Click Details to view execution details for each stage of the workflow.

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Data Source | dropdown | Yes | — | Lists available API data sources that have completed full indexing. |
| Index Table | text_input | Yes | — | Displays the index table associated with the selected data source. |
| Index Version to Restore | text_input | Yes | — | Shows the specific index version to be restored. |

### Configure OBJECT Field Type

**Navigation**: Console > OpenSearch > Instances > Configure Index Schema

**Prerequisites**:
- An OpenSearch instance must be created and running
- Access to the OpenSearch console with appropriate permissions

1. Navigate to the index configuration step  
   - Element: **Configure Index Schema** (button) — main content area

2. Select **OBJECT** as the field type for the target field  
   - Element: **OBJECT** (dropdown) — field type selection panel

3. Click the **Schema Configuration** option  
   - Element: **Schema Configuration** (button) — top-right corner  
   - Notes: This option appears after selecting OBJECT type.

4. Enter the complete JSON schema for the field structure  
   - Element: **Schema Editor** (text_input) — center panel  
   - Notes: Paste the full JSON schema as shown in the sample. Include nested fields like 'tags' and set 'is_nested' to true if needed.

5. Save the changes to the schema  
   - Element: **Save** (button) — bottom of the form

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Field Type | dropdown | Yes | STRING, INTEGER, FLOAT, BOOLEAN, OBJECT, ARRAY | Specifies the data type of the field. Select OBJECT to store nested structured data. |
| Is Nested | checkbox | No | — | When enabled, each nested object is indexed and queried independently. Required for accurate querying of nested arrays. |
| Analyzer | dropdown | No | standard, whitespace, keyword, simple, stop, pattern | Defines how text in the field is analyzed during indexing. Not supported for OBJECT type fields. |
| Multi Value | checkbox | No | — | Determines whether the field can contain multiple values. Ignored for non-nested fields; treated as true. |

### Commit Documents Batch

**Navigation**: Console > OpenSearch > Applications > [Application Name] > Data Management

**Prerequisites**:
- An OpenSearch application with a configured table
- The application name (appName), table name (tableName), and OpenSearch API endpoint (host) for your region
- An AccessKey ID and AccessKey secret from a RAM user with required OpenSearch permissions

1. Set environment variables for credentials  
   - Element: **export ALIBABA_CLOUD_ACCESS_KEY_ID=... export ALIBABA_CLOUD_ACCESS_KEY_SECRET=...** (text_input) — terminal/command line  
   - Notes: Replace placeholders with actual RAM user credentials. On Windows, create an environment variable file and restart the system.

2. Build document data as a Map object  
   - Element: **Map<String, Object> doc1 = Maps.newLinkedHashMap();** (text_input) — Java code editor  
   - Notes: Include all required fields such as 'id', 'name', 'phone', arrays, etc., using appropriate data types.

3. Stage the document using **add()**, **update()**, or **remove()**  
   - Element: **documentClient1.add(doc1)** (button) — Java code editor  
   - Notes: This stages the operation in the client buffer without sending it to OpenSearch yet.

4. Submit all buffered documents with **commit(appName, tableName)**  
   - Element: **documentClient1.commit(appName, tableName)** (button) — Java code editor  
   - Notes: After calling commit, wait at least 10 seconds before querying due to asynchronous indexing.

5. Query OpenSearch to verify the result  
   - Element: **searcherClient2.execute(searchParams)** (button) — Java code editor  
   - Notes: Use primary key query to confirm document was added, updated, or deleted.

### Pause Specific Cluster Operations

**Navigation**: Console > O&M Center > O&M Management

**Prerequisites**:
- Access to the OpenSearch console
- An active OpenSearch instance

1. Log on to the **OpenSearch console**  
   - Element: **OpenSearch console** (link) — top navigation

2. Select the instance that you want to manage  
   - Element: **instance** (dropdown) — main content area

3. Choose **O&M Center > O&M Management** in the left-side navigation pane  
   - Element: **O&M Center &gt; O&M Management** (menu) — left navigation panel

4. Click **Pause Switch** on the O&M Management page  
   - Element: **Pause Switch** (button) — main content area

5. Turn on or turn off the specific pause switches based on your requirements  
   - Element: **Pause Switch panel** (text_input) — center of the page  
   - Notes: Each switch controls a different layer of the indexing pipeline.

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Pause Full Index Update | checkbox | No | — | Blocks the new full index version from being applied to the online cluster while reindexing continues. |
| Pause Real-time Data Indexing | checkbox | No | — | Prevents indexes from being applied to the online cluster, but data continues to be pushed via API. |
| Pause Incremental Data Indexing | checkbox | No | — | Stops incremental indexes from being applied to Searcher workers. |
| Pause Configuration | checkbox | No | — | Prevents configuration changes from being applied to the cluster. |
| Pause All | checkbox | No | — | Pauses all the above operations simultaneously. |

## FAQ

Q: Where do I find the option to create a new index table?
A: Navigate to Configuration Center > Index Schema and click the "Add Index Table" button in the top-right corner.

Q: What happens if I leave the "Remark" field empty when publishing an index version?
A: The "Remark" field is optional when saving a version, but you must provide a remark when publishing. Leaving it empty during publish will prevent the action from completing.

Q: Can I modify the Index Table name after creation?
A: No. The Index Table name cannot be modified after creation. You would need to create a new index table if a name change is required.

Q: What permissions do I need to modify an index schema and trigger reindexing?
A: You need permissions to modify index configurations in the Configuration Center and to execute operations in the O&M Management section, including "Push Configuration" and "Trigger Index Rebuild".

Q: Why can't I delete an index table version that is currently in use?
A: Versions with "In Use" status cannot be deleted directly. You must first unsubscribe the index table in Deployment Management, then return to Index Schema to delete it.

## Pricing & Billing

### Billing Model
per_instance_hour

### Price Reference
| Tier | Input Price | Output Price |
|------|-------------|--------------|
| Standard | 0.01 / | 0.01 / |
| Standard | 0.05 / | 0.05 / |
| Standard | 0.08 / | — |

### Free Tier

### Billing Notes

- Index rebuilds are billed during execution time; minimum charge is 1 hour.
- Reindexing operations are billed per request. The free tier applies to all index modifications.