# opensearch-vector

Part of **OPENSEARCH**

# OpenSearch Vector Search Console Guide

## Operations Overview

| Operation | Console Navigation Path | Prerequisites | Description |
|----------|------------------------|---------------|-------------|
| Perform Vector Operations | Console > OpenSearch > Vector Search Edition > Query Test | Running OpenSearch Vector Search Edition instance; appropriate permissions | Execute primary key-based, vector-based, or hybrid queries using a codeless UI with support for text, images, and developer mode |
| Insert or Delete Data | Console > OpenSearch > Vector Search > Data Management | Running OpenSearch Vector Search instance; Data Management access | Insert or delete vector data via table or raw JSON input, with real-time metrics and verification |
| Lock Vector Index in Memory | Console > OpenSearch > Deployment Management > Searcher Worker Configurations | OpenSearch Retrieval Engine Edition instance with ≥1 Searcher worker; sufficient free memory | Configure mmap-based memory locking to reduce query latency by keeping vector indexes in RAM |
| Create Vector Image Search Engine | Console > OpenSearch > Retrieval Engine Edition > Create Instance > Configure Table | Alibaba Cloud account; OSS bucket with images; OpenSearch Retrieval Engine Edition instance | Set up an end-to-end image search engine using MaxCompute + OSS + OpenSearch architecture |
| Configure Vector Table for Text Search | Console > OpenSearch > Instances > Configure Table | OpenSearch Vector Search instance in 'Pending Configuration' status; MaxCompute project/table ready | Define table schema, field types, vector models (e.g., ops-text-embedding-000), and indexes for text vector search |
| Configure Vector Index Advanced Settings | Console > Elasticsearch > Index Management > Create Index > Index Settings | Elasticsearch cluster with vector search enabled; knowledge of embedding model dimensions | Fine-tune vector index parameters including distance_type, vector_index_type (Qc/HNSW/Linear), quantization, and real-time indexing |
| Configure Quantized Clustering | Console > OpenSearch > Index Management > Configure Quantized Clustering | OpenSearch Retrieval Engine Edition instance; existing index supporting QC | Optimize quantized clustering (QC) performance via QcBuilder and QcSearcher parameters like centroid_count and scan_ratio |

## Step-by-Step Instructions

### Perform Vector Operations

**Navigation**: Console > OpenSearch > Vector Search Edition > Query Test

**Prerequisites**:
- An OpenSearch Vector Search Edition instance must be created and running
- Access to the OpenSearch console with appropriate permissions

1. Navigate to the **Query Test** page  
   - Element: **Query Test** (link) — left navigation panel

2. Select the query type from the dropdown  
   - Element: **Query Type** (dropdown) — main content area  
   - Notes: Options include 'Primary Key', 'Vector', and 'Hybrid'

3. Upload text or image data for vectorization (if needed)  
   - Element: **Upload File** (button) — top-right corner  
   - Notes: Supports text files and image files for automatic embedding

4. Enter query parameters in the form fields  
   - Element: **Query Parameters** (text_input) — main content area  
   - Notes: Includes fields for search criteria and similarity thresholds

5. Switch to developer mode for raw JSON query editing (optional)  
   - Element: **Developer Mode** (toggle) — top-right corner  
   - Notes: Enables direct input of HA syntax or raw JSON queries

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Query Type | dropdown | Yes | Primary Key, Vector, Hybrid | Specifies the type of query to execute |
| Vector Threshold | number_input | No | — | Sets the similarity threshold for vector-based matching (default: 0.7) |
| Max Results | number_input | No | — | Limits the number of returned results (default: 10) |

### Insert or Delete Data

**Navigation**: Console > OpenSearch > Vector Search > Data Management

**Prerequisites**:
- An OpenSearch Vector Search instance must be created and running
- User must have permissions to access the Data Management page

1. Navigate to the **Data Management** page  
   - Element: **Data Management** (link) — left navigation panel

2. Click the **Insert Data** button to begin data insertion  
   - Element: **Insert Data** (button) — top-right corner  
   - Notes: Available in both Table Mode and Developer Mode

3. Enter data using table fields or switch to Developer Mode for raw JSON  
   - Element: **Table Fields / Developer Mode Toggle** (tab) — main content area  
   - Notes: In Developer Mode, paste JSON-formatted documents for bulk insertion

4. Submit the data to the index  
   - Element: **Submit** (button) — bottom of the form  
   - Notes: After submission, real-time document count appears on the Table Metrics page

5. Verify inserted data via query test  
   - Element: **Query Test** (link) — main content area  
   - Notes: Run sample queries to confirm successful indexing and searchability

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Index Name | dropdown | Yes | default_index, test_index_01, prod_index_v1 | Select the target index for data insertion |
| Document ID | text_input | No | — | Optional unique ID; UUID generated if omitted |
| Data Content | text_input | Yes | — | JSON-formatted data (single or multiple documents) |

### Lock Vector Index in Memory

**Navigation**: Console > OpenSearch > Deployment Management > Searcher Worker Configurations

**Prerequisites**:
- An OpenSearch Retrieval Engine Edition instance with at least one Searcher worker
- Enough free memory on the Searcher worker to hold the vector indexes you want to lock

1. Click a Searcher worker on the **Deployment Management** page  
   - Element: **Deployment Management** (link) — main content area

2. Open the **Online Table Configurations** tab  
   - Element: **Online Table Configurations** (tab) — Searcher Worker Configurations panel

3. Select an index table and click **Modify**  
   - Element: **Index Table** (dropdown) — Searcher Worker Configurations panel

4. Replace the `load_config` section with the provided JSON configuration  
   - Element: **Modify** (button) — Searcher Worker Configurations panel  
   - Notes: Replace `${Vector index name}` with the actual index name; ensure all parameters are correct

5. Confirm the rolling upgrade starts after configuration change  
   - Element: **Online Configurations** (tab) — top navigation panel  
   - Notes: For production clusters, ensure minimum service ratio > 0 and sufficient workers during upgrade

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Index Table | dropdown | Yes | — | Selects the index table to configure for memory locking |

### Create Vector Image Search Engine

**Navigation**: Console > OpenSearch > Retrieval Engine Edition > Create Instance > Configure Table

**Prerequisites**:
- An Alibaba Cloud account with AccessKey ID and AccessKey Secret
- An OSS bucket with images uploaded (e.g., /test/images/ in test-image-vector bucket)
- An OpenSearch Retrieval Engine Edition instance

1. Click **Add Data Source** and select MaxCompute  
   - Element: **Add Data Source** (button) — main content area

2. Fill in MaxCompute connection details  
   - Element: **project** (text_input) — form fields

3. Enable **Automatic Index Rebuild** (optional)  
   - Element: **Automatic Index Rebuild** (checkbox) — form fields

4. Confirm the configuration  
   - Element: **Confirm** (button) — bottom of form  
   - Notes: Monitor progress in Function Extension > Change History

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Table Name | text_input | Yes | — | The name of the table to be created |
| Number of Shards | number_input | Yes | — | The number of shards for the table |
| Number of Data Update Resources | number_input | Yes | — | Beyond 2 are charged |
| Project | text_input | Yes | — | The MaxCompute project name |
| accessKeyId | text_input | Yes | — | The AccessKey ID for MaxCompute access |
| accessKeySecret | text_input | Yes | — | The AccessKey Secret for MaxCompute access |
| Table | text_input | Yes | — | The MaxCompute table name |
| partition key | text_input | No | — | The partition key for the MaxCompute table |

### Configure Vector Table for Text Search

**Navigation**: Console > OpenSearch > Instances > Configure Table

**Prerequisites**:
- An OpenSearch Vector Search instance has been purchased and is in 'Pending Configuration' status
- AccessKey ID and Secret for Alibaba Cloud account or RAM user
- MaxCompute project and table name ready for data source connection

1. Navigate to the **Instance List** page  
   - Element: **Instance List** (link) — top navigation panel

2. Locate the instance with status 'Pending Configuration'  
   - Element: (text) — main content area

3. Click **Configure** in the Actions column  
   - Element: **Configure** (button) — Actions column

4. Enter table name and set shard/update resource counts  
   - Element: **Table name** (text_input) — Basic table information section  
   - Notes: Two free data update resources; additional ones are billed

5. Select MaxCompute + API data source and enter credentials  
   - Element: **AccessKey ID** (text_input) — Data synchronization section  
   - Notes: Partition key required (e.g., ds=20170626)

6. Configure field settings: set `id` as primary key (INT64), `vector_source_text` as STRING, and `vector` as multi-value STRING with comma delimiter  
   - Element: **Primary Key** (checkbox) — Field configuration section

7. Set vector model parameters  
   - Element: **vector_model** (dropdown) — Advanced settings > Configure  
   - Notes: Models include ops-text-embedding-000, ops-text-embedding-en-000, etc.

8. Create primary key and vector indexes  
   - Element: **Primary Key Index** (tab) — Index settings section  
   - Notes: Vector index must be CUSTOMIZED type and include primary key + vector fields

9. Set dimension to match the selected model (e.g., 768)  
   - Element: **dimension** (text_input) — Advanced settings > View/Modify

10. Finalize table creation  
    - Element: **Confirm Creation** (button) — bottom of form

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Table name | text_input | No | — | Custom name for the table |
| Number of data shards | number_input | Yes | — | Positive integer up to 256 |
| Number of data update resources | number_input | Yes | — | Two free; additional billed |
| AccessKey ID | text_input | Yes | — | Alibaba Cloud or RAM user AccessKey ID |
| AccessKey Secret | text_input | Yes | — | Corresponding AccessKey Secret |
| Project | text_input | Yes | — | Target MaxCompute project name |
| Table | text_input | Yes | — | Target MaxCompute table name |
| Partition | text_input | Yes | ds=20170626 | Partition key for MaxCompute |
| Automatic index rebuilding | checkbox | No | Enabled, Disabled | Rebuild index when data source changes |
| Field type | dropdown | Yes | INT64, STRING | Type for the field |
| Primary Key | checkbox | No | — | Marks field as primary key |
| Multi-value | checkbox | No | — | Allows multiple values |
| Delimiter | text_input | No | , | Separator for multi-value fields |
| vector_model | dropdown | Yes | ops-text-embedding-000, ops-text-embedding-en-000, ops-text-embedding-1024-000-20231001, ops-text-embedding-512-000-20231001, ops-text-embedding-128-000-20231001, ops-text-embedding-512-en-000-20231001, ops-text-embedding-128-en-000-20231001 | Model for text-to-vector conversion |
| vector_modal | dropdown | Yes | text | Type of vector (always 'text' for text vectors) |
| vector_source_field | dropdown | Yes | vector_source_text | Field to be vectorized |
| Index type | dropdown | Yes | PRIMARYKEY64, CUSTOMIZED | Type of index to create |
| Dimension | number_input | Yes | 768 | Must match vector model output dimension |

### Configure Vector Index Advanced Settings

**Navigation**: Console > Elasticsearch > Index Management > Create Index > Index Settings

**Prerequisites**:
- An Elasticsearch cluster with vector search capability enabled
- Understanding of embedding models and their output dimensions
- Knowledge of the data format used for vector embeddings

1. Open the **Index Settings** tab  
   - Element: **Index Settings** (tab) — main content area

2. Set vector dimension  
   - Element: **dimension** (text_input) — Index Settings form  
   - Notes: Must be integer > 1; match embedding model output

3. Select distance metric  
   - Element: **distance_type** (dropdown) — Index Settings form  
   - Notes: Options: SquareEuclidean, InnerProduct

4. Choose indexing algorithm  
   - Element: **vector_index_type** (dropdown) — Index Settings form  
   - Notes: Options: Qc, HNSW, Linear

5. Enable real-time indexing if needed  
   - Element: **enable_rt_build** (toggle) — Index Settings form  
   - Notes: If enabled with quantization, set is_embedding_saved = true

6. Configure offline build parameters  
   - Element: **build_index_params** (text_input) — Index Settings form  
   - Notes: Use defaults unless custom optimization needed

7. Set search parameters  
   - Element: **search_index_params** (text_input) — Index Settings form  
   - Notes: Default: {"proxima.qc.searcher.scan_ratio": 0.01}

8. Specify vector delimiter  
   - Element: **embedding_delimiter** (text_input) — Index Settings form  
   - Notes: Default is comma (',')

9. Select matrix storage order  
   - Element: **major_order** (dropdown) — Index Settings form  
   - Notes: col = column-major, row = row-major

10. Set LinearBuilder threshold  
    - Element: **linear_build_threshold** (number_input) — Index Settings form  
    - Notes: Default: 5000 documents

11. Adjust minimum scan count  
    - Element: **min_scan_doc_cnt** (number_input) — Index Settings form  
    - Notes: Default: 20000

12. Enable recall reporting  
    - Element: **enable_recall_report** (toggle) — Index Settings form  
    - Notes: Enabled by default

13. Decide whether to store original vectors  
    - Element: **is_embedding_saved** (toggle) — Index Settings form  
    - Notes: Set to true for INT8/FP16 quantization with real-time retrieval

14. Skip invalid documents during indexing  
    - Element: **ignore_invalid_doc** (toggle) — Index Settings form  
    - Notes: Enabled by default

15. Confirm indexer plugin  
    - Element: **indexer** (text_input) — Index Settings form  
    - Notes: Must be set to aitheta2_indexer

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| dimension | number_input | Yes | — | Vector dimension; must match embedding model |
| distance_type | dropdown | Yes | SquareEuclidean, InnerProduct | Distance metric for similarity |
| vector_index_type | dropdown | Yes | Qc, HNSW, Linear | Indexing algorithm |
| enable_rt_build | toggle | No | true, false | Enable real-time indexing |
| rt_index_params | text_input | No | {"proxima.oswg.streamer.segment_size": 2048} | Real-time index builder params |
| build_index_params | text_input | No | See Build index parameters section | Offline index builder params |
| search_index_params | text_input | No | {"proxima.qc.searcher.scan_ratio": 0.01} | Index searcher params |
| embedding_delimiter | text_input | No | , | Delimiter for vector strings |
| major_order | dropdown | No | col, row | Matrix storage order |
| linear_build_threshold | number_input | No | 5000 | Threshold to use LinearBuilder |
| min_scan_doc_cnt | number_input | No | 20000 | Min retrieval candidate sets |
| enable_recall_report | toggle | No | true, false | Report recall rate |
| is_embedding_saved | toggle | No | true, false | Store original vectors |
| ignore_invalid_doc | toggle | No | true, false | Skip invalid vector docs |
| indexer | text_input | Yes | aitheta2_indexer | Plug-in for vector index building |

### Configure Quantized Clustering

**Navigation**: Console > OpenSearch > Index Management > Configure Quantized Clustering

**Prerequisites**:
- OpenSearch Retrieval Engine Edition instance deployed
- Index already created and ready for configuration

1. Navigate to **Index Management**  
   - Element: **Index Management** (link) — left navigation panel

2. Select target index and click **Configure Quantized Clustering**  
   - Element: **Configure Quantized Clustering** (button) — main content area  
   - Notes: Button appears only if index supports QC

3. Set training sample count  
   - Element: **train_sample_count** (text_input) — QcBuilder configuration form  
   - Notes: Enter 0 to use all documents

4. Adjust build thread count  
   - Element: **thread_count** (text_input) — QcBuilder configuration form  
   - Notes: Set to 0 to auto-detect CPU cores

5. Define centroid count for hierarchical clustering  
   - Element: **centroid_count** (text_input) — QcBuilder configuration form  
   - Notes: Use format like '100*100'; leave blank for auto-inference

6. Select quantizer class  
   - Element: **quantizer_class** (dropdown) — QcBuilder configuration form  
   - Notes: Options: Int8QuantizerConverter, HalfFloatConverter, DoubleBitConverter

7. Enable quantize-by-centroid (if using Int8QuantizerConverter)  
   - Element: **quantize_by_centroid** (checkbox) — QcBuilder configuration form

8. Set query-time scan ratio  
   - Element: **scan_ratio** (text_input) — QcSearcher configuration form  
   - Notes: Higher = better recall but higher latency

9. Set brute force threshold  
   - Element: **brute_force_threshold** (text_input) — QcSearcher configuration form  
   - Notes: Below this doc count, use linear search instead of clustering

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| train_sample_count | number_input | No | 0 | Docs used for training; 0 = all |
| thread_count | number_input | No | 0 | Build threads; 0 = auto-detect CPU |
| centroid_count | text_input | No | — | Centroids per level (e.g., '100*100') |
| quantizer_class | dropdown | No | Int8QuantizerConverter, HalfFloatConverter, DoubleBitConverter | Vector quantizer |
| quantize_by_centroid | checkbox | No | — | Quantize relative to centroid (Int8 only) |
| scan_ratio | number_input | No | 0.01 | Fraction of docs scanned per query |
| brute_force_threshold | number_input | No | 1000 | Doc count threshold for linear search |

## FAQ

Q: Where can I test vector queries without writing code?
A: Use the **Query Test** page under Console > OpenSearch > Vector Search Edition. It provides a codeless UI for primary key, vector, and hybrid queries with support for text and image uploads.

Q: Can I modify vector index settings after creation?
A: Some settings (like dimension or vector_model) require reindexing and cannot be changed directly. Others (like search_index_params or scan_ratio) can be updated via the index configuration panels without full rebuild.

Q: What happens if I leave the Document ID field empty during data insertion?
A: The system automatically generates a UUID for the document. You can still retrieve it later using vector or hybrid queries, but not by primary key unless you track the generated ID.

Q: Do I need to manually lock vector indexes in memory for low-latency queries?
A: Yes, for high-throughput workloads, you should configure memory locking via the Searcher Worker Configurations panel using mmap-based load policies to keep indexes resident in RAM.

Q: Are data deletion operations billed in the console?
A: No, data deletion operations are not billed. Only data insertion requests and vector queries incur charges based on usage tiers.

## Pricing & Billing

### Billing Model
Billing is primarily **per request** for queries and data insertion, and **per resource** for additional data update resources beyond the free tier.

### Price Reference
- Standard vector queries: 0.0001 / (input), 0.0002 / (output)
- Data insertion: 0.0001 / (input), 0.0002 / (output)
- Image/text vector processing: 0.002 /tokens (input), 0.003 /tokens (output)
- Vector index creation: 0.0001 / (input), 0.0002 / (output)

### Free Tier
- 1000 free vector queries per month
- 1000 free data insertion requests per month
- 100 free vector index creations per month
- 100 free tokens per month for image/text vector processing
- 2 free data update resources per table

### Billing Notes
- Queries in developer mode are billed at the same rate as standard queries
- Async tasks are not supported in the console interface
- Batch insertions exceeding 1000 documents are billed per actual document count
- Additional data update resources beyond two are billed as (n - 2)
- Memory usage for locked indexes may affect overall cost based on instance size
- Quantized clustering and real-time indexing increase compute resource consumption