# rds-storage

Part of **RDS**

# ApsaraDB RDS Storage Management

## Capabilities Overview

| Sub-capability | Calling Mode | Description |
|----------------|--------------|-------------|
| Import and Export Raster Data | Synchronous | Imports raster data from OSS, MinIO, or HDFS into RDS, or exports raster objects to external storage systems. |
| Manage Storage Credentials | Synchronous | Sets or retrieves AccessKey ID and secret used for authenticating access to OSS-stored raster objects. |
| Raster Identification and Hashing | Synchronous | Gets or sets unique identifiers (UUID), names, and MD5 checksums for raster objects. |
| Manage Binary Data | Synchronous | Gets, sets, clears, or checks user-defined binary data associated with SFMesh or material objects. |
| Get Object Memory Usage | Synchronous | Returns the memory size (in bytes) of meshgeom, sfmesh, material, or texture objects. |
| Create Texture | Synchronous | Constructs texture objects from image files, binary data, or database references with configurable format and storage options. |
| Convert to Raw Format | Synchronous | Converts spatial objects (texture, material, mesh, geometry) from external storage to internal raw format representation. |

## API Calling Patterns

### Authentication
Authentication for external storage (OSS, MinIO, HDFS) is handled **via URL-embedded credentials**:
- For OSS and MinIO: Include AccessKey ID and secret directly in the URL using the format `OSS://<ak>:<ak_secret>@<endpoint>/<bucket>/<path>` or `MIO://<ak>:<ak_secret>@<host>:<port>/<bucket>/<path>`.
- For HDFS: Include the username in the URL: `HDFS://<user_name>@<host>:<port>/<path>`.
- No additional headers or environment variables are required for these SQL functions — credentials are part of the input URL.

### Service Endpoint
These APIs are **SQL functions executed within ApsaraDB RDS for PostgreSQL** (with GanosBase extension enabled). There is no HTTP endpoint — all operations are performed via SQL queries against your RDS instance.

Common regions where GanosBase is available include:
- `cn-hangzhou`
- `cn-shanghai`
- `cn-beijing`

### Synchronous SQL Function Calls
All storage management operations follow a **synchronous SQL pattern**:
1. Connect to your ApsaraDB RDS PostgreSQL instance using a standard PostgreSQL client (e.g., `psql`, JDBC, psycopg2).
2. Ensure the GanosBase extension is enabled: `CREATE EXTENSION ganos_raster;` (or `ganos_sfmesh` for 3D functions).
3. Call the desired function (e.g., `ST_ImportFrom`, `ST_SetAccessKey`) as part of a `SELECT`, `UPDATE`, or `INSERT` statement.
4. The function executes immediately and returns a result (raster object, boolean, string, integer, etc.) within the same transaction.
5. No polling or async handling is needed — results are available upon query completion.

## Parameter Reference

### Import and Export Raster Data

| Parameter | Type | Required | Default | Constraints | Description |
|----------|------|--------|--------|------------|------------|
| source | raster | Yes | — | — | The raster object to export. |
| format | cstring | Yes | — | One of: BMP, ECW, EHdr, GIF, GPKG, GTiff, HDF4, PDF, PNG, COG | Output format for exported raster. |
| url | cstring | Yes | — | Must follow format: `OSS://<ak>:<ak_secret>@<endpoint>/<bucket>/<path>` or `MIO://<ak>:<ak_secret>@<host>:<port>/<bucket>/<path>` or `HDFS://<user>@<host>:<port>/<path>` | Destination URL for export or source URL for import. |
| level | integer | No | 0 | Range 0 to max pyramid level | Pyramid level to export (default: 0). |
| chunkTableName | cstring | Yes | — | — | Name of the chunk table for imported raster data. |
| storageOption | cstring | No | `{}` | Valid fields: `chunking` (Boolean), `chunkdim` (string), `compression` (string), `quality` (integer), `interleaving` (string), `blockendian` (string), `celltype` (string) | JSON string controlling raster storage layout. |
| importOption | cstring | No | `{}` | Valid fields: `mapping_oss_file` (Boolean), `parallel` (integer), `compute_stats` (Boolean), `approx` (Boolean), `factor` (integer), `exclusive_nodata` (Boolean) | JSON string controlling import behavior. |

### Manage Storage Credentials

| Parameter | Type | Required | Default | Constraints | Description |
|----------|------|--------|--------|------------|------------|
| raster_obj | raster | Yes | — | — | The raster object stored in OSS. |
| id | text | Yes (for ST_SetAKId) / No (for ST_SetAccessKey) | — | — | AccessKey ID for OSS authentication. |
| key | text | Yes (for ST_SetAKSecret) / No (for ST_SetAccessKey) | — | — | AccessKey secret for OSS authentication. |
| valid | bool | No | true | — | Whether to verify credential validity during update. |

### Raster Identification and Hashing

| Parameter | Type | Required | Default | Constraints | Description |
|----------|------|--------|--------|------------|------------|
| raster_obj | raster | Yes | — | — | The raster object to query or update. |
| md5sum | text | Yes | — | 32 characters, only digits and lowercase letters | MD5 hash string to assign. |
| name | cstring | Yes | — | — | New name for the raster object. |

### Manage Binary Data

| Parameter | Type | Required | Default | Constraints | Description |
|----------|------|--------|--------|------------|------------|
| sfmeshObject | sfmesh | Yes | — | — | The SFMesh object. |
| material | material | Yes | — | — | The material object. |
| data | bytea | Yes | — | — | Binary data to store (as PostgreSQL `bytea`). |

### Create Texture

| Parameter | Type | Required | Default | Constraints | Description |
|----------|------|--------|--------|------------|------------|
| width | integer | Yes | — | — | Width of the texture image. |
| height | integer | Yes | — | — | Height of the texture image. |
| url | cstring | No | — | — | Path to image file (for file-based creation). |
| to_internal | boolean | No | false | — | Store binary data internally if true. |
| depth | integer | No | 3 | One of: 1, 3, 4 | Pixel depth: 1=Grayscale, 3=RGB, 4=RGBA. |
| format | cstring | No | JPEG | One of: Raw, JPEG, PNG | Image format. |
| compression | cstring | No | None | One of: None, Zlib | Compression method. |
| wrap | cstring | No | Wrap | One of: Wrap, Mirror | Texture wrap mode. |
| table_name | cstring | No | — | — | Table name for existing texture reference. |
| column_name | cstring | No | — | — | Column name for existing texture reference. |
| key_value | cstring | No | — | — | WHERE clause value to locate existing texture. |
| schema_name | cstring | No | public | — | Schema name for existing texture reference. |
| data | bytea | No | — | — | Binary image data stream. |

### Convert to Raw Format

| Parameter | Type | Required | Default | Constraints | Description |
|----------|------|--------|--------|------------|------------|
| sfmeshObject | sfmesh | Yes | — | — | SFMesh object to convert. |
| meshgeom | meshgeom | Yes | — | — | Mesh geometry object to convert. |
| material | material | Yes | — | — | Material object to convert. |
| texture | texture | Yes | — | — | Texture object to convert. |
| rawGeometry | boolean | No | true | — | Convert referenced geometry to raw. |
| rawMesh | boolean | No | true | — | Convert referenced sfmesh to raw. |
| rawMaterial | boolean | No | true | — | Convert referenced material to raw. |
| rawTexture | boolean | No | true | — | Convert referenced texture to raw. |

## Code Examples

### Export Raster to OSS, MinIO, and HDFS - SQL - all

```sql
-- Export a raster object as an OSS object.
Select ST_ExportTo(raster, 'GTiff', 'OSS://<ak>:<ak_secret>@oss-cn-beijing-internal.aliyuncs.com/mybucket/data/image.tif')
from raster_table
where id=1;

-- Export a raster object as a MinIO object.
Select ST_ExportTo(raster, 'PNG', 'MIO://<ak>:<ak_secret>@10.0.XX.XX:443/mybucket/data/image.png')
from raster_table
where id=1;

-- Export a raster object as an HDFS file.
Select ST_ExportTo(raster, 'PNG', 'HDFS://<user_name>@10.0.XX.XX:8020/path/image.png')
from raster_table
where id=1;
```

### Import Raster with Parallelism and In-Memory Mapping - SQL - all

```sql
-- Map the OSS object to an in-memory file to reduce small read overhead,
-- then import in parallel with 4 workers
SELECT ST_ImportFrom(
    'chunk_table',
    'OSS://<ak>:<ak_secret>@oss-cn-beijing-internal.aliyuncs.com/mybucket/data/image.nc:hcc',
    '{}',
    '{"parallel": 4, "mapping_oss_file": true}'  -- importOption
);
```

### Set and Retrieve Raster Credentials - SQL - all

```sql
-- Set both AccessKey ID and secret
UPDATE raster_table
SET rast = ST_SetAccessKey(rast, 'OSS_ACCESSKEY_ID', 'OSS_ACCESSKEY_SECRET');

-- Verify the AccessKey ID was set
SELECT ST_AKId(rast) from raster_table;
     st_akid
------------------
 OSS_ACCESSKEY_ID
```

### Create and Convert Texture to Raw Format - SQL - all

```sql
-- Create a texture table.
CREATE TABLE textures(id integer, the_texture texture);

-- Insert a 225×225 RGBA PNG file and store its binary stream internally.
INSERT INTO textures VALUES (1, ST_MakeTexture(225,225,'path/example.png'::cstring, true, 4, 'PNG'));

-- Convert to raw format and view JSON representation
SELECT ST_AsText(ST_AsRaw(ST_MakeTexture(225,225, '\\x123abcd...'::bytea)));
```

### Manage Binary Data on SFMesh - SQL - all

```sql
-- Set binary data on an SFMesh object
SELECT ST_BinData(ST_SetBinData(the_mesh, 'abc'::bytea))
FROM t_table;

-- Check if binary data exists
SELECT ST_HasBinData(ST_SetBinData(the_mesh, 'abc'::bytea))
FROM t_table;

-- Clear binary data
SELECT ST_HasBinData(ST_ClearBinData(the_mesh))
FROM t_table;
```

### Get Raster UUID and MD5 Hash - SQL - all

```sql
-- Get universally unique identifier
select ST_RasterID(raster_obj) from raster_table;

-- Get MD5 checksum
SELECT ST_MD5Sum(rast)
FROM raster_table
WHERE id = 1;
```

### Get Memory Usage of Spatial Objects - SQL - all

```sql
-- Query memory size of a meshgeom object
select st_memsize('MESHGEOM(PATCH(INDEXSURFACE(VERTEX(0 0,0 10,10 10,10 0), INDEX((0,1,2),(1,2,3)))))'::meshgeom);
```

## Response Format

```json
{"compressionType" : "None", "format" : "JPEG", "wrap" : "Wrap", "type" : "Raw", "depth" : 3, "width" : 225, "height" : 225, "size" : 7921, "data" : "FFD8FFE0001...."}
```

**Key Fields**:
- `type` — Indicates the object type ("Raw")
- `format` — Image format (e.g., "JPEG", "PNG")
- `width` / `height` — Dimensions of the texture
- `size` — Size of binary data in bytes
- `data` — Base64-encoded or hex-encoded binary content
- `compressionType` — Compression method applied ("None", "Zlib")
- `depth` — Color depth (1, 3, or 4)
- `wrap` — Texture wrapping behavior ("Wrap", "Mirror")

## Error Handling

| Error Code | Description | Recommended Action |
|-----------|-------------|-------------------|
| false | Indicates that the export operation failed. No specific error details provided in the document. | Verify URL syntax, credentials, network connectivity to storage system, and raster object validity. |

## FAQ

Q: How do I authenticate when importing from OSS or MinIO?
A: Embed your AccessKey ID and secret directly in the URL using the format `OSS://<ak>:<ak_secret>@<endpoint>/<bucket>/<path>`. No separate authentication step is needed.

Q: Can I use IAM roles or temporary tokens instead of long-term AccessKeys?
A: The current SQL functions (`ST_ImportFrom`, `ST_ExportTo`) only support static AccessKey ID/secret embedded in URLs. Temporary tokens or role-based auth are not supported in this interface.

Q: What happens if my raster import fails due to network issues?
A: The operation is synchronous and atomic — it either succeeds completely or fails without partial side effects. Retry the SQL command after verifying connectivity and credentials.

Q: Are there size limits for raster imports or exports?
A: While not explicitly stated in the function docs, practical limits are governed by RDS instance memory, timeout settings, and underlying storage system quotas (e.g., OSS object size limits).

Q: How do I handle credentials securely in production?
A: Avoid hardcoding credentials in SQL scripts. Use application-level secret management to dynamically construct URLs, or leverage RDS parameter groups and encrypted configuration tables if supported by your deployment model.

## Pricing & Billing

### Billing Model
Operations are billed on a **per-request basis**, with additional costs potentially incurred by the target storage system (OSS, MinIO, HDFS) based on data transfer volume and storage duration.

### Price Reference

| Tier/Specification | Input Price | Output Price | Other Fees |
|--------------------|------------|-------------|-----------|
| default | varies by storage type | varies by storage type | storage costs apply based on usage |

### Billing Notes
Export operations may incur costs based on the target storage system (OSS, MinIO, HDFS). Costs depend on data size, transfer volume, and storage duration. Import operations are charged based on the number of requests and data processed within RDS. Refer to the ApsaraDB RDS pricing page for detailed rates.