DaaS / Products / Real-Time Data Pipeline to Search

Real-Time Data Pipeline to Search

Use DataWorks to orchestrate batch ETL jobs that land transformed data into OSS/MaxCompute, configure EventBridge to stream real-time events into the same OSS landing zone, and then configure OpenSearch to ingest from those OSS/MaxCompute sources for unified search and analytics over both batch and streaming data.

Products involved

Scenario

Use DataWorks to orchestrate batch ETL jobs that land transformed data into OSS/MaxCompute, configure EventBridge to stream real-time events into the same OSS landing zone, and then configure OpenSearch to ingest from those OSS/MaxCompute sources for unified search and analytics over both batch and streaming data.

How the products combine

  1. dataworks · dataworks-onboard — DataWorks 模块选择 / Module Selection Routing

    2. See dataworks/dataworks-onboard.

  2. eb · eb-configure-streaming — EventBridge — Configure real-time event streaming

    3. See eb/eb-configure-streaming.

  3. opensearch · opensearch-manage-sources — OpenSearch — Manage data sources for ingestion

    4. See opensearch/opensearch-manage-sources.

Typical questions