Document-Aware App with RAG and Recommendations

Scenario

Use this integration when building a document-centric application requiring precise full-text search, context-grounded RAG Q&A, and behavior-driven content recommendations. It is ideal for knowledge management or technical documentation platforms where developers must unify structured metadata (Supabase), extracted document content (Bailian), keyword indexing (Elasticsearch), semantic retrieval (OpenSearch), and personalized discovery (AIRec) into a single workflow.

Integration steps

1. Upload to OSS: Push raw PDFs/images to an OSS bucket. Attach AliyunOSSReadOnlyAccess to Bailian and OpenSearch service roles.
2. Extract via Bailian: Route to bailian-extract-documents for OCR, then call DashScope:

POST https://dashscope.aliyuncs.com/api/v1/services/aigc/document-processing/generation Payload: {"model": "doc-parser-v2", "input": {"file_url": "oss://bucket/doc.pdf"}, "parameters": {"enable_ocr": true, "layout_analysis": true}}

3. Persist in Supabase: Insert extracted JSON: POST /rest/v1/documents with apikey and Authorization headers. Body: {"title": "...", "ocr_text": "...", "metadata": {...}}.
4. Index in Elasticsearch: Sync for BM25 search via _bulk:

POST /documents/_bulk → {"index":{"_id":"doc_1"}}\n{"title":"...","content":"...","type":"structured"}

5. Vectorize & Index in OpenSearch: Generate embeddings using text-embedding-v3, then ingest via OpenSearch _bulk with knn mapping: "vector_field": {"type": "knn_vector", "dimension": 1024}.
6. Configure AIRec: Initialize instance. Push interactions: POST /v2/openapi/instances/{id}/actions/bulk → {"action_type": "click", "item_id": "doc_1", "user_id": "u_123"}.
7. Query Pipeline: Route exact matches to ES _search, semantic/RAG to OpenSearch _knn_search, and discovery to AIRec GET /v2/openapi/instances/{id}/recommendations.

Architecture

Raw documents reside in OSS. Bailian acts as the extraction engine, parsing text and layout into structured JSON. Supabase serves as the system of record for CRUD operations and metadata. A dual-indexing strategy routes extracted content to Elasticsearch for keyword search and to OpenSearch for dense vector storage powering RAG. AIRec consumes user interaction logs and document metadata to generate personalized recommendation feeds. The frontend orchestrates queries across ES (precision), OpenSearch (semantic context), and AIRec (discovery).

Prerequisites

• Alibaba Cloud account with OSS, Elasticsearch, OpenSearch, AIRec, and Bailian instances provisioned
• Supabase project with REST API enabled and RLS configured
• IAM roles granting cross-service read/write access (OSS → Bailian, ES/OpenSearch)
• Valid API keys for DashScope (Bailian) and AIRec

Common pitfalls

• Layout fragmentation during OCR: Bailian’s layout_analysis must be enabled; otherwise, table/multi-column text merges incorrectly, degrading RAG chunking.
• Vector/keyword mismatch: Using different embedding models for OpenSearch and Bailian extraction causes semantic drift. Standardize on text-embedding-v3.
• AIRec cold start: Recommendations fail until ~1,000 interaction events are pushed. Seed with historical click/view logs before launch.
• Sync latency: Supabase-to-ES sync via webhooks can drop records under high throughput. Implement idempotent _bulk retries with exponential backoff.

Typical questions

• build document app with search and recommendations
• Supabase OCR Elasticsearch RAG recommendation pipeline
• upload PDFs extract index and recommend
• full-stack document intelligence platform
• 上传文档提取内容建立搜索和智能推荐
• 文档感知应用加RAG加个性化推荐
• Bailian Supabase ES AIRec end-to-end
• document CRUD plus vector search plus recommendations

Q: How can I build a document-aware application with search and recommendations? A: You can build this application by combining Supabase for structured CRUD, Bailian OCR for document extraction, Elasticsearch for keyword indexing, and a RAG vector knowledge base paired with AIRec for semantic recommendations. This architecture delivers both traditional search and AI-driven personalized discovery over a unified document corpus.