# pai-pipeline_management

Part of **PAI**

# Platform for AI (PAI) Pipeline & Workflow Management Console Guide

## Operations Overview

| Operation | Console Entry | Prerequisites | Description |
|------|-----------|---------|------|
| Create Workflow | Console > PAI > Designer | Create a workspace | Create a new ML workflow using preset templates, blank canvas, or custom templates |
| Create Pipeline from Template | PAI Console > Visualized Modeling (Designer) | PAI activated; workspace associated with MaxCompute resources | Build a pipeline using a pre-defined Heart Disease Prediction template |
| Build Machine Learning Pipeline | Console > PAI > Designer > Use Cases | Access to PAI Designer; data available; permissions granted | Explore and implement end-to-end ML pipelines for recommendation, risk control, and more |
| Create Custom ML Pipeline | Console > PAI > Machine Learning Designer | PAI activated; workspace linked to MaxCompute | Construct a full custom pipeline with data prep, training, evaluation, and EAS deployment |
| Configure Global Variable | Console > PAI > Designer > Pipelines > Create/Manage Pipeline | A PAI Designer pipeline created; access to DataWorks; MaxCompute table ready | Define pipeline-level variables for reuse across components and integration with DataWorks scheduling |
| Schedule Pipeline Run | Console > PAI > Pipelines > Schedule Pipeline Runs | Pipeline created; appropriate permissions | Set up recurring executions of a pipeline and optionally update dependent services |

## Step-by-Step Instructions

### Create Workflow

**Navigation**: Console > PAI > Designer

**Prerequisites**:
- Create a workspace

1. Log on to the PAI console  
   - Element: **PAI console** (link) — top navigation panel

2. Select your target workspace and go to the Designer page  
   - Element: **Designer** (tab) — main content area

3. Choose to create a workflow using a preset template  
   - Element: **Use a preset template** (tab) — top navigation panel

4. Enter a custom workflow name  
   - Element: **Workflow name** (text_input) — form fields

5. Configure workflow data storage path  
   - Element: **Workflow data storage** (text_input) — form fields  
   - Notes: If not configured, uses default workspace storage.

6. Set visibility to private or public  
   - Element: **Visibility** (radio) — form fields  
   - Notes: Private: visible only to you and workspace administrators. Public: visible to everyone in the workspace.

7. Click Create  
   - Element: **Create** (button) — bottom of form

8. For custom templates, double-click a successfully run workflow name to open its details page  
   - Element: **Double-click workflow name** (text_input) — workflow list

9. Click Release template above the canvas  
   - Element: **Release template** (button) — top-right corner

10. Configure template name, description, and optional link  
    - Element: **Template name** (text_input) — dialog box

11. Click Release  
    - Element: **Release** (button) — dialog box

12. Click Next to view the created template  
    - Element: **Next** (button) — dialog box

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Workflow name | text | Yes | — | Custom workflow name. |
| Workflow data storage | text | No | — | OSS Bucket storage path for temporary data and models generated during runtime. If not configured, uses default workspace storage. |
| Visibility | radio | Yes | Private, Public | Controls who can see the workflow. Private is visible only to you and workspace administrators. Public is visible to everyone in the workspace. |
| Template name | text | Yes | — | Name of the custom template being created. |
| Template description | text | No | — | Description of the custom template. |
| Template link | text | No | — | Optional accessible HTTP address that describes the custom template. |

### Create Pipeline from Template

**Navigation**: PAI Console > Visualized Modeling (Designer)

**Prerequisites**:
- Platform for AI (PAI) is activated and a default workspace is created
- The workspace is associated with MaxCompute resources

1. Log on to the PAI console  
   - Element: **PAI console** (link) — top navigation

2. Choose Visualized Modeling (Designer) from the left-side navigation pane  
   - Element: **Visualized Modeling (Designer)** (menu) — left-side navigation pane

3. Select a workspace and click Enter Visualized Modeling (Designer)  
   - Element: **Enter Visualized Modeling (Designer)** (button) — workspace selection panel

4. Click the Preset Templates tab, find the Heart Disease Prediction template, and click Create  
   - Element: **Preset Templates** (tab) — top navigation of canvas

5. Configure parameters in the Create Pipeline dialog box and click OK  
   - Element: **OK** (button) — bottom of dialog box  
   - Notes: For more information about parameters, refer to external documentation.

6. Click Open to view the pipeline on the canvas  
   - Element: **Open** (button) — top-left corner of canvas

7. Click the Run icon at the top left of the canvas to execute the pipeline  
   - Element: **Run icon** (icon) — top-left corner of canvas

8. Right-click a component and select View Data or Visual Analysis to inspect outputs  
   - Element: **View Data** (context menu option) — right-click context menu  
   - Notes: For evaluation components, you can also use the visualization icon at the top of the canvas.

9. If a component fails, right-click and select View Log to troubleshoot  
   - Element: **View Log** (context menu option) — right-click context menu

10. After successful pipeline execution, go to the Models panel and click Deploy in EAS  
    - Element: **Deploy in EAS** (button) — Models panel in left panel

11. On the Deploy Service page, confirm parameters and click Deploy  
    - Element: **Deploy** (button) — bottom of page

12. Wait for the service status to change from Creating to Running  
    - Element: **Running** (status indicator) — Actions column  
    - Notes: To avoid unnecessary costs, stop the service when not in use.

### Build Machine Learning Pipeline

**Navigation**: Console > PAI > Designer > Use Cases

**Prerequisites**:
- Access to PAI Designer console
- Basic understanding of machine learning concepts
- Data available for training (e.g., MaxCompute tables, CSV files)
- Appropriate permissions to create and deploy models

1. Navigate to the PAI Designer section in the console  
   - Element: **PAI** (menu) — left navigation panel

2. Select 'Designer' from the product list  
   - Element: **Designer** (link) — main content area

3. Click on 'Use Cases' to view the list of pre-built pipelines  
   - Element: **Use Cases** (tab) — top navigation bar

4. Select a specific use case to view its detailed guide  
   - Element: **End-to-end solution for recommendation services** (link) — use case table  
   - Notes: Each link opens a new page with a step-by-step tutorial

### Create Custom ML Pipeline

**Navigation**: Console > PAI > Machine Learning Designer

**Prerequisites**:
- Activate PAI and create a workspace
- Associate the workspace with MaxCompute resources

1. Navigate to Machine Learning Designer and select a workspace  
   - Element: **Machine Learning Designer** (link) — top navigation panel

2. Create and open a new pipeline  
   - Element: **Create New Pipeline** (button) — main content area

3. Add a Data Source/Target component and drag Read Table onto the canvas  
   - Element: **Data Source/Target** (menu) — component list

4. Configure the table name in the Read Table node  
   - Element: **Table Name** (text_input) — right panel  
   - Notes: Enter 'pai_online_project.heart_disease_prediction'

5. Search for and drag SQL Script component onto the canvas  
   - Element: **SQL Script** (menu) — component list

6. Connect Read Table-1 as input source for SQL Script-1  
   - Element: **t1** (dropdown) — Parameters Setting

7. Enter SQL code in the SQL Script node  
   - Element: **SQL Script-1** (code_editor) — right panel

8. Save the pipeline configuration  
   - Element: **Save** (button) — upper-left corner

9. Run the pipeline from root node to SQL Script-1  
   - Element: **Run from Root Node To Here** (context_menu) — right-click menu  
   - Notes: Check for success icon (✓) after execution

10. Right-click SQL Script-1 and view output data  
    - Element: **View Data** (context_menu) — right-click menu  
    - Notes: Select 'Prediction Result Output Port' to view results

11. Drag Data Type Conversion component and connect it as descendant of SQL Script-1  
    - Element: **Data Type Conversion** (menu) — component list

12. Select all fields to convert to double type  
    - Element: **Select Fields** (button) — Fields Setting

13. Drag Normalization component and connect it as descendant of Data Type Conversion-1  
    - Element: **Normalization** (menu) — component list

14. Select all fields for normalization  
    - Element: **Fields Setting** (text_input) — right panel

15. Drag Split component and connect it as descendant of Normalization-1  
    - Element: **Split** (menu) — component list

16. Configure split ratio on Parameters Setting tab  
    - Element: **Split Ratio** (number_input) — Parameters Setting  
    - Notes: Default is 4:1 ratio

17. Run remaining nodes starting from Data Type Conversion-1  
    - Element: **Run from Here** (context_menu) — right-click menu

18. Drag Binary Logistic Regression component and connect it to Output Table 1 from Split-1  
    - Element: **Binary Logistic Regression** (menu) — component list

19. Set Target Column to ifhealth and Training Feature Columns to all other columns  
    - Element: **Fields Setting** (text_input) — right panel

20. Run the binary logistic regression node  
    - Element: **Run** (button) — upper-left corner

21. Drag Prediction component and connect it to Output Table 2 from Split-1 and Logistic Regression node  
    - Element: **Prediction** (menu) — component list

22. Set Reserved Columns to ifhealth and Feature Columns to all other fields  
    - Element: **Fields Setting** (text_input) — right panel

23. Run prediction node and view results  
    - Element: **View Data** (context_menu) — right-click menu  
    - Notes: Select 'Prediction Result Output Port'

24. Drag Binary Classification Evaluation component and connect it to Prediction-1  
    - Element: **Binary Classification Evaluation** (menu) — component list

25. Set Original Label Column to ifhealth  
    - Element: **Original Label Column** (dropdown) — Fields Setting

26. Run evaluation node and view visual analysis  
    - Element: **Visual Analysis** (context_menu) — right-click menu

27. Click Model List and select model to deploy  
    - Element: **Model List** (tab) — top navigation

28. Click Deploy to EAS  
    - Element: **Deploy to EAS** (button) — model details panel

29. Confirm deployment parameters and click Deploy  
    - Element: **Deploy** (button) — bottom of page  
    - Notes: Wait until Service Status changes from Creating to Running

30. Stop service if no longer needed to avoid costs  
    - Element: **Stop** (button) — Operation column  
    - Notes: Important: Avoid unnecessary charges

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Table Name | text | Yes | — | Enter the MaxCompute table name to read data from |
| Split Ratio | number_input | No | — | Ratio to split data into training and prediction sets |
| Target Column | dropdown | Yes | ifhealth | The column containing the target label for binary classification |
| Training Feature Columns | checkbox | Yes | — | All columns except the target column used for training |
| Reserved Columns | checkbox | Yes | — | Columns to preserve in the prediction output |
| Feature Columns | checkbox | Yes | — | All columns used as input features for prediction |
| Original Label Column | dropdown | Yes | ifhealth | The column containing the actual labels for evaluation |

### Configure Global Variable

**Navigation**: Console > PAI > Designer > Pipelines > Create/Manage Pipeline

**Prerequisites**:
- A PAI Designer pipeline created
- Access to DataWorks console for scheduling setup
- MaxCompute table already created (for example 2)

1. Click a blank area on the canvas to open the right panel  
   - Element: **Add Global Variable** (button) — right panel

2. Enter a variable name and value in the form fields  
   - Element: **Variable Name** (text_input) — right panel

3. Reference the variable in a component's field using ${variableName}  
   - Element: **SQL Script** (text_input) — component configuration  
   - Notes: The variable must be referenced exactly as defined, including case sensitivity

4. Click Periodic Scheduling in the upper-left corner of the canvas  
   - Element: **Periodic Scheduling** (button) — upper-left corner of canvas

5. Click Create Scheduling Node to open DataWorks  
   - Element: **Create Scheduling Node** (button) — Periodic Scheduling section

6. In the Properties panel, add a scheduling parameter named gDate with value $bizdate  
   - Element: **Scheduling Parameter** (text_input) — right side of screen  
   - Notes: The parameter name must exactly match the global variable name (case-sensitive)

7. Save and submit the node using the save icons  
 - Element: **** (image_button) — toolbar 
   - Notes: Two save icons are used: one for saving locally, one for submitting to DataWorks

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Variable Name | text | Yes | — | The name of the global variable used for substitution in components |
| Variable Value | text | Yes | — | The value assigned to the global variable, which will replace all ${variableName} references |
| Scheduling Parameter | text | Yes | — | The name of the DataWorks scheduling parameter that maps to the global variable |
| Schedule | dropdown | No | Allow Regardless of Running Status | Controls whether the pipeline can be rerun even if currently running |
| Dependencies | checkbox | No | Add Root Node | Specifies that this pipeline depends on a root node for execution |

### Schedule Pipeline Run

**Navigation**: Console > PAI > Pipelines > Schedule Pipeline Runs

**Prerequisites**:
- A pipeline has been created and is available in the PAI console
- User has appropriate permissions to schedule pipelines and update services

1. Navigate to the Pipelines section in the PAI console  
   - Element: **Pipelines** (menu) — left navigation panel

2. Select the pipeline you want to schedule  
   - Element: **Pipeline name** (link) — main content area  
   - Notes: Click on the pipeline name to open its details page

3. Click the 'Schedule' button to configure a recurring run  
   - Element: **Schedule** (button) — top-right corner  
   - Notes: This opens the scheduling configuration panel

4. Set the recurrence pattern (e.g., daily, weekly, custom cron)  
   - Element: **Recurrence** (dropdown) — scheduling configuration panel  
   - Notes: Use the cron expression editor for advanced scheduling

5. Configure service update settings if needed  
   - Element: **Update Services** (checkbox) — scheduling configuration panel  
   - Notes: Enable this option to automatically update associated services after each run

6. Save the schedule configuration  
   - Element: **Save** (button) — bottom of the form  
   - Notes: The pipeline will now run according to the defined schedule

| Parameter | Type | Required | Options/Values | Description |
|-----------|------|----------|----------------|-------------|
| Pipeline Name | text | Yes | — | The name of the pipeline to be scheduled |
| Recurrence | dropdown | Yes | Daily, Weekly, Monthly, Custom Cron | Define how often the pipeline should run |
| Start Time | datetime | Yes | — | The date and time when the first scheduled run should begin |
| Update Services | checkbox | No | — | Whether to automatically update dependent services after each pipeline run |

## FAQ

Q: Where do I find the option to create a new workflow in PAI Designer?
A: After logging into the PAI console and selecting your workspace, go to the Designer page. You can create a workflow using the "Use a preset template" tab or by importing an existing workflow.

Q: Can I modify global variables after creating a pipeline?
A: Yes, you can edit global variables by clicking a blank area on the canvas to open the right panel, where you can update variable names and values.

Q: What happens if I leave the "Workflow data storage" field empty when creating a workflow?
A: If left unconfigured, the workflow will use the default OSS storage path associated with your workspace for temporary data and model artifacts.

Q: Do I need special permissions to schedule pipeline runs?
A: Yes, you must have appropriate permissions to schedule pipelines and update services. Contact your workspace administrator if you encounter access issues.

Q: Can I deploy a model without running the full pipeline first?
A: No, you must successfully run the pipeline to generate a trained model before it appears in the Model List for deployment to EAS.

## Pricing & Billing

### Billing Model
per_request

### Price Reference
| Tier | Input Price | Output Price |
|------|-------------|--------------|
| standard | 0.001 / | 0.002 / |

### Free Tier
 100 

### Billing Notes