Workload Isolation

Background

Tacnode is a cloud-native data warehouse system that fully leverages the compute-storage separation architecture. Its storage and compute layers are independently decoupled, enabling flexible compute resource management and robust workload isolation. This document details approaches to implement effective workload isolation for scenarios involving multiple users and complex mixed workloads.

In data warehouse scenarios, workload isolation means logically separating resources and tasks to prevent interference between users or jobs, thus ensuring performance and execution accuracy. Tacnode’s compute-storage separation enables efficient support for these scenarios via shared storage, independent compute resources.

A Nodegroup is Tacnode’s compute resource abstraction. Allocate different Nodegroups to different business units for precise workload isolation. Each Nodegroup supports independent data processing and querying. Concurrent data updates from multiple Nodegroups ensure serialized consistency.

Catalog is Tacnode’s metadata abstraction. All Databases under a single Catalog are accessible to Nodegroups. The Catalog is also the atomic unit for instance upgrade; all Nodegroups under the same Catalog are upgraded simultaneously to prevent metadata version inconsistencies.

When a Database is accessed by multiple Nodegroups, you must specify its bound Nodegroup. The bound Nodegroup handles local, direct reads and writes (typical for TP scenarios), delivering superior performance. The bound Nodegroup must be running for other Nodegroups to access the Database. All Nodegroups are functionally equivalent; however, non-bound Nodegroups use remote read/write which in TP scenarios yields lower performance and consumes compute resources on the bound Nodegroup. Each Database can be bound to only one Nodegroup at a time.

Design guideline for workload isolation:

• Place Databases to be shared within the same Catalog.
• Isolate workloads using different Nodegroups.
• Bind the primary write/short-query Nodegroup to the Database directly.
• Realize agile isolation using the Shared Data architecture.

Note: Multi-Nodegroup shared Database isolation is a "flagship edition" feature—upgrade to access this capability.

Typical Workload Isolation Scenarios

1. Development and Production Isolation

   Separate development/testing and production workloads, ensuring test loads cannot impact critical production tasks.

2. Multi-Team Workload Segregation

   Separate team or project tasks to prevent resource contention.

3. High-Priority Task Protection

   Dedicate stable resources to high-priority workloads.

4. Read/Write Separation

   Isolate read and write operations to address different I/O and access privilege requirements through independent resources.

Nodegroup-Database Relationship Design

Default Bound Nodegroup

DataCloud supports multiple Catalogs. When creating a Nodegroup, either create or select an existing Catalog.

When a Nodegroup is created, the system generates default Databases (postgres, template0, template1), which are automatically bound to the Nodegroup. If the bound Nodegroup goes offline, another Nodegroup is automatically elected to ensure system metadata availability.

Users can create additional Databases by executing the CREATE DATABASE command on any Nodegroup. The Nodegroup that created the Database becomes the bound Nodegroup by default. Warehouse bindings can be viewed via the Database listing page. It is recommended to create separate Databases per business context and not use system Databases (which handle metadata duties) for user data.

If the bound Nodegroup stops or a Database is not linked to a Nodegroup, the Database becomes unavailable/invisible.

Nodegroup Access to Databases

Any Nodegroup can connect—using standard database connection strings—to any Database under its Catalog, supporting both read and write requests. Write requests consume Owner Nodegroup resources (observable via monitoring metrics). Read operations utilize the respective Nodegroup's compute resources independently. Resource adjustment for one Nodegroup does not affect others.

Rebinding a Nodegroup

Bound Nodegroups bear responsibility for direct, local I/O on the associated Database. Recommended: assign long-running, write-heavy, or short-query Nodegroups as bound Nodegroups.

To change the bound Nodegroup:

• Unbind the current Nodegroup
• Select the new Nodegroup to bind (specify new Database name; typically, use original name for continuity)

Nodegroups that are offline cannot be unbound. Only running Nodegroups support unbinding operations for their Databases.

Best Practices

Database Partitioning

Database boundaries should reflect responsibility boundaries. Data within a common access scope should reside in the same Database, facilitating consistent permission control. Typical practice: assign a dedicated Database per team, with a team security officer managing Database access.

Data warehouse teams may aggregate, clean, apply data QA, and develop models across multiple data sources, with warehousing assets (bronze/silver zones) distributed by team or domain.

For organizations with multiple, disjoint business domains, isolate each domain in distinct Databases to control visibility and security per business vertical.

Note: The default system Databases (postgres, template0, template1) are reserved for metadata management—do not store business/user data here. Create dedicated Databases as per your use case.

Database Authorization

All Nodegroups under the same Catalog can see all Databases but may not access them: actual data visibility depends on user permissions.

• Global Users: must be explicitly added to each Database (except for creator, who is auto-granted access)
• Local Users: created per Database with CREATE USER. Local user credentials are not shared between Databases.

Nodegroup Partitioning

Nodegroup partitioning is flexible, leveraging their stateless, elastic nature—divide per use case for optimal isolation.

• Assign persistent, write-heavy, or low-latency workloads to Nodegroups bound to the corresponding Databases, maintaining stable compute allocation.
• Use specialized ETL Nodegroups for data processing tasks with bursty/variable demand; dynamically scale compute to optimize for workload patterns and cost.
• For data consumption and services, allocate separate Nodegroups for production/online services and internal/team users—online services typically demand higher stability and stronger SLAs; team Nodegroups can privilege flexibility and local administration.

This approach supports precise operational control and superior resource isolation for enterprise-grade data warehousing workloads.