all-things-risingwave

How to ensure that data in every fragment plan is separated in a distributed computing environment?

In a distributed computing environment where stream plans are divided into different fragments and executed on different compute nodes, how can we ensure that the data in each fragment plan is separated and does not intersect with each other?

JJ

JJ

Asked on May 24, 2022

  • In a distributed computing environment, data separation in fragment plans is ensured through hash partitioning.
  • Hash partitioning guarantees that data in each fragment plan is distinct and does not overlap with data in other fragment plans.
  • For hash joins involving multiple tables, data redistribution based on the join key is necessary before performing the join operation.
  • Ensuring data separation in fragment plans is critical for performance optimization in distributed computing environments.
May 24, 2022Edited by