Skip to main content

​
Intelligent DataPart Merging

We have developed policies and algorithms to intelligently merge DataParts in the background. The goal is to:
  • Automatically compact multiple DataParts into fewer ones
  • Detect hot paths and frequently accessed subgraphs
  • Optimize for query throughput and storage locality
Commits start as multiple data parts for fast ingestion and converge toward compact forms for analytical speed, combining the best of both worlds.

​
Summary

FeatureBenefit
Immutable DataPartsSafe versioning and reuse
Parallel write ingestionHigh-performance batch processing
Shared storage across commitsLower memory usage, fast snapshots
Merge roadmapCompact layout for ultimate read speed
TuringDB uses DataParts to balance high-speed writes, versioned safety, and read-optimized performance, all in a single, cohesive engine.
  • ClickHouse: Parts , A similar model used in high-performance columnar stores to enable immutability, versioning, and efficient compaction.