Internals on Jin Cong Ho | ML & Analytics with HPC

Paper Notes: Yellowbrick: An Elastic Data Warehouse on Kubernetes

Thu, 31 Jul 2025 22:08:00 +0100

Yellowbrick: An Elastic Data Warehouse on Kubernetes, 2024 VLDB

1 Key Design

Yellowbrick Data Warehou delivers efficient, scalable and resilient data warehousing in public clouds and in private data centers.

2 Architecture

Storage is separated from compute and data is persisted in object storage as column-oriented, compressed files known as shards.

Microservices Architectrue

Deployment Approach

3 Software Optimizations

3.1 Database Optimizations

parallel query plans, cost-based optimization, workload mangaement and parallel query execution

Paper Notes: Amazon Redshift and the Case for Simpler Data Warehouses

Sun, 27 Apr 2025 18:38:10 +0100

Amazon Redshift and the Case for Simpler Data Warehouses, 2015 SIGMOD

1 Key Design

Redshift is a fast, fully managed, petabyte-scale data warehouse solution that makes it simple and cost-effective to efficiently analyze large volumes of data. It uses familiar data warehousing techniques, including columnar layout, per-column compression, co-locating compute and data, co-locating joins, compilation to machine code and scale-out MPP processing. It also had a number of additional design goals:

Paper Notes: ClickHouse - Lightning Fast Analytics for Everyone

Sun, 27 Apr 2025 16:21:04 +0100

ClickHouse - Lightning Fast Analytics for Everyone, 2024 PVLDB

ClickHouse is an OLAP database designed for high-performance analytics over petabyte-scale data sets with high ingestion rates.

1 Key Design

ClikcHouse is designed to address 5 key challenges of modern analytical data management:

Huge data sets with high ingestion rates
Many simultaneous queries with an expectation of low latencies: ad-hoc and recurring queries, pruning techniques allow optimizing frequent queries. Managing shared system resources.

Paper Notes: The Snowflake Elastic Data Warehouse

Thu, 24 Apr 2025 21:54:23 +0100

The Snowflake Elastic Data Warehouse, 2016 ACM

1 Key Design

Snowflake is an enterprise-ready data warehousing solution for the cloud.

Cloud promises increased economies of scale, extreme scalability and availability and a pay-as you go cost model — but it can only be captured if the software itself is able to scale elastically over the pool of commodity resources in the cloud.

Meanwhile, Saas brings enterprise-class systems to users who previously could not afford them. Snowflake key features includes: relational model, semi-structured data, elastic compute and storage, highly available, durable, cost-efficient and secure.