ddia/content/en/toc.md

title	linkTitle	weight	breadcrumbs
Table of Content	Table of Content	10	false

Preface

1. Trade-offs in Data Systems Architecture

• Analytical versus Operational Systems
  • Characterizing Transaction Processing and Analytics
  • Data Warehousing
  • Systems of Record and Derived Data
• Cloud versus Self-Hosting
  • Pros and Cons of Cloud Services
  • Cloud-Native System Architecture
  • Operations in the Cloud Era
• Distributed versus Single-Node Systems
  • Problems with Distributed Systems
  • Microservices and Serverless
  • Cloud Computing versus Supercomputing
• Data Systems, Law, and Society
• Summary

2. Defining Nonfunctional Requirements

• Case Study: Social Network Home Timelines
  • Representing Users, Posts, and Follows
  • Materializing and Updating Timelines
• Describing Performance
  • Latency and Response Time
  • Average, Median, and Percentiles
  • Use of Response Time Metrics
• Reliability and Fault Tolerance
  • Fault Tolerance
  • Hardware and Software Faults
  • Humans and Reliability
• Scalability
  • Describing Load
  • Shared-Memory, Shared-Disk, and Shared-Nothing Architecture
  • Principles for Scalability
• Maintainability
  • Operability: Making Life Easy for Operations
  • Simplicity: Managing Complexity
  • Evolvability: Making Change Easy
• Summary

3. Data Models and Query Languages

• Relational Model versus Document Model
  • The Object-Relational Mismatch
  • Normalization, Denormalization, and Joins
  • Many-to-One and Many-to-Many Relationships
  • Stars and Snowflakes: Schemas for Analytics
  • When to Use Which Model
• Graph-Like Data Models
  • Property Graphs
  • The Cypher Query Language
  • Graph Queries in SQL
  • Triple-Stores and SPARQL
  • Datalog: Recursive Relational Queries
  • GraphQL
• Event Sourcing and CQRS
• Dataframes, Matrices, and Arrays
• Summary

4. Storage and Retrieval

• Storage and Indexing for OLTP
  • Log-Structured Storage
  • B-Trees
  • Comparing B-Trees and LSM-Trees
  • Multi-Column and Secondary Indexes
  • Keeping everything in memory
• Data Storage for Analytics
  • Cloud Data Warehouses
  • Column-Oriented Storage
  • Query Execution: Compilation and Vectorization
  • Materialized Views and Data Cubes
• Multidimensional and Full-Text Indexes
  • Full-Text Search
  • Vector Embeddings
• Summary

5. Encoding and Evolution

• Formats for Encoding Data
  • Language-Specific Formats
  • JSON, XML, and Binary Variants
  • Protocol Buffers
  • Avro
  • The Merits of Schemas
• Modes of Dataflow
  • Dataflow Through Databases
  • Dataflow Through Services: REST and RPC
  • Durable Execution and Workflows
  • Event-Driven Architectures
• Summary

6. Replication

• Single-Leader Replication
  • Synchronous Versus Asynchronous Replication
  • Setting Up New Followers
  • Handling Node Outages
  • Implementation of Replication Logs
• Problems with Replication Lag
  • Reading Your Own Writes
  • Monotonic Reads
  • Consistent Prefix Reads
  • Solutions for Replication Lag
• Multi-Leader Replication
  • Geographically Distributed Operation
  • Sync Engines and Local-First Software
  • Dealing with Conflicting Writes
  • CRDTs and Operational Transformation
• Leaderless Replication
  • Writing to the Database When a Node Is Down
  • Limitations of Quorum Consistency
  • Single-Leader vs. Leaderless Replication Performance
  • Detecting Concurrent Writes
• Summary

7. Sharding

• Pros and Cons of Sharding
  • Sharding for Multitenancy
• Sharding of Key-Value Data
  • Sharding by Key Range
  • Sharding by Hash of Key
  • Skewed Workloads and Relieving Hot Spots
  • Operations: Automatic or Manual Rebalancing
• Request Routing
• Sharding and Secondary Indexes
  • Local Secondary Indexes
  • Global Secondary Indexes
• Summary

8. Transactions

• What Exactly Is a Transaction?
  • The Meaning of ACID
  • Single-Object and Multi-Object Operations
• Weak Isolation Levels
  • Read Committed
  • Snapshot Isolation and Repeatable Read
  • Preventing Lost Updates
  • Write Skew and Phantoms
• Serializability
  • Actual Serial Execution
  • Two-Phase Locking (2PL)
  • Serializable Snapshot Isolation (SSI)
• Distributed Transactions
  • Two-Phase Commit (2PC)
  • Distributed Transactions Across Different Systems
  • Database-internal Distributed Transactions
• Summary

9. The Trouble with Distributed Systems

• Faults and Partial Failures
• Unreliable Networks
  • The Limitations of TCP
  • Network Faults in Practice
  • Detecting Faults
  • Timeouts and Unbounded Delays
  • Synchronous Versus Asynchronous Networks
• Unreliable Clocks
  • Monotonic Versus Time-of-Day Clocks
  • Clock Synchronization and Accuracy
  • Relying on Synchronized Clocks
  • Process Pauses
• Knowledge, Truth, and Lies
  • The Majority Rules
  • Distributed Locks and Leases
  • Byzantine Faults
  • System Model and Reality
  • Formal Methods and Randomized Testing
• Summary

10. Consistency and Consensus

• Linearizability
  • What Makes a System Linearizable?
  • Relying on Linearizability
  • Implementing Linearizable Systems
  • The Cost of Linearizability
• ID Generators and Logical Clocks
  • Logical Clocks
  • Linearizable ID Generators
• Consensus
  • The Many Faces of Consensus
  • Consensus in Practice
  • Coordination Services
• Summary

11. Batch Processing (WIP)

12. Stream Processing (WIP)

13. Doing the Right Thing (WIP)

Glossary

Colophon