Spanner Performance
Changing Google’s view of distributed-database performance
Changing Google’s View of Spanner Performance
During Spanner’s early years, many engineers viewed its strong transactional guarantees as coming with an unavoidable performance cost. Bigtable was often assumed to be the faster choice, while Spanner was selected only when global consistency and transactions justified the tradeoff.
I set out to test that assumption directly.
Establishing the Performance Case
I designed and ran comparative benchmarks between Spanner and Bigtable using representative cloud workloads. The results showed that Spanner was not merely competitive: for read-heavy workloads, which are common in cloud applications, Spanner could deliver lower latency and higher throughput than Bigtable while also providing stronger transactional semantics.
We also investigated the architectural differences underlying the results. Bigtable’s write-optimized, log-structured storage design is highly effective for write-heavy workloads. Its latest logical state, however, may be distributed across an in-memory memtable and multiple immutable SSTables, requiring reads to locate and reconcile the relevant data across these structures.
Spanner uses timestamped, multiversion storage. A snapshot read requests the most recent version visible at a specified timestamp and, when the requested timestamp is available at a replica, can be served without participating in a read-write transaction or acquiring locks. For the read-heavy workloads we evaluated, these architectural differences enabled Spanner to deliver substantially better latency and throughput.
This shifted the internal discussion. The question was no longer whether teams had to sacrifice performance to adopt Spanner, but which workloads and system designs could benefit from both its consistency model and its performance characteristics.
The benchmarks helped shift Google’s view of Spanner from a specialized database accepted despite its performance tradeoffs. Today, Spanner is the default database choice for a broad range of applications.
Exploring Future Transaction Capabilities
Beyond the direct comparison with Bigtable, I also studied how proposed Spanner capabilities could improve transaction throughput.
Three mechanisms had been designed but not yet implemented:
- read-your-writes;
- distributed counters;
- atomic writes.
I simulated these proposed capabilities in a TPC-C-like benchmark to evaluate their potential before they were available in the production system. In combination, they increased simulated transaction throughput by approximately 3×.
This work provided quantitative evidence for the value of the proposed features and helped clarify how future system capabilities and application designs could work together to achieve substantially higher transaction throughput.
My Role
I designed and conducted the comparative Spanner–Bigtable performance study, analyzed the architectural causes behind the results, and connected the findings to internal workload-selection decisions. I also simulated three designed-but-unimplemented transaction capabilities in a TPC-C-like benchmark, showing the potential for approximately 3× higher throughput.
What This Work Demonstrated
This work reinforced a principle that has shaped my career: important technical decisions are often constrained by assumptions that have not been tested under the right conditions.
Spanner’s transactional model was widely understood. Its performance envelope was not.
By building the right benchmarks, identifying the workloads where Spanner had an advantage, and connecting the results to real adoption decisions, I helped replace an inherited perception with empirical evidence.
The broader lesson was that technical leadership is not only about improving a system. It is also about changing how an organization understands what the system makes possible.