[EN] re:Invent 2023 - Monday Night Live with Peter DeSantis

Keyword: Amazon Web Services re:Invent 2023, Aurora Limitless Database, Serverless Computing, Quantum Computing, Database Sharding, Caching, Data Warehouse Optimization

Words: 2000, Reading time: 14 minutes

Video

Introduction

Peter DeSantis, Senior Vice President of AWS Utility Computing, continues the Monday Night Live tradition of diving deep into the engineering that powers AWS services. He will provide a closer look at how AWS' unique approach and culture of innovation help create leading-edge solutions across the entire spectrum, from silicon to services—without compromising on performance or cost.

Highlights of the Speech

The following is a summary of the highlights from this speech, around 2000 words, estimated reading time is 12 minutes. If you would like to further understand the speech content or watch the full speech, please view the complete speech video or refer to the original speech text below.

As Peter DeSantis delved deeper into the intricacies of AWS's serverless journey, he emphasized the pivotal role played by services like S3, Lambda, and DynamoDB in delivering on the promise of serverless computing. These services, he revealed, were built on top of AWS's massive infrastructure, enabling them to take the six most important attributes of a cloud computing service to new heights.

DeSantis elaborated on these attributes, highlighting how serverless services leveraged AWS's vast infrastructure to achieve unparalleled elasticity, with each customer workload representing a mere fraction of the overall capacity. This shared capacity model not only ensured cost-effectiveness, as customers only paid for what they used, but also contributed to better sustainability by optimizing power consumption – a testament to AWS's commitment to environmental responsibility.

Moreover, DeSantis underscored the inherent security and availability advantages of serverless services, which were built from the ground up to run on AWS's infrastructure, taking full advantage of native capabilities like AWS Nitro and the Availability Zone architecture.

Turning his attention to the challenges that had hindered the widespread adoption of serverless computing, DeSantis acknowledged the familiarity with legacy code as a significant hurdle. He drew a parallel to the enduring presence of mainframes, noting that their longevity was not a result of developers' love for the platform or its innovation, but rather the difficulty and expense associated with migrating away from these entrenched systems.

DeSantis also highlighted the initial limitations of serverless offerings, which deliberately introduced more targeted product offerings to deliver on the promise of serverless capabilities. He cited the example of DynamoDB, which at launch offered high-performance reads and writes with minimal query semantics, a far cry from the capabilities of traditional SQL databases at the time.

However, AWS's commitment to innovation was unwavering, and DeSantis proudly shared that the company had accelerated its pace of innovation, introducing new features and capabilities that fundamentally changed the way these services could be used. He pointed to the addition of transactions to DynamoDB in 2018, which transformed it into a much more viable replacement for traditional relational databases.

Furthermore, DeSantis celebrated the launch of entirely new serverless capabilities, including EFS (Elastic File System), Lambda (which pioneered serverless computing as a service), and Fargate (which enabled serverless containers).

As he delved deeper into the journey of making relational databases more serverless, DeSantis highlighted the pivotal role played by Amazon Aurora, which introduced a groundbreaking database-optimized distributed storage system called "Grover." This innovative approach, he revealed, allowed Aurora to disaggregate storage from compute, providing high durability by replicating logs across multiple Availability Zones.

The true power of Grover, however, lay in its ability to reduce the I/O demands on the database by a staggering 80%, a feat that contributed to Aurora's impressive 3 to 5 times price-performance advantage over equivalent open-source managed databases.

DeSantis also lauded Grover's ability to provide the durability of multiple Availability Zones without the need for complex database replication setups, further simplifying the management overhead for customers.

Despite these advancements, DeSantis acknowledged that Aurora still fell short of being a truly serverless service, as scaling out read capacity required manually adding read replicas, and increasing write capacity necessitated a failover or resizing of the server – a process that was far from serverless.

This realization led to the launch of Aurora Serverless, a solution that could scale up and down seamlessly as database load changed, without the need for resizing or failovers. DeSantis delved into the technical details, explaining how Aurora Serverless leveraged a new hypervisor technology called "Caspian" to achieve this feat.

Caspian, he revealed, employed a technique called "cooperative oversubscription," which allowed it to run multiple databases on a single physical host while dynamically allocating resources based on actual needs. This approach ensured efficient resource utilization while maintaining the security and isolation benefits of a hypervisor – a critical requirement for AWS.

DeSantis provided a detailed example, illustrating how a Caspian instance could be configured to support the maximum amount of memory available on the host, such as 256 gigabytes. However, unlike traditional hypervisors, physical memory was allocated separately based on the actual needs of the database running on the instance, a process controlled by Caspian's heat management system.

In one scenario, DeSantis demonstrated how a single Caspian instance running on a host with 256 gigabytes of memory could be allocated only 16 gigabytes to run its database, showcasing the efficiency of the cooperative oversubscription model.

As the narrative progressed, DeSantis acknowledged that even Caspian had its limitations, as it could not scale beyond the physical constraints of the host it was running on. This realization led to the announcement of Aurora Limitless, a groundbreaking sharded, autoscaling database service that could scale virtually infinitely while maintaining transactional consistency across shards.

Aurora Limitless, DeSantis explained, employed a unique approach to making cross-shard transactions perform exceptionally well, leveraging the custom time synchronization technology he had previously discussed. This technology, which delivered timing signals accurate to nanoseconds from atomic clocks, enabled the ordering of hundreds of thousands of events per second, a critical requirement for distributed transactions in Aurora Limitless.

Shifting gears, DeSantis turned his attention to AWS's efforts to make caches more serverless, introducing the new ElastiCache Serverless offering. Like Aurora Serverless, ElastiCache Serverless leveraged Caspian instances to automatically scale cache clusters up to an impressive 5 terabytes without the need for manual management overhead.

DeSantis also highlighted the impressive performance of ElastiCache Serverless, revealing that the median latency of an ElastiCache lookup was a mere half a millisecond, with exceptional outlier latency as well.

As the evening progressed, DeSantis invited Brent Rich, the Head of Global Infrastructure and Operations at Riot Games, to share their journey on AWS. Rich's enthusiasm was palpable as he recounted Riot Games' decision to go "all in on cloud" with AWS, driven by the need to keep pace with the demands of their rapidly growing player base of over 100 million players worldwide.

Rich shared the challenges they faced, such as safely deploying and configuring over 30 microservices, and how AWS delivered solutions to address their needs. He also highlighted the unique challenge of mitigating "peeker's advantage" in their tactical shooter game, Valorant, which required a server tick rate of at least 128 per second and network latency under 35 milliseconds to players – a feat made possible by leveraging AWS's global infrastructure.

Rich also emphasized the benefits Riot Games experienced by migrating to AWS, including improved uptime, with outages turning from multi-hour events into mere hiccups that players barely noticed. Additionally, he lauded the improved visibility and ease of data retrieval, with most information now accessible through simple API calls.

In a testament to the scale of their migration, Rich revealed that Riot Games had successfully migrated 14 data centers, modernized their decade-old game for hundreds of millions of loyal players, and launched several global games in the cloud – all within a span of 36 months.

As DeSantis transitioned to discussing AWS's efforts to reinvent data warehousing, he introduced the next generation of Amazon Redshift Serverless, a solution that employed AI-driven proactive scaling based on forecasted workloads. This innovative approach, he explained, used machine learning models trained on historical data to automatically adjust capacity based on anticipated query load.

However, DeSantis acknowledged that even the most accurate forecasting models could not account for every scenario, which is why Redshift Serverless also employed intelligent query routing to separate large and small queries, scheduling complex, long-running queries on dedicated capacity to avoid bogging down the production cluster.

To achieve this, Redshift Serverless employed a query analyzer that created a feature embedding of each query, taking into account over 50 unique features, such as query structure, types of joins used, and dataset statistics. This approach enabled Redshift Serverless to accurately identify the complexity of each query and optimize its execution accordingly.

DeSantis revealed that a staggering 80% of the queries that run on Redshift Serverless had been seen before, allowing the system to quickly look up information about the query and execute it with high confidence. For new queries, Redshift Serverless employed a small model trained on all queries ever run on the data warehouse to generate a reasonable prediction about execution time.

However, for complex queries, Redshift Serverless employed a much larger model trained on all data warehouse queries ever run, enabling it to understand the likely performance of the query in greater detail. This additional analysis, DeSantis explained, was crucial because query scaling could exhibit sublinear, linear, or even super-linear behavior, depending on factors such as memory constraints.

To address this challenge, Redshift Serverless provided customers with the ability to specify their cost and performance sweet spot, allowing the data warehouse to optimize resource allocation accordingly. Customers could choose to run in a cost-optimized mode, adding resources only if it was cost-neutral, or opt for more aggressive scaling, even if it incurred additional costs, based on their business needs.

DeSantis proudly proclaimed that the new Redshift Serverless could deliver up to 10x better price-performance while automating capacity management, data layout optimization, and other operations that previously required manual tuning – a significant step forward in AWS's mission to alleviate the burden of undifferentiated heavy lifting for its customers.

As the evening drew to a close, DeSantis provided a tantalizing glimpse into AWS's groundbreaking work on quantum computing and error correction at the AWS Center for Quantum Computing. He explained the key challenges of noise and error rates in quantum computing, noting that while current quantum computers with hundreds or even thousands of qubits were being produced, the fine print revealed that they were noisy and prone to error.

DeSantis delved into the intricacies of error correction in classical computing, where bit flips (a zero turning into a one or vice versa) were relatively rare and could be protected against with a low overhead, such as using a single parity bit for every eight bits of data.

However, in the quantum world, noise was much harder to mitigate, as quantum objects were far more sensitive to environmental noise, and qubits could experience errors not just in the form of bit flips but also phase flips – a two-dimensional challenge.

DeSantis shared the progress made in minimizing qubit error rates, revealing that 15 years ago, the state of the art was one error every 10 quantum operations, while today, the best systems could achieve one error per 1,000 quantum operations – a 100x improvement in 15 years.

Despite this progress, DeSantis acknowledged that qubit error rates were still far too high to be useful for the quantum algorithms that researchers were excited about, which required billions of operations without an error.

To address this challenge, DeSantis explained that error correction could be employed by encoding a block of physical qubits into a logical qubit. However, due to the high error rate of physical qubits, each logical qubit currently required a few thousand physical qubits – a significant overhead.

DeSantis then shared a glimpse of AWS's innovative approach to quantum error correction, revealing that the team had developed a custom quantum chip that separated bit flips from phase flips. This approach enabled a remarkable 6x more efficient quantum error correction than standard approaches, a significant step towards developing the hardware-efficient and scalable quantum error correction needed to solve interesting problems on a quantum computer.

Throughout his speech, DeSantis wove a captivating narrative that showcased AWS's relentless pursuit of innovation, driven by a commitment to making its services more serverless across databases, caches, data warehouses, and even the frontiers of quantum computing. He highlighted the key enablers that underpinned this journey, including AWS's massive infrastructure, custom silicon like Nitro and Graviton, and pioneering approaches like Caspian hypervisors and Grover distributed storage.

As the auditorium lights came back on, the audience was left with a profound sense of awe and anticipation for the future of cloud computing. Peter DeSantis's speech had not only provided a comprehensive overview of AWS's current offerings but also offered a tantalizing glimpse into the innovations that lay ahead, innovations that promised to redefine the boundaries of what was possible in the realm of serverless computing and beyond.
 

Here are some exciting moments of the speech:

The leader greets the audience at Monday Night Live, emphasizing the event's unique aspects like loggers, laughter, launches, learning, and more loggers.


The leader explains that legacy systems and developer familiarity make it hard to quickly adopt new technologies like serverless computing.


AWS believes using a hypervisor is the only secure way to isolate customer workloads when sharing server resources.


The Nitro hypervisor provides consistent performance for EC2 instances by reserving memory and CPU resources.


AWS Caspian allows easy database sharding and scaling to support limitless database growth.


You can now specify cost and performance preferences to optimize Redshift queries.


The leader emphasizes AWS's commitment to continuous innovation across emerging technologies like quantum computing to help customers build better experiences.

Summary

The talk focused on AWS's journey to reinvent traditional infrastructure into more serverless offerings. Three key points were made:

First, AWS has developed innovative technologies like Caspian and Grover to enable traditional databases like Aurora and ElastiCache to seamlessly scale up and down based on demand. This removes the need for capacity planning and makes the services truly serverless.

Second, AWS announced major improvements to Redshift Serverless, including using AI and machine learning to optimize query performance. This enables up to 10x better price-performance by intelligently routing queries and right-sizing clusters.

Finally, there was an overview of progress towards quantum computing. AWS is developing new quantum error correction techniques to create logical qubits orders of magnitude more reliable. While there is still a long way to go, these innovations bring the possibility of quantum advantage closer.