Powering high-speed workloads with Amazon MemoryDB for Redis

Hello, everyone. You are here because you're developing an application on AWS and you're looking for the right database to store your data. If you're looking for a low latency database with flexible open source APIs that can scale to millions of requests per second, you are in the right room.

In this session, we will talk about three customer cases - customers who had a similar problem. We will show you how Amazon MemoryDB for Redis is the lowest latency database that AWS offers today to help these customers solve their problem.

My name is Itai, I'm the General Manager of In Memory Databases. With me is Kevin, the Principal Software Engineer who led the architecture of MemoryDB.

So in this session, we will:

• Talk about why use Amazon MemoryDB and introduce the service
• Talk about the customer use cases
• Later on dive deep into MemoryDB and see how we achieved durability, consistency, performance and scale
• And then we will have some time for questions

We will talk about the following three use cases and what's unique about them or what's common across them. And even though each one of them uses MemoryDB in a different way for a different use case, the common theme that you will see is that they all needed a very fast, durable database and that they all use MemoryDB for data structures.

Our first example is Twilio Segment. Twilio Segment is a leading customer data platform used by 25,000 businesses to collect, clean and activate AI-ready data. They use MemoryDB for GDPR deletion orchestration.

Our second use case is HUD. This is the largest open metaverse platform with millions of players around the world. They use MemoryDB for gaming sessions and for user profiles and hot data like maps, likes and reviews.

Our third customer use case that we will review today is Mediaset Infinity, which is a streaming platform for streaming content over the internet, both live and on demand. And they use MemoryDB as a cache for a reality TV voting system.

So something we see common across all of these use cases is that modern applications need near real-time performance. Data grows exponentially year over year and this is why they need high scalability to support more data, more requests per second and up to millions of requests per second.

Modern applications also use microservices, and when you do that, the latency accumulates up the stack. And this is why you need a very low latency for each one of the microservices. And this is why you need microsecond latency databases.

And the other thing that we see common is that modern applications a lot of times choose open, flexible APIs for their applications.

Customers like you are adopting new modern architectures for building applications to drive agility, innovation and efficiency. And developers are looking for the right database that meets their applications' unique requirements across data model, functionality, performance and scale.

And with traditional databases, customers see several issues across performance, scalability, lack of developer flexibility and a data model that does not meet their application requirements. This leads to additional cost, both infrastructure cost, as well as development costs and maintenance costs. The one size fits all approach where you use a traditional relational database for everything just no longer works in order to meet modern applications demand.

AWS offers a wide range of purpose built databases. With the explosion of cloud native applications that require different access patterns, data models, performance and scale, you can use the wide range of purpose built databases that meets your application's unique requirements.

And based on the application that you are building today, we'll talk about MemoryDB.

So what is Amazon MemoryDB for Redis? It is the fastest, durable database that AWS offers today. It has microsecond read latencies and low single digit millisecond write latencies. We'll elaborate more on performance later on.

Today it is Redis compatible. So you get all of the rich data structures that Redis has with lists, sets, sorted sets, hashes and you can get Redis client libraries in over 50 languages and benefit from the rich Redis ecosystem. You essentially get all of the benefits from Redis with the durability and managed by AWS.

MemoryDB was designed for zero data loss. One of the key innovations that we built into MemoryDB is using a multi-availability zone transactional log. Every write is acknowledged after being written to at least three different copies and at least two different availability zones. This is using the same underlying technology that is used by S3 and by Amazon.com ordering data.

MemoryDB also offers high availability. And when you use our highly available configuration with one replica per shard, we offer four nines of availability SLA. We take care of all of the monitoring, replacements of replicas and failovers similar to other services that we offer.

MemoryDB is fully managed. So we take care of all of the heavy lifting of installation, configuration, patching, monitoring, repeating snapshots automatically and so on. So you can focus on what's better for your application, for your business.

MemoryDB is highly scalable. A cluster can scale up to 500 nodes in a single cluster. This means 100+ terabytes of data in memory. And when you use data tiering, which we are not going to elaborate a lot in this session, when you use data tiering you can get up to 250 terabytes. This is really massive scale.

And at this scale, you can also use MemoryDB to scale to millions of requests per second. MemoryDB has security and compliance. We have encryption in transit, encryption at rest, role based access control for authorization and authentication.

We have integrations with IAM for authentication and we have compliance with multiple programs like FedRAMP, PCI DSS, SOC and it's also HIPAA eligible.

MemoryDB was built from the ground up for performance. Let's look at the write performance. As we said, every write is committed to the multi-availability zone transactional log after being written to three copies and at least two different availability zones. It's only then that you get the acknowledgement. Nevertheless, you get low single digit millisecond write latencies.

A single shard or single primary node can do up to 100,000 requests per second or 100 megabytes per second. And that's on a single shard.

When it comes to read, the performance is really crazy because you get all the data is in memory and you get microsecond read latency. And a single node, either primary or replica, can go up to 470,000 requests per second or you can get to 1.3 gigabytes per second on a single primary node or a single replica node. This is really, really fast reads.

But what's interesting about this is that you have also massive scale. So you get this really great performance at a very large scale. And with 500 nodes in a single cluster, this means millions of requests per second. This means that MemoryDB can be used for your most demanding and largest applications.

So let's talk about the customer use cases. Let's first talk about microservices. Customers use MemoryDB when they have a microservices architecture and they sometimes use it as a way to synchronize between the different microservices. Maybe as a message queue to pass data or messages between the microservices. Sometimes as a durable data store for a single microservice and sometimes as a shared database for multiple microservices. These are just different design patterns that customers use with microservices and MemoryDB.

With web and mobile applications, customers are using MemoryDB for storing content, data for session management, for chat, for message queues and they benefit from the rich data structures that MemoryDB offers - whether it's lists for queues, hashes for session stores or geospatial commands for the geospatial indexing. We will elaborate more and show you concrete examples on how to use these data structures in just a few more minutes.

We have retail customers using MemoryDB for storing customer profiles and for storing inventory tracking and fulfillment.

We have large customers in the gaming industry and we'll talk about HUD in just a few more minutes. Similar to retail customers, we have customers using MemoryDB for storing profiles, history, preferences and likes for a full 360 view on the customer.

In the banking and finance industry, we have customers using MemoryDB for transactions and for fraud detection.

We have several customers in the media and entertainment and we see them using MemoryDB for storing data stores and real time streaming. We will elaborate on the Mediaset Infinity example in a few more minutes.

And last we have customers in the IoT using MemoryDB for streaming device data and operational insights.

So let's dive deeper. I will hand it off to Kevin.

Kevin: Hello everyone. I'm going to talk a little bit more about those three case studies that Itai introduced earlier and we'll talk through how the customers use MemoryDB.

Though the use cases and success stories are real, the actual code examples that I'll show are just simplified examples and may not represent exactly what the customer's code looks like.

So let's talk through Twilio Segment first. As Itai introduced, Twilio Segment is a leading customer data platform used by over 25,000 businesses to collect, clean and activate their AI-ready data. Their primary use case for MemoryDB is to back their General Data Protection Regulation or GDPR deletion orchestration system where users can come in and request that their data be deleted, which then propagates to a number of different downstream platforms.

Their requirements were, and you'll see this repeated throughout the different use case studies that we have today, were low latency reads and writes with very high throughput. They wanted their data to be stored across multiple availability zones redundantly to be able to tolerate availability zone failure. They wanted to have multiple different efficient data structures to organize their data within the single database. And additionally, they wanted backup capabilities for disaster recovery.

And so MemoryDB allows you to export your data into your own S3 buckets that you can retrieve later on in another region if you so choose.

So let's look at an example architecture of how this might work. Here you see user Alice on the left hand side that can make a request to the front end API service to delete her data. What we'll show is that this request will be stored inside MemoryDB as a JSON document, which is one of the data structures that MemoryDB supports and will be used throughout this use case to store the state of that request as it is processed by multiple different downstream services.

The second half of the use case will be the front end service actually acting on that request and communicating with downstream services that may have that user's data to perform the deletion. And we'll show with this how you can use a list as a job queue for the downstream services to pull and take action on that deletion request.

So let's look at the first half of how we store this request. With MemoryDB, you can use the JSON set command and this is just a CLI command that is used by multiple different clients. And all of the different requests take a key name as the primary object that it operates on.

Here we're going to call it "job:alice". So the job ID would be "alice" and we'll store all the different request metadata associated with the user's request - the request type, start time, and the user ID.

Once we send the command, it will be stored durably across multiple availability zones at which point it is acknowledged with an "OK".

So the first thing that we want the front end to do is to set it to "in progress". And so similar to setting the entire document, you can set different fields using the JSON set command. And so here we'll set the "status" field to "in progress". We'll get an "OK" back.

And if we were to use the JSON get command to retrieve the entire document, we'll see that it has the initial metadata plus the status. And so we can append and read different fields from this JSON document as we continue to process this request.

So let's look at the second half of this use case, which is now the front end wants to communicate with different downstream services to do processing on this request. For this, we'll use a list. And so this is similar to from computer science, a linked list that's implemented inside Redis and MemoryDB. You can have a variety of different commands to operate on this list, either on the left hand side, the head of the list, or the right hand side, which is the tail of the list.

Here we want FIFO semantics, so the first in, first out. And so we'll push to the right hand side of the list, the tail, and pull from the left hand. So our "RPUSH" stands for push to the right side of the list. It operates on a list object. We'll name our list particularly to the downstream service that we want to communicate this job to.

So you can have multiple different downstream, each of which might have the data. Uh so we'll just say downstream one, we'll push the job Alice. Once you do this, you'll get back the, the number of elements in the list and it'll create this one element list.

Other jobs can come in, say Bob wants to delete his data and it'll update that list and the tail pointer. So you can efficiently operate on the front and the, and the, the back of the list. So from the downstream side, they want to be able to pull elements off this list and then operate on them.

So from that, we can use the LPOP command which will remove from the left hand side and in FIFO semantics. And so once we execute this, it will return Alice and remove it from the list.

Similarly, if we then pop again or remove Bob, you'll get, you're left with an empty list. And if you pop for an empty list, you get a nil object back. Uh and so you know that there's nothing left to do.

However, we don't want our downstream services to just continue in an infinite loop, getting back an empty object and sort of wasting CPU cycles by uh requesting Memory DB uh significant rate. So we can use what's called a blocking command.

So uh the Redis API supports a variety of different long pole commands where you can say I want to wait for data to be pushed. So similar to the LPOP command, you just prefix it with this B letter which is a blocking list, pop and you add an additional argument which is a time out here. Zero is blocked forever. Wait until uh data is added, but you can add a certain time out at which point it would return nil as well.

So if we do this, then it'll wait until a producer comes in and adds an item to the list and then it'll immediately return the list name and the the item that was added and remove it.

So Patrick Sandquist, the the engineering lead in Twilio Segment said that we are impressed by Memory DB's, high throughput and low latencies. Memory DB has enabled us to have a durable primary database with the performance benefits of Redis.

The next use case I want to talk through is that of Bud Technologies. So as I mentioned, Bud's, the largest open meta first platform with millions of players and user generated content Bud uses Memory DB for a variety of different use cases. Some of which we'll talk through here.

They use it for a primary database for their game session data such as leader boards. They use it for storage of hot frequently changing dynamic data like maps, likes and reviews that are core to their platform and they also use it as storage for their user profile data.

So all of this data can be co located in a single Memory DB cluster so that it can be acted upon from the same locations. Their requirements were similar to what we saw before low latency reads and writes with very high throughput to support the millions of users and and tens of millions of digital assets.

They wanted durability of these assets across multiple availability zones within a region. They wanted to, as we talked about the variety of use cases, they wanted to be able to represent those in the same database. So have flexible data storage and access.

They also wanted to be able to uh support uh spikes in terms of of read throughput. And so, uh with, with Memory DB, you're able to read data across multiple different availability zones by having replicas of a given uh data. And that allows for parallel execution if say there's a hot item that uh or hot uh mini game inside the the metaverse that becomes super popular.

This is their high level architecture. You can see that they use Amazon GameLift for their actual online service hosting. Uh and for the back end services, they use a variety of different databases. One of which is Amazon MemoryDB for Redis, which we'll talk through da

So let's talk through some of the example data structures you could use to represent those use cases. We talked before about before we'll show how to use a counter within the Redis API to count likes for different content. A hash data structure for storing reviews of content and a sorted set to represent leader boards for mini games inside the metaverse.

The simplest thing is at a like which is just an intent from a given user that I like this piece of content. You know, I want a plus one and we'll count the number of plus ones for, for the content ID. So with this, you can use a counter within the Redis API which allows you to increment and decrement. This case will just allow increments.

And so the command within the Redis API says is INCR for increment and you can name your key. However, you'd like here, we're saying likes colon the content ID. When it's empty, the first increment will increment it to one and then subsequent increments will continue to increment, it returns the current value.

And so if you wanted to display this uh say on, on the actual page, you could use the GET command similar to how you would get a normal string and you get back the current value. And there's a variety of different commands to increment by certain amounts or, or decrement uh that you can manipulate this counter with very high throughput of um you know, hundreds of thousands of concurrent users.

The second data structure we want to talk through is backing review content where now in addition to just having a plus one, we want some uh data associated with that relationship. So different users can come in and review a piece of content. And this, we can use a hash which is basically just a hash table that you would be familiar with with uh from uh your, your applications. But it's now a remote data structure that can be used across a variety of different client applications.

And here HSET is the, the command within the Redis API that allows you to manipulate or add data, key value pairs to a different hash structure just like our likes key. We'll call it reviews, colon content ID. And we'll say that the user, Alice has a review of this made me cry. It'll create the hash table if it's not initialized and other users can come in and add data to that hash as well.

There's a variety of different commands to iterate through and read the hash here. We'll just do HGETALL, if we want to print out all the reviews and we'll get the users as well as the reviews that they posted.

The last use case I wanna talk through that is that of leader boards. So within the metaverse, there can be different little mini games that users are able to interact with and have scores associated with it. And for this, we can use a sorted set, which is kind of a special data structure that is uh combined some of the properties of a set where you have a set of unique users that uh are part of it, but they have a score associated a numeric score.

Uh and that uh it's a internally, it's a combination of two different data structures, uh uh a set as well as a list to maintain sorted order of by score of these different users. So let's look at how you would use it to populate and read through a leader board.

So within the Redis API the Z prefix is used to indicate sorted set operations. So you'd call ZADD to add an entry to a sorted set. And so we can add Alice's score of 31 and it creates this data a couple of different data structures. One of which here is on the bottom a list which has the user name as well as the score associated with it.

And so as we continue adding scores at addition time, Memory DB adds it in sordid order from smallest to greatest. So here we have four different scores each time we add it, it adds to the list.

So now we want to be able to read from this to actually display our our leader board. And with that, we can use the ZRANGE command. And again, there's a variety of different options here to display exactly what you want to do. This will show the entire leader board from highest score to lowest score.

And you can see that Doris has the high score of 56 and Charlie has the lowest score with 18.

So Steve Zoe, the, the head of the technical team of Bud said that Amazon MemoryDB for Redis which delivers microsecond, read and single digit millisecond response times and handles millions of transactions per second has helped the Bud team to quadruple its research and development efficiency.

The last use case that I want to talk through is that of MediaSet Infinity, which is an Italian streaming platform for viewing streaming content via the internet, both live and on demand.

So MediaSet Infinity uses Memory DB to back their reality television voting system. So they have uh a real uh live streaming shows that, that air uh where users can then log in and vote for different contestants at, at certain times during the, the, the show and those results are then broadcast after the, the voting is closed.

The requirements for this were similar to the last two use cases, low latency reads and writes with very high throughput so that millions of users could log in and vote in the short time intervals. Uh during the show's airing, they want a durability of these results across multiple availability zones.

They want to be able to easily, easily and seamlessly scale to handle these expected peaks during the show's air time. And I will talk a little bit further later on about how Memory DB scales.

So let's look at their high-level voting system architecture. There's a lot going on here, but we'll walk through uh how it uh the sequence uh for the the voting system.

The first step is that an administrator who's running the live event would log in and open the voting session and say voting is now allowed for the different users to come in. They can't cast votes beforehand and they'll interact with the back end to do this.

Then as soon as that happens, the users on a variety of different interfaces can go in and, and to the front end service and cast their votes, the votes are persisted within Amazon CloudWatch ultimately to S3. But the real magic happens on the MemoryDB side where the votes are then validated and aggregated.

And after the voting is done, the time period is closed, those results are then retrieved from me Memory DB and broadcast on air.

So if we zoom into the Memory DB requirements a little bit further, we want to make sure that the user can only cast a vote during an open session. In this case, we won't allow the user to cast perhaps multiple votes.

So we'll show an example of four during each session, we wanna be able to support millions of different users concurrently during each session. So we want to make sure that the results are consistent and that we don't drop votes or double count them.

And to actually support this, we'll use a few data structures. One is a string to store the global state for which session is open at a given point in time. We use a counter like we saw before to actually count the votes both for the contestant as well as the number of, of uh votes cast by each user.

And we will package this up in Lua script, which is a scripting language that the MemoryDB supports to be able to do multiple operations atomically so that the results are consistent.

So let's look through an example of a Lua script. And you don't need to understand every single line here, i'll walk through it uh because Lua is a language that many of you might not be familiar with.

So the first section of it will be to determine if the vote is valid. And so here we'll retrieve a, a key from Redis, for example, the current session and compare it against the past an argument. So only if the administrator has set the current session appropriately will proceed. Otherwise, it will air out, we then check if the user has cast all of their votes.

So we look at a user specific key and in this case, if the votes cast is greater than equal to four, we'll air out. Now, we've proceeded with the past the validation and we can then check to see or actually perform the actions associated with the voting.

So here we'll increment the keys, vote uh cast by the user. And so this will be the the user specific vote key and then we'll vote for the actual selected option. So for example, the singer of, of carroll and return as a hint to the client, the number of votes they have remaining.

So this can be run by millions of users concurrently. It'll be sequenced by Memory DB one after the other so that each of these blocks happens atomically.

So let's look at this and how it would work in practice after you have this script, we'll have the administrator come in and set this current session key with att l. So uh MemoryDB supports expiration, we'll set a 10 minute validity so that votes will only be allowed for 10 minutes as soon as we get the ok back, we then see a variety of different uh script commands come in.

And that within the, the Redis API is the EVALSHA command where you have loaded a script and then execute it with a variety of different user names and contestant IDs. And so these, you'd see uh an influx of these at that point in time and after the 10 minutes are over and the script is or the, the key is expired for the session, you can then retrieve the aggregate results for all the different contestants and report them back.

So Daniele Ky, the software engineer and solution architect at MediaSet said that using Amazon Memory DB for Redis viewers had a very good experience, could express their votes quickly and didn't encounter any errors.

Next, I want to talk about the Memory DB architecture and how it actually achieves this multi, a durability with the blazingly fast performance that i talked about earlier.

So as you might or might not be familiar with in terms of uh a Redis cluster, each shard is composed of a single primary and perhaps multiple replicas, each of which represent uh an in memory copy of the data.

However, what's special about Memory DB is that it has this multi transaction log, which is used as a replication bus connecting the different nodes in the shard. So when a write from a client comes in say to set a to 100 it manipulates the in memory state first but uh then it synchronously writes to this multi transaction log and only once it's durably written in two AZs to disk, do we get an acknowledgement back from the transaction log and then acknowledge the write back to the client?

So this this ensures that you have multi durability for every single write, the replica nodes still asynchronously consume this data, but it's guaranteed to be consumed by the replicas that asynchronous consumption by the replicas means that replicas reads from replicas are eventually consistent.

So how that would look like is immediately after you get the acknowledgement. for write, you can read the data from the primary and this will be served from memory directly in the primary. However, if you were to read the rep uh from the replica before it's propagated, it just returns the in memory state, you might see an empty record. But after the data is propagated, usually on the order of milliseconds, you'd be able to read that data from the memory of the replicas.

So how do we ensure that during failures we maintain durability and consistency

Let's take another use case, slightly more advanced, where we have multiple sets that are occurring on the primary. We get acknowledgement back for all of them from the transaction log. But let's say we only have one of them propagated to the replica. And at that point in time, the primary were to fail.

And we want to be able to plan for failures of either single nodes or availability zones. In order to maintain high availability, we need to fail over to one of the replicas, but we're not yet caught up.

So what MemoryDB does is it ensures that we read to the tip of the transaction log and the transaction log actually mediates the fail over itself. And so as soon as we've consumed all of those sets in memory only, then do we transition to a primary and start accepting writes from additional clients?

So that ensures that now the the primary, the new primary state matches the old primary state and we can continue where we left off and replace the, the old node as a replica. And then other replicas that are in other availability zones can continue to consume from it asynchronously. And that allows zero data loss during fail over.

Now, I want to turn it back over to Et to talk more about the scaling characteristics of MemoryDB.

Thank you, Kevin. All right. So we talked about elasticity and scale. Let's see how this works.

So here you can see an example of a MemoryDB cluster. It has three shards and every shard has a, has one primary node and two replicas. We will use this example in the next few minutes.

When we talk about skilling, the small colorful squares that you see here represent data and more accurately, you can think about every square as a group of data items. Each with a key and a value in MemoryDB, each group of data items is called a slot, which is a Redis term. And every key is uniquely mapped to the slot.

You can see in this slide that there are multiple slots in every shard and that every primary node and every replica has the same has the same slots within the shard.

So like any other system, when you get close to the capacity, you need to scale either the memory capacity or the throughput capacity. So let's see how scaling works and the scaling ways to do with MemoryDB.

So we have three scaling methods. You can scale just you read throughput by adding or removing replicas. You can scale vertically by scaling the nodes to bigger instance types or smaller instance types, which gives you primarily more memory but also more cpu and network bandwidth. And you can scale horizontally by adding or removing shards. This is the most common and most recommended way to do scaling.

So let's start at the fir with the first one which is scaling the read throughput by scaling the replicas. So when is this recommended? So when you have a meaningful increase or decrease in read throughput, but you don't have a meaningful change in the amount of data and you don't have a meaningful change in the amount of write throughput. This is where you wanna go and scale your read replicas. When you do that, you will get more read throughput.

So in our example, with two replicas per shard, we can just add another replica. So you can see that the new replicas here have additional copies of your data. These copies of the data as Kevin explained earlier are continuously updated from the multi availability zone transactional log. Once you have more copies, that means you can get more read throughput with these new copies.

The second way to scale is to scale up vertically. And you would typically use this scaling method only in use cases where horizontal scaling is insufficient. The two common examples are you have an increase in throughput in a lot into a subset of keys. For example, you have very hot keys or when you have very large data collections, for example, tens of gigabytes in a single key. Both of these are not recommended. We generally recommend that you split the load more evenly across keys and split the data more evenly across keys. However, we recognize that it's not always doable and this is exactly why we're able to scale up into bigger instances uh and scale vertically.

So how do here is an example. For example, you have an r six or r seven gx large and you wanna move to r seven g to x large, all right. So when you do that, you get double the amount of memory, you get double the amount of cp us. But you also pay double the cost.

When you do that, when you, when we do the scaling up, we first provision new instances, they do a full synchronization from the old instances for or from the multi a transactional log as Kevin explained. And once all the data is copied, we go and delete the unused instances.

Let's look at the third way of doing scaling which is horizontal scaling. This is the most common and most recommended way to do scaling with MemoryDB you scale horizontally by adding or removing shards. When you do that, you can scale both the reads and the rights. You can scale the network throughput and you can scale the memory capacity. And what you get out of this is a lot more granular scale scaling.

For example, let's say you have a um a cluster with 10 shards and now you need 10% more capacity, then you can simply add another shard, which will give you this 10% more capacity. When you have vertical scaling, you cannot just scale by 10%. And this is why vertical scaling is less flexible and less granular. And this is why customers prefer horizontal scaling.

So let's see how horizontal scaling works. Here is an example of a cluster with three shards. And we want to scale it to six shards MemoryDB first provision, new ec2 instances and new transactional logs for the new shards. Then MemoryDB decides which slots to move to which shards we call this process resharding.

During the resharding, the keys are moved atomically without any interruptions and without any data loss. In addition, when you do multi key operations, for example, the Lua script that Kevin presented a few minutes ago, when you do those operations on a single slot, then this happens without any interruption during the rearing scaling in is quite similar.

We initially do the shouting and the shouting is done in the exact same way, every key is move atomically without any interruptions. And without any data loss. Once we finish the rearing, we go ahead and delete the unused instances and transactional and transactional logs.

So let's talk about performance. This is gonna be the last part of the session. So our customers asked us for extreme performance. In fact, this is one of the reasons they love MemoryDB, they love the extreme performance. And over the years, we've built multiple performance features and performance optimizations in this session. We will talk about one of them, which is our enhanced IO library that primarily optimizes MemoryDB's read path.

We initially built this library for Elastic Cache. And then we ported it to MemoryDB, we will talk about two primary optimizations. The first optimization is enhancer itself which boosts MemoryDB read, read throughput by up to 3x and reduces the latencies by up to 75%. This is a massive performance improvement.

But then we thought can we improve it even more? So we did enhancer multiplexing which further added 46% more read throughput and 21% lower read latency for use cases that have many client connections.

So let's understand how we did it. So the command execution in the engine is really really fast. And if you think about simple commands like a re command that has, you know, a few hundreds of bytes of data, it happens typically in one or two microseconds. So the engine is really really fast in handling commands.

But what happens when you have multiple client connections, the old dio is serialized on the same engine thread. And then this shares times this year, the time the cpu time with the main engine thread and the command processing.

So we thought, you know, when we did the profiling of the engine, we saw that most of the time is spent in the network aisle. And this reduced significantly the amount of time that was available for s for handling command processing.

So we thought, why won't we offload this to a separate thread? And this is exactly what we did. So we moved all of the socket communication to a dedicated thread. Later on, we also moved all of the encryption, all of the tls handshake and encryption to also this thread to upload more work out of the main engine thread.

And of course, when you have one connection, it's not that useful because there's just contact switch. But when you have many client connections, which is what customers usually do and they all send traffic at the same time, we're able to run all of this io in parallel. And that's and also all of the encryption in parallel, which gave back a lot of cpu time back to the main engine and was able to boost performance considerably considerably.

It also improved latencies quite a lot because now the client connections don't need to wait in line in order to get their commands processed. So here are the performance result, right? And we see the re throughput increase by up to 3x and we see the latency, both the p 50 the p 99 latency go down very significantly up to 75%. This is a really massive performance improvement.

But then we thought, how can we optimize this even more? So we looked at the how the handle, how the engine handles batching commands. And then in one of the performance cycles, when we did this, we saw that it handles it very, very efficiently.

So we thought that instead of opening a channel for each client, we can have each enhancer or thread combined commands from multiple multiple uh clients together into a single channel to the engine. So this is exactly how it works with enhancer multiplexing. Every each dedicated network io thread combines commands from multiple client connections into a single channel to the engine. Taking advantage of the engine's ability to handle batching commands very efficiently.

So this gave additional performance boost. And we saw that up to 46% better throughput and 21% lower re lacs for ca use cases that have many client connections. Besides enhance io we have multiple chapters in our performance journey. We're not gonna cover all of them today. But what's unique about them that all of them increase the value that use of memory out of MemoryDB? We do not charge extra for this. So you can just benefit and get in more performance and more throughput, lower latencies from your MemoryDB clusters.

So we will conclude here. There's uh some another session tomorrow. What's new with me, amazon MemoryDB for it is it's a very different content than this one and i encourage you to go and you uh join that session. It's tomorrow at 2pm at the MGM Grand and there's a an overflow at the Win.

So thank you for listening. Uh one last comment is that MemoryDB has a free trial. You can try it out for free for two months. So it doesn't cost you anything to try it out. Go ahead and give it a try. We will answer questions in the next few minutes. And if we don't get to your question, feel free to email us to memorydb dash help at amazon.com, or you can uh email us to directly to the these email addresses behind us. And also don't forget to please fill the survey.

So thank you and we'll take questions now.