Evolve your web application delivery with Amazon CloudFront

However, if you're still relatively new, we'll do our best to bring you along. All right. So let's get started.

So in case you weren't already familiar, CloudFront is AWS's content delivery network service, which essentially means it's a service that focuses on helping you accelerate and protect web content delivered over the public internet.

Now, it just so happens this month, the service reached its 15 year anniversary, which is a huge milestone for us. And as you can see from this timeline, it's grown quite a bit since the original launch in 2008. We launched with only 14 edge locations around the world. And over time, we've continually added more locations and added features to the network.

Today, we're actually in over 100...sorry. Let me start over. We have over 600 edge locations deployed around the world. We're in over 100 cities in 50 countries around the world. We also have 13 regional edge caches that are deployed as a mid-tier cache within our network architecture. On top of this, between the edge locations and our AWS regions, we connect them using the AWS Global Network. This is a privately managed network managed by AWS mainly to provide better performance and reliability when routing traffic around the world.

Now, as you can see, we've grown the network quite a bit since the original launch. And a lot of this growth really has to do with the growing number of customers running critical workloads on CloudFront.

So as you can see from this slide, we have customers across a wide variety of industries, from media, entertainment to gaming, financial services, retail, I mean, you name it right. This is just really a short list of some of the customers and hopefully you recognize some of these brands. But ultimately, it's all of our customers that dictate how we continue to evolve the service, adding more features and capabilities over time.

So in today's session, I'm gonna dive into some of those core requirements that many of our customers are really looking for when they come to us asking for help with web delivery. And the first one is caching. Caching is basically table stakes for any CDN. You have to be able to do it, you have to be able to do it well. And the reasons that you want to cache at the edge in a globally distributed network has largely remained consistent over time.

However, what we have seen change is the content that's being served over the network. In the early days, many of our customers were really serving largely static content because they were using their company websites just to share information broadly with their customers. But over time, they've also evolved to have way more interactive web experiences, way more dynamic content and way more personalized content for us. What that means is we had to adapt as well. We're seeing a change in terms of the access patterns, the volume content that needs to be cached and the expectations of the cache itself.

So in 2008, when we launched, we had 14 edge locations and as we added more customers to the network, we continued to expand the network, adding more locations, it extended our reach, increased our capacity. However, something that we quickly realized was to provide better performance caching performance specifically for our customers. We couldn't depend specifically on the cache width that's available at each of those independent locations.

So in 2016, we added regional edge caches, regional edge caches are mid-tier caches that we've deployed around the world in our AWS regions. Essentially, they're super POPs with really large cache widths and what they allow us to do is collapse any request that's a cache miss from the edge POP to a nearby regional edge cache. When we deployed, it was great customers instantly saw an increase in cache hit rate for us. It also provided a number of benefits. We could add more edge locations without increasing load on our customer origins.

So today we have over 600 of those in 2020 we introduced origin shield origin shield is a feature that you can actually configure to turn one of those regional edge caches into a third tier of caching. So whenever there's a cache miss at the regional edge cache, that request would then collapse to the origin shield location. This is great for customers with either really large content catalogs like video libraries looking for better cache hit rates or customers with sensitive origins looking for more offload.

Now, when we work with customers on a caching strategy, we're really looking at a number of different things. The first thing that we look at is the popularity of the content and how that might affect its chances of getting a cache hit when a request is made. So I just showed you within the CloudFront architecture, we rely on a tiered caching strategy to increase the chances of less popular content to be available in the cache. When we're working with customers, we're looking beyond our network, we're looking from the client all the way to the origin servers at places where you might want to be able to cache that content.

The other thing that we're we're always looking for when we're working with our customers is opportunities to leverage the cache to provide the most performance and efficiency for your application as possible, right? Within the HTTP standard, there's capabilities that you can leverage and there's leniency depending on the workload to potentially serve content that maybe, maybe have expired and still exists in cache. If you're having issues with your origin, maybe you want to leverage negative caching to provide a good consistent experience for your end users. If you're not super strict, in terms of the expiration of a TTL, maybe we want to serve slightly stale content while we're refreshing that in the cache.

So all of these attributes are things that we're looking at and it's really a combination of them that you can consider when you're coming up with a strategy for your application.

So the next thing I want to talk about is dynamic content acceleration. In 2012, we launched dynamic content acceleration as our network got bigger, we had more reach and we realized there's actually a lot of value in serving content across large distances over our network rather than, you know, going direct to origin specifically when I say dynamic content, I mean content that is not cacheable.

Now, the first thing we always get asked is like, how does this really work? Right, because we're adding an additional hop to the request response path, which is a fair, a fair observation. And the answer is we do a number of things. The first thing that we do is we optimize the connection by reducing the distance to establish a TLS connection between the client and the server. Now, if you're familiar with TLS and TCP, usually there's a number of round trips that have to happen. And when you're doing that over a large distance, you're incurring a lot of latency before you can even send any request packets.

The second thing that we do is we maintain persistent connections from the edge POPs to your origin servers. Now, when we do this, we're actually able to reuse the connections, not just across a single client, but across multiple clients. And on top of that, we're always tuning these connections so that we can scale them up faster and provide better throughput.

The last capability that we have really distinguishes us from other providers. In that if you're hosting an origin with AWS, we're able to use the AWS Global Network to route the traffic to the origin. Now, when you route traffic over the public internet, you're often bound to the conditions of the network at that point in time, the routes may change and there may be some packet loss and congestion here and there. With the AWS network, our network engineers are constantly monitoring this network forecasting the capacity demands based on the observations we see from customer traffic over AWS border and scaling accordingly. And we take this investment very seriously.

In 2012, we're actually one of the first providers to move to 100 gigabit ethernet connections at our peering connections. 10 years after that, which was last year, we already announced that we're moving a 400 gigabit ethernet connections. And when we monitor the network. we've actually seen quite a bit of a net advantage over long distances. We've seen a reduction of up to 2x in network jitter and 30% in network latency.

So in the last slide, I talked about, you know, optimizing the connection by reducing the distance to establish the connection. Now, let's talk about how you can improve the performance of your application just by using the right protocol or even sometimes the latest version of the protocol.

HTTP, TCP - all these protocols have been around for a while which each with each iteration they've gotten better, right? They've become more efficient or more secure and something that you can kind of do right out of the gate is just use the latest protocol to and test it out to see if it's a good fit for your application.

For example, within CloudFront alone, when we moved from TLS 1.2 to TLS 1.3 we instantly saw a 20% reduction in first byte latency. And when we turned on session resumption, it jumped to 50% improvement right now.

One thing that's worth noting is while the protocols are out there and anyone can implement them implementing them is an investment. A good example of this is last year, we announced this work for HTTP/3 over QUIC protocol, which is a protocol built on top of UDP. And to take advantage of some of those connection migration capabilities in single round trip connection establishment. We ended up building our own termination layer to really optimize it for the best performance possible.

The nice thing about using CloudFront is you can offload all that work to us, right? And you can deploy, test and tune and figure out what's the right protocol for you and testing. It is important.

Having the ability to configure something at the CDN, maintain your existing protocol from the edge POP to the origin server is something you can take advantage of. For instance, we see customers today still who are attempting to move from HTTP 1.1 to 2.0 to take advantage of some of those multiplexing capabilities to stream multiple requests over a single connection. When they turn it on, something that they might learn is their application servers aren't scaled to handle the spikes and requests of, you know, the flood of traffic that's concurrently happening over that stream. So they can quickly roll back or build queuing over time and then turn it back on.

So these are all things that you can consider. Another trend that we're also seeing continually grow over over the years is the growing need for bidirectional communication.

Now, Web Socket protocol is a protocol that we've supported for some time now and it's pretty much the standard because it's supported by a lot of the clients such as browsers, right? Something that um is worth noting out just by the way, some examples of these use cases, live scoring updates, chat applications, interactive in game experiences. These are all kind of things we're hearing from customers on the use cases for these protocols. But we're always listening to our customers, right? And last week, we actually announced support for gRPC. gRPC is a protocol that's also been around for a while. It's built on HTTP/2. It's great for low latency network connections. And um it's built on a binary format protocol. So, you know, traditionally this is was really designed for server to server communication. But as we've spoken to some of our customers, they've explained that, you know, for custom applications such as application, mobile devices, um this is a protocol they really want to take advantage of. So we're proud to share that we're supporting that now and it, it actually just launched this past week.

Next, I want to talk about um some of the functional capabilities that customers are expecting from a modern day CDN, right? And adaptive web delivery is becoming a pretty common trend these days. So clients are changing, they're continually involved. There's huge variety of them, network conditions are are always changing. There's more mobile networks, the reach is getting longer over that last mile than ever. And for application owners, what they want to know is what exactly the client environment is like. They wanna be able to respond to these environments to provide the best end user experience as possible for their customers. What that means for the CDN is we need to be able to provide the, provide the capabilities to help them with that. As we're working with our customers, we're learning that application architecture is no longer just within the origin. It's extending to the edge, it's extending to the CDN. So we have to provide the granular configuration capabilities, compute capabilities at the edge and it all has to be developer friendly. So CDN is no longer an administrative task. It's now part of application architecture in itself. With CloudFront. we have a number of these capabilities built in natively to the service. However, for the sophisticated use cases, we have Edge Functions in 2017, we launched Lambda@Edge, Lambda. Edge Edge is an ability to invoke Lambda functions that are replicated around the world from your CloudFront configuration on the request and response path of that request flow, right? Um it's designed for sophisticated use cases. You write some Python or Node.js code, you can integrate third party software into your CloudFront configuration. You can make network calls. In 2021 we added CloudFront Functions. CloudFront Functions are lightweight functions. They're JavaScript functions designed for high volume low latency use cases. They can scale up to millions of requests per second, they're designed to execute in sub millisecond time. Um and you know, there's a lot of value in being able to do things like request manipulation on the fly directly at the edge, they're deployed directly in our edge PoPs.

Last week, we also announced a new feature called Key Value Store, which was a huge rush to get to re:Invent by the way. But um Key Value Store is exactly how it's described. It's a data store that's deployed directly at at the edge location and it allows CloudFront Functions to make calls to the Key Value Store. Now with Key Key Value Store, we unlock a number of capabilities. You can now decouple your your application data from the function code itself. Uh this allows you to do a number of things. You can make those calls in really efficient time frames like microseconds. And you know, from our customer standpoint, here's the summary, the use cases that we've seen them use rather than deploying function code every time you need to make a configuration change or update a feature flag. You can now make an API call directly to the edge PoPs and they'll get updated and propagated around the world.

Another interesting thing about Key Value Store is with the launch of Key Valley Store, we introduced a new runtime. This new runtime actually allows you to write more efficient JavaScript code. It opens up the capability to do asynchronous operations using some of the common JavaScript models. So to give you an idea of how customers are using CloudFront and Edge Functions to optimize their web delivery. I'm gonna walk through a few use cases now. And the first use case that we see is one around network conditions. Oftentimes we talk to customers and they say, hey, if our, if our end user is on a spotty network connection, we want to serve a degraded experience to them so that they can at least have some functional use case. But we need to be able to understand when they're experiencing that and integrate it into our edge architecture. So how do we do that within many of the client protocol libraries today? You can actually make calls to determine how expensive that network path is. For instance, this is a call that we pulled from an iOS library. Now, if you get a response that says, hey, we don't really have a good network connection. One way that you can let the server know to serve, maybe a degraded experience is to use the same data header with the Save-Data header. We can now pass that along with the index request, right? And on the CloudFront function itself, we write a simple if statement and say, hey, let's rewrite this URL to a path that actually serves that bandwidth saving version of the site.

Another example is we have customers using Edge Functions to optimize for latency and performance. One of our customers IMDb actually identified a use case for this where if you're familiar with CORS, CORS is basically a standard that's used to make sure that cross domain sharing of assets is is allowed. And part of this protocol is to send a preflight request to validate that, hey, this domain has access to this asset hosted on another domain, right? If you have a network call that could potentially affect the performance of the server, it's actually required for for the browser to send this preflight request. Now, what our customer realized was as more and more customers started hitting their endpoint, there was more preflight requests and their origin was hosted in one AWS region. So as a result of that, they incurred an additional round trip before any actual request could go through to serve the actual data. So they experimented with it, they tried caching it, they saw some benefits there. But you know, the variety of the requests and the domains that came in was good. It it helped. But then they started experimenting with Edge Functions. And when they did that, they realized that they could shave 200 milliseconds off of the request.

Now to expand on this architecture. If you think about Key Value Store, when you're dealing with a use case like this, if you ever had to dynamically change those domains that you wanna approve, you can now do an API call to the Key Key Value Store rather than updating the function code.

The last use case I want to talk about is client consideration. As I mentioned, we now have clients of all different types from smart TVs to web apps to, you know, Android or iOS. You know, the the experience could be different depending on what client the end user is using. And within CloudFront, we have native capabilities to tag and identify that by inserting an additional header, some of the headers that you see here right now for really advanced use cases, sometimes our customers want to optimize for things like search engine crawlers. We have some customers that are still doing, you know, client side rendering with their web applications and they may, they may rely on CloudFront and CloudFront functions to do things like dynamic rendering. So in this example, if CloudFront detects that it's a crawler and I'll talk a little bit more about bot detection later on in presentation. But with ABS Bot Management integrated into the CloudFront, you can actually detect specifically what type of bot is hitting your endpoint from there. You can make a Lambda@Edge function call to determine where you want to route that traffic. Maybe you'll route it to an SEO optimized version of the website.

Alright. So with that said, I wanna hand the mic over to Harsh Kalia who will share how Capital One has been able to leverage some of these capabilities for their workloads.

So users can view the contact much faster without having to wait for javascript or cs s files to load. And as I said, why do we need server side renting because of the c requirement, which is the search engine optimization. When a page is server side render, it's very effective when a user is actually having or having trouble with the slow network connection. And when the page or the content of the page is rendered before even the page is loaded, the search engines can easily crawl and index the page, the social media crawlers or the search engine bots, they don't usually like the javascript rendered content. So since social media sharing was an important part of our marketing strategy, the search engine or the server side rendering was a better option.

The first requirement was the great user experience which is clear and input navigation. So whatever the variation we are rendering, it should be easily navigable by the by the customer. The responsive designs as more and more customers are using mobile. So responsive design is a very critical requirement. And third was the engaging content. So that content which aligns with the user needs and expectation is very critical for us.

So what all, so what are things basically or what i'll be tested using to implement this ab testing. So the first thing was like a in house solution. So the clients will make a call, they will get a variant version as a part of the response. The second thing was contextual bandits. The difference between contextual bandit and a simple a b testing is the contextual bandits implement some machine learning algorithms and divert the traffic to high performing to high performing variations and it reduces the traffic where the variations are under performing. The third was third part of the shelf tools. All these solutions were great. There was one major drawback with all these solutions. This all these solutions were a p based. So just to retrieve a particular variant version, it took a lot of time as you can see basically the network hop, it has to travel. So suppose you're in vegas, you are getting routed to us, west. And then from us was i'm making a call to the a p which is hosted on east because we are running active, passive and it's not direct call actually consider the war, the alb and the route 53 it just keeps adding to the latency and to other thing is what we are doing is on that a p solution. It's not the variant, it's just the variant version. So we are building the entire code base into one. So whatever the variant version is getting retrieved, everything is bundled in a single code base doesn't matter what a variant is getting rendered.

So what we were looking for is basically to ability to test independent variations which can lead to true comparisons. So to talk about that, i will hand over the mic to kevin and he will discuss the implementation details.

Thanks rish. Ok. Now for the fun part, so user experience testing, i'm i'm i'm gonna dive just a little bit deeper into the actual problems that we were seeing. Ok. So one of the things about um everything bundled up as, as harish was saying, right is that you know, size is a killer to performance, ok? And when you have larger packages, all the variants are burned in all of that stuff, it took longer to download, took longer to interpret and it took longer to render, right? And with that comes hits, right? You're gonna take an seo hit, you're gonna take a user experience hit. Uh google was just given up on us. We we, we had zero indexing. You couldn't find us on the internet. It, it, it, it was a bad day, right? Uh the user experience, you know, again, it was definitely one of those like less than impressive. Ok.

So, so what did we do? We said, ok, we got to get the server side rendering in place, right? We we gotta get sco we we we people have to be able to find us, right? So step one, let's go, we're gonna do some server side rendering. Well, we started server side rendering and what we found was, wow, control always one. Why is that? Well, because when you only server side render and you rehydrate or you still have the rehydration requirement right? On the on the front end. Um it works well when control equals control, right? But every single time you deliver control and then the variant is selected. Yeah, that kind of looks pretty messed up, right? Like people bailed on the variant every single time because it looked like, hey, i got delivered one page and all of a sudden it just disappeared and i got this brand new page, right? It made people very nervous and so people we we had tremendous bailout rates, ok? For any variant.

So here we're thinking, wow, control must be really solid because no one uses a variant, right? So once we learned what was actually causing such aaa high uh bias toward uh control, we went ahead and said, ok, we need to implement something right? And what we decided to do was leverage call from functions. Ok. So what we did is we literally took advantage of the 600 nodes, right? That, that aws has and we wrote a a function that is deterministic, right? If given the same input, it will always generate the same output. Ok? Then, then what we did was we put that at the function level, ok? We built a config so that we could dynamically allocate whatever we need. We stored that on s3, we had to create an entire pipeline to push both the function and the s3 because it had to be burned in. And we'll get to why i love key life. But in, in our current solution, what we have is the synchronization between the file that's on s3 and the function itself, you have to take that same config right? That, that same jason object and you have to burn it into the, the config itself, ok? So with limits of, of, you know, character size and execution time and all this other stuff, you know, you, you kind of burning the the the uh candle at bolt ends in, in, in that case, but it was a beautiful solution, right? As it comes in, what we do is we modify the path, right? And so when anybody asks for index, we kind of roll our guys, we make that we, we, we kind of roll that algorithm. Um if they don't have a, a sticky cookie, then we will generate one on the fly, we attach it to the request, it'll go back to be set at the client. So the next time that they come in, they are literally stuck to that same variant because again, deterministic. So same in same out, right?

So then what we did is we took that exact same algorithm and we stuck it in the client itself, right? So when the client goes to render, it looked now every single time the client goes to render, we've always already set that cookie, right? So it looked at the cookie, same thing determines everything is the same, it, it was a beautiful solution. It, it, it was so good that we gave it to product and they went nuts, i mean nuts. Ok? They have run hundreds of tests. Ok? We've, we've tried all sorts of different types of, of home pages and a a and all that stuff, right? Uh everything that harish was talking about about, you know, prequels and all this other stuff, right? Um even to the point where they wanted to geoloc things. So, you know, if they wanted vegas to be the only city that gets a certain variant, we could do that, you know, and, and, and that's the beauty of the call from function. It's not just this a b you, you, you have access to a lot of things and it opens the door right to, to, to being able to, to do some very sophisticated testing.

So this is our full page testing. And again, i referred to the fact that the, the one on the left, right, that was controlled, the one that always won, right? And so we always thought, hey, rich, a rich experience like that is gonna be the simple experience every single time until we finally separated out and actually applied actual speed to the two. And what we found out speed winds every time, ok. The variant crushed control, i mean, crushed it like not, not even a race. Ok.

Um so with that man, we improved click through rates. Ok? Our our conversions went up, i mean, product was incredibly happy, right? It made for faster response. People engaged more in our content, right? And so what were the results of that when we increase performance? Right? By, by making everything smaller and by by being able to break out all these things smaller packages again, simple, render all of that stuff performance went up by three times and when that happened and we could finally start getting index by google and all of that stuff, organic traffic, right? I mean, people who are literally searching, right? And coming to our site directly went up by four x and the big one was time on site, right? The time on site went up 600%. Ok. People were spending six times longer on our sites, right? And when you spend that amount of time, you're engaged, when you become engaged, you're more likely to prequalify and, and, and, and look at cars and, and, and really kind of get into what we can deliver. You start using all of our tools that we've spent hundreds of hours developing, right? And um it actually drove business up, ok? It, it drove business up. We, we started generating a lot more leads uh uh to the point where uh we, we're actually mon as of this year, we're actually monetizing our platform. Yeah.

So, so we've actually been able to be good enough to make this an actual monetti product. So we're selling it today. Ok. So that part is the current story, but i'm super excited about key value store. Now, we were lucky, we were lucky enough to preview key value store, you know, and it's probably because the first time i heard it, i got on the phone with, with alex and, and tino, and i begged, i mean, i begged for it. I emailed them. I, i wouldn't stop, i wouldn't stop pestering them for, for months and months and months. Ok? Because this is what we see as kind of the next level of what we would like to do with this a b testing. Ok.

One of the things that we know is that we have five megs, right? That we can remove from the actual function itself, ok? We're no longer burning the candle at both ends, right? We can put far more functionality into the function itself because we're not worried about the exponential growth of the config file, right? Because you need to handle the parameter, then you need to to to house the parameter, right? Well, with that separation, we can now become way more flexible in in what we want to do in the function and not worry about how big the config gets, right? Because we have plenty of room for that. Ok?

So the other beauty is there's no s3 there anymore, right? No s3. I don't need a pipeline. I don't need to to release this right? I just need to make an api call, right? Same thing that we're talking about the response body stuff right now, i can guarantee you there's there's zero, i should say it's 100% synchron synchronization, right? When that key value store gets updated, the function and the return body will always be in sync, right? My client and my function will always be in sync. I'll never have to worry about invalidating cash. I won't have to worry about anything. That's that, that's being called on the on the browser you know, all of that goes away. Right. And it's a single api call, right.

So, as we were talking about all those cool office shelf api driven all of these things. Right. Ml and, and, and, and a i and all of these things which we want, but with all the re, the, the r tt, the, the, the return time, right, just we couldn't use it. Right. It, it was unusable if we wanted to give those performance and, and, and those enhancements. But now we could put that on the back side, right? So instead of api s driving the determination, what we can now do is have those same systems, any one of those that we like best and they can make an api call and as often as we want can dynamically allocate those tables globally anytime we need it, right? Once a day, once an hour, whatever it doesn't matter, right? So we see this as game changing as far as what we can do, right? And we're also leveraging the fact that, you know, because we're at the edge, right? We, we're not talking west, right? We're not talking us west, right? You wouldn't leave vegas, right? That's how fast you're gonna get these responses back, right?

So what, what does that mean? Ok. What do we see about? What good is it? Well, here's something, how about no code testing? How about straight html injection? Ok. Now, our product has this thing called the ct a card, ok? That's a call to action. Ok? And we love testing new variations of this, right? Different colors, different, you know, all these things, right? So as you can see, that's the default one, the one that's, that's circled in, in, in red, right at the time of render because we can get that config back so fast and that we have so much room in the config that we can actually just in line html, right? When this goes to render, we can swap it out, we can roll the dice know exactly which allocation that we should determine them into and we can just directly inject any of those possible variants html, right? In that same square real time. Now, your code doesn't have to go, you don't actually have to re release your mfe your micro front end and yet you can run all sorts of cool experiments because you're doing it offline in the key value store with your function that you can again update with an api call, right?

Another thing that we look at is component testing. So there are sometimes whether it's time to market, right? Or mfes that, that, that need to render on the client anyways, right? Where you have those um a single package that has both variants, right? As where as like when it just can't be helped. Ok? And we do have business rules, you know that, that, that do um have this because we can again get that config back so fast, we can determine which one of these variants is actually going to be rendered before rendering, which means you don't get one and then it gets replaced, right? Ie api calls, right? You can do this and you'll know before that component ever touches the screen

And that's kind of where we get very excited about key value store and cannot wait to get it into production because we can see how this is will explode our, our um experimentations. So thank you. And I'm gonna go ahead and pass back to Tina.

Awesome. Thank you, Kevin and Haresh. Um that's definitely a great example of how you can use CloudFront and Edge Functions to experiment and make gather the data that you really need to make business driving, impacting decisions for your, your your product. They literally grew the Auto Navigator service from scratch and uh I think it's monetti now, right? Awesome.

Um ok. So with that said, you know, we talked about caching, we talked about serving dynamic content, we talked about extending application logic to the edge. As you can see here, the trend is more and more is moving to the edge and CloudFront and the CDN is becoming that front door to the internet for most applications uh so much so that it's the best practice now to serve any HTTP public facing asset out of CloudFront.

Now, what that means is we have to think about the threats at internet scale. Right. So security has always been a top priority for AWS. And as we're able to see more and more of the internet data that's coming in and out of the service, we're able to make advanced decisions, continue to grow the service to the point now where it's actually a security service in itself.

When we think about security within AWS or within cloud for, I should say we're looking at it across a number of different pillars. We're looking at the infrastructure layer, we have to have protections in place to protect against things like DDoS attacks, attacks that will take down your resources and essentially your application right over time in 2012, actually, we started seeing a lot of DOS attacks at the network layer and our engineers started building capabilities that have actually turned into services themselves such as Shield Shield is our, our DDoS protection offering.

The next area that we're really looking at is application security. We have to provide the capabilities for you to implement mitigations unique to your applications. At layer seven. In 2015, we launched AWS WAF WAF has been integrated into CloudFront since the beginning. Since then, between the two services, we're well integrated, we've added sophisticated capabilities that I'll talk about in the next slide and we've made it easy to use for our customers.

There's also secure connections that we need to think about encryption is important. I already shared that we cover TLS and we provide support for all the different protocols. But we have to make it easier for, for customers to use. A good example of this is ACM Amazon Certificate Manager is integrated with CloudFront. And when you want to serve HTTPS traffic at a CloudFront, we actually give you those certificates for free.

The next thing that we look at is secure access. We also have to make sure that we provide the capabilities needed to protect your resources against unwanted or unauthorized users, right? So let's take a look at how this looks in execution.

When customers often come to us looking to protect their endpoints, they usually have an origin that's either in an on prem environment or maybe it's in an AWS region. And if it's HTTP public facing endpoint, we say put CloudFront in front of it and there's a number of reasons for that. At layer three and layer four, we have advanced mitigation capabilities built into the network at our edge locations. On top of every rack, we have a service called Blackwatch.

Blackwatch is a homegrown service that we built that inspects every network packet as it comes into our border and detects whether or not we should potentially drop that packet, right. We're looking at different profiles such as if it's not an HTTP request. What's it doing here? Let's get rid of it. The other thing that we do when you use CloudFront is we're doing that inline inspection versus a reactive inspection and then mitigate right. We inspect and then we drop right away if we detect that there's an issue.

At layer seven, I mentioned we're deeply integrated with AWS WFWf has progressed tremendously. Over recent years. We have now sophisticated capabilities for bot detection rate limiting. You can create custom rules. We have AWS managed rules and partner managed rules to mitigate against common threats like SQL injection, cross site scripting, you name it, right? It's well integrated within the service and we've designed it in a way where it's easy to use.

The last area that I wanna talk about is CloudFront is always looking at our traffic. If you think about it, it's the service that's internet facing. And we're looking at largely most of the web application traffic that's happening over the internet. When we do that, we're able to identify things. We're constantly evaluating the service for server efficiency and identifying vulnerabilities before they actually become threats.

A good example of this is the HTTP two reset attack that recently happened a few months ago, many months before that. As part of an efficiency exercise, our engineers were looking at the, the code and identified, hey, there might be a vulnerability here. Let's put a mitigation in place to where it can't consume all of our resources. When the attack actually happened, we knew exactly what was happening and we were able to mitigate it before there was any major impact to our customers.

Now, when it comes to application security, you know, one of the things that somebody told me was security has to be easy otherwise it's unusable. And when we talk to our customers, something that we learned is there's a diff a couple of different perspectives that to look at. When you think about security, you have your security teams that look horizontally across a lot, large number of applications. They have very strict requirements and they want to get in there deep. And do you know deep custom customizations in terms of their security configurations?

Then we have our application owners, our application owners, they're looking holistically at their application stack and for them they value speed of on boarding and a unified experience. So we took that feedback and we looked at what we have, we have AWS WAF that provides all of those customizations that you can implement in detail. And then within our CloudFront console, we introduced one click security integration where you can actually configure a WAF configuration directly within the CloudFront console. Within a click from there. We're actually looking at your configuration with the name clot and coming up with recommendations that you can actually enable on the on the fly, you can start in a monitor mode to evaluate if these rules are working for your use case. We can detect, detect if you're serving something like a WordPress application and go ahead and deploy that.

On top of that, we introduced a security dashboard directly within the console. The security dashboard adds capabilities that make it a lot easier to investigate and understand what type of traffic you're receiving. You can look at things that like where are your end users coming from? What percentage of your traffic is bought traffic? And then once you look at those things, you can actually take actions on them. You can decide if you wanna block them rate, limit them number of capabilities like that directly within CloudFront. Ok?

So the last requirement that I'm gonna go over is really more one that we're always looking at internally, right? The internet's not gonna stop, it's gonna get bigger and bigger people are gonna wanna do more edge compute, there's gonna be more data to cache. Um the volume of traffic is always growing and we have to find creative ways to innovate to continue to grow our network beyond adding POPs um to provide better experiences for end users.

So over the past few years, we've actually been working on a new type of POP called the embedded POP. And we talked a little bit about this at in previous sessions as well. Um but embedded POPs are POPs designed specifically for by heavy cacheable content such as video streaming workloads or software downloads, popular software downloads.

The challenge with delivering these workloads is they're very spiky. We've seen the traffic scale to up to like 12 times the average traffic pattern. Right now, when we look at the capacity of our network, we realize that the constraint really isn't within the CloudFront network itself, but it's at the internet exchange points where we peer with local ISP providers. And because of the nature of the traffic being so spiky, it's not always economically viable to scale up these connections for us or the the providers. However, it's very important because you know, these bottlenecks can greatly affect a major event like streaming the Super Bowl or you know, launching the next video game release, right?

So embedded POPs addresses this problem and how they work is with embedded POPs, we deliver a rack unit, an appliance essentially to the local ISP. They partner with us, they deploy them within their data centers, these devices run CloudFront software and we manage that software. What it allows us to do is get closer to the end users serve content really deep within ISP networks and relieve all that pressure off of the network. For both the ISP and for the interconnect, it also increases our capacity and reach and this is something we're actively scaling out over the next few months.

Now, in terms of the value of these, these locations beyond the capacity requirement, if you think about the last mile, we're limited to how far we are from the end user from that internet exchange or that peering connection, right. And the internet is deep, depending on the network, it could run really, really deep. So, as we have observed this traffic, we've actually realized by deploying these POPs, we actually reduce latency for up to 65% in terms of first bit latency for some use cases. And we're, we're able to offload pressure on the network network of up to 95% where the real innovation actually is is it's actually not in the caching technology itself or how we deploy the POPs. It's how we've actually built the POPs.

If you think about it, these are devices that we ship to at an ISP, we have no physical access to them like typical AWS infrastructure. So we have to build them to AWS security standards. So all of these boxes are built with really cool capabilities like host authentication, malware detection, firmware validation, all directly within them. We even have capabilities to actually self destruct the device physically if we detect any type of tampering.

Lastly, um one of the things that we've been working on over the past few years as well is reinventing our CloudFront architecture. So on here, I have a QR code and you can, if you scan that, you can visit one of the talks we did in the past that goes into deep detail undercovers of how we've built CloudFront as a network and software architecture.

When we first launched CloudFront in 2008, we were using what we deem to be, you know, cutting edge technology with Nginx and Squid and it's worked great over the years. But something that we've realized over time is as new protocols advance, we need to be able to provide support for those protocols much quicker and reviewing large pieces of code that might not be super relevant to CDN use cases isn't always best when you're looking to be really agile and deliver new features for our customers.

So something that we've actually been working on over the last few years is building our own web server specifically for the CDN use case. And this web server is actually built on a new runtime that's actually managed by AWS engineers and it's designed to be multi threaded to scale and allow for edge compute use cases over time, we envision it to integrate well with native AWS capabilities like CloudWatch and S3. And we've already started deploying this across our POP architecture.

So with the launch of HTTP3 and QUIC, we actually started with that at the termination layer and over the last few months and near future, we're working through the our connection pooling layer as well. We're seeing already performance improvements of up to 100 milliseconds for some of these requests.

Alright. So with that said, um that's the presentation that we have for you and we have roughly eight minutes. So we'll open up for questions.