AI/ML: A trigger for innovation at speed and scale

Hey, everybody. Uh thanks for coming to our session. Um sometimes I just started at how like far in the corner off we are in the session. It's um Thursday morning. So really appreciate you kind of making the last effort to kind of join us at this session today.

Um you know, it's uh you know, I don't think that you've heard enough about AI/ML in the conference this week and you're here for yet another session. But hopefully, we can make this even more relevant and fun for you because we're here to share some insights on Amazon's approach to AI/ML and how we use AI/ML to really transform our customers experiences.

So for those of you who caught, you know, Werner's keynote, he did mention a few kind of approaching examples from Amazon. So maybe it's a, you know, an interesting way to think about this as an extension kind of um of and building on kind of the Amazon stories here.

So um just to get a sense of the audience here, um you know, how many of you have actually kind of downloaded ChatGBT and play with it? Ok, almost everyone. Um and then how many of you have actually tried to build uh machine learning, you know, kind of enabled solutions. Don't care what flavor it is great, some of you there. So, and how many of you guys work in organizations where you guys run ML solutions in production? Great, great. Ok. Then this is the right audience uh because obviously it's a way for us to compare notes a little bit to see how Amazon's approach, both from a business strategy perspective, but also the lessons that we learn from technology is relevant here.

And um you know, in order for us to have this broader conversation, we want to make sure that we brought in a diverse set of voices that represent different areas. So we have three speakers here today and we're kind of go in this sequence. Uh so my name is Cheung. Uh I lead a program uh in AWS Innovations called Learning from Amazon and that's what we're here to do. So I kind of kick off for the first section to share some use case examples from Amazon's retail on how to use AI to improve customer experiences.

I'm Ricard Allen. I'm leading the innovation transformation team for Latin America.

Uh and my name is Kendre Reed. I'm part of the AWS data strategy team. Um so I help a lot of customers like yourselves think about their data strategy in order to build those ML models and general AI pieces.

Yep. So I will kind of cover what Werner kind of called the good old AI use cases. Uh Ricardo has a fun job of covering some Gen AI uh thinking approaches. And Kendre has the most important job to share some lessons learned. So you can hear from all of us through this session today. So, gentlemen.

Alright. So let's dive in. Um I just want to share a bit of a context uh behind all of these things before we get into use cases. Uh and we start by appearing to the minds of Amazon's founder, obviously Jeff Bezos, uh when he kind of thought about what does Amazon.com stand for about 25 years ago, you know, annually, he writes a letter to the shareholders, obviously in the last few years. Uh andy jesse, who used to be the CEO of AWS. Now the CEO of Amazon has been kind of writing on annual letter to shareholders and 25 years ago, um Jeff basically told all the shareholders and this was the second shareholder letter that he wrote. He was just a small time start up founder running an online bookshop, but he has some audacious thinking uh that you see here.

And um you know, some of the things are really notable, you know, the idea of customer sensitivity isn't just one of our core values. It is the mission of Amazon. So it's really interesting that our mission is to be the world's most customer centric company. And then basically how we define success in terms of what that actually means, translated into practical terms is that when we are able to deliver the best possible experience for the customers, that's what it means to be successful in being the world's most customer centric company.

And you also read this kind of thing and there's a little bit of paranoia almost in a sense of urgency behind all of this. And when you think about this, you know, you know, we Amazon competes against some of the world's biggest and the best walmart is not an easy competitor to compete against. Um netflix was also one of our biggest customer, but our customers are choosing between streaming something on netflix or prime video or hulu that they're all AWS customers. But there's a bit of this idea that in this digital age, customers can make choices and they can change their mind very quickly.

So it's not about really just being good enough or just being, you know, fast enough or being quirky and pleasant enough. It's really about setting the highest bar and consistently pushing the boundaries of what the customer should expect. And that's kind of the context behind why Amazon really has been applying AI/ML at scale for more than 15 years.

So the mission is clear, but the extra dimension of the challenge that Amazon has is this idea that they want to do this at Amazon's speed and scale. And as you guys know, you know, achieving both is incredibly difficult for any company, let alone Amazon. So the uh the the kind of the thesis of this session today is that for Amazon at the sweden scales that it wants to interact with its customers delivering the best possible experience is simply impossible without the practical applications of AI/ML everywhere. And I kind of enjoy talking about the good old AI/ML because frankly, that's what Amazon runs on and I'm pretty sure that that's what Amazon continue to run and grow on for a very long time. What we all collectively learn more about then. What does GI mean for us in the future?

So, um let's dive in now to set the context for um um Amazon's business. How many of you have actually traveled from international to be here today? Great, welcome. Thanks for being here. Um how do you actually shop on amazon? Ok. All of you and the ones from international. Thanks for being Amazon dot com's customers. For those of you who live in the states and these are american statistics. This will feel kind of close to home. Uh I didn't know about this until we had to pull up the stats.

Basically uh six out of 10 of us. Uh our household has Amazon Prime membership and for those of you, because I'm slightly older who are between the ages of 3039. Um 80% of you have an Amazon Prime membership, which is a more significant and Amazon can deliver packages anywhere in the world in two days or less. So sometimes your orders get takes longer, sometimes it's shorter. Our average global uh average for package delivery is 1.9 days. That's more than twice the fastest of any comparable operator of our size. So it is very significant and one of the most um important aspects of kind of Amazon shopping experience is not the fact that we deliver packages fast, but we actually make a promise to you when you're looking, consider buying a product. And the customer promise, this feature is literally called the customer promises. If you complete the purchase of this product in the next two and a two minutes, two hours and eight minutes, we guarantee delivery by tomorrow.

Now, we may not meet that promise 100% of the time, but we try our darn best to make it as close to 100% as possible because making a promise is pretty useless. If we keep failing you at some point, you either get cynical and don't believe in this. Um and then you start your shopping somewhere else. But hopefully, if you're like me, this is almost like a swiss train running on time. Like i will basically take my camping to uh kids to camping this weekend and you know, the camping stove that i order right now, maybe after the session, it better be at my house tomorrow morning before we set out for the mountains. And that's kind of how reliable it needs to be.

Now, that seems pretty simple in a micro scale. But think what the macro scale, what Amazon has to operate and there's a lot of complexity behind what we do. Did you know that Amazon sells more than 400 million unique products? So when you think about it, there are literally billions of products that are kind of slushing around in a supply chain, either coming into our warehouses or what we call fulfillment centers or on various stages of delivery or various stages of handling within our centers. So there's a lot of moving pieces that involve, you know, millions of people vehicles and other things to orchestrate and in order for us to be able to make a promise to you and keep it. And as Werner said, and be very cost efficient about it, we gotta be incredibly good at managing this complexity using data and machine learning.

And the idea is this the only way that you can actually stay on top of the complexity instead of being crushed by it is the idea that if you're able to understand the state of everything that you operate or and everything that matters to you. And in our case, where is the package at any given second, what is the operating condition of our warehouses in that given. second, what is the condition of our third party logistics provider? What is the condition of traffic in certain places? What is the weather condition in places where we expect to deliver packages by tomorrow? If we know all of this information and if we can actually apply intelligence and automate our decisions on top of that, what are the kind of opportunities and solution opportunities that we can actually go after? And that this question or this line of thinking is what i want you to keep in mind as we now dive into those use cases.

So here's the first one, let's say that i live in chicago, i do live in chicago and i just decided to get by camping stove great which fulfillment center we have hundreds of them should actually originate the product to deliver to me

Now, if I don't have a lot of intelligence or I'm not very sophisticated, sophisticated, maybe the most obvious answer is whatever is the fulfillment center that is closest to Chung's house should be the one where a person would find the item and then put it in a van or trailer and send it to Chung's house. So that's solving for the obvious without data.

But what if you were able to solve for these kind of parameters? And this is not a comprehensive list. But what if we knew, actually knew the unit handling cost by fulfillment center by shift in a given period and say, "Hey, Cheung, the one that is closest to your house might be on the Wisconsin border. And the unit cost handling of that center might actually be a lot higher than the performance center that might be far, far away."

There's also shipping cost and you surely there's a function of distance. There's also a function of who is actually handling the shipping. How efficient are they, what kind of vehicles are they using? Are there some instances where we actually have to get two different products? Because Chung said for some reason, he needs a camping stove and he's a little headlamp for the kids and they have to come from different places. But somehow Chung said, I also want that in the same package. So now they have to collect it together and send it to me.

And what is the opportunity cost of shipping a package from one place? But at the same time, there might be a peaking demand for that same product in that area where we wish that we held back some items to be able to do that. So let's kind of get into this scenario of how Amazon solves this.

And in this case, let's say that it's a customer in Washington DC that says I want to buy those anti bacterial soaps and you promise me that you get it to me by tomorrow. So what you see here is kind of a light up of all the different performance centers that can potentially deliver to the customer. The ones in green can actually meet our promise. And within a threshold of cost kind of tolerance, the ones in orange can be two days, but it's going to cost us a lot. The ones in red when I miss kind of the delivery and what Amazon does is that it computes all the different cost factors and basically decides in this instance that instead of the five or six different performance centers that might be really close to the customer in Washington DC, they might be in Virginia or New Jersey or Philadelphia.

Interestingly, the fulfillment center in Akron, Ohio might be the best center to originate the item. And for anyone who's a basketball fan, you know, LeBron James, um, Akron Ohio, that's 1000 kilometers away from where the customer is. Now, why is that in this specific instance, Amazon knows that there's actually a trailer that's going to head out of that performance center in the next five hours that's going to actually head out to a kind of a last mile delivery depot by the customer. And if we can get the package onto the truck in the next five hours, and if you know that the customer can complete the purchase in two hours, because our SLA for package handling from when the order comes in to when the item is actually found packaged, um, and then shipping label put on and put onto an outbound vehicle is within less than three hours, we can make the promise and it's going to be significantly cheaper than other options.

So that's how Amazon calculates that that thing. By the way, this happens twice as this customer is trying to buy the soap. First time when the customer actually brings up the detailed description and we make the promise and the second time when the customer actually complete the purchase. So maybe within the three minutes or 10 minutes that the customers took from adding the item to the cart or when they browsed the item to when they complete the purchase, the conditions might have shifted. And when we compute this, the graphic might be different. And this time, maybe we might have missed that, that, that, that the trailer, it might have come from somewhere different. We make the computation again so that we can actually optimize for what we know in the very given second when the customers make the decision.

Now, I'm just gonna cover two more use cases relatively quickly because you probably want to hear about AI as well. Amazon also has been applying machine learning to make sure that the items that you're purchasing are going to be. Um there's no risk related to food or product safety risk. My kid has a peanut allergy. So whenever I buy some food product, I look at the label of the ingredient like the bible to make sure that there's no nut items. But sometimes the manufacturers miss labeling one very important thing that says, "Hey, this item does not have nut products, but it was actually handled in a facility that handles nut products, buyers, beware," but sometimes there might be that label missing and if I was to buy it, then my kid is in trouble.

Now how Amazon used to handle this before machine learning was actually fairly straightforward. We'll figure out if there's some significant issues with a lot of customer complaints or something in the newspaper that says, oh my goodness, there might be an elevated risk with this product that we're selling, then go tag the item in our system to have somebody investigate and then figure out if it is true and there is an issue and you handle it.

Now, what's the limitation of this approach? The thing is that there's only a limited amount of things that we can actually apply this process. We handle more than we sell more than 400 million unique items. So which one do you look at? And everything is reactive, somebody was harmed and it was big enough or frequent enough so that it became a priority for us to resolve. And unfortunately, when they did this with a machine learning, 90% of the things that we thought had the issue did not, there was a false alarm.

So Amazon is only able to basically handle and remove actual issues for hundreds of products a year with machine learning based solutions that they implemented production five years ago. Now, they can actually handle significantly more like literally hundreds and thousands of more volumes of products. 35% of the things that this model actually tags are predictive and 90% turns out to have an elevated issue. So that means hundreds of thousands of products are basically taken off our shelf at any given year so that you guys can be safe.

And imagine that if you sell hundreds of that product in a given year, the impact for our customer is not just millions but billions of instances where we kept our customer safe.

Now, another kind of example and my final example before I hand over the mic to Ricardo is shipping packaging. So for those of you who's been shopping with Amazon for a while, hopefully, you would have noticed that we're sending you a lot less of these cardboard boxes versus a lot more of these flexible envelope mailers. Now, why do we do that? Because these velo mailers are significantly cheaper. And when I say cheaper and again, going back to what Bernard said, cost and carbon go together when we reduce the cost, carbon is reduced. When we're able to ship packages, there are half the weight, uh half the size and just 1/4 of the weight, we can literally send you more packages without adding a single truck.

So the Amazon's volume, even if we're able to actually divert 10% of our of your orders from these packages from these cardboard to the packages, we can literally ship 30% more items to you without adding an extra truck on the road. So that's really important.

So how do we decide when Chen orders this camping stove for camping tomorrow, whether that thing goes into cardboard or velo mailer? So let's kind of use your input to decide what we do. Here's a 20 ounce kind of a bottle of shampoo. I chose it because embarrassingly, I have some dandruff issues. Now by show of hands, cardboard, tin, cardboard or team envelope, who thinks that this liquid filled 20 ounce bottle plastic should go into cardboard if you were to order this item and you want to make sure that this item comes to your home safely. Cardboard. Great envelope, mailers. Great. It's like literally half and half toss a coin and maybe we'll make that decision.

And that was exactly the way that Amazon used to make decisions depending on whether it's John or Chung or Maria make being given the assignment to say, "Hey, we have this new product, where should we go?" They will say, "Yeah, yeah, I've seen things break before it's gonna go into cardboard." They're gonna enter that into a system. And henceforth, any Head and Shoulders product that is 20 ounces is gonna go into cardboard forever until somebody else has a shift to change their mind.

What's more impactful? This might be the leading dandruff shampoo, but we might be selling, you know, 200 other skews or products of same dimensions. That rule just got to apply to 200 other products. That's how because you cannot inspect every every single item.

So same thing, Amazon was only able to make judgments on a fraction of the items, fraction of percentage of my item that it sells. We have a lot of customer input feedback from customers and things. None of that was used by Chung when he decided. "Yeah, I think I'm gonna put it into a cardboard." 100% rely on human judgment.

So what does that mean? If I'm not sure I'm gonna choose cardboard? I don't think I'm gonna get fired for putting things on cardboard and that's what happened. Now. Amazon has been applying machine learning uh on models built on stage maker to solve for this. Now, 100% of the decision is being made by the model. 100% based on data intelligence based on the information that it reads that our customers have told us our associates know they're in the system to make an optimized decision.

So, and here's some things it got so good that you can actually solve for other things other than just fragility. Here's an amazing book of Shakespeare's best works, collected items beautifully bound and Amazon is amazing. This is cheaper than the latest paperback from Noah Roberts.

For those of you who know Nora Roberts, you're kind of my tribe. But where should this book go again? Cardboard, flexible embler mailers. Most of you set the latter. What I know is when I buy this, I am not buying this to read. I am buying this to put on my bookshelf to impress my smarter friends that I have these kind of books. This is not a book, it's a decoration. And Amazon knows the intent that when customers buy this book, it cannot be dented when we deliver because this is meant to be an ornament, not a book.

Here's another one where should this go? Uh boxes or mailers, boxes, mailers or half an f for my children. This is just a toy. So when it comes to rip off in the box and start building it for some lego collectors and know that the last time that they had like this national mission thing was back in 2001 and it's the first time they released it, this might be a collector's item. So in this case, we might put this into a box even though 49 other products that have similar kind of dimensions may actually go up and, and end up in a, in a, in an envelope mailer because it's a toy.

So the business impact is real and compelling in the last seven years that amazon has really been trying to do this and we actually had some significant unlocks with machine learning. Amazon was able to basically replace half the packages they were sending with cardboard. Now with this flexible envelope mailers. So this year, you should be seeing a lot more of your products being delivered in these envelope mailers versus uh cardboard boxes. So there's a win for you because you have less things to throw away win for amazon because we put less trucks on the road and win for the environment because we're creating less carbon and less landfill to do the same thing for you.

So the final things I want to say is look, meeting some of these things that customers want from an online retailer when you meet these basic needs. Yes, it's great. It satisfies customer needs. But when you're able to understand, have the data and be able to apply intelligent top of the data, we can actually solve for opportunities to truly delight customers. So instead of just saying, hey, we can deliver the package to you by next day. If you order in the next two hours and eight minutes, we can also say, hey, look, we certainly can do that. But if you're willing to wait another day, we have found ways to reduce the carbons related to the shipping, the item for you by half, would you be willing to make that choice with us? And when we can opt you into these kind of choices we can make because we can give the data and intelligence, we can make that truly delights customers and sets our customers experience apart from any other vendors they can do business with because they know and they trust and they believe that they we can, you can be opted into our good intents for the environment.

So those are the good old kind of use cases. There's significantly more as you can imagine, our speed and scale runs on these use cases. But we're just as excited about gen a i as you guys are. So we want to also take an opportunity to share some insights on how amazon approaches gen a i. So here's ricardo.

Thank you. Uh we are not talking about today. How deeply is the gen a i generative a i has been all over all the news. We are talking this all these days and uh it's a no difference from any other meeting that we probably having that we are using the word of gen a i generative a i. But we are talking here about how amazon applied it and how we can thinking on uh use cases that bring it uh more ideas how to solve the business problems.

Amazon always looking for opportunities to improve our business. Imagine this size of the business that ch was telling us and we'd be able to improve this by 1% 2% in things that we are doing this optimized a huge amount of things and this, it will be able to change the needle to move the needle. So when we thinking on gen a i is part of our a iml for a long time history is another two, is another line of thinking that we are going deeper on that for people who are not very used it to gen a i or if you wanted to remember thinking on generative a i as part of uh uh intelligent artificial intelligence that you can create the things you can use it just to build uh pro of the images of products that it even doesn't exist. You can use this to build uh movies, uh small films and whatever you want. Uh summarize text building new images, new things using generative a i can help us to think on how to collect the data, but it doesn't need to have all the information up front, use a long uh large amount of data that have been collected. Then we call this a part of the foundation models.

So this foundation models that is part of the amazon aws or other uh companies that are partner of us, uh build these foundation models for you to use this and to create this uh solutions around it. When you're talking about uh the interest of uh a iml or generative a i at amazon, we are talking about main three areas. The first one is how generative a i can impact and it can influence the services that is basically typically what we are seeing here at the ws in all these days. Uh amazon bedrock, uh titans and many others. And we are launching other services and systems like an amazon q and things that we are using to build this foundation models.

Uh directly or with partners that help us to elaborate more these large language models that runs on top of the foundation models that help us to imagine new things. And it's sort of like the same philosophy as aws. We are democratizing the access of the, the generative a i for people who wanna so the things and build the things. So if you are interested like to play with party rock at uh dot aws, you see that how you can use these models to experiment and around it.

Another area that we experiment at amazon is how to uh sorry, how to leverage user uh end user tools and user tools can help the individual productivity at amazon. Uh we use it at the amazon code whisper that help us uh that runs on aws that help us the developers, the engineering teams to develop a code uh 57% faster and 27% more likely to, to be accurate to be more secure. So that is not only a tool to help it, it should be to develop faster, but also to deliver it more secure. And that is an opportunity to leverage the individuals uh uh in this.

And the third one is how we augmented the services, how we can uh experiment, how we can extend the customer services, the customer experience and we'll be focusing on this part today. So when you're talking about the the business services in the platform level we're thinking on different efforts that it uh artificial intelligence, machine learning can help us generative a i is just an extension on this. But uh unleash a lot of other things as we are talking, the difference between the foundation models and the traditional models are uh where we had to uh started from the machine learning, the traditional machine learning with the questions.

So if you don't have it at the, the data, if you don't have it at the train, the model, if you don't apply the model, you cannot use it, but you have to do this by each of the tasks. The tasks level is one of the one thing that it determined all the whole chain. When you're talking about the generative a i, the foundation models rules, the foundation models are labeled data that uh build the uh foundation models that you can use it to adapt to the answer.

So you have an untrained data, you have it at uh the foundation models that have happen, help you to address different questions and the models helps you to run these tasks that are not even sometimes uh predicted. So you can use this to build uh images, you can use this to summarize the text. You can use this for different things. And even the users can think, you know, new ways that you are not aware of that the same foundation models be able to support applying this to our reality at amazon, we always thinking about the opportunity versus the risk.

We cannot play it with the, the risk, we cannot play it with the, the customer uh services we cannot play. Uh that is with the, the trust from the customers. And we apply this in three main areas. So for us, things that are non negotiable are these three points that are no providence. Uh the cost of implications and the unpredictable behaviors. We cannot afford that a model be uh having a hallucination and generated things that are not true.

In order to use this quad rails as our opportunities, we developed three strategies to unbound this to uh this quad rails. Uh we generated the, the response as much as using the a i. We are focusing on our opportunities and we give the users the opportunity to make the decisions. These three strategies runs on different uh uses different cases.

We have it here. Uh a few illustrative examples that are coming through and we expected that it become true very fast. So for example, for the, the uh agent at a customer service, uh imagine that they have a, a huge amount of data and then you can summarize it all the calls and you can summarize it, calls that are received it from the other people for other uh colleagues and they can extract it in, in alternatives and they can give it to you like an opportunity or give it to you ideas about what to do. So, but at the end of the day, they are giving you suggestions, not making decisions, the end user doing that. This is well connected with the presentations that adam did at the keynote talking about the connect.

Uh other example is how we be able to give it insights, not only for the customers but for the sellers. So how can the sellers can use the data that they have about their product to make it a decisions if they act it or not? So, in this model, we can use the data, we can present opportunities like you can be more efficient. If you, if you put uh put like a people are more willing to buy like two products together and put together than one things that like that. But i'm the seller, i'm deciding if i you adapted for these suggestions that the dna i is giving me or not. Uh it's also coming through, we are having a lot of experiences on that.

Now, another illustrative examples is imagine a, a small business that you are running that you are selling, putting your products and in our marketplace. But you don't have a lot of money to invest in on the marketing to do like a beautiful campaigns, videos and using the social media. And we can use it as services from amazon that helps you to get the, the uh picture that you took it from your product. And build it like a very beautiful 30 seconds video that you can upload it online and help you to increase the sales.

But I'm the owner of the business and I can decide if this video is good enough or if I needed to change the background because they put the, the product in the background that I don't like it so we can make these decisions and it's also becoming true.

Yeah, another example that I have that is became the uh implemented already that was uh communicated. Uh is you, you, you as a buyer, we have a lot of people here that is buying aws. Oh, sorry in amazon and we read did a lot of the customer reviews, right? So imagine that you have like a, a summary of the customer reviews and that's all already in production that it has an uh uh a comment here that it was generated by uh generative a i. But it, it's your choice to uh use it or not, but it summarize it like a 3000 reviews in uh a few lines.

To recap it, the possible experiences that we have and we know that invention innovation is not new for amazon for a iml, but we use it a generative a i in ways to invent a new things to extend it. What we know to enrich the experience for the users, using these boundaries to augmented what we knows to make it the suggestions to extend the knowledge, to personalize these experiences going further and to automatize the uh all the solutions.

So you get it more and use it to review the models, reveal the things that we have it to buil it. And now let's shift to, to kendra to talk a little bit more about our lessons learned and the most important parts that we learned along these more than 515 years. Thank you, ken.

Thanks ricardo. So you learned today a lot of the things of how, you know, one we're reducing costs, us am l how we are, you know, improving sustainability and ultimately improving the overall customer experience that you're having. But through that old process, there's a lot of learnings that we've gotten through this journey, right? And so we're gonna talk a little bit about this now um and getting into, you know, what you can do as an organization to kinda operate at scale when it comes to leveraging a iml.

But before we get into that, let's take a little look, take a look at, you know, some of the blockers when it comes to leveraging mls scale, right? This is a survey done by o'reilly uh just last year on, you know, the adoption of a i within the enterprise, right? And i can assume that, you know, by a raise of hands, you know, how many of you all have experienced some of these same issues when looking to leverage ml within your organization, right?

And so you see here that it really comes down to really, you know, four of these key areas here around, you know, the skills and people that we have in place, having the actual data and the quality issues intact, having, you know, the mindset and the culture in place to really leverage this technology to operate a scale. And this is some of the same issues within amazon that we had to face as well when we first started on this journey.

But what we found was there's kind of a systematic approach that you can take to really solving these, right. It really starts with, you know, having that mindset first, you know, looking at an executive level on, you know, what are our, you know, culture and mindset around, how are we gonna leverage machine learning and using that to kind of set the goals we have as an organization and then allowing that to flow down throughout the rest of the teams within the organization.

And once you start to do that, really enabling the teams to take on that ownership of building out these different ml experiences. And once you start to gain momentum there, you start to realize, all right now that we're, we're doing these pieces, we really have to have the data in place and having that good foun foundation around data in order to really enable and operate at scale.

And then the most critical piece with that is really finding the right first project. You want something there that can be done quickly, you know, 60 to 90 days, but done in a way that provides substantial impact, right? And so what that does is it allows the business to really see the value of leveraging that technology and overall improves the culture of saying, hey, we need to continue to leverage this and continue to expand this and scale it out throughout the rest of the organization.

And so the way that we really set that mindset and culture within our organization in amazon was really asking the question, how we will use machine learning. And this is a question that is asked of each and every team of amazon, you know, thousands and thousands of teams are asking this question each and every year as they go through their annual planning process.

And what this really does is really sets the mindset for those teams on not when or if we're gonna use machine learning, but how and so what that does is really gets them to think about who is our customer and how we're gonna apply to machine learning to improve those experiences for those customers. And with that beginning to set our short term and long term goals around how we're gonna leverage this technology.

And once you start to do that, it really starts to begin to think about, you know, how are we gonna support and enable our teams to be able to leverage this technology. One of the key pieces there is how we are organized and structured within the organization, you know, typically within a lot of the organizations that i talk with, you know, they have their business teams, they have their technical teams, but neither know what each other does, right?

And so there's a lot of uh you know, smoking mirrors or, you know, shadows on what actually happens behind the scenes when it comes to leveraging a lot of the technologies. But what we did as amazon was really sort to think about breaking down those silo teams and bringing them together in what we call cross functional teams, really having these domain experts, those that knows the business as well as those technical experts, those that really know the technical tools needed and bringing them together and start to create teams that are a mixture of both.

And what that does is, you know, this is a really critical piece where each side understands what the other needs in order to enable that customer experience. And so now what they start to do is thinking about, you know, what is those tangible business outcomes that we wanna have? And how do we enable those we're working together once we have, you know, that alignment from a mindset, and once we start to think about, you know how we structure that organization, we also need to think about you know what tools are we gonna utilize to really enable our teams.

And you know, this here really shows a breadth of the stack of different services that we have within amazon that are leveraged. But you'll see here that there's a diverse set of tools, some that are simple yet powerful to use, but also some that require a little bit more expertise and it can get a little bit more granular.

And while all these services are used, each team is on a different journey, wherever they start is different from another team and how they grow and continue on that journey is different from team to team within the organization. But the real, you know, key here is not choosing a particular tool just because of, you know, they love this widget or love how this tool operates.

But really looking at, you know, what is that business problem we're trying to solve, what is that customer experience we're trying to prove upon and then working backwards from that and really identifying what tools helps us achieve the outcome that we're looking for when solving that business problem.

One of the things that we said in our earlier days uh within amazon was, you know, data beats intuition. And essentially what that means is, you know, if i have the right data, i can really solve those customer problems. And so when you start to build out these teams and start to build out these ml algorithms and things, you really need the data to really, you know, scale that and make it possible.

And many organizations, you know, this is where, you know, we've done these po cs and now we're ready to scale. But then we realize, oh, we don't have the data foundation in place to really enable this right now. I like to think of this as an iceberg. You know, we have, you know, the generative a i model at the tip of the iceberg, we have that ml model. But beneath the surface, it's really that mindset how the teams are organized, the data foundation we have in place and how we're gonna solve those customer experiences.

And so the data piece is really key and it was key within amazon. We found that, you know, in our legacy enterprise data warehouse, the way that we operated, how we were structured internally, wasn't going to be able to scale and meet the experiences that we, that we had, right?

So just, you know, just seven years ago, we really went under a transformation of how we leverage data internally. You know, we were more of a monolithic type organization, centralized it team and you know, they, you know, was slow to really operate in a lot of areas because there was a bottleneck on enabling a lot of these experiences that these line of businesses wanted to enable.

So what we did was really rethink of how we are going to enable our line of businesses to utilize this technology in order to improve the overall experiences of their end customers. So there's three areas we wanna look at there one being, what is the security mechanism we're gonna use, right? Security is job number one within amazon. So we wanted to make sure we had our data secure, but also having it on a platform that is able to scale as we continue to scale and grow within our businesses.

The other thing was thinking about how we are structured, right? Instead of having a monolithic organization with one central it team really breaking that down and having what we call producers and consumers within our organization. And having those that are responsible for these transactional systems really also on turning those transactional data into analytical data that can be leveraged for things like ml algorithms.

And then on the other hand, having those consumers to really own their own data engineering and processes and building of those eml models as well. And then the last piece was really removing a lot of those heavy lifting tasks, right? Instead of me needing to set up a lot of custom manual things in order to create those data pipelines, getting access to the data, you know, waiting weeks to get access to the data, uh really removing a lot of that and implementing a lot of automation within that process.

And so when we did that, what we really did was start to rethink of how we were structured and the technologies that we were using. So from the security and performance at scale, we utilized s3, right? They gave us the security out of the box with some of the security mechanisms that we have within amazon s3. And also they gave us the scalability and performance that we wanted when accessing the data.

The second thing was we started to operate in what we call a data community, right? With those producers and consumers, and we created what we call a data marketplace all in this marketplace. You can think of as just like amazon.com users within this marketplace can come in search for data products and go and you know, in look into that data product and understand who makes it how often it's updated. When's the next update it happening, all of these pieces that you expect when it comes to understanding what a data product is and done it in such a way that anyone within an organization can understand it, business or technical.

And then the other piece was removing a lot of that heavy lifting with providing automation around no matter if i'm using amazon red shift glue emr any computer service that i'm leveraging, i should be able to get access to the data and complete and utilize it in any fashion that i want. And so when we created this piece here, we really found that it really allowed us to scale and meet our demands that we wanted.

And so as a result of this marketplace that we built, we created what, you know, as amazon datazone, amazon datazone is really a child of what we have internally and built internally through the years of our learnings within amazon.

Now, once we have, you know, that momentum going around building our teams have that mindset and we actually have the foundation in place. We really wanna start thinking about what is that first uh project that we're gonna go after? Right? And we really wanna start thinking about, you know, how do we tackle that? And what is the definition of success we're gonna have with that first project.

And the key piece here is really balancing the effort with the success chance and the impact that you're gonna have within your business and your overall customer experience. And these here are just some common examples of questions that you wanna ask to understand, you know, what is the right project for us to do and how we can go about implementing their first key project. The first project, excuse me is very key and to show the actual impact that the overall ml is gonna have and establishing that culture within your organization.

The other piece that you want to start thinking about as you start to scale is really removing a lot of those bottlenecks, right? And this is really thinking about um how we're gonna scale overall, right? I like to call this year, the year of the poc everyone i've talked to is doing a poc around general a i or just ml in general, right? But what you will see is as more and more adorn po cs, less than half are actually making it into production, right?

The reason for that is a lot of these uh a lot of these pieces that we talked about prior to this around that mindset, the culture, how the teams are structured the foundation and that's really critical in understanding, you know, how can we scale, making sure those pieces are in place so that we are able to see the full ability of machine learning and investment is realized when we are looking to leverage this technology.

So this year is just really a illustrative example of some of those key activities that go into the ml development life cycle, right? Um you know, you can see that there's a significant amount of uh inspiration alignment that goes into that and it's a real big investment um in the in the amazon. You know, there's been a lot of learnings around, you know, the platforms and tools that have been built, a lot of it being custom, right? Just because of where we were at the time and how we grew, we built a lot of custom pieces.

But what that also caused was a lot of these und dere tasks around manual scripts that had to be run manual upkeep monitoring of different areas of the tools that we're utilizing. And so what that forces us to do is really pay what we call this hidden cost, right? The cost that we're spending on the upkeep and maintenance of a lot of platforms where we can, whereas spending time on improving the overall customer experience.

And so what we've done really was start to leverage a lot of these modern platforms. And what this does is allows us to really focus more on improving that customer experience, how we're using machine learning and us utilizing either sage maker or bedrock in order to improve those experiences while also uh improving that speed and agility.

And it's not just only about the speed and agility that we're trying to do here, you know, we really want to think about how can we improve the processes of a custom of ourselves internally as well. A great example of this is, you know, initially, when we started on this journey, you know, it was four engineers to one data scientist within each team by us, really removing a lot of these bottlenecks, improving our automation, being able to operate a scale. We've now gone to more of a 1 to 1 ratio in those teams and how we operate.

So in in in closing here, we really want to highlight, you know, amazon is all about customer session, right. Chon started off talking about how we utilize customer session is really the driving force in all that we do, right? And that really keys in on how we identify, you know, what those customer needs are and how we can improve the experiences. Using that approach, we kind of work backwards and understand how we're gonna utilize machine learning and setting that culture on how we can improve those overall experiences and utilizing the technology more.

So as the scaling mechanism in order to do that at broad, at scale within all of our lines of businesses. And as we're doing that, one of the key things that we're doing is building and influencing a lot of the products that we offer within aws itself. So as we continue to utilize these products and find where enhancements can be made, we do so not only to benefit us internally but you as well as a customer.

Great examples are sage maker bedrock or even amazon q that we um highlighted this earlier this week. So taking these same considerations in place what we've highlighted throughout this presentation and some of the things you can do within your organization as well when you're thinking about scaling and creating these new machine learning experiences.

So as we wrap up here, i definitely wanna bring back on chung and ricardo to talk through uh any questions that you may have around some of the pieces that we talked through within this presentation