Augment creative thinking & boost productivity with AWS generative AI

Oscar: Good afternoon, everyone. I hope your re:Invent Tuesday is coming to a very good positive end. My name is Oscar Ziel. I'm a senior AI/ML product marketing manager now with Generative AI with AWS. With me is Sylvia Prieto who is a senior AI/ML specialist at AWS. And we both are very honored and proud of having with us on the stage Jaron Mina who is Chief SO Architect with Chopped.

The three of us will be walking you through the following agenda:

• I'm gonna show you some market statistics and the impact that generative AI is having on people because at the end of the day, regardless of our position in the organization, we need to convince the stakeholders, we need to ignite that spark in teams to join the AI movement and deploy solutions that will help us all do better work.

• I'll take a deep dive into the marketing and sales use case just because, and I'll give you the reasons why these two use cases - I think that these are the use cases where you can demonstrate the most value that you can grab from AI.

• Then I'll invite Jaron to share the Chopped success story. You'll get inspired about what they've done. You'll get ideas about how to make it happen for your applications, for your platforms, for your solutions.

• Then Julia, our senior specialist, will take from there and talk about bringing all that you've heard that will hurt at that point into technical reality. And what do we have to offer on AWS for generative AI and foundational models.

• And then as we've worked in different capacities with AI, we're gonna put everything together to give you the five quick, easy ways to get started to see what you can grab from there also to start that revolution in your own organization.

So let's get started from the very beginning of things about generative AI. What's generative AI? We know it's that branch of AI capable of creating images, sounds, music. These days, people use it for everything - generate puppies with hats or do real work. I've seen people generate bedtime stories for their children to tell them at nighttime. I'm guilty there. And I'm seeing people delivering global campaigns with the tools that we all have seen so far delivered in the market, some of them very basic, some of them more complex, but they are both being used for either serious or not that serious use cases.

Generative AI is powered by large models pre-trained on vast corpus of data. And they are referred commonly as foundation models or FMs. And they are the ones that have been trained and to enable people to generate images, like "I want to see a lemur wearing a hoodie" or "Show me a fox in a purple suit" or "Maybe show me how my product would look in a campaign using the branding guidelines of 2023". Things like that. It's using natural language to generate content.

Now, when we talk about generating content, we have to talk about people and that's where I usually like to start speaking and talking when I refer to generative AI because it is who's going to be impacted. Internally at AWS we'd be enabling different teams to use generative AI and you can feel the anxiety and you can feel the doubts and the thoughts that people have about job security and so forth.

I found research from MIT done this year where they took professionals that are using or planning to use generative AI and they asked them about it - what do you think about it? Are you seeing a benefit? And a large majority reported that AI is having a positive impact on their work. AI is not only helping them to do more work, but better work - work that people feel proud about.

Now, as a Spanish native speaker, I've been using generative AI to polish the way that I communicate with others. And these days, I'm more proud of what I say, what I show, because it's of higher quality.

It can also reduce those tasks that people usually seem to be boring and tedious. And we know that all these metrics together have a direct impact on professional fulfillment and personal satisfaction.

Now, when we look at the market, we right away see that the shift has been nothing less than seismic. According to Bloomberg, the market is going to grow to $1.3 trillion in the next 10 years. Now, remember that in January 2022 the generative value market was estimated to be $40 billion and today is at $1.3 trillion.

Now, from that $1.3 suspected that at least $250 billion will come in the shape of new software, new applications that we haven't even seen yet. That's why Gartner estimates that at least 100 million humans will be interacting with what they call "robo colleagues" as early as by 2026 and that's something that we are actually doing right now. At AWS these days, we are interacting with models through Amazon CodeWhisperer, just generating marketing content, asking questions and so forth. So we are already living that life that Gartner predicts many workers around the globe will be living very soon.

And when we look at the use cases that bring the most value, there's a piece of research from McKinsey where they look, they asked the organizations that are using AI and they ask, which are the use cases that are bringing the most value to you? And they identify five, that bring 75% of the total value that AI brings to organizations, but only from those five marketing and sales represent at least 20% of that total value.

So that's why I'm saying that the impact that you can create by enabling people in marketing and sales with AI will show the organization the future that they can live when they take then AI, generative AI to manufacturing, to new product design, to new business development, to all the functions that can leverage it, but you need to sell the case inside the organization. So that's why marketing and sales are proving to be very, very effective.

Now, why generative AI for marketing? When we look at our marketers, we can see that generative AI can augment the way that they create content. They can generate far more ideas than before. In my case, when I had an idea for a good marketing activation, I could come up with 3 to 5 ways to do it in a week and then you have to share with different people. People give you feedback on documents and so on and so forth. Now, just with pressing a button, I can generate another workflow of assets to see my ideas in the market. And I can do that exercise for as long as I need to.

It can transform the way that you engage with your customers. Not only because the content that you can generate can be highly personalized when you rely on your internal data and your historical data from salesforce, from all your platforms to create highly personalized content. But also for marketers to create, generate new ways to go to market, you know, challenge the models to tell you, this is the way that we traditionally go to market, and here are the stages where we see we've been effective, create five traditional and three highly creative ways to take this piece of content to market. And you see what, what the models can come up and gain, inspire from there.

It definitely boosts productivity. You know, for some teams, it will create speed. I can now generate a draft for a new press release document in a matter of seconds by just entering a few ideas about a new product. For other things, it can reduce the number of cycles that takes to revise a piece of content.

It can foster confidentiality and compliance. Because when you create your own marketing solution internally, you are not forcing your marketers to go outside and share by mistake, confidential data in the shape of prompts or even outputs. So you keep everything inside and compliance because generative AI can check on more content and more often at a lower price helping you achieve compliance, let's say in content moderation across geo regions.

Now why for sales? You know, we want our sellers selling, we want our sellers talking to customers and closing deals. So generative AI can help you move sellers from the keyboard and into meeting rooms because they now don't have to spend too much time generating content for emails and contracts, proposals and so on and so forth. They can spin up a quick version of something to share with teams for approval and feedback.

They can reimagine their own customer relationships. In AWS, I've seen very creative sellers where they with prototypes that we create, they enter a few details, anecdotal information of all their customers to tailor emails based on customers that are in very specific industry segments. But also what they talk with their customer. Perhaps they talk about the last soccer match from last night. That's something that it could be included. If you have the right implementations to enable your sellers to become more personal. In that way, it can strengthen your sellers' talents because now your sellers can look at the camera and deliver their pitches and have generative AI give them real time feedback.

"Hey Oscar, you're going too fast. Slow down. Your audience is in this part of the world. According to our database, you need to slow down at least 25% Oscar. You shouldn't be using this term Oscar. You should..."

Little by little, they can have a session expert giving them feedback and become better sellers. And it can unlock patterns and insights in AI not only at a great speed but with greater accuracy because now you're feeding all your information to tell sellers:

"This account is at this stage, they have these tickets open. If you help them close this ticket, it's gonna help to close a deal."

Now, when you draft a proposal, this proposal has had impact in these regions and because it's automotive, you should be using this email that is shown to be also impactful, and so forth.

So these are insights that for a seller could have taken days or weeks and we don't want sellers spending days or weeks in front of a computer and not talking to customers.

Now, envisioning a workflow like this in practice, it's something like this:

I wake up with an idea, you know, as a product marketing manager. Let's say I work in a shoe factory. I wake up with, you know, people in Seattle where I'm from should have shoes that are like this - reflective and comfortable.

And so I can just jump on my marketing solution and enter a product description. And my solution will be relying on brand guidelines and company information to make sure that what it generates fits all those guidelines and I can even go further and generate things like images.

So if my engine knows what type of products, materials requirements, geos that I'm selling my product at it can generate an image about it. It can give me a description that I can send right away to people. "Hey, what do you think about a shoe that help you experience Seattle with the comfort and durability of XYZ?"

That's great. We need a press release.

Well, I have it because I generated it already. So, here's a draft. What if it's not durable? What about if it's just reflective? If I just change the word reflective on my prompt, I can regenerate the whole marketing workflow in a snap.

Now when it come times when, when the time comes to sell it. Then my seller can take that press release that I created, consume it and, and, and summarize it in a few sentences so that person can understand it without having to read the whole thing. And I've seen creative sellers generate images, for example, based on the brand guidelines of their customers. So they are in this case, they are saying "Dear Laura, great we met, this is a shoe that is going to sell great in your shelves" and look how it would look like in your shelves and it matches your tone in the market. It's, you know, it's immediately impactful for a customer to see that type of creativity being, being implemented.

And by relying on the historical information of success in your platforms, you can even draft contracts. And I've seen also implementations with where now these contracts are being parsed through systems that simulate negotiations, information that you can pass to your seller. If the customer says this, here's the most economic and valuable answer that you can give to that customer to close that deal, things like that.

Now, I mentioned that we've worked in AWS with a few teams and at one point, my product marketing team asked us to show the art of the possible with, with AI and we created this prototype at the time, it was an internal prototype that quickly moved into what today we know as Party Rock. No, an Amazon playground. I recommend that you all give it a try.

We created this prototype when we surveyed the PM organization, they said it's just we, we have to create so many documents for a single new product release. So we created this where with three basic user inputs, you generate as many documents as you want in any single shot. So let's see it in, in action.

So this is the basic UI of Party Rock. As you know today, my three wide fields are where I enter the information. So my product name is Walker Shoes, a new line of comfortable reflective shoe. Then I enter my launch date and right away the tool starts generating content. Now note that behind these different outputs I have prompts about, I need you to write like an Amazonian based on the 10 writing principles of Amazon. I need you to create linking posts of 400 characters because that's the guideline for my AWS social media team and the webinar needs to follow these three guidelines because that's what AWS events team say those are the guidelines for our tool.

We created an extra column to even challenge the model to give a suggestion on what the model wrote. So now that you're finished generating content, go take a look and give me five suggestions to make it even better, right? So those are suggestions that PMs can look at and say like, oh, that's interesting. I'll make it part of my document. And that prototype, it took us, it took us one hour to create, five hours to, to prompt engineer and is now being used by many, many teams in AWS. And it relies just on a handful of capabilities that we have on foundational models.

With foundational models, you can turn good coders into great coders. And in marketing and sales, we have coders when you have to go in, in the back end and, and change wiki code for your wiki page or go into that widget on your internet to deliver a widget for, for, for our internet page. Sometimes you need to code, it could be HTML, it could be CSS, it could be something else. But you can send that code to the, to the engine and say, you know, redo this code, this is what I need to do and it will do it, it will do it for you.

You can create question answering widgets where you know that in the enterprise, we have different answers for the same question and some of them are as old as my seven year old and some others more relevant. We want to give people the, the, the, the relevant answer. And so that's what those widgets can do.

We can transform our information into mathematical interpretation through embeddings. So we uncover insights that we haven't seen before and that's exactly what our customers are doing today, right?

So we have Analyticom, a group with Omnicom that is creating end to end advertising solutions supported by generative AI on, on AWS, you know, creating segmentation, automatic segmentation and, and advertising assets that a human team will review before going to market and but reducing the number of cycles in a go to market activity.

We have Coda who is making hundreds of teams all around the world more effective by integrating a platform of 600 plus tools with generative AI. And we have Choppah, a great example of what I mentioned that health sellers become better sellers and close more deals. But you know, to tell the Choppah story, there's definitely no one better than Chief Architect Jerome Mina at Choppah.

Thank you Oscar. That's working. Yes. Yes, fantastic. Thank you. Yes, now it is working. So, hello everyone. My name is Yo, I am Chief Architect at Choppah. And so today I want to talk a bit about our Genie story, what we've built and how we've built it. But first, let's take a step back, look at sales and marketing and from our, from our perspective as Choppah.

What we see are two main challenges. On one hand, we see a seller that has a hard time finding the most relevant marketing content that's out there. And the second challenge is that we see sales reps having struggles with training and coaching and specifically to turn that information, the marketing content into an engaging conversation with a buyer.

And so what happens is you get horrible buyer experiences, you get terrible and missed opportunities and lost revenue. And so with Choppah we're trying to fix that, we're trying to provide a platform where marketing, sales and buyers can come together so sales can have more engaging conversations to buyers. They can see more effective results from talking to buyers and in general, they can see more opportunities turning into deals that are closing into more revenue.

We do that by ensuring that marketeers have a content platform where they can upload their content into Choppah so that sellers can use that content and tools and coaching to share that with customers. And we give buyers and customers a form of a digital deal room where they can collaborate together and get to a deal.

Very high level Choppah is a Belgian based company with about 500 people. And as we're approaching our $100 million annual recurring revenue mark, we're serving about 1400 customers in about 50 countries all over the world with one sales platform. And in terms of activity, we have about 6.7 million monthly buyers that are interacting on our platform with sellers.

We're also entirely built on top of AWS. So we have about 90 services, a bit more actually. I took the time to take all the icons and put them on here on this slide so that you can, can see which ones we're using. But specifically the ones that we're using today, I shouldn't find icon for because it's Bedrock and that's what we will focus on a bit later today. But we're here to talk about AI.

So let's take a step back and look at the things we did for AI this year. So this only started a few months ago where we said, ok, we have to go and look at some use cases and instead of focusing on creating another chat bot like so many do. And we said, ok, what are those really essential and unique sales use cases that can help us seller in the field and can also make sure that those sellers can stop doing things that are very hard for them to do, but very easy for them to verify. So they can still build a trust relationship with their customer.

And so today, I'm gonna present you four use cases, not just one, we were able to build four and I will take you through them to show you the power of what we've built. So the first use case is something we call AI generated asset summaries. And this builds upon a very common use case of generative AI of creating a summary of a document or a piece of text.

What we've done is we've taken that idea, but given it a sales flavor, we basically said, ok, it's gonna be very hard for a sales rep to read through a paragraph as a customer is asking them questions. So what we did is we intelligently formatted that as an FAQ. So we took the document and asked the, the generative AI to generate a set of frequently asked questions based on that document.

So that when a buyer and a seller are talking to each other, the seller can easily take the buyer's question and return that into an answer through email, WhatsApp or anything like that. But then we took it a step further because if you can generate questions on one document, why don't we just allow you to ask any question about all your documents? And that's what we do with AI powered search.

We take a question and we translate that to your document base and surface the answer in a written piece of text. So you can easily copy in, in your email and we reference all the source documents that it found that information in.

But what if you don't have your laptop? What if you don't have your computer in front of you and you're going for? Well, you're in Vegas. So you're going to play golf with your customer, right? Or you're going to do, going for a fancy dinner with your customer. Well, in those cases, you want to prepare offline, maybe you wanna do and create yourself an exam based on some of the content that you've seen. So you want to create yourself a multiple choice exam. And that's something we also provide with AI generated test questions.

We use the AI to generate a question, a right answer and a couple of detractors so that you can optimally prepare for that face to face, meeting that you have with your customer. And then finally, not exactly generative AI, but I'd like to mention it as well once you're ready and you know all the content, you have all the information that you're ready to share with our Pitch AI feature.

You can record yourself, you can record your pitch and we'll give you information about your, your tone of voice, your body language, the number of smiles, the number of wiggles that you're doing as you're talking to your customer, right? And we intelligently give you advice on how to perform better in front of your customer.

So those are four features, four use cases that we've built. But what's the secret sauce? Right? How did we get to these features? What are the, the core ingredients of the recipe that allowed us to ship this?

So this is how we're going to get a bit more technical by the way. And essentially there's only three, it's surrounding ourselves with an environment of tools, of data, of experts and expertise. It's making sure that we have an architecture of loosely coupled components so that we can reuse features. That's how we got from 1 to 4. And it's having a good set of guiding principles that allow us to apply very mature software engineering practices.

So let's dive into each one of those. Your environment is super important, right? Because you want to create your AI applications in an environment that's essentially AI friendly. So we started off making, making sure that we can give the right people the right access to data. So what we've done is we've essentially built a whole data mesh platform and together with strong governance, we can actually make sure that there's a single source of truth from which you can develop your applications on, on top of that, we built our data platform on top of AWS.

So when it comes down to using multiple services, combining them, we have one cloud native platform that's fully serverless and cost optimized so that we can actually provide the most effective solution to our customers. And finally, we surrounded ourselves with expertise, we upscale our own engineers, first of all, but second of all, we also work strongly together together with the guys from AWS and we also partnered with external partners to bring in some external expertise just to make sure that we're set up for success.

And that brings me to architecture. That's the second ingredient. And like I said, architecture is all about those loosely coupled components and making sure that we can reuse them so that your FAQ turns into your AI powered search and it starts from the bottom, it starts from having a very good machine learning infrastructure, having MLOps available for your engineers so that when it comes down to deploying your your foundational model or your large language model, it's just a click of a button.

And we also added context retrieval as well. So we can augment the foundational models with knowledge, for example, extra knowledge that we have in Choppah, extra data from our customers, for example, and we also added a component for monitoring and feedback. So once this is in production, we're continuously measuring how well it's performing. So we can make immediate changes if there happens to be something going well or something going wrong.

And that's all combined in orchestration layer and that orchestration layer ultimately connects it to the application that serves it into nice and beautiful UX to customers. Now, I can take talk days about this and give you a bit more depth. But today, I'm only gonna focus on the foundational model side of things, the the large language model side of things. If you have any questions about the others. Please let me know after the session.

So let's dive in.

You might might notice today if you look at the landscape of solutions, especially compared to last year, it's pretty big and there's quite a large range of solutions for you out there from out of the box solutions like open a i that provide you with an api with everything pre trained to solution like data bricks that now allows you to manage these models and deploy them, but you still have to self train them two on the other axis, something very managed like amazon bedrock to your own custom models that you self hosts and you self train.

So there's a wide array of, of solutions for you. That also means there's a wide range of questions. But for sure part, we get narrowed down pretty quickly uh because we're selling to large scale enterprise. We, we have quite a lot of compliance constraints for our customers. Some vendors just aren't an option for us because they require us to move data. And that's something that we traditionally don't like to do.

Also developing custom models for every one of our 1400 customers is also not something scalable. It's something that we can do in full detail and therefore also that's excluded from the landscape for us. But with what amazon really offers for us is this ability to cover that that entire landscape with the services that they have some from going from something like amazon stage maker jumpstart, which allows you to deploy a model very, very quickly onto your own machine or something super managed like amazon bedrock, that is just very much plug and play.

And the reason i'm highlighting this is because we're pretty much all over this landscape with our solution we use where the bits that we want and that we need to deploy our applications. So what does that look like? Well, first of all, no reinvented breakout session is complete without a good architectural diagram. So here's the first one, sylvia has a couple of couple more ready for you. But the key thing i want you to, well, there's actually three things i want you to remember from the slide and that's on the left hand side.

What's super important is those traditional api mechanics like authentication, authorization, those still need to be in place. Gen e i doesn't solve it. And you, so you still need to have the engineering, you still need to have the framework to deploy that. In.

The second thing i want to highlight your attention to is on the right hand side of the picture where you see that we're using multiple models, right? We're using amazon bedrock for more conversational applications. We're using sage maker, for example, for making the summaries, we're deploying lama, for example, with jumpstart. So we are just with that lambda in between, have a bit of a switchboard where we pick and choose what we want depending on the application.

And on the bottom, you can see how we're integrating with opensearch to augment the large language models with extra knowledge from our customers and the index information that we have in there.

Now last but not least, the third ingredient are our guiding principles. It's what brings it all together. It's what allows us to bring these products to our customers in a mature way. And so we have six at chopin, i will go through them very, very quickly and then focus on one of them.

So it starts with human agency where we believe that in an in an era in a in a time of a i, we still want there to be human control, we want there to be human audits. So we want to be human overview, especially in enterprise. And it also means for us and from our perspective, a show pad, even though we are using something like amazon bedrock when we're using uh a i models like cloud and lama, for example, we still see us as responsible for delivering that service to customers. So accountability is key in this game.

Transparency is also key right, indicating and educating your customers on where a i is used and what data is being used is essentially also very important to gain trust with your customers. Then obviously, robustness is required any software application that goes to production needs to be. Well, architected needs to adhere to these best practices that have been working for decades on as a software industry.

And then there's inclusivity. I'll zoom in on this one in just a second. But inclusivity is all about fairness. It's about non discriminating. It's about making sure everyone that uses these applications feels like they, they are using it in the right way or feels like they're being fairly treated by it.

And then finally, there's integrity, integrity basically means for us that we're respecting the legal, the social and the compliance landscape that essentially forms the foundation of our relationship with customers.

So very quickly, let's focus on inclusivity. Again, if you have more questions on this, let me know later. Um and i want to take the example of pit a i that we used before. So when you're recording your pitch, obviously, you're doing that in a certain language, maybe you're saying it in english, you're saying it in french, maybe you're saying it in polish. But what's very important is every single language has their own pace. A language like polish tends to be slower. A language like french or spanish tends to be a lot faster.

So one of the things we do is in our application, we detect the language and then we translate that to a language based benchmark. So that depending on the language you're speaking, we're grading you according what the benchmark is to that language. And we're not just treating everyone as equal.

And so with that, i think some final thoughts from my side before i give it to, to sylvia, we will talk a lot about bedrock and the technical details there. We've been on an amazing journey in just a few months time. We've had a couple of really core ingredients that allowed us to bring these applications to production very, very quickly and with a lot of production, production, great quality. And we're very excited to get this to the next level and see what this year brings. So sylvia up to you. Thank you.

Thanks. Hi. Yes. Ok. Well, thanks everybody for being here today. Uh thanks you heard from uh oscar about the art of the possible um how the market is booming your own uh great representation about how choppa is actually using this technology. But now we're gonna get to technical uh one level down.

Great. So i want to introduce you to amazon bedrock. So amazon bedrock is a fully managed service which allows you to interact and build applications using a single api through this api you have access to state of the art models. Uh we keep adding models as the time goes by and these models become more famous. Um and are being asked by our customers.

We have built bedrock. Amazon bedrock from the ground up in a private and secure way. This is very, very important. Why? Because you want to leverage this foundation models with your own data. So you want to make sure that the data is not being shared with others. There is no leak uh data leakage uh or no is being used for retraining of the model. So you can make sure that this is really happening with bedrock.

Um we don't want you to also uh work infrastructure management. This is all taken care of. So you, you don't have to worry about any of this and what models do we have. So we have anthropic. Anthropic is a model that's been trained with uh constitutional a i principles, meaning in a, in a uh ethical and responsible way.

Stability a i um is a state of the art, text to image and image, to image. So some of the things that oscar was showing before about creating your content out of the box, this will be the model that will help you generate those images a a 21 laps.

Um and it's a great model for generating text and generating emails. For example, then we have amazon titan, our own uh own uh models generated and produced by aws. Uh we have texts and embeddings. In this case, we have cohere and command, cohere. They are models who have been built particularly for enterprises and support more than 100 different languages.

So if you have customers across different regions and countries, this will be a great model for you to use. And lastly, um it's meta llama two, it is the first open source model that you can access through an api.

Now, when i present these models to my clients, they say, but which model shall i use? Uh and the answer is it depends, it depends on your use case. So three things that you need to consider is speed, accuracy and cost. This will determine which one you have to use.

So for example, smaller models are really fast, they are also cheap. But the curacy is not the same as the large language models. On the contrary, large models are really accurate, but the cost is high and the speed is not there. So you need to balance this.

Now, when i explain this, always someone asks, but i want it all and don't, we don't, we all always want it all. And the answer is yes, you can have it all if you fine tune you fine tune for a task. And then yes, you take a smaller model, you put your data and then boom, you have something speed cheap and accurate.

But let me talk a little bit more about what else you can do with bedrock. You can find tune, which is exactly what i just described. Um we also have uh agents for amazon bedrock. So what are agents? Agents is a way to automate tasks.

So you can configure agents uh give them access through api s to other solutions. So the services uh agents, what we do is uh get your question, your request, analyze, get information through api through other systems reason what the best next action is and execute, right?

For example, think about you want to send an email to remind a particular set of your customers about a new product that you know that they really want. You can get the agents to do it automatically.

And then uh also we have provision throughput. So you wanna use this model at scale and not have to worry about capacity. You will, you will use this feature as well.

And last, but not least, some of the things that uh um and jon was saying on his presentation, you know, chat to your documents, we have created uh an integrated service list uh vector database integration with uh read this opensearch and pinecone where you can automatically create this vector embedding for your documents in a in a service way.

I'll talk a little bit more about this in a second. Now l ms is, is, is obviously key, but they are part of a much wider integration. You don't build applications just with l ms, you need more things

And there are different ways on how you can optimize the use of of, of these LLMs. And I want to talk to you today about which there are four strategies that you should bear in mind when you want to improve uh the performance and the accuracy of it.

First, one is prompt engineering. What is prompting engineering? So uh Oscar was explaining a little bit about it behind when he was talking about the Party Rock application. Proper engineering is when you tell the model, how it needs to behave as part of the question. Sometimes you do this behind the scenes, you don't let the user see it, but it's something that your prompt engineers will have to configure. This helps improve accuracy of the model by giving examples of what a good answer looks like.

So let, let me give you an example here. Uh here we have just a normal question with no context. We're just using the information that the model is being trying enough to answer the question. So here we have uh the prompt is bright attack line for a new snack fruit and the answer is fueled by nature. Ok.

Now, if I give the prompt a little bit more information, it will give me something better. Um so it will say burst them with real fruit and nutrition, grains, grains, sorry. So this is a little bit better than the one before still using the knowledge of the training. But imagine here that we can also say into the model. Hey, you are a marketing specialist, you work in the food industry. And these are three or four examples of all the taglines that have been done before. This is how you will improve it through prompt engineering.

Second technique. Retrieval of and generation commonly known as RAG. What is RAG? RAG is when you talk to your documents. So here you are not leveraging the information that the model is being trained on. You are just using the model to summarize and and response to questions of your own documentation. Why is this great? Because you reduced hallucinations considerably because you are not using the training data, you're using your data.

Give me another example exactly the same prompt as before. But here what we're telling the model is I want you to check the brand guidelines before you answer my question. And the what the answer is wholesome snacking naturally and naturally is a word that appears on the brand guideline. And that's why the model is is giving you back this answer and the answer is on document brand guidelines page 20. So you can actually go and check the document and see if that's actually true. So this creates trust with your users because they can go and double check or read more if they wanted to.

Now, how do we put this into practice? I'll show you an architecture diagram. So step number one is that we need to transform the text into vectors. So what are vectors? Vectors are numeric representations of the text that have the semantic meaning of the words. So they are stored in a 3D database. In this case is Amazon OpenSearch, this is our vector database. So you do this first uh before you start and you do it once you will have to repeat it if the document changes or, or you add new documents to it.

Now, the day of the uh a user comes through a web application, uh application of your choice and send a query that where the query is then sent to an API gateway that hits a lambda. The lambda does and takes that question and convert, convert it into vectors. So here you're using one of the embedding models on Amazon uh BER it could be the Coherent one. It could be the Titan one. The response is sent back and sent to the vector database.

So what happens now is that Amazon Search is gonna give you back the paragraphs where the answer to that question is and those paragraphs plus the question is gonna be sent to the text model. So the text model now will generate an answer and the answer is gonna be sent back to the user. And this is how you do RAG.

Next. So you wanna go back. Yeah, sure, no problem. Opportunity for everybody to take a picture here. Ok? Perfect. Right.

Next strategy for implementing um LLMs is fine tuning, right? So now you say ok, I've done prompt engineering. I'm done RAG uh but I still want more. What do I want? I want the model to speak the style of my company, the jargon, the, the, the voice you know how we write our our marketing without having to refer to the brand guidelines. I just wanted to speak my language. This is why you will use fine tuning.

For example, the other option is that you want to really improve a specific task. That will be another reason why you do fine tuning. And this is an example. So again, the same prompt just to so you can see the variation. So here the answer is simply fruit and grain, simply delicious. Why is the model using the word simply? Well, because we have been fed this model with previous press releases with the ceo speeches with uh the old board and information that you give to your new employees when they join the company, uh previous marketing material and the model has learned what the words are and how the best way of speaking is.

So, so what if you wanna do it in Bedrock? How would you do it? It's very simple. You just put your data on three packet, you choose a base model here, you can be Titan, it can be Cohere, it can be Lamma two and soon it will be also the Tropic model. It would take a few hours to generate the new model, the version of the model. And then you get your new fine model and then you can start doing prompted.

So now you can take the architecture that I showed you before and just use this model rather than one of the past models. So the architecture before stays the same. The only thing that you need to change is the name of the model in the API called for Bedrock, which is very simple.

So imagine how easy it is to change models in the future when you just need to change the name in, in, in, in the same API that strategy is retraining. So fine tuning is great when you have hundreds uh dozens of documents. But if you have thousands of documents, then you need to use a different tool. Um and this tool is called uh well, sorry, let me show you an sample first.

And so again, this is a different prom here we are saying we are so social media post. Um and then the of the of the box, let's say the mother has not been trained. We are not using R we are not using prompt engineering. Just the answer as it is is we we have launched a new granola bar made with os, learn more the find. uh the, the, the retraining version is much better. It says new blueberry plus granola bars, bursting with juicy, blueberries and crunchy oats. These bars are crispy, delicious and made with high quality ingredients.

How does the model know that it's high quality ingredients? Well, because some of the information that we fed, it talks about uh you know, tracking the supply chain working with ethical farmers ma to ensure that, you know, we, we, we spend time on sustainable um a activities. Uh we pride ourselves and our brand with crispy kind of bars. You know, that's one of the things that we uh try to sell in previous marketing campaigns. So the mo automatically generates something without doing anything other than asking, that is really aligned to the company style and voice and branding.

Now you wanna do this yourself. So as I said, this kind of retraining requires data science skills. Bedrock is for developers and for any builder. But for data scientists, you need to use a different tool which is Amazon SageMaker, which is our uh data uh science uh service. It's very similar though you would take a pre trainin foundation model, let's say Lama two, for example, or another open source model, you will take your thousands of examples. You will use um the tools within SageMaker and then you will get a a aaa free train model that then you can again use again in the same architectural RAG.

So all these techniques are not exclusive, then you can al always mix them together and use them to, to, to get the best performance out of it. Now, some customers will also want to do themselves the fine tuning that is also an option to do fine tuning. So if you wanna do parameter efficient, fine tuning PE and some of those techniques are Lora and Kora, you can do this. Also on SageMaker, there is gonna be all the sessions uh this week on how to do fine tuning on SageMaker. I'll really invite you if you are, if you want to get into the weeds on how to do this to, to visit some of those sessions, right?

So to summarize, we have four techniques. Uh as I said, these are not exclusive, you can put, use them together. And don't you worry like 12 and three will do 90% of your requirements. Even the first two will do most of uh the use cases that you probably come across. So I really invite you to start checking.

But now let me give you a bit of guidance and this is my last slide uh about how i would recommend after, you know, doing many POCs and putting some of these models into production with my, with, with my clients.

First is, you know, get used to these models, you know, try and play with them, how they react, what they're good at and, and, and get a, a bit of a flavor of, of, of how they they work, then find, find a good use case, not just uh let's see if it works kind of use case and science experiment, find a use case that will really solve a problem for your team, for your company and something that will create business impact and, and something that is visible and that you do, then you can go and showcase and get uh you know, the funding that you need to put this into production.

Number three, build a prototype but with a prototype thinking end to end, not just let's see the model, does this, you know, how would this look like from the user point of view? Who's gonna be the user of the tool and build that prototype? And the view end to end that always from the user experience. How can you do this easy? You can use Bedrock, you know, there's no simpler way than an API call and get access to six different models. Plus, there are different sizes of those models as well. We have called repositories. We have Party Rock. So there's many ways that you can interact with this and then you might also want to think, do i really wanna build this myself or maybe there's already an off the shelf solution like Showcase you know, for your sales teams that you can really leverage.

So, but now you have a much informed decision whether this is this is the right tool for you and, and the right use case to, to move forward and that's it. Uh thank you very much. I hope you find this session useful and, and thanks for, for being here. It's, it's late in the day.