Bring the power of generative AI to your employees with Amazon Q

I've always been fascinated about conversational interfaces and the ability to interact with computers using natural language. For example, I was always amazed how my three year old could just call me whenever he wanted by just saying, "Alexa call that" with generative AI. We have the potential to revolutionize the workplace by enabling every single employee to excel at their job by using natural language interfaces.

So today we're gonna talk about how you can leverage generative AI to boost productivity for every employee in your business. I'm Vikraman Barragan, I'm General Manager for the Amazon Q service and I'd like to welcome you all to today's session.

Uh but before we start, uh can I have a show of hands? What how many of you have tried to build your own generative AI powered assistant uh by using tools that are available out there? Ok. Quite a few of you. All right.

So today, this is what we'll be covering and I'll start with the opportunity in front of us. Here's just a sample of all the various types of roles in an enterprise and different types of employees that could exist in an enterprise so let's take the example of a marketing manager. They need to, let's say they're launching a new product. They need to figure out. Ok, how do I succinctly summarize what are the main capabilities of this product? They need to write product detail pages, they need to generate blogs, optimize their pages for search engine optimization and so on, a research scientist would need to look through all the prior results, look, understand what was done in what research compare the side effects of all of those trials and then figure out what the next steps need to be a sales manager. Maybe they have a new account that they've inherited. They need to understand what happened in the past, what the customer wants and what their approach needs to be in order to successfully handle this new customer and so on and so forth.

The opportunity with generative AI is that we can make the life of all of these employees better and make them all more efficient. So in a recent study, McKinsey estimated that AI can simplify work activities that take up up to 25% of your daily time. So that's about two hours of savings for every employee.

Now that we see that this is a great opportunity in front of us. What are the challenges? What's stopping us from leveraging gene? We see three big challenges.

First accuracy, finding the most accurate result from across millions of documents in your enterprise. is a very challenging task.

Number two, security and control over the behavior of these LMs. How do you ensure that all your employees are only seeing the documents that they have access to? How do you integrate this with your directory server to make sure the groups and the access control and all of that is maintained across all of your employees.

Number three is time to value in today's world. It's important to move fast because your competition is moving fast. They're quickly launching these new capabilities and rolling out cool new features for the customers. So how do you make your employees more efficient quickly rather than starting projects that last months on end? And you don't know if it will be successful at the end of it.

And so we launched Amazon Q yesterday in Adam's keynote and what Q does is it's your generative AI powered AI assistant that is specifically designed for work and it can help you with your business, with your data, with your code and with your operations. And to give you a broad picture of what we've launched here with Amazon Q, we have embedded Q in across all of AWS.

So for example, we start with Amazon Q for your business where you can deploy Q with your own data in your enterprise. But then if your developers are building on in AWS, you can use Amazon Q to build better and get help on all of the AWS tools as they are using the AWS console and operating various different services on AWS.

You also have Q in Amazon QuickSight where you can do natural language powered BI business intelligence by just asking questions like, "Hey, who are my top 10 customers?" or "Can you show me which region had the most sales?" and so on?

We also have Q in Amazon Connect, which is our contact center software to help agents be more efficient while helping customers and soon to come. We'll also have Q in AWS Supply Chain.

In today's session, I'm gonna focus more on Q for your business and how you can build a digital assistant for your business and all the different things that you can leverage it for to be more efficient.

Some of the key features of Q can be divided into features for admins and features for end users. From an end user perspective, Q can do conventional question answering where you are able to ask questions across all of your enterprise content. And Q is able to give you accurate answers along with citations and ground rules so that you can verify these answers and make sure it's not hallucinating.

You can upload files and analyze the content because you know your admin may not set up all of the files in, in your centralized index. You may have many files just that are just on your local laptop that you may want to upload into Q and analyze them, compare them and so on.

You can even execute actions on various systems like Salesforce or Z like cutting tickets in those various systems. And you can generate content like writing emails or writing blogs or marketing copy or product detail pages, our memos and so on.

From an admin perspective, it's a fully managed solution. It has 40 plus prebuilt connectors comes with its own vector index and an end user application. So you know, this goes back to the time to value that I was talking about, you can deploy it very quickly with no need for any development work.

It's permissions aware. So it's able to look at your access controllers and then compare that at run time with the permissions that the user has and only show you the documents that you have access to. And then most importantly, from an administration perspective, it gives you controls and guardrails using which you can ensure that Q was behaving in a way that you wanted to.

So for example, maybe you don't want it to handle questions on certain topics or maybe you wanted to use a certain data source to handle certain topics and so on.

Uh my colleague Nikil will now take you through these features and the details of each of these features.

Thank you, Vikram. Hello everyone. Uh welcome again. Thank you so much for joining this session. I'm Nikil Shetty. I'm a senior product manager for Amazon Q and I'm here to talk about. Very excited to talk about all the features that we've built in Q for you. In fact, over the past few months, I've been using Q myself and I made it my personal assistant and my manager suddenly tells me I'm a high performer. So I'm very, very excited to see how you can make this your own personal assistant.

Let's talk about the key features. So the first one, conversational Q&A. So what you're looking at on the screen is actually a screenshot of what the end user experience will look like, which is integrated with your identity provider. When your users are asking a question, what Q does is it looks at all the documents across all of your data sources. Be it SharePoint, be it Salesforce, be it OneDrive, Google Drive looks at all of the relevant documents that will be important or relevant to the query, put all of them together, combine the information and give you a very, very comprehensive answer along with the sources so that you can go and fact check the responses.

So you look at a response, you're not sure whether it's accurate, you look at the sources and make sure that the facts and the accuracy is right. In fact, for every sentence that we generate, we make sure that we give you citations for that sentence. So that if there's a long response, you can always go back and see for every sentence you can see well, if this fact is correct or not just like a natural conversation, you can continue this conversation in the context. You can ask follow up questions you can ask for summarization and you can ask for even some actions as we will see later and it will understand and maintain the context.

We also store the conversation history for you so that you can go back in time, see what you asked for before and even continue the conversation to pick it up at a later point. And all of this happens with user permissions in mind. So users will never get any answers from documents that they don't have access to.

Users can also upload files and ask questions. So for example, you can upload a file and ask Q to summarize the information in that file or you could ask questions about the content that is in the file. Or let's say if you have some numerical data, you can upload an Excel file, ask questions, do some analysis on that data, maybe even asked to write a narrative.

You could even take, let's say two documents together, upload those two documents and say compare these two documents and give me a summary of all the differences. We support multiple file types for the file upload feature. So you can use PDFs, CSV, PowerPoint text and more.

So think about some scenarios where something like this will be useful. We talked about a marketing manager earlier, let's say you are a marketing manager and you get a new product spec and you've been asked to write, let's say an advertising copy or a marketing copy. In a very short time, you don't have time to read the entire document. You can just use Q, upload the document to Q, ask it to summarize, tell it what the key feature, what the key features in the product are. Tell it who your audience is or target customer is and just ask it to draft the first copy of the marketing copy of the advertising copy. And that's your starting point.

Or let's say you're working on contracts and you've, you've gone back and forth with the customer and you're not sure what the first contract is. What's the difference in the latest version? You can upload those two versions and ask Q and say, what is the difference between these two? Just summarize everything for me and we'll do that for you.

Now, when employees are working in enterprises, they are usually hopping from one application to another. As an example, someone might start with finding some information, ask questions, summarize everything they found out and then go and file a ticket or someone may start with brainstorming ideas, summarize all those ideas and they may want to share those ideas over chat or email with your colleagues or someone may even want to update some enterprise applications. Let's say they've had a meeting, they want to organize all the notes and they want to update it to an enterprise application for all such activities. We want Q to be an enabler or a starting point to be able to start to be able to enable the employees to use natural language to make these tasks happen.

So starting today, we are announcing support for creating tickets in Zendesk, Salesforce, Jira and ServiceNow. Now, so for example, a user can say summarize the information and file a ticket in Jira and open a Jira interface for you where you can quickly file, submit the information and create a ticket.

So talking about safety and security, top of mind for enterprise admins and we talked about connectors that support document access control, meaning this ensures that your users never get answers from documents that they don't have access to.

Now let's take a look at how that works. So when you connect your data sources into Amazon Q think Salesforce, SharePoint, Google Drive Confluence, whatever have you. What Q does is the connectors will go and crawl these data sources, take the content out and put it into a semantic index for search.

Now, when we do that, we not only crawl that document content but we also crawl the document permissions. Now, what these permissions contain is a list of users and groups that have access to that document.

Now, when a user is running a query against Amazon Q. We gather information about the users, of course, the user ID. But since we are integrated with their identity provider, we also find information about the groups that they are part of and we use all of this to make sure that they only those documents are pulled from the index to which they have access to.

So now you don't have to worry about any documents, any sensitive documents being accidentally leaked to your users through Amazon Q, it is all supported out of the box.

We also have guardrails for admins. So we have four types of guardrails. First, we have default guard rails that protect against queries that could generate offensive, discriminatory or illegal content. This comes out of the box without any configuration required. So Q will just respond with, "Hey, sorry, I cannot help with that." if there is a question that matches with, that's either offensive or discriminatory or toxic in general.

Now, second, we also give you the ability to restrict responses to enterprise content only. Now we hear this from customers all the time. They say, well, we love the the properties of these LMs, the fact that they are able to generate answers, but they don't want to hear answers from what's been trained, what the LM has been trained with, they want answers only from enterprise content and that's what we enable with this feature.

So for example, if you enable, if you restricted to enterprise content and the user ask, tell me some ways to improve. So for my web page and let's say your enterprise repository does not contain the answer. Amazon Q will just respond with sorry, I do not have enough information to answer.

Now, let's see, on the other hand, if you do enable, if you do enable it to go outside of enterprise content, in that case, it will come back and say based on my knowledge, here are some tips to do that. So in this case, the user knows that this is not coming from the enterprise sources, but it's coming from its own knowledge so that they can go and make sure and verify that all of the responses are actually correct.

Then we have third thing is specifying blocked words or phrases. So you can specify a list of blocked words or phrases that should never appear in the response. Imagine you have a secret project in the company, very sensitive documents, sensitive subject or keywords that sure you'll have access control to make sure those documents are not accessible to everyone.

But you know, Pick Enterprises is always a straight document that talks about something that people should not know. And so with this, you make sure that you you can say well for this top secret Project X make sure that this never appears in any responses and we have checks and balances in place that any time it encounters any topic like that it will just block it and make sure that it doesn't, it's not shown to users that don't have access to it.

Then we also have topic level guardrails and there's so what you can do with topic guard rails is you can specify topics or special topics and you can define treatments for those topics. And we have four types of treatments.

The first is you can respond with a predefined message. So a query comes in, we identify that it's related to our topic that you have defined. We will respond with a predefined message that you have defined. It will say here's what it is or you could restrict it to enterprise content only same situation query comes in but it's only responding from enterprise content, not going anywhere else.

Third treatment is you can restrict responses to enterprise content but with merida filters. So maybe you want to say for these type of queries only go to sharepoint or only go to google drive or you may have some other merida to identify your content only restricted to those sources. And you can apply these topic filters to specific users and groups and enterprises.

Now let's take an example scenario where this will be useful and say you have a security team that has discovered vulnerabilities in your website. They've created some documents about it. That's all in your enterprise corpus. Now, you want to make sure that these are not opened up to all the employees. Again, of course, there's access control but like we said, pick enterprises very hard to control these things.

So what you can do is you can create a topic and say security vr is in the website and say, and you can define a few sample utterances to, to identify what kind of questions could be asked to get to that topic. And then you can say, block this for all of my users. Well, wait, hold on though. You don't want this to block for everyone. You want to block it for people who are not in the security team. So you can go a step further and say block this for everyone. But for the security team restricted to only content in your enterprise repositories so that they will access those documents, everybody else will get a predefined response.

So we talked about connectors. We have 40 plus connectors for different enterprise data sources that take out the tragedy and the heavy lifting involved in bringing all of your data together into an index, extracting the content out of this, parsing everything, chunking them into sizes, taking out permissions, information metadata, all of that is completely built in. All you have to do is you go to the console, you point it to the uh the data source and you start the injection process and we do everything for you.

Our connectors also have features like periodic scheduling. So you can say, you know, sync the run the sync operations every month, every hour, whatever works best for you, you can run incremental syncs or full syncs. So incremental sync is in case you want to make sure that only the updated documents uh flow through and you make sure that your index is up to date.

But there's a very, very important feature that our connectors also have, which we refer to as the identity crawler. Now many data sources like sharepoint slack have the concept of local users and groups. What this is is is basically a permissions group that is defined within the, within the application but not in your identity provider. Now, it is extremely important for any generative assistant to be knowledgeable about these things. And our connectors make sure we have identity crawlers that make sure that they take the information about local users and groups as well so that the acs or the access control permissions restrictions that we have are enforced for every scenario. So you are covered from all fronts as far as making sure users cannot access information that they are not allowed to access.

And so let's look at the console and the experience for admins and what it takes to build an experience like this, right?

So this is what the console looks like on the landing page. You start with creating an application, so you specify some basic parameters, create the application.

Next step you select a retriever. This is where all of the documents will be ingested for semantic retrieval. Later on, we have two options, we have the native retriever. But if you are already invested in something like Kendra, you can bring that in with a one click integration.

Then comes the part about connecting your data sources. As I mentioned earlier, we have 40 plus data sources. In this case, we will use Amazon S3. You can see the configuration page for S3. Uh we got a lot of different features we talked about earlier, the sync mode. Um you can also specify metadata that you know what metadata should be mapped to, to what in the S3 to make the search better.

So you've created the S3 uh data source now and you're finished, your application is now created. Now, you want to go and check whether the content is actually ingested into uh q into your instance. So the connector is building up once it's ready, you hit sync now to make sure all of the content that is in S3 is actually indexed into the index.

Once that is done, we have a preview web experience for admins just to check and see what the end user experience will look like. And, and, and that's it. This is the process for you to get started. Admins can start asking questions, start getting responses.

Now, if you want to take this experience, deploy to your end users, all you need to do is whatever identity provider you have - Octa, Azure, a Ping Identity - you need to connect it to that and you get a url that you just deploy to your end users and they can get started quickly.

So that's the end of the demo section. And with that, I will hand it back to Vikram to talk about the challenges that we introduce. How are we solving those with all everything that you saw?

Thank you Nicole. That was an awesome demo. All right, we spoke about the three challenges at the beginning. Now, let's look at how Q addresses each of these challenges for accuracy. As you saw, we are able to use your enterprise content to give you accurate answers and also show you the sources so that you can verify that it's not hallucinating and give you specific citations for every point in the answer that he is giving you. Plus we also give you flexibility of uploading your own files and analyzing them for security and control.

We spoke about how we respect the access control lists as well as the permissions and groups that have been set up within your enterprise and the ability to configure and the guard rails at a really nuanced level so that it behaves exactly the way you want it in your enterprise.

From a time to value perspective, you get the 40 connectors so you can quickly index any repository across your enterprise. And we also have an SDK in case there is a repository which is not among the 40 that we support, you can quickly build a connector for your own repositories as well. It comes with the inbuilt intelligence built in all of the indexing and you know, all the work that goes into it that you know, a lot of you that have tried to build such a system, you know, might have realized and then the integration with SSO which again makes it super easy for you to deploy this in your enterprise. Imagine you just connect it to Octa and now this is another SSO enabled application that any of your employees can just log into and start using.

So not to bring it all together. You know, we started with the work that the marketing manager has to do and now all they need to do is open up Q and say Q, summarize this product requirement document. They get the list Q write me a blog that focuses on all of these benefits for customers. They can also ask you to help optimize the search engine optimization strategy for their blogs. Ask them to ask you to write advertising, copy and so on. And similarly, the research R&D scientists that we spoke about can do all of that analysis of trials comparing the side effects of various trials, writing up their findings as a summary, sending and sharing it with their colleagues. All of those use cases can be now done with Amazon Q.

With that. I'd like to invite on stage our customer Sri from Experian to talk about how they have used Q in their organization.

Welcome. Great. Thank you, Vikram. Hi, everyone. I'm uh Sri Sanam, uh EVP of Analytics and AI at Experian. Many of you may know us as uh a global leader in information services and technology. We host among the most valuable data sets on the planet. 1.5 billion people that's the number of people whose consumer information we handle. Uh as Vikram talked about privacy, security responsibility is paramount for us. We have information on 200 million plus businesses and span over 40 countries and a core part of our mission is financial inclusion. It's an area which cuts pretty close to me personally as an immigrant coming from India growing up as a child. Uh my parents had very limited means to uh perform some of the major life challenges. We had uh financial access was quite limited and we had to rely largely on networks. So financial inclusion is paramount for us. And we see that as an important mission and for, for decades now, we've been using AI to further that cause with generative AI, we see that as having a tremendous amount of potential to amplify that mission for financial inclusion.

We once the the meteor hit earth around gender of AI about a year ago, we've been looking at a number of use cases on how gender of AI can help us with products, productivity and across the enterprise. But uh a little bit of a story as we think about our strategy, I have an 11 year old son and this was about nine months ago. And as, as most 11 year old kids, he's uh he's quite fascinated and addicted to digital. He's got a Nintendo Switch and uh watches all of the shows you'd expect. And uh we don't like that. Of course, as many of our parents would imagine here and he's not allowed to use his Nintendo during the week. But one day that's about six months ago, he comes and sits us down, uh sits, uh me and my wife down and um, and he's heard about uh Amazon's PR FAQs and their documents. And he's, he's 11 years old and he presents this three page essay titled the case for 10 minutes of screen time during the week in which he talks about uh dopamine addiction. He argues both sides and he presents that. And I look at that and say, like I could not have written that essay even when I was in college. And here you have a child who has figured out that this is a way to influence two of his parents speaks their language. And my wife was like, I was actually ok with it, but my wife was convinced. And uh and if, if he can do that, uh what we realized as a company that the biggest thing we needed to do with generative AI was what you see on the right hand side, grassroots innovation. We were not going to unlock this by getting the 10 smartest people in the company locking them in a conference room and saying, figuring it out, this really was going to happen by letting people across the company. The 22,000 people we had use this understand how it can be used, develop use cases. Of course, the left hand side was paramount. Our focus is impact and financial inclusion and benefit for our businesses and customers and consumers. But promoting grassroots innovation was key and that's why we see Amazon Q uh coming in as a pretty important vehicle.

Three big reasons why Amazon Q resonated with us one. And I'll try and tie back to Vikram's three things. I think it was accuracy, security and time to value which, which really made sense for us. First, it's a fairly low technical lift, fully managed application. We could get up and running within a matter of hours, which made a huge lot of sense to us because we had 250 plus use cases. Second, uh the time to value we without this, we were looking at weeks, days, months, even for some of these use cases. And we needed to handle this in a secure and responsible way. And third uh unlocking the value of our, our core assets and our data, we have some of the most valuable data sets on the planet and not just structured data but also unstructured data and this is very important for us. So Amazon Q certainly resonated with us.

Here are some examples of the sort of things we did with Amazon Q uh one for the spanner heads and the crew here. Uh the engineers will appreciate we have uh as your coding and uh and, and committing code, there's often uh detailed technical documentation as we build applications with infrastructure code, uh dev ops code that you have to follow. And I've had folks in our team say they spend at times 45 minutes an hour reading through specifics on how uh even five line of code need to be written just to understand it. And, and Amazon Q really allows you to have a dialogue with that whole document and allows you to skip that whole 45 minutes and understand what you're looking for, understand the specifics. So quite a much more than a than a mundane look up or search. It's a thoughtful connection of various forms of language, technical documentation which comes out with results.

Uh Vikram and Nikil talked about product and sales and uh and we also see it as an important use case for us in in coaching our technical and non technical employees. So uh early days, but a number of powerful examples and ultimately, we'd love to have uh uh a plateau of agents who help us unlock a lot of our productivity and value.

So what we built uh you're seeing here is an example of one of those use cases. Uh what we particularly liked here was uh the secure connectivity to some of our existing data sources, the whole engineering layer which you can often underestimate when you're trying to take ca use cases like these into production. So with secure access control, uh appropriate permissions connecting via Octa to Confluence. And some of our internal web pages was able to do a nice job and, and this one give or take took like a few hours. The the core portion of this took about 30 minutes but relatively short time. And, and ultimately, for us early days, but we're seeing business impact in three key areas.

First, increased engagement. We expect our customers, our employees to be able to engage much better with our products. Our services amplify the impact, uh foster time to value. Certainly, it allows for a lot more experimentation. It allows us to reduce the cost of trying many of these things. So it, it drives innovation in many ways and finally secure and safe. As I said, we have among the most valuable data sets on the planet. We have all your data and security and safety is paramount for us. So we want this to be handled in a safe, secure and responsible manner.

So we're, we're excited about what we'll see and more to come over here. So look forward to the partnership with Vikram and team.

Thank you, Vikram back to you.

Thank you Sri.

So that brings us to the end of today's session. Uh hope you all enjoyed it. Uh that's a list of uh other related sessions that uh you should check out, feel free to take a picture if you like. Uh please uh fill out uh the, the survey as well at the end of it and here's a few links uh to help you get started. I'll leave it on for a moment in case uh you want to take pictures of our web page, console developer guide and uh with that. Thank you all. Uh and uh we'll be here uh if you have uh further questions. So uh