New Amazon EC2 generative AI capabilities in AWS Management Console

One. Thanks for coming and joining us on a Friday here. I wanna thank everybody who decided to sit real close to me right in the center. Uh you know, but uh uh thanks everybody for coming out here. I'm Art BDO. I'm a Principal Product Marketing Manager here at AWS. And joining me today is uh Sakeet Gupta.

Sakeet: I work as a Principal Product Manager in EC2 in AWS.

Yeah. All right. So, uh for those who don't know, we're gonna be covering uh new, I'm gonna just read from the slide for a second, new Amazon EC2 generative AI capabilities in the AWS Management Console. So, what does it mean? Uh what we're going to be covering are uh three launches that happened this week. So, uh two of them are part of the Amazon Q family and the other is I would say a uh close cousin of that. So we're gonna first cover uh how to select use Q to do instant selection. We're gonna talk a little bit about uh console to code and then we're gonna finish up by talking about uh one of the network management features that you can use Q for as well.

So I'm really excited that all of you join. And the best part is, is, is, uh it won't entirely be us talking with slide, although I'm sure that's what you all came for. Uh we're also going to do a demo as well here. So we're gonna try to real time do it. So bear with us if we uh make an error, uh we're error prone just like everyone else and we'll try to walk through and then, uh we'll have a little bit of time most likely at the end if you have any questions for either of the two of us. And uh let's get started.

All right. So I first wanna just start by uh some basics a little bit about how we got where we are and, and uh a little bit about some of that naming stuff. And so we won't spend a lot of time on this. But, you know, we've, we've uh been focused with on EC2 about, about trying to kind of reimagine in our, for ourselves, the cloud and make it even easier uh to use that. And one of the things that we've, we've noticed over time, um for those who, who have been here before, have worked with us is this, we realized back in 2012, 2013, that fully 25% of the servers at that we, we had who are being used or consumed for management functions at AWS. And that prompted us to realize that this is not a projection that could go on. And so how could we reimagine what we were doing? And this resulted in us creating the a what's called the Nitro system or the AWS Nitro system?

What the AWS Nitro system allowed us to do is, is separate all a lot of the management functions that we had been doing uh on the CPU hardware that we were working with uh off to the side to separate chips, a whole separate set of chips. Uh and Nitro cards that we distributed. And to date, we've uh actually produced and moved into our own, our own cloud, excuse me, over 20 million Nitro chips at this point in time. So it speaks to the breadth and depth of what we're, what's happening. But the advantage to everyone out there is, is, is and how is it applicable to this conversation about gen AI and the work we're going to do is is we've had a lot of learnings uh in the cloud. But part of this is this has allowed us to actually deploy more and more instances.

Uh and those instances are actually not just some random thing that Sacket and I like to do on a Thursday afternoon, uh create random instances. These instances are the result of the direct result in product of conversations with customers who have asked us for more and more. And asked us for changes. And so some of the things we're also trying to act on is how to make it even simpler to use the instances that we have. We think they're, we've tried to make them as simple as possible, but we're trying to make them even more simple here.

And just a couple of other quick things is, is that to date, you know, a little bit of statistics, you know, it took us about 2.5 years when Amazon got started in 2006 for the 1st 1 million instances to be launched. And today, over 100 million instances are launched every single day, translating into billions of instances launched per year. And that gives you an idea of the scale that we operate at and that you get when you use Amazon here. And it also gives you the i an idea of the planning that we have to do that goes into that and making that available to everyone.

So we continue to have two goals here to provide all of you customers here with tools and services to reliably run virtually any workload. And we're going to discuss how you can do that today and to do continuous innovation to increase performance. As I mentioned before. Uh today, we have virtually, we have over 750 instances available for virtually any workload and the features we're gonna talk about today.

And we're going to start with here is is to talk about how you can use uh these tools that we have to make it even easier to select an instance. So ok, it's all right.

Sakeet: So we're gonna talk first about the instant selection feature. Um so EC2 provides a wide portfolio of instance offerings they are made for, you know, exhaustive list of use cases which customer has. We also have specialized instance offerings which, which goes to customer for specialized or unique workload requirements. Now what happens is like having a 750 plus portfolio of instances, it gives her some complexity to the customer to select like what is the right instance for me? right? And what is the right instance for my workload?

So we have, we have solved that using Amazon Q. Amazon Q is our generative AI powered assistant and it analyzes our ears of data on instance offerings and then based on simple input from you on your workload requirements or you know anything about workload. Like you can tell us that hey, you are working in a gaming industry and you are you want to spin up an encoding or transcoding workload, right? Or you are working in a say an machine learning product and you want to create a deep learning workload. You tell us that an Amazon Q gonna suggest you instances based on your requirement.

So yeah, let's quickly have a look on a couple of use cases right here and then i'm going to switch gears to, to console also to walk you through a couple of flows where you can actually engage and make best out of amazon q for the instant selection.

Art: Hey, just a sec. Before we move forward, how many folks are familiar with the AWS console here? Hopefully. All right, good. Hey, that's a laugh. But i, i ran a builder session earlier in the week where someone didn't know what we were even meeting about. So i just wanna make sure we're all familiar when you see it come up here. All right, let's, well, that's a good question. A good follow question would be like how many of you have tried Q already after the huge green went? All right, great. Ok. Cool. Fine.

Sakeet: So let's, let's quickly see like how i as a customer with a couple of use cases can interact with Q and make the best out of it to get down to an instant selection for, for my workload.

So yeah, like i'm a customer, i want to spend up an RDBMS workload for my web app. So you can define your use case. You can define your application type. You can also define something like what's your priority? Are you looking for low cost? Are you looking for performance or availability? right? Or you're looking for price performance? These are very simple inputs which any new customer who hasn't done extensive study on their CPU or memory requirements? They would know that right?

So you can define that, you can talk to Q give these requirements and then Q gonna you know, analyze our databases and our our instance details and then suggest use some of the recommendations here. So in this case, you can see we are suggesting R7A R6A latest generation instances, we are also explaining a Q is also explaining that why we are suggesting this right? And, and, and also not just you to do or take the next step. So it guides you also like with these suggestions, what are the next steps you need to do to kind of finalize? What is that final one instance type you want to work on? right?

Art: No, it also it's also in priority order uh according to the query that you have provided it. So this is what it thinks is a priority based upon the query that you started in there. So i think that that that's the other thing like uh the two things i like about it is one, it provides you the priority order, but also provides you at the bottom. That paragraph is a discussion. And i can assure you, we just for simplicity, we, we took out the que we have uh this is a query and response directly from Q. If you submit this question, you will see this answer as well. We're not, this isn't. So it just, it didn't, it was smaller don't worry, i'm gonna show everyone in the console also.

Sakeet: Yeah, cool. Ok. So, so while doing this query, maybe like i have some more information like i got to know that, hey, my existing workload is using CPU, you know, intel CPU as my CPU preference, right? So what i do is on this query, i just give this query like i, i tell you like, ok, based on the context i gave you just tell me like, what are the intel CPU based instances? And then it's gonna, you know, give me a return answer out of it.

So it's a standard like, you know, interaction canvas using natural language. And you can tell you, you can give additional information to narrow down your search and you get to the final selection. Let's take another example and i will be quick with this one and this is like the gaming example or use case i talked about, i want to spin up a video encoding workload for my gaming application. And again, my preference is performance, right? So i gave that and you gonna analyze the the instance data and we will present this answer. It has the specialized V1 instance also, but the other ones and again, with the explanation and you know, what are the guidance, what are the next steps?

So it guides you through that. And then like maybe like if, if i'm thinking of more of from a workload perspective, i decided that maybe i have a second priority also like i first need the performant instance types. But availability also matters to me a lot. It's a gaming application. I want to be available. I cannot have my users facing a downtime error or like in between with their, you know, stick like they, they, they are not able to play more.

So yeah, like i give that and then what it does is it takes the context from before identifies dig deep and get to the availability information for these instances and then recommend me with proper reasoning, the instant suggestions, right?

So, so this is these are the two use cases. I wanted to talk to you about to give you an idea of how you can interact and get answers. Let's go to AWS Management Console and, and actually see how it's working.

All right, great. It's working. Awesome. Ok. Number one. So before you know, talking through the console, i want to give you some context like so in in AWS Management Console, the ask or the need for the instance type decision is very contextual to your actions, right? So you're taking, you're launching a EC2 instance or you're launching a launch template and you see an instance type selection module in that and that is where you need to take a call, right?

Obviously, there is a lot of pre decision making which happens before you know, you think and start going in the console, but console is a prototyping tool and you know, it's a simple canvas where you play and learn and, and that's where we we wanted customers to, you know, kind of get these suggestions and work through the final selection.

So what we have done is like, so i'm in a of management console, i will go to EC2 and this is EC2 landing page and now i am launching an instance, right? So this is my action where potentially like i have the need to take citizens type decision. So in this floor, i can, i can fill this name and maybe move forward with army and architecture as defaults. But when i get to that instance type section, i have to take a call and here you're gonna see the list of our 750 plus offerings, which is not much helpful, right?

So what we have done is we have provided this get advice, you know, button and again, you can get advice here also. And if you click on it, we have produced an initial model. And this is like basically for a very new customer who doesn't even know like how to take a call for an instance type. Like what is the mental model or what is the framework i should follow to take a call, this model kind of kind of give them a hint or you know, kind of bring them into a framework and then they can work forward with it.

So you present this model and then you can make some simple selection. So like i am, i have a high performance computing use case i want to spin up a Hadoop application performance is important to me. And maybe like i'm looking for say Graviton, Graviton instances and i click on get advice, it takes me to Amazon Q and it will show an answer to you.

So you can see like it's just R7G, which is the latest Graviton instance and proper reasoning, some links to you know, as a reference for you like, hey, these are the documents we used, you can read more about these suggestions in these documents and this is how we interact with Q and then like you can use this box to, you know, further give your requirements, give your queries to narrow down your search, right? And get to the final instance type you want to choose, get back to your launch flow and keep going with launching an instance, right?

So that is a kind of contextual flow we we have tried to achieve in the console, you can any time go to Q directly, right in console and ask any question about instance, suggestion, right? So that flow is always there for you in AWS Management Console. Ok?

I think the best part is is you can continuously follow it up. Um i don't know if other people end up being like me when you're in the console that you realize. Oh, wait, i have another requirement. So that's why we did our example with the follow up questions here. This one was a little bit simpler since you filled out the template at the beginning. But it's great, especially if you're working with people in your own organization who have to, who haven't done or haven't worked with some of the console aspect before.

It's a great way to get the feedback uh as well. At least I think so. Um right. Right. And it, you know, we were, when uh Zack and I were coming here today there, somebody was analyzing our questions and like, oh, this, this isn't spelled precisely right. You know, this should be this way, but we, this is natural language. So literally, it will try to make a determination, you know, you don't have to be perfect uh here as well uh to, to get a response. So it's kind of, right.

So one more thing I can say here is like when generating these answers, we try to get more context and, you know, try to understand you as a customer, right? So if you have asked something else before and q related to your workload, maybe it's like, you know, what are the im roles? We should, we should, I should think of for by this workload, it has that context. So, so, so it carries that context and you know, it, it trying to build the answer based on you know what what you created before also in the queue. So so these kind of machine learning intelligence is has been built in the queue. But yeah, like these are the two flows, you can directly go to q you can go through launch experience and you will find it in launch templates. Also when you are comparing in this types, there is a compare in types experience also in our aws management console there. Also you can you know get a suggestion, get back, see details on the specification, take a call. So we have tried to make it simpler for for our customers to to get to a instance type decision quickly, right? Yeah.

Alright. Ok, cool. So with that, let's jump to our next launch. Uh um do you wanna bring up a slide? Let's split up the slide. Ok, look forward. Yeah. So this is the second launch we did and re invent. This is aws console to code. It's a capability for our developers that makes it simple fast and cost effective to move from prototyping in console to running production workload using code. So, so let me give an example. Like console provides a simplified interface to take action, right? When you go and launch an experience, it's a very simple ui with inbuilt validation checks, guard rails and best practices, right? It forces you to take an action with best practices so that you know after any configuration change or provisioning, you get into less errors. But what we have heard from our customer is like when they do this prototype, they learn about a service and an action and then when they want to apply it to their production workload or any add scale operation, they have to start from scratch and writing the code for, for what they did in console.

So what aws console to code does is with the help of the generative a i models, it understands what actions you took. And then based on your selection of your actions, it generate a synthesized code for you in your preferred format. So let's take an example of a persona like devops, right? One of the task which i heard a lot when talking to customer was like they have to launch an instance with a specific configuration and then they have to enable monitoring with a certain configuration and specification. Very simple like launching instance, enabling specific monitoring and it's a repeat task for all their instance, launches for a production workload, right?

So what we could do here or what they could do here is they can come to console, launch instance in an easy canvas, do the configuration for monitoring and then they can come to console to code, select those two action and then we're gonna generate a cdk or a cloud formation code for you. And this is not just limited to mutating actions which i talked about which you know, kind of provision or configuration changes. It's it also expand to non mutating actions like filtering. So scenario where you want to terminate instances which have a specific legacy army, right? Maybe that's a scenario. So there the actions you're gonna take is you're gonna go to instances pages in aws management console. You're gonna add a simple filter through two clicks for your army and then you're gonna just click on terminate to delete those instances which showed up as part of the filter. So the first part, the the read only with the filter, the conditional non mutating part is also captured as an action. So you can do all these permutation combinations in console to code experience and generate a synthesized code for it, which will give you a good start and you can take that code, go to your coding environment, mature it even more and then run your production workload based on that.

So yeah, let's uh ok. So maybe like let's switch to console aws management console first and then i'm gonna get to the slide. Ok. Ok. I just ok. Yeah. So i'm gonna demo the whole flow right, starting from the two actions like launching instances and enabling monitoring. So we were here last right in the launch flow. Let's just use this. So i'm gonna launch oh this instance uh for my account i'm gonna just use without keep it and just keep it simple and ok. So this is my first action i took. Then i went to instances and what they did is went to monitoring and just to keep things simple, i just uh ok, this has enabled, so what i'm gonna do is i'm gonna disable and enable it back. But the idea is like they go and enable monitoring configuration. So this enable detailed monitoring. And then i see this console to code in the same nav this is a new feature so you can read about this and go to console to code and i see all these actions which i took.

So i first went to launch instance, then i you know, disabled the monitoring and then i enabled it back. So it's all captured here and this is under mutating actions pocket, but i have read only also and then all the actions i took. So the whole list. So, so as a customer, i'm gonna select launching and enabling monitoring. Then here these are the code formats we support right now. So you can choose any of these code format, right? So our buckets right now are cdk and cloud formation. But yeah, like we are working on evolving this even further. And in this case, like i just selected cdk, python curl and i clicked on it and it generates the code for uh for your actions.

So just a few things while it generates like it actually takes the a p commands which console is making when you take the console action. So it preserves the goodness of console, your defaults your checks and validations and your guard rails, which made console a simpler canvas, it preserves that and using that, it generates the code, right? So, so it's, it's we, we call it as a like a well architected code because we try to, you know, replicate well architected practices in console. So, so that's how, you know, the code generation helps you in in making it a good code rather than a standard code for your different actions, right?

So yeah, this code is generated. it's a synthesized code, combining your two actions. So it's a single code snippet you can run and the both the action is gonna happen. And in here you will see some explanation on the code also like what does it do? Because if you want to share this, you want a copy and download and share this as a file. It has everything which will help your ecosystem to you know, kind of collaborate and mature this code. Yeah, like uh the other things it has is uh yeah, a copy and download. These are the two controls we have provided, you can copy, you can download and share it. Uh you can copy and paste it in your coding environment and start working on it or modify it. You need it. Yeah. Yeah. Modify it, enhance it based on your requirement and then use it production ready before you implement it.

And then like we also have here c commands, these are the commands for the two actions i selected. So, so you would know like the individual commands for the actions which contributed to that synthesized code in this page. It has all the artifacts you require to get, get to understand what it has generated for you uh uh as as possible.

Alright. Ok. Cool. So that's, that was the announcement. Number two, we have the third announcement, which i'm gonna talk about me back to slides and you can skip this. Yeah, we had a, just in case we had a demo thing, we decided to put a big chances. So give us a sec to let me go through the slides here, right?

Ok. So our final feature that we're talking about is is uh amazon q for network troubleshooting. So once again, it uses large language models to check connectivity for troubleshooting and answers broad questions. So as a lot of folks who deal with networking stuff, a lot of anytime there's a problem, you know, the first question is usually to the network administrators or other people like what's going on. And so we're trying to allow you to ask plain text questions and do some consolidation uh to boost productivity here as well for all of you who or those of you who deal with or work with network issues.

So similar to before uh i put in some questions into it based upon questions that we received feedback for uh before. And so my first question is pretty basic here. It was, uh you know, why can't i ssh uh you know, a lot of, i'm not so sure all network administrators need to answer this question, but a lot of people have the trouble. Ok. I think we're ok. Um so why can't i ssh enter in the question? And so it then provides uh an initial response. It sounds like you're unable to ssh into the network and it could indicate a problem with perhaps security networking ac ls or some routing table information.

Alright. So then, you know, also though, doesn't stop there with this, this being a more networking oriented feature, we provide some information here about the path details to give you some perhaps information about why you can't uh ssh right now and follow it up with some additional uh text. It says, you know, i chose the internet as your source and it gives you some information about what you could try. So it's both this particular feature gives you kind of a natural language processing uh advice on what action to take. It does not take the action for you. It uh offers you the ability to do that here or the suggestion, depending upon your network here, it gives you this information.

And then finally, uh i i put, we put together another question here that, you know, may, may come up is a little bit more complicated uh in orientation is, is, is, do i have any instances that can access the internet? This is i call a uh trust but verify kind of question. Maybe we have a policy or, you know, everybody has told me that they cannot access the internet in their instances and i don't know about other people, but sometimes there's when you're looking at thousands of instances or lots of instances, maybe there is someone that has kind of violated our policy or what have you and i want to do a check.

So first thing it does is is it's gonna give me a quick response. It sounds like you want to check if any of your instances have uh deployment. And then it gives me some information back here. So it looks like, you know, you have an instance that actually can reach the internet in this question here. It gives you the details about the instance and then it will also, it's doing this analysis specific to your network as well. So the one of the, the things about here is, is the network troubleshooting is specific to your network as well and specific to your setup.

And then it provides, it also will tell you some information about that specific vpc in this case, because you someone uh named it web server, web servers frequently need access to uh the internet. And it, so it gives you an idea of that uh of what may be out there. And now you have the specific information, the ip address the naming and you can kind of go back and talk to the instance uh owner as well to get more information and say, hey, you know, have you configured this, you know, different to our policy or what have you.

So, uh these are the three features uh in the console with a i generative a i learned language models that we're here to discuss today. So one on the ask uh for the instance finder, the other are uh console to code and then here are the amazon q network troubleshooting. And so all three are in preview right now and i'd highly encourage everybody to take a look and try them out uh either in the console or, or using directly using amazon q which is pretty simple to find. It's at aws, it's aws.com.