New AWS generative AI features and tools for developers

You may or may not be familiar with these terms. Some of them have been around a while in the realm of machine learning and artificial intelligence. Some have just been introduced in the last few years. At AWS, we've been working with these technologies for many years, applying them throughout our business to help our customers.

While for many of us, the last two years may feel like an AI revolution, artificial intelligence has actually been around since as far back as 1943, with it being formally recognized in 1956. The advent of large language models in 2018 and foundation models in 2021 are simply the latest evolution in artificial intelligence.

Today, we'll discuss how AWS is leveraging the newest artificial intelligence techniques to make software engineering more powerful and enable more builders to exchange, engage in the software development process.

My name is Emile Lerch and with me today I have my colleague from AWS Adnan, Bewani and Guida Ramachandran from FINRA. So let's get started.

Usually when I have conversations with customers, I don't want to do it in a presentation mode like this. I wanna have an actual conversation this, that's hard to replicate in a venue like this. Um but we, we're gonna try today.

So what I'd like everybody to do if you don't mind is pull out your phones, grab the QR code there or go to the uh URL and put in uh the uh polling question ID. Um that'll give you a question. And what I'm looking for is just a general overall understanding of AI where you're at in your journey so that we can tailor the session a little bit um toward that piece of the audience. I give everybody 45 or 60 seconds to do that.

Alright, we'll close in about 10 seconds.

Alright. Um yeah, this is pretty cool. This is the first time I've done this. So uh yeah, we see a lot intermediate, a few advanced uh we'll hear from gua a little bit later who's pretty far in their journey in exploring these technologies. Um and a number of foundational. And so we, we will cover some foundational concepts uh as well in uh in this presentation.

So when we talk about generative AI, uh what we really want to know in terms of uh the software development process is, is this uh technology valuable for me? How happy are my developers using it? And what is it that um uh can we use this technology right now?

To answer the first question around the genera, the impact of generative AI in software development, McKinsey and Company did a study to try to uh measure that impact and they looked at a number of different types of industries and where the generative AI um can have an impact uh both in a pure dollar amount as well as a percentage of an organizational spend.

And what you can see if you, uh look at the chart, we have two buckets of software engineering way up in the upper right hand corner. So large amount of impact in terms of dollar uh dollar spend and large amount of impact in terms of percentage of overall organizational spend. Uh so in the upper right, we see software engineering for corporate IT, this is internal applications that you might be building to improve efficiency and logistics, your HR or your, your legal processes, software engineering for product development. However, are the applications that you yourself would build for your customers. And so we see a huge amount of impact potential impact of generative AI technologies in both of those areas.

The second question that I mentioned is how happy are developers using this technology? And to uh to answer that question, we also turned to McKinsey who did a study this year um on that particular subject and you can look at this side. But what I'd like to draw your attention to is the the middle part of this side i was able to focus on satisfying and meaningful work. And we see a, a huge uh improvement when you're using generative AI. The reason for that is the generative AI can really take away a lot of the undifferentiated heavy lifting of software development.

If we imagine for a moment, a developer that needs to do something simple like parson integer, that's actually not so simple. They may have been working in five different microservices that day. Each one of them with a different programming language and a different standard library. Each one of which are doing something different to uh to parse that integer. If for instance, the uh the text doesn't parse correctly to an integer. Um maybe a null value is returned or maybe an exception is thrown or maybe there's another mechanism to uh to determine that the standard libraries are gonna name the function calls differently and they may have different parameters. Uh one language may take a radix and another 1 may assume uh that we're looking at the decimal uh in, in a decimal format. So there's a number of different nuances, all of which require memorization of all the different standard libraries that they might be using. Whereas if they're using generative AI, they just say, hey, i intend to parse an integer here and i'm gonna need to handle errors and the generative AI can produce that code leaving them to work on the actual business processes.

So the third question uh that I want to uh address is, is this technology ready today? We see a lot of generative AI in the last 18 months or so. Um and so, you know, is this stuff really ready for, for use? Now, it's still a pretty new technology gartner helped us answer that question and they went out and, and did a study to determine when uh generative AI is useful to us in software engineering. And what you can see is that right now, this is uh something that we can apply um to the core dev secs uh portion of uh the software engineering landscape.

So at this point, I'd like to kind of go back and we had that polling question, there were a number of people that um are just getting started in the journey. So, uh with generative AI and foundational models all being talked about a lot in the last 18 months. Um let's talk a little bit about what that is versus what we've been talking about for several years. In terms of machine learning.

When you look at a traditional machine learning model, you wanna take labeled data, you have a use case in mind. So you take that labeled data and you train it. Um and you'll uh end up producing a machine learning model. When you're happy with the performance of that model, it's really dialed into exactly the use case uh that you're looking for at that point. Uh you'll deploy the uh deploy the model and let the machine learning inference take over in uh your production system.

So everything that I've talked about is specific to a task. So if I want to extract text from images, uh like Amazon Textract does, I would build the machine learning model um based on that particular use case. And then I would have a system that extracts the uh the text from images. But I'm not gonna ask that system to then turn around and become a chatbot or ask questions about uh AWS. I'm going to only use that system uh specifically for the task. It was intended foundational models. On the other hand, um are a single very large model uh that can then be reused for multiple different use cases.

In this case, I would use a single foundational model to generate text or to uh summarize things that have already been written or uh become a chatbot. So that foundational model can adapt to various use cases. If I have uh specific needs that are unique to my organization or my customers use cases, I can fine tune that model by tweaking some parameters in in the foundational model itself or i can um provide more context to the foundational model. Um and let it retrieve data from that context as it's as answering questions. And that's a process that we call retrieval augmented generation or rag.

So using, we can use a combination of both traditional ml models and uh foundational models to be able to accelerate the software engineering journey. And we have a number of different services that some have been around for a while. Others announced just this week to address um software engineering uh within uh using both machine learn, traditional machine learning and uh uh foundational models.

So dev ops guru is a s a service that you can use to um help uh achieve higher availability for your service and production. It looks at the live running of your um of your application and then it will uh provide you with uh availability uh information codeguru profiler um also looks at your live running application um but it uh is focused on optimization. So it'll give you optimization recommendations.

Codeguru security uh will look at your actual source code and then uh tell you when you may have security issues in that source code uh so that you can then go and uh remediate that codeguru reviewer uh uses machine learning to provide uh specific recommendations for your code. Uh as though you have a senior developer sitting next to you um doing a code review.

And then this year we announced Amazon code whisper in general availability around April uh code whisper is your AI based coding assistant. It lives right alongside uh you as you work in your IDE um jet brains and VS code. Um it exists within the aws extension for those products. Um and it will uh allow you to say things like i talked about before where i want to parse an integer today and it will um provide you with a function to be able to do just that works in many different languages.

Um two weeks ago, we announced party rock, uh where you can go to today. Partyrock.aws is the, is the url and sign in with any social media uh log in. But when you go to uh party rock, this is a playground. Um it works for uh works well for prototyping. It works well for experimentation and um as well as education, it's kind of a fun and intuitive UI system to be able to do that. It actually comes from a tool that we built for our own engineers um to be able to learn about creating generative AI based applications.

So with party rock, um you'll describe the application that you want to build that's uh using generative AI and it will then provide you with context. Here's a foundational model to use. Um this is the type of prompt to uh to use and then you can add widgets or modify things, change the foundational model. If you don't want to use um clawed, you'd rather use titan uh foundational models. Uh titan is amazon's uh foundational model. Um claw and cloud v two come from anthropic and are all, all of them are available uh within party rock for your use.

Amazon Q was just released this week. Um and we're gonna see some demos today and I'll remind you again at the end, but there is a session tomorrow morning at 1030 in caesar's forum um to go more in depth in all things. Amazon Q uh by one of our senior principal engineers, Claire Lauri. Uh so amazon q is uh your aws assistant um that can follow you throughout your, your journey.

So within the aws console, uh you can be asking questions around uh what types of uh designs you might want for your application, what type of two instances to use? But then you can also use it within your IDE with code whisper along with code whisper to actually say, hey, i need a new module or i want to refactor this code or translate this code from java 8 to java 17. Amazon Q is available in many forms to be able to do many things and it's designed to be your assistant throughout your aws journey.

So we talked a lot about a lot about different point solutions. But when we think about generative AI and the software development process, uh what we're interested in is focused not just on the task of writing code. Um a lot of times when we about generative AI and software engineering, we imagine a developer sitting there typing out code, but we really want to think holistically about the software development life cycle.

So from idea to pull request to operating that code in production and to working with the users, we want to have generative AI along with us throughout the way to make that a more efficient, enjoyable experience for, for everybody. When gua comes up and talks about uh finra, she'll share finra's specific example of how they thought about generative AI throughout all of these stages in the, in the sdlc, um i mentioned code whisper and some of the things specific to uh software development.

Um but some other use cases might be better filled by using the amazon bedrock or sage maker for a specific use cases using machine learning technology. But the point is that we want to improve the pace of innovation, uh we want to be able to provide more consistency in terms of our engineering resources. Um and we want to make them more efficient. As i mentioned, we, we saw the mckinsey study that showed um uh developers are happier using these technologies uh which will also feed into talent acquisition and retention uh for your organization.

So I'll pause here again. Um and we're gonna do another, another poll.

We've got three total. Uh so, uh the, the second one is really around how you'd like to use generative AI. Is it to build generative AI applications or to use uh generative AI uh within the software development process or perhaps it's both. Alright, maybe we'll give another 10 seconds here. A lot of both. Alright. A lot of both. Um good. So it sounds like Party Rock was on point to talk about and we'll uh we'll definitely talk about Q in terms of um uh you know, developer productivity. Um so we'll, we'll talk a little bit about uh both of those. But uh when we talk about generative AI in, in general and how uh AWS can help. Uh we, I wanted to show kind of the stack that uh we, we think about in terms of uh various AWS services that uh that can help.

Um so clearly, we have uh some of the developer specific tools. Uh but if you are building um applications using uh your generative AI, um you're gonna wanna be familiar with um all, all the stages of the, of the stack. So at the very bottom, obviously, AWS provides a lot of base infrastructure um building blocks for you. So AWS Trainium specific CPU that is designed for training machine learning models and AWS Inference. So once those models are in production, being able to provide efficient um recommendations based on the, the machine learning model, um you need to have uh base compute. So uh EC2 with GPU instances, if you were in the keynote earlier uh earlier this week where we talked about the partnership with Nvidia, you'll know that we're bringing the latest Nvidia GPUs into uh into EC2 uh working side by side with Nvidia. Um and then uh we also have uh you need to have your data someplace, right? So we have um all kinds of um you know, data services available S3 DynamoDB, DocumentDB and Neptune was talked about during Swami's keynote. Um so, a number of different um ways to tailor uh storage directly for uh uh AI and ML needs ideally though we'd be looking up the stack and we wouldn't have to worry as much about the infrastructure because we can worry more about our business problem. And so we have a number of services that are developed around machine learning technology for specific use cases. So I've got a number of them here listed. I'm not going to go through all of them, but we did talk about Textract. We're going to come back to Textract a little bit in one of the demos that we have. But we have other things like Rekognition for objects and images and video. Amazon Comprehend and Comprehend Medical, Translate those types of things. So if they match your use case, absolutely go use them if they don't match your use case. And you want a specific machine learning model for that use case. That's when you turn to uh SageMaker and uh SageMaker has a number of different services within the SageMaker umbrella uh that can help you uh quickly create and maintain uh machine learning models. And then the newest on the block is uh Amazon Bedrock that will allow you that focuses on the use of foundational models and provides you access um to foundational models in a consistent way so that you can build your generative AI applications. And then we get into the developer tools and many of these I already covered so I won't cover them again. But Amazon CodeGuru, the tools in SDK is to allow you to operate or work with AWS services. Amazon CodeCatalyst, which we'll see a demo of in a few minutes, which is our unified development service that has issues, source code management workflows to be able to deploy applications. Um and then Code Whisperer, Party Rock and Q that I've covered.

Um but I've been talking a lot and I'm, I'm sure a lot of people are excited to see some demos. Um so at this point, I'd like to bring Adnan Bilwani up um to show you some of these things in action, Anna.

Awesome. Hey, thank you Emil. So as I was speaking about Amazon Q, uh the interactive assistant that we have, um let's look at a few examples of how Q can really help us as we walk through the SCLC itself. We'll start with a console experience, right? So whether you're dating on a new application or um you're just exploring new services in your sandbox accounts or, you know, uh just trying to figure out how to add new features into your existing applications, right? Q is your interactive assistant um in the console and to act as your guide. So let's take a look real quick.

So I'm gonna go ahead and talk about, let's see if we can go ahead and add and you know, talk about our idea on adding a new application here, right? So let's start by asking you, hey, I'd love to build a new application. Um I want to build an application where users can basically um upload pdf documents um to extract all the data and make it searchable, right? I'd love to do that. Um I'd also want to keep my costs low as possible. I'd be able to but at the same time, make sure that, you know, i can control the traffic spikes as well, right? And I expect to have like thousands of requests per second at its peak last but not least, hey, can I do a vendor in architecture as well?

So we feed all the data to the assistant that's available in the console. It takes a year to figure out what to do and it comes back and says, hey, you know, you can introduce me basically to Amazon Textract and says, hey, you could use that to extract all your metadata um from the pdf documents. It scales very well. It takes out the documents once you're done extracting it, you can store the data in Amazon DynamoDB. And then hey, you want to have users upload that data. Let's create an API using Amazon API Gateway. You can use Lambda on the back end and basically call Textract from there. Sounds like a good idea. What else does it give me? Well, it gives me some additional details as well. It says, hey, if you're going to store those pdf documents, you can potentially do it with a on S3 and once your app is up, right, you can actually use CloudWatch and monitor stuff. Good start. Let me ask you some more information, right?

Great. I'm downloading and extracting pdf data. What if I wanted to do more than that? Right? What if i want to process document types like word documents, for example, so I can continue having this conversation with Q and give it that input and it comes back and says, hey, you know what you can use the same architecture you have. I'm gonna give you the list of formats that's available with Textract, give you some details. There gives me some additional details around word documents as well. Um and then it tells me I can extract what kind of data I can extract and then some size limit restrictions as well that I should be aware of as I'm dating on creating that new application as well.

So, all in, all right, the idea is to a great assistant to have to kind of idea with in the console as you go through now. That's great. But you've got the great console experience, but it doesn't stop there, right? Um most of us developers are in the IDE, not in the console and that's where we spend most of our time. But how does it work in the console itself? Well, as part of the IDE, you see Q as part of the experience that as well where you can ask it similar questions that I did in the console, but not only that you can actually ask it to explain code to you as well. Um and provide details.

So let's take an example. Um I'm on boarding onto a new team. I've got a, you know, they're, they're working on this digital wallet application that i started uh looking at and a digital wallet is basically meaning it's called wallets, stores, different currencies. I can move them back and forth. Great. I'm on boarding onto the team. Um and they're signing, getting assigned issues, right? So I'm in my IDE. How do I sort of, you know, make or help Q or get assistance from Q to be more productive?

So I can go in, I'm opening up my project. I'm gonna go, hey, let me go ahead and select my code that I've got. Now, I could go into the next cube and ask my buddy what's going on. But I could also ask you to explain that selected code to me. I'll go ahead and, you know, take a minute and then come back and say, hey, look, I get it. It looks like it's a python file. You've got um a class in here that looks like it basically is responsible for doing DynamoDB credit operations at the moment in time. Um and then it actually goes ahead and point by point, explains me what all the functions are. So it kind of talks about the constructor, the initialize, um then talks about the get item API right? And then the list transactions, wallet functions as well, talks about basically how it takes a wallet id, right, or a transaction id and kind of queries the DynamoDB and gets the data going.

So as you can see, right, it's Amazon Q is really helping me accelerate my individual developer productivity and the coin and hey, right now and explain code. Uh what's coming very soon is it actually going to be providing code for you as well? And the IDE e and helping you sort of make that happen as well? So, really excited to look for that.

Ok. So wonderful. We've idea it in a console. We've gone ahead and leveraged it in the IDE. But what about our pipeline, right? Um what can we do about development capabilities is being offered within the pipeline itself? And that's where Amazon Q comes into play within Amazon CodeCatalyst, our unified development experience as well.

So with Amazon Q developers can pretty much assign an issue in CodeCatalyst, you can provide the details of what you would like it to do. It takes a look at your repository, it figures out sort of what your repository is made of. It goes ahead and then figures out an approach in order to fulfill your request of what it does, convert it to code and then get it ready for a pull request itself.

So in this case, for example, let's use that digital wallet application that the team has been working on. Um and I wanna add a new um function that will implement, you know, getting new transactions by wallet, right? So that's what I'm asking you to do.

So basically, i'm in the CodeCatalyst um uh platform. I go into issues. I go ahead and click on create issues. I'm gonna start by creating a title and say, hey, you know what, I'd love to implement a feature to list all transactions for a given wallet. I'm gonna go ahead and give it some details. I'm gonna say, hey, I'm gonna give you a wallet id, go ahead and list me all the transactions associated with that um particular um wallet. I'm also wanting to give you, give me some unit tests as we go through and then let me give you some more details, right? I actually really want this as a lamb to function um which basically is taking data from an API Gateway event. Um it's according to DynamoDB. And then at that point in time, can you go ahead and please return me a json for all the data that you've received as well?

So we go into that, we assign this to Q and go ahead and create the issue as we go forward. And by the way, the reason i recorded this is because it takes about 2025 minutes for this to go through. So i know you don't want to sit here for that. So, but it goes ahead and uh the first thing it does is it looks at the code, right? Provides me a background of what it saw in the code. So it says, hey, i get it, python 80 k stack. You've got lambda functions basically for creating and updating all the wallets and processing the draws

I see you've got some model space in there and it looks like you're using DynamoDB to do all that. Once it's done with that, it then provides me an approach of what it's going to do or how it's going to actually do what I'm asking it to do. So it says I'm going to create a new Lambda function. I'm gonna import the proper files and I'm gonna go ahead and create the file that you need to basically do the transactions and also gonna create a file for the unit testing as well, right?

So go ahead and give me the detail if I agree with that. Um I'm gonna say, ok, go ahead and do that. I'll go ahead 20 minutes later, magic abracadabra. It basically comes up with a pull request that I can then review and see what it's done. So first of all, it comes up with a file, it has actually set up that Lambda handler for me. Um go go ahead and put the details in and in this, in addition to that, it's also done that unit test file I had requested as well.

So think about it right from a perspective of where it's at. If it's gotten you there, let's say not even 100% you can go ahead and make the changes you need in the pull request, right? Um m make sure you can update it to what you need it to be. But even if it gets you there, 60% or 70% of the way, right? It's done a lot of that undifferentiated heavy lifting that you're looking to help, get help with as well. Awesome.

All right. So I hope the demo is very exciting. Um and give you an idea of some of the possibilities that Amazon Q can do for you as you think about expanding or accelerating your developer productivity as well.

Um I leave you with one of the exciting features that I actually think is kind of really exciting, but I'll be honest with all the announcements this week, I didn't get a chance to really play with this a whole lot. Um but I am looking forward to doing that post re invent. It is the Amazon Q code transformation capability. So starting with Java, this is available within your code whisper experience and in your ID and you can basically go ahead and upgrade your old um Java code, let's say Java seven or eight to Java 17 with just a few clicks utilizing Amazon Q. We've also got.NET, by the way, coming very soon. So you can actually take your.NET framework and convert it to cross platform.NET as well. So you can actually port your code from windows to linux um with a lot more ease as well. So really looking forward to this and playing with this actually next week for myself as well.

Ok. So as gen a I changes and how we think about software development life cycles, right? It'd be really interesting to sort of know where you are with your gen a I journey. You know, we kind of talked about it a lot would be really cool to sort of understand where you guys stand. Um and uh or at least to have everybody kind of understand where everyone sort of stands, right? And what the deal is. So would love for you guys to take a few seconds to see if you guys can fill out that poll as well. I like this life bowling thing by the way. It's kind of cool. All right, we'll give you guys say 10 more seconds. Awesome.

So, I think that's almost 50% of, you know, hey, we're actively researching genny. I, we aren't sure about operational it. exactly yet is sort of a bucket that I think i fit, fit, feel that most people fit in and, and I think that makes sense. right. So, I think I'm hoping this reinvents and some of the stuff that we're showing you hopefully helps you sort of get to the point of where to operationalize it. And I know I'm able to talk in the end about sort of some of the things that you should look to when you are actually going through that process as well.

Awesome. Well, you know, we've talked about sort of a lot about what AWS has done and what AWS s offerings are, what we think things are going and, and how generative a I and that developer productivity is being affected as well. But it would be awesome to basically hear this from an actual real customer as to how they are actually exploring this. That's why I'm really, really glad to have Gita from Finner here um on the stage to talk about how Finner is exploring genitive a I in software engineering.

Thank you. Thank you. Welcome everyone and good afternoon. I'm very happy to be here with you all in this breakout session in today's presentation. I'm gonna be sharing how Finra is approaching, advancing developer productivity or r and d explorations of leveraging generative a i technology within software development life cycle and our evaluations of AWS services such as code whisper,

I'm Geeta Ramachandran, Senior Director at Finra Technology leading application engineering and data science platforms. Like to give a little intro about Fin a Fin a stands for Financial Industry Regulatory Authority. We regulate the US, brokerage industry and the stock markets. Our mission is to protect investors and promote market integrity. We detect the wrongdoings in the US markets, bring disciplinary actions against them for their unethical behavior and deter any misconduct actions by enforcing rules.

Let's see, Finra in numbers, Finra is named as the best mid-sized companies to work in it for the 10th time. As part of our regulatory operations, we analyze and oversee activities of 620,000 broker dealers. At 3200 securities firms. We collect and surveil almost 99% of equities and options, market transactions. We collect just process and analyze this volume of data at scale to detect insider trading fraud and market manipulation on a day to day basis. It's a lot of data to process. So Finra processes up to 662 billion market events per day. Let me repeat. That's a per day number. That's the volume and scale we operate with.

As we continue our presentation, I'd like to walk you through our journey of software. Development life cycle at Finra a journey that is marked by continuous improvement and innovation. Our journey began in 2007 with significant investments in automation such as integrated version control test, automation, embracing c id platforms. Setting the stage for a more responsive development environment.

Fast forward to 2013, we embraced on a significant shift which is starting to move to cloud. We implemented infrastructure as code built our c id strategies and started defining what develops meant for us which was pivotal in our journey towards operational excellence.

In 2016, we witnessed a full scale des transformation. We build self service tools for on boarding and provisioning, established des experts for advocacy and best practices. This transformation was about empowering teams, streamlining processes and fostering a more collaborative environment.

In 2019, our journey took us deeper into more cloud native architectures such as containers and serverless. We also trained our technology teams to be cloud and dev ops aware.

In 2021 we entered the realm of advanced analytics, data science became an enabler for our enterprise. We built tools and platforms for advanced analytics using mission learning solutions that we that we saw earlier. We defined policy and procedures for governing a i and machine learning solutions. Thus beginning our uh sdlc journey to a more dynamic stlc and md lc for models that is insights l and data driven.

Now at 2023 we are standing at the forefront of another technological frontier which is generative a i. We are currently exploring this technology to leverage within product development operations and delivery of applications. Building on our journey of embracing generative a i and new technologies.

I'd like to talk about the goals of our project which is generative a i in sdlc. This is a research and development project within Finra's r and d initiative. Our first goal is to investigate how large language models treat software code as input and a software code base as its context. We have all seen how these models work with a natural language form of input from a chat interface. So this exploration is for us to understand how these models interpret, generate and optimized code.

Our second goal is to experiment with these large language models to fine tune and align them with our domain specific code and artifacts and integration within delivery tooling by tailoring these models to our domain specific needs. We can enhance their effectiveness within our specific development environment.

The third goal is to identify opportunities and use cases within the software development process where our teams can focus more on creative and complex aspects of software development and reduce the repetitive tasks, the time spent on those tasks.

As we innovate, we remain vigilant about the risk controls such as data privacy, legal standards and security compliance and identify new risks that may emerge from the use of this technology and potential mitigating controls that we should consider.

Our final goal is to assess the state of current state of the industry in software development tooling that leverages generative a i by doing so, this will help us understand where the industry is heading and also benchmark our experiments against industry practices and standards.

Having established the goals of our project. Let's look at what are the anticipated outcomes and the value we expect to derive first is enhanced productivity with faster development cycles and an improved overall developer experience. This means providing our teams with tools that will make their work more efficient but particularly more enjoyable leading to higher productivity and innovation.

Second is improved software quality and reliability. Our hypothesis is that this technology can help teams to build fundamentally build the solutions and systems that are fundamentally more reliable and stable. This will translate to lower maintenance costs and increased customer satisfaction and finally, with a highly productive the and a good quality product, one can react and respond to changes more quickly and capitalize on emerging opportunities.

So let's look at what are some of the generative a i use cases within the software development life cycle where this technology can be applied catering to the specific user personas across these stages. And these are the use cases that we identified as part of our r and d project as the use cases for us to evaluate.

First, this can help in generating user stories and analyzing functional specifications. And product review documents for consistency and comprehensiveness, aiding a complete support for product managers.

Next. During design, these models can help in generating design and architecture artifacts, analyzing the artifacts for coherence in the design and enabling rapid prototyping.

Then in design help in generating context aware optimized code and generating boilerplate code and unit test code.

During code review, these models can analyze the code for issues and ensure compliance for coding standards that are set at the organization level. This step can be particularly helpful for architects, developers, testers and security engineers to help them maintain a high quality, secure optimized code throughout the development process.

This can be a useful tool for q a engineers where it can help in generating integration test code, test plans and test data. And this can streamline their testing processes and ensure enough thorough test coverage in operations

This can assist in analyzing errors in production and even generating business requirements by analyzing operational events that occur in production, which will be a useful valuable add on to DevOps and SRE professionals for proactive system monitoring and continuous improvement.

So that is in a nutshell all the use cases that we have identified as part of this project to evaluate.

Now, given these use cases across different stages of the software development life cycle, let's look at what is the intersection of AWS services that can meet these use cases and also the user personas who can gain, you know, efficiency by leveraging these services.

These are the AWS services that we identified to leverage across these SDLC stages that we hope to evaluate as part of our R&D project:

• Amazon CodeWhisperer to analyze functional specifications. The models in CodeWhisperer are very suitable to analyze documents such as PRDs and user stories and functional specifications for consistency and comprehensiveness. Also use LLM agents that is available in CodeWhisperer to automate commonly performed tasks using a domain specific knowledge base.

• CodeGuru can be used across design, development, code review and testing stages.

• DevOps Guru for analyzing operational events and errors in production.

Having said that, as part of our project, we are currently evaluating CodeWhisperer for development and testing use cases. So I'd like to focus more on and speak to what our evaluations progress is.

As you know, CodeWhisperer provides foundation models like what Adnan shared. These foundation models can be used by users as a coding companion.

Now, as we are thinking about leveraging these large language models for our coding process, we have to be considering and mindful of what controls these models have from a responsible AI perspective. So let's talk some about those.

The foundation models are trained on open source datasets, open source repos that have acceptable and permissible open source licenses, which is very important from a license compliance standpoint.

And these models provide reference tracking when they generate piece of code that looks exactly the same as what is available in an open source repository for visibility and tracking so that the teams can add attributions or eliminate those suggestions based on their organization's policy.

Secondly, these models have built in bias avoidance so that they do not generate inherent biased coding structures.

And finally, the foundation models are trained on repositories that are scanned for security vulnerabilities.

Now, in addition to that, the developers can also run security scans right in their IDE using CodeWhisperer. So as they get generations and suggestions from the CodeWhisperer service, and if they are augmenting that with their additional code, they have the ability to run security scans right before they commit.

So this is an added security layer apart from other security controls that we would typically have in a DevOps pipeline such as static analysis, dynamic analysis and software composition analysis.

So we've talked enough about the foundation models. Let's talk about the CodeWhisperer customization workflow.

As I said, one of our project's goals was to fine tune a foundation model to our domain specific code and artifacts and customize it to improve the effectiveness of those models.

CodeWhisperer's customization capability is available in preview and we participated in that preview program late summer this year. So I'd like to talk about what that approach looked like and how we went about designing and architecting the customization workflow.

This is a two part process. So the first step is to create customization and this is usually performed by a CodeWhisperer administrator called CodeWhisperer admin role.

So this is an IAM role that has specific permissions in an AWS account to create customizations for CodeWhisperer.

This CodeWhisperer admin provides a training set or a set of packages for code repos from an internal package store that the team would identify that they would like to customize on and provide that to a KMS encrypted S3 bucket.

So this bucket is within FINRA's AWS account that is encrypted using our customer managed KMS key and kicks off the CodeWhisperer customization process.

This customization process will run and the service will assign an eval score on the customized model based on the evaluation score. The CodeWhisperer admin can then assign users and groups to have access to this model to further evaluate.

So the second step is when a developer or tester tries to connect to this customized model using CodeWhisperer using their development environment, usually a local IDE using AWS Toolkit plugin.

The first step here is an authorization step. The service first authorizes the users via AWS Identity Center which is running in a FINRA's Identity Center account.

The goal here is to make sure that the users have the right set of entitlements and approved access to access the customized model.

Now, once the authorization is complete, the developers can access the customized model and continue with their development process.

Now note here that in this setup, the Identity Center is a key authorization piece that needs to be set up in our FINRA's AWS organization first and managed by an IDC administrator.

Some of the security controls that we were able to achieve with this design and approach is data encryption. The data that we used to customize the model is fully encrypted with an S3 bucket and the access to these models are authorized via our own federated identity provider, which the AWS Identity Center communicates with.

And most importantly, both the data and model did not leave FINRA's AWS environment and both of them were within FINRA's VPCs.

We are also opted out of data sharing both on the service side and on the client side. What does this mean is the data that we use to train or customize the foundation model does not get used by the CodeWhisperer service to train the foundation models, right? So this is used only within our own customization process.

So we have seen enough about our CodeWhisperer customization workflow. So let's see a quick demo of CodeWhisperer in action.

We are going to see three use cases:

• Code completion using AWS SDK
• Code completion using a domain specific SDK
• Code completion for unit tests

Now, for the purposes of this demo, I have used FINRA's open source repositories. You are going to see terms like Gatekeeper and Fidelis throughout the demo and you can check them out in FINRA OS GitHub project page.

Ok, let's get started.

So we'll see the first use case in action - CodeWhisperer using AWS SDK. I have got Gatekeeper in my IDE and I'm connected to CodeWhisperer through Identity Center and I have a customized model here.

So I'm for the purpose of this use case, I'm going to open a class called RDSLookupService. And this class has methods to interact with RDS service. And the two methods that we see here to interact with Aurora RDS cluster, global cluster already.

So I'm starting to write a comment that says method that takes Aurora RDS cluster, global cluster as input and returns its reader cluster as output. And when I hit enter CodeWhisperer should be able to provide suggestions.

Ok, it has suggested a method signature that seems to be right with inputs and output. And it has also generated the logic for the code which seems to be right. So we'll go ahead and accept that and proceed with testing further.

The second use case is code completion for domain specific SDK. So this is using a FINRA specific credential management SDK called Fidelis.

So we are asking to write a method to return credential from Fidelis by passing a set of inputs.

Alright, so it has generated a getCredential method and has a suggestion to use FidelisClient. But the second suggestion is even better, which is more having robust error handling. So we'll go ahead and accept that.

So that's the demo for the second use case.

Now, the third use case is code completion for unit tests. So we have a class here and a method called putCredential. So we'll try to write a unit test for this putCredential method.

And I'm opening another class file which is a test class file, starting to write my test annotation and will start writing the unit test method here which is putCredentialWithoutComponent ok?

And once I finish writing my method signature CodeWhisperer has generated the code for unit testing that method, it has set the properties did not set the component and it has three suggestions.

Let's look at the second suggestion which is setting a blank component. And the third suggestion is setting a null component. So all of them are valid use cases, but we'll accept the first test case because that was our intent and we'll go ahead and run the unit test, unit test passed.

So in this test case, we have seen how CodeWhisperer is able to understand its context where the main method that we were trying to unit test on was on a different file and we were trying to write the unit test on a test class file and it was able to understand the properties that the method took and also marked the service and written the assert statements.

What is more interesting is that it was able to identify the edge cases that we typically would have tested for, right, not setting the property, setting a blank property and setting a null property. So one can easily generate these type of test cases with an assistant like CodeWhisperer.

Now, in terms of next steps for our project, we would like to complete the tool evaluations for the range of models and tools including the AWS services along with open source models and vendor tools. Our goal is to assess effectiveness, integration capabilities and the potential impact on developer experience and productivity.

Following the evaluations will propose a specific set of next steps for usage and impact measurement.

So in conclusion, today, we have journeyed through our evolving landscape of software engineering from the strides we have made in SDL to the latest explorations of R&D.

I hope this was insightful for you and your teams and thank you for your time today.