The pragmatic serverless Python developer

Hello, everyone. Thanks for coming. I don’ know if you are an early bird or the jet lag, like, like me forced me to come down here early. You know. So thanks for coming. It means a lot to us just in case you're in the right place. This is the open source session, uh the pragmatic service python developer. So either if you either you like open source or python the best language in the world or something else. So thanks for coming again.

I'm meola, I'm a chief architect for power to sway ws uh tool that we're gonna see later. And with me, i have my friend run from cyber arc which he presents uh soon enough.

Let me walk you through on what we've got. So this is the agenda we have today. We want you to walk you through and explain to you what pragmatism actually is in the first place and then to apply this pragmatism, this idea of making decisions, we, we thought i use case would be better. So we apply this into multiple cases and we, in this use case, we're gonna use a synchronous part which is a product api and then we're gonna use asynchronous part to show you how the initial ideas that look good will look terrible if you're doing async fashion and how we have to change.

And then lastly, we look into some of the tools you could use for production as you go to production with this or something similar. And the ones that we're gonna show you are handling roughly 50 billion transactions a week. So it should be production ready for most of you. And then we wrap up when it comes to pragmatism, what we're trying to mean in this entire session is decisions that you're about to make. Whether you are in a project alone or in a team, the skills that you have or the tools you're gonna use all of this matter. Someone could tell you that these are the best practices and this is the only way to do things, but they not might make sense to you in the team you have or the setting that you have. So taking content into account is important. So that's what we're going to use this in this session here.

When you start on your first project, you might be thinking should i use s am as my infrastructure is called framework. Should i use cd k? Should i use amplify or maybe bash scripts? They could also work depends on what you're trying to work on the context you have and should i use a single function for everything or, or multiple minor micro functions in there. Should i use domain driven design or something else entirely? Maybe i don't need any of these things in the first place as you answer these questions, other questions will pop up. Should i do synchronous api should i do asynchronous from the get go. Should i use open source tools? And so which should i use? How do i even test for something that's asynchronous? That's something that's gonna go across the border of boundary of my service. So these questions are common and typically you get into this analysis paralysis that you don't really know what decisions you gotta make.

And this session is about coming some of this for you and trying to make a framework so you can make those decisions easier. We're gonna cover the open source tools and some of the leading practices and what we typically learn from open source that we can apply in our projects for this session. There's so many open source tools that will be hard to cover all of them. So we're gonna be focused on this for if you never heard of them, make do material is the engine that drives the documentation that makes writing documentation fun. Actually, it's something we use for power tools for the last nearly four years. And p test is to run your task that we write units integrations and show, which will show you identic is what makes it easier for us to do data validation and defining our models and what our data should look like and so forth and power to ve brass lambda. I hope you're using it is the framework that makes all these best practices easier and improves your developer experience.

But i would be remiss if i didn't tell you that there's so much more that i wish we could cover in the session. These are just some of what we're using it. And at the end of the session, we're gonna give you uh the project itself and a treat at the end of it as well. If you stay long enough, hopefully you will.

What can we learn from open source though? All of those open source tools typically have something similar to this. When you're trying to deploy something uh on github or bit bucket, you name it in there. You typically have all these phases. Someone has to set up your project for the first time. Imagine you're on boarding someone for your project, they have to know how to check out the code virtual environments. How do i find dependencies and so forth and then eventually in those pre commit hooks, which is a fantastic open source framework called pre commit that saves you from a lot of trouble. Imagine open source, you have people from all sorts of skills trying to make contributions. So you've got to have some baseline to allow them to make those contributions more easily.

So pre pre commit checks can easily check from anything from security, potential leaking credentials to incorrect files or basically not even meeting governance for instance. And then when it comes to pre, before you make a request, there's even more checks in there and they have documentation linking most of us eventually end up pushing some documentation that's broken. The links don't work or something that doesn't really fit into the message that we're trying to say. And then lastly, we have the pull request checks. We have governance which we do in this project and the power tools as well. And you have security checks that you might want to run on your dependencies or whatnot, no matter which project you see that's popular and open source. They all have something at least like this in power tools. We go just over 20,000 c runs a week to give an idea of how important these things are. Every time someone makes a commit or code or open a pull request, we run through this, go into thousands of this a week.

But when it comes to prag pragmatism for us, this is something you can copy and paste. We work for everyone. But this will not if you have to test something. How do you run your test? Whether you run the test only for files that have changed or you run unit test, integration tests and so forth. These are different. So you gotta have some application to be able to run these things. Otherwise there's nothing really for us to talk.

So this session, we're gonna go through this in more detail, how you can run these things. So let's get to the use case first. We wanted something that isn't a hello word, but isn't something too complex that you wouldn't be able to understand in one hour or everything about it in one hour. So imagine you have a product api that you add products, delete products and so forth. You gotta use a database and then eventually whatever you add it, you emit an event.

So if you're using brass terms, we're gonna have api gateway with a bunch of functions that handle all these operations, create a delete and so forth the database. We're gonna be using dynam with the b as our nosql managed solution and asynchronous part. We're going to be using a mix of batch processing and event bridge to emit these events. The idea for this is that this part on the left is what most people typically go through. You have a synchronous flow, you send a request, you wait for a response. Good to go. The asynchronous part is a bit difficult when it thinks about testing. If you make an api call to the first api gateway and wait for the response, you test, it looks good. But how do you test? It goes all the way through now that the event was ted and everything else. It's a bit different.

But let me allow you to meet, run to go to the first part and i'll come back into the streaming.

Thank you, hater. Hi, everybody. My name is ran isenberg. I'm a principal software architect at cyber arc at the platform engineering group at ed service hero and the owner and the maintainer of the website ronda builder dot cloud where i talk about everything at us and service in general.

Right. Right. So we're gonna focus on the products api the synchronous part of the service, the api gateway, the lambda functions and the time of the bt, we're gonna dissect it into several practices. The project practice, we're gonna talk about project structure infra code, the lambda handler. How do we write the lambda handler? How do you write the business domain code? How do you write the integration code, the code, it interacts with any of those resources and how do we test it all the unit test, the integration tests and end to end tests.

So first of all the way i see it, a server service is a synergy between four elements, the infrastructures code, the domain code, the tests and the production readiness elements. Now, the infrastructures code is basically the resources that we deploy to the cloud or the api gateway to the table and the other functions. Now the rama functions they hold and contain the business domain code, the business domain code on the other hand, it interacts with the resources that we deploy such as the internal b table in the test section in the test elements, we're gonna test basically both the infra the infrastructure code, the api gateway and our business domain code and the production readiness elements. Well, they're pretty much spread across all the other elements in the in infrastructure as code. The production elements are manifested as configuration for scale security backups, things like that in the domain code, it is manifested as input validation, authentication, checks, authorization checks. And in the testing, you might probably have test, testing for security.

So as you can see everything is basically connected together. Now, i've been advocating for dev ops culture for a very long time now. And the way i see it, developers, they need to own these um elements from the development stages, all the way to production. It makes them basically more dependent and it enables them to deploy much faster to production.

Now, in order for all these um synergy to occur, you need all the elements to reside in the same project structure. So that's where we what we're gonna talk in a second. We're gonna present you an opinionated version of the project structure.

Ok? So first of all, we're gonna have a product folder. That's the uh basically the pro the folder that's gonna hold our business domain code product is the name of our service. We're gonna have a sub uh sub folder crowd for the domain or crowd api. And then we're gonna have the handlers for the business domain code. And schemas schemas is very important. It's the, basically the pedantic models that we're gonna use for pic is an open source utility that allows you to do input validation and parsing. So we're gonna basically uh contain all the models there.

We're gonna have a testing folder. So end to end integration unit. And like i said before, all the production radiance elements are gonna be spread across the other folders and we're gonna have a pi project thermo file. Now we chose poetry um over pip because we think it's, it's uh much faster and we're gonna basically manage all the dependencies, both the infrastructure is code and the business domain code in one file.

Now, i've seen people use multiple files to hold the dependencies, usually multiple files per lambda handler. And while that is possible, i think it makes things more complicated as we're gonna see later on in the testing. We want to have um a local environment to set up it much easier. So having one file is just simple. So keep it simple, just one file.

And lastly, we're gonna have a folder for the infrastructure as code. So you can choose either cd k, sam turf and whatever you want to choose. But we chose cd ki, i love cd k. Um i have loved it for a long time and we're gonna use one cdc application up dot py one stack service stack dot py. And we're gonna have 11 construct a domain driven approach to building our cd k uh construct.

So one construct, the crowd api construct is gonna build for us, the api gateway, the lambda functions and all the, the aly table and all the resources that are required to make everything work together. So in order to understand that, let's talk about writing our first lambda function.

Let's say that we want to write the create product lambda function. We're going to implement the http put slash product interface. It's gonna save the product into the demo b table. And now comes the question, what is the best way to write a lambda function? Is there a best way to do that?

So back in the day when i started to write several services, what i did is basically write everything in the handler itself. So it looks something like this. Just one file uh handler create product. And first of all, you gonna do some input validation. So i'm using power tools, the amazing power tools library. I'm gonna use the parcel utility to do input validation in conjunction with pic.

So i'm going to define the create product request model. I'm going to get the input, then i'm gonna use b to basically put the item, the product that i want to insert from the input to the table. And then i'm going to return the response back to the color and sterilize it and this is how the code looks all together.

And as you can see, it's very simple, right? From the beginning to end, very simple. But is it the best way though? Right. So as with everything in life context is what matters? So if this were a chrome job, a nightly batch job, sure, that's perfectly fine. It's not gonna change that often

You can write it like that, it's perfectly fine. But we're running a service or STP service with logic and it's going to evolve over time. And as you can see, our service is now evolved and now we're required to write three more lambda functions like get products, get product and delete product.

Now, let's say that we're gonna write it in the same way, right? We're gonna write everything in the handlers. So now we have four models, four separate models, they're not showing any code, but now we have new problems. So we have code replication. Again, it's not showing any code for different models. Testing becomes much harder because we're doing basically the same four times. And in some cases, we might even have code readability issues because some handles are maybe more than 300 lines. For me, that's a magic number that tells me one, if you have more than 300 lines, you should probably refactor it and make some changes.

So as you can see this does not scale. So can we do better? Yes, we can introducing architectural layers, which is a fancy term that architects love. But apparently it's gonna make sense in a CR so we're gonna take our, our handler, we're gonna split it into two. We're gonna have the handler layer and we're gonna have the domain layer. The handler is gonna be basically zero domain code and it's gonna have very clear responsibilities. The domain error is gonna be where we have all the business domain logic and the other three calls and it's gonna be shared across multiple handlers. And we're gonna have isolated tests which also solves a problem that we saw before.

So the handler, what is gonna be, what, what is it gonna do? It's gonna parse configuration, either dynamic or static. It's gonna parse and validate environment variables. It's gonna do input validation, authentication checks, authorization. It's gonna pass the input to the domain layer. It's gonna get back a response and it's gonna serialize it back to the caller. And this is from a high level overview.

You have the um API gateway in invokes our lambda function. The handler does this thing passes the input to the domain, layer the domain layer accesses dynamoDB table returns with the output and the handler serialize it back to the caller.

So let's write some code. So our handler is not gonna change that much it, but still we're gonna do the inter validation as before. But now, instead of doing the boto3 call, we're gonna call a function in the domain there, it's called create product. We're gonna pass it a identic model of the product that you want to basically save in the database. We're gonna get back a response and we're gonna serialize it as before, back to the caller. And this is all the code together as you can see. Um it's very short, very clear zero domain code now as for the domain layer.

So now we need to implement a new function, the create uh the great product function. So i, i chose a functional way to do that, but you can also do it with object oriented and implement it with interfaces and classes for this. i i don't think it really makes a difference. So it's a matter of style.

So uh we're gonna, we're gonna build the product entry, which again is a pedantic model that uh simulates uh represents what we have in the database itself. And then you're gonna see the same photo three code as before. We're gonna save it into the database and we're gonna return the response uh back to the handler layer.

Now, we're gonna need to have more function like this, right? We need to have the product list, product and delete product interface functions. And as you can see, this is the only place that it's gonna basically access the the dino db table right now, let's talk about integration practice.

What is inte integration practice? What does it mean? So in this uh in our example, integration is basically any code that accesses dynamoDB table. And in more general terms, it's basically any code that interacts with any of those resources or calls an external API.

Now, if you recall from just a second ago, uh this is our dome there and it has basically integration code, right? It has about three code into it uh written in, in, into it. So is it, is it ok though, right? Can it backfire?

So as before, it really depends. So i'm gonna show you this case that happened to my team where it did uh where it did make a difference. So over the time, um we had, we started with da the bt, we had a schema, everything is working fine. Over time, we had new requirements for the product team to add the new, more and more advanced queries. And our schema just didn't match and we ran queries and it just took too long and uh the queries are too, too slow. At some point, we realized that the solution would be to basically replace the dod b table. We have an amazon aurora database.

Now, since the uh um integration code was pretty much hard coded into the domain there, it was really hard to factor it out and switch to overall database and we didn't have specific testing for that database section.

So as before, can we do better? Yes, we can introducing again another architectural layer surprise. So we're go, we're, we're still gonna have the handler layer remain basically the same. We're gonna have the domain layer, but now it's gonna have zero integration code and we're going to have a new integration there. It's gonna basically implement an adapter pattern. It's gonna adapt whatever logic we have into a complete implementation of the database. And this is the only place where you're gonna have any apr code or and data database code.

Ok. So again, in high level overview, you have a API gateway invokes the handler and does this thing passes the input to the domain layer domain layer calls the interface function of the integration layer. The the integration layer access to the down to b table returns with the output, the domain layer. The, the domain lay converts the output to whatever the output the heller requires and the heller cs it back to the color.

So let's write it, let's write the interface we're gonna use, we're gonna create the db handler interface and we're gonna have a function for all, all their needs. So create products, lea products, delete product and get product and now we're going to write an implementation.

So now let's write the dino db handler implementation. So it's gonna implement, it's gonna inherit from the db handler class and we're gonna move our beta three code into the create product function. So as you can see, this is basically the only class that's gonna access our demo db table and now we can have isolated testing, which is something that we really needed.

Now, let's say that we want to switch um and write the new aurora handler instead. So all we need to do is basically write a new class inherit from the db handler and implement all the functions. And then we need to make the switch in the domer, right? The domer needs to call the new aurora handler instead of the dino db handler.

So let's see how we can do that. So we're gonna refractor our domain layer. So as you can see, there's no more be three code here. Instead you have a g db handler function, a getter and we're gonna get a concrete implementation the db handler and we're gonna call the create product function. Now, the domain doesn't really care what implementation it is and it doesn't even know all it cares about is that it has the great product function.

Now, if we were to make a switch, all, all we need to do now is basically change implementation of g db handler from download b to the oral handler. That's it very simple now that we understand how to uh build the st the project structure, right? The handlers on all the layers, let's talk about testing.

So as you know, cus testing is not easy. But for me, what guides me is a very specific developer experience that i aim for. And that's basically not to never leave the id. I want to work in the id as much as possible. I want to locally debug my lambda functions to have breakpoints in my id. And most importantly, I want to work with real cloud resources. Let me stress that again, real cloud resources. Because if you work with real cloud resources, you can gain more confidence that the code actually works, right? i don't wanna work with mocks. i want to work with the real thing.

So we're gonna cover the testing pyramid on the service testing pyramid. We're gonna start at the bottom unit test. So unit tests are quick. Um they require no deployment of your service and you can run them locally in your id e. So it's gonna be something like this. We're gonna use p test, we're gonna prepare the test data, run the tests and run some assertions.

So this is an example of one of the tasks that we have. We're gonna build um um test for pent schemas. We have lots of pent schemas as you saw and uh create product body. That's basically the input model that we have for the create product function, the handler. So we want to make sure that if we get to create a product for, if we get an input with an invalid name, an empty name, we're gonna get a validation error. And this is basically what the, the test does.

Now, another use case for unit test is where you have a very specific class that takes very specific input, returns your very specific output and does not require any resources in the cloud. So you can use that to really make sure that your logic there w works properly.

Now, let's talk about integration tests and these are gonna be pretty much the majority of your tests in your, in your project. Now, they, they require you to deploy your stack to the service to the, to the cloud because you're gonna run locally in the id e but you're gonna access cloud resources. That's why you need to deploy it first.

So the way it's gonna work, we're gonna use python, we're gonna generate our test data and this is, it's gonna be uh an api gateway uh input the envelope that it's uh gonna send to our handler, the way it expects it in the, in the cloud and we're gonna send it a context object. Same way it's gonna get it in the cloud and then we're gonna work in the id e at break points. And uh we're gonna access our download db table, which is the resource in the cloud that we have here.

Then we're gonna verify it and run some assertions. So first of all, we need to generate the api g event. So you see i have a function returns the dictionary with all the metadata that you're gonna get. And in line seven, you have the body of the business payload that you're gonna send the customer is gonna send us.

So this really makes it easy to run uh a repeatable testing. And you might ask yourself, well, how did you get, how did you find this dictionary? Where did you get the definition from? So we have several options. First of all, you can print, you can simulate the events and print them and get them from cloud logs or you can use power tools for AWS lambda. That's what i did actually. And they have test json files for multiple services. So you can take the examples from there and just make the changes or you can refer to the aws service commutation where they tell you this is how the event looks like.

So let's write a test. We're gonna write the integration test for the create product uh handler. So we're gonna generate the event in line two. And then we're gonna call that our lambda function entry point in the id, which i call lambda handler. I know very original. We're gonna pass the event and then we're gonna pass the context, then you can add breakpoints in the id e. You're gonna work against the cloud, you're gonna access down with the b table, you're gonna get the response back and then you're gonna assert the response so we want to make sure that we have status code 200. Ok? And want to check any side effects.

Now, our great product function save uh save the product to the database. So what we want to check that um an entry is actually inserted to the database with the correct value. So that's what we're gonna do. We're gonna use better to basically fetch the item and see that all the values match up. And this is all the test altogether. I think this is a great way to do TDD. That's usually how i develop new lambda handler. So i would write an empty skeleton for a lambda handler and iw i would write the test and i would use the test to basically uh debug my function and start to implement it until it uh the test passes.

Now, there are some cases where you want to simulate some failures. In our case, we want to make sure that if we have a put item failure, right? It can fail. Sometimes we want to see that our uh handle returns a 500 response code. So i use stubborn from bottle core to basically simulate that the failure. And this is how it looks like.

Now you might ask yourself, well, you're not run, you're not using any cloud resources in this case

so why did you write it as an integration test? why not unit test? and to be honest, you do have some point here. but from my experience, the the more the service becomes more complicated, you have more and more cloud resources that you interact with.

so uh you might have at the beginning some app config configuration that you loaded loads and maybe um m parameter store uh parameter or secret manager secret. so now if you read it as a unit test, you need to also start to mock all the other resources. so you spend more time to mock resources instead of just focusing on the test.

so for me, a better developer experience is to mock only what i want to fail and to work against the cloud and let the other resources just load from the cloud.

lastly, we're gonna talk about end to end tests. again, these tests, they require deployment to the cloud, but they're gonna run specifically on the cloud and not on your id e. now you might ask well run, you're already working as a cloud in the integration test. so why do you need the end to end test?

so the reason is um that i need them because the lambda functions when they run in the cloud, they run a bit differently, right? they have their own role with different permissions, you might have mis configured the environment variables or other other configurations. so in order to make really sure that it works, even though your integration test work locally, you need to make sure that also the end to end tests are properly, that they work properly. so everything is configured properly in the cloud.

and this is how it's gonna look, you're gonna use python to generate any taste data. and we're gonna simulate the same input that any customer is gonna use our api we're gonna call our api gateway in the cloud. it's gonna invoke our lambda function and then we're gonna uh run some assertions on the response that we get back.

so let's write the end to end task for the create product function uh lama handler. so we're gonna generate in line to our um body for the requests uh open source library. we're gonna call, call the put h put product api, we're gonna pass the input again. it's gonna trigger the api gateway. it's gonna run on a plus 100% and we're gonna run some assertions. we're gonna get the hp studies. ok? we're gonna assert that response back and again, it's just gonna run on 100%.

and lastly, we're gonna check for some unhappy path. now, you can't really mark the cloud, right? because it runs on the cloud, but we can control some things like bad input. so if we're going to pass a bad input, missing some of the in line to the, the input is missing in every parameter. so i want to make sure that i'm gonna get a 400 response back uh response card back from our lambda function.

and there are also other se uh important test tests that you need to do such as api security test. so if you have authentication authorization enabled, you want to make sure that it's really working. so this is another example of entering tests that you should have back to you hater. ok. thank you.

all right. so you saw the asynchronous parts, the api gateway, which you probably have seen it before a few times. but how about the asynchronous parts? now we're going to go through the same things. but the asynchronous part, there are certain advices that wouldn't actually work in here. so i'm gonna work you through and why that's the case when it comes to the project part of things.

it's exactly as ron showed you before. we have our domain logic. all all of our business logics under the main, the main lambda handlers integrations. and such, one of the nice pieces about thinking about in asynchronous parts is that for batch processing or events and such, you might want to have a single lambda handler for one event, another, another handler for another event. and you can easily separate by having these folders. it makes it easier for you to see them what they're for. so nothing changes the lambda handler, which is very similar to what run actually had.

we have a function that's called process string, which is our lambda handler. but we're going to have to do things very differently now, from what run did one thing that not very, not very many, not many, not many people actually know you can do this lambda function is just a function that the lambda services will call and say here's the event, here's a lambda context. that's it. that's your lambda function. everything else you can extend this function to add defaults, to have dependencies, you have everything you want. and this is what gonna make your task much, much easier.

what we do everything else is the same. we take the events coming from dynamo to b. we use power to validate everything and then we gonna use our domain to come up with product changes. as it happens from here, we're on inside. we're only actually converting what's our input from dynamo to b and converting to our domain. so there's nothing really magical in here. it's very simple. no domain logic. if you will, what's interesting about the asynchronous parts which is gonna make our testing much easier later is if it's called by lambda, it's a real invocation, then this event handler with a dependent i will show you soon it's always gonna be none.

so in this case, i'm gonna create a new dependency and use the event source that this lambda function is supposed to be sending the event from and then the event bus. what this allows us is that in the testing later, we can inject this dependency and we can test the entire code with just one line and the return notified product updates is essentially our domain because it's a very simple product. we're not actually doing any business logic, anything difficult. our domain looks as simple as we take the dependency and we simply emit an event based on that domain. that's it, nothing really complicated. we're using depend in version to make it easier for test and to make it easier to add other pieces. now is an event, maybe it could be three later, maybe it could be something else as well. so it's easy to extend the bulk of this part.

now is integration. if you think from the integration point of view, you have the inputs that came from the domain and then this integration there has to talk to eventbridge. but now we're using eventbridge. what if we want to use sns later or what if we want to use something else as well? that's when things got to be tricky.

so what we do here is we, that event handler injection that you saw earlier, it's gonna have a few responsibilities. one of them is to make it so seamless that you can create event out of anything. we're gonna take any byd model and convert into an actual event that we define. and that event we're gonna standardize if you work in an enterprise company or if you have multiple events or multiple microservices. one of the easiest ways to make a mistake in the synchronous or event driven architectures is to simply emit an event without thinking what an event should look like across the entire company. because more events you add, the more complicated it becomes it's easy to get lost.

so in here, the event handler will be responsible to standardize any event that we create. so it becomes exactly the same everywhere. and for operational pieces, if our intro production, you want to think about observable and how do you know when something goes wrong? so event handler will add additional metadata, custom or not. so you know correlation id, when was it generated from who, which version? what if there was a breaking change? so event handler handles all that for us, but you have to send this somewhere.

so now we have a event provider that will be responsible to take this event that we just generated and convert into a put events api to eventbridge. so if you later want to swap to sns, everything else remains exactly the same inside this event provider, what we do is some interesting pieces we know now from this event provider that eventbridge has some limitations on how many events you can send, what's the payload size and everything else, all this knowledge goes inside this provider. so if i want to swap again, i have a specific one for s ms or kfc or whatever else you want to use and the error handling as well. events bridge handles error differently. even if you actually send a one event only or if you send a batch of events, eventbridge will simply say some of these methods fail and you have to know how to handle this error handling as well.

and last but not least what makes it possible for you to swap this event breach? sns kafka momento, you name it is that we have a contract which whatever event we send, we return and received. so now you know which event was successful, which one actually failed and you can handle these errors better.

how does it look like? in practice? it looks like this. this is a patent model where we say a product change modification had a product id, its title. and when it was created in which we default to utc, optionally, we can define the version of our model which we can view one v two, you name it. and this is gonna become a, let me go back here, an event. that's our standard event. every model will become something like this inside this event. we're gonna have our model inside this key called data which came from the idea from lego back in the days and the metadata is gonna come anything else operationally wise.

so we can standardize whether you use eventbridge or not, you might already recognize some of these things. what's the event name, the event source, the version correlation id. there's a ton more fields in here that we can use. if we ever swap from our eventbridge to sns or whatever, you always know that this data is gonna look like this. the payload will obviously be different. but if i want to create product change notification now to an event or i want to create something else like um order change modification, it's gonna be exactly the same by antic model and event out.

and then for eventbridge at the end, the api will look like this. if you ever use eventbridge, you already know some of these fields. so we use the metadata and the event standard that we created to fulfill some of these fields, detail type. what's the data inside? what's the event buzz name, et cetera testing. this becomes much easier. but we're now gonna be using some of these techniques to use more unit test and then make our integration test easier and convert something that's asynchronous into synchronous.

if you remember our al lambda handler, we receive a event handler and we use this and we pass it on. so we inject the dependency and keep passing and we use it. so if you come from java or the languages, this is kind of a uh easy for you. what we want to test is the dynamo db event came in truly and it was validated correctly. we want to make sure the event handler was created correctly. and we could also take into account the domain event that was also validated as it should.

so how do we test this first before we test anything? the first thing you should do, which we couldn't call it earlier is as you start with your lambda function testing, you might have bo three or aws sdk calls or you import a bunch of these things. and as soon as you run, test, it may work locally, it may work slowly. but when you work running c i, it may not pass. why that is when you use the wssdk, we're gonna create sessions automatically and this is a way to which were also something we do at amazon and also at open source in general, we want to block any connection outside of our unit test unit. i shouldn't be calling http end points or any sockets at all.

so if you ever, if you add this as a p test fixture at the beginning of our code, any test that runs unit test will automatically fail. so if you import bow to three initialized clients at the outside of the handler and all these things, which is common practice, it will fail all your tests. and now you know where the side effects are coming from, which makes things so much easier. so it's a, it's a plug in code p test sockets from uh mic. so if you're watching things for doing the work for us, one thing we also can do now which we couldn't before is to create a test doubles mocks is something we try to avoid at all costs in power tools because we have over i think 9 900 tests at the moment, it becomes tricky to have mocks everywhere in some cases.

so what we're gonna do is if the event handler is a dependency, like anything else we have a contract. so we create a new one called fake event handler, which we can handle everything in memory, which makes this if we should assert as well inside this event handler, all we can test. now, we can say we generate the dynamo to events. we initialize this fake event handler which i'll show you in a minute, a bit more and we call our alum the handler with that dependency already. so there's not gonna be any call to eventbridge or anything at all because we control everything in here.

and what's better is that because we control this fake event handler, we can add python magic methods or dender methods to make assertion easier. if we just implement len or contains or equals our assertion becomes much easier inside all we're doing is we say if the number of dynamo records should have been emitted from this fake event handler. and just by overriding python's contracts, we can make this possible inside the domain we want to test our business role.

and if it actually uses the integration, the piece that's interesting. now about the depend is that our test just becomes online. all we're trying to test is that our domain notified product updates is using our fake event handle again. and it's actually creating this event received that i told you before the domain should call this integration and return and receipt that an event was successful or failed. again. there's no io here you can, you can now unit test anything you want very specifically the entire lambda handler to domain integration or simply the domain integration or simply the integration, which i'll show you.

now, now it becomes a bit tricky. we need to test the integration. one thing you can do before they go crazy on the synchronous test is to test the contract that it actually works. one thing i promise you is that we take any byte model and we turn into a event

So we take a simple notification like this and we simply initialize it. And now the event handle, which I didn't show you all the code inside because it would be too much now. But we're going to give you a link that you can see all the details.

We have a method that called build events from model. We pass one or multiple pent models. What's the event sources should be? And now we have an event, the standardized event that you saw earlier. And then lastly, we need to convert this into eventbridge.

So now that we're testing, the eventbridge creates the put events request as it should. We simply initialize our provider, eventbridge, which is hidden in the event handler. If there's no provider, we initialize eventbridge, that's why how we could swap the sns and everything without changing any of our code anymore.

And then we simply build that put events request and we can assert that, that a that eventbridge put request is exactly as it should and we can look into our event, everything is fully typed and everything we want, we can even use aws sdk. stubborn to do even more deeper validation. Have we passed the right types? Have we passed the right values and such which we also have uh in our github, which i will show you that was relatively easy. That's the hard part. And every customer hand solves this differently.

So we wanted to have an opinion either way, but we recently launched another tool which i'll give you a link as well to make this even easier. The challenge we have is that once we have an event that's been emitted by eventbridge, it's lost. We can't really assert anything because it's already gone, we can't really look into it. What was the event that was emitted?

We now know that event handler and the event, which provider works exactly as it should. But we don't know if you have permissions to actually send the event or if the event was ever captured by anyone in that case. So the only way to test asynchronous architectures is by introducing something in between.

So the way you can do here is by adding additional infrastructure and we did a little bit more opinionated than usual because i didn't want to maintain all of the functions again. And this, what we did was we create a new ruling event bridge to say whenever you see a test event capture this and store into a dynamo db table which we can assert if that event was ever emitted in a shape that we actually wanted.

Some people. We use uh web sockets, which is perfect as well. And in this case, i needed to do that in less than an hour. So that was the quickest solution i could find. And it's also super, super cheap and fast as well. All of this testing takes less than a second, actually less than 400 milliseconds should be more precise inside this dynamo db table.

We use the principle of single db, a single table design if you like, which we use the name of our test as our primary key. So we always know that the test, the event that was admitted came from this particular test. Otherwise we have all mixed up, it would be difficult to query and then we use this issue received. So we know that test was emitted and we can take this exact test from this exact events from this exact issue received internally.

The trick is to use a not very known featuring p test called request. If we ever change the name of our test, our test will fail eventually. So what we want to get is dynamically take the name of the test and use as an event source. So now we know every event that was ever generated that came from a test would always have test underscore because that's how p test would actually work.

So you can now make it easier for you to have infrastructure in there. I had to build this code. But now last week, like two weeks ago, we announced aws iatk which is integrated application testing toolkit, which does this thing for you. But if you're curious on how we did it, we had to do because there was no tooling and maybe we do it next year in power tools. I don't know. Let's see, we created an event fetcher that will handle all this thing about what is the event that was omitted, go to the dynamo db table and confirm that was actually in place. Was this one event? 2, 10 events, how many failed and so forth?

Ok. Let's go into production now. And getting close to the end, you saw us talking a lot about power tools. If you never heard it before, we couldn't show you all the code in the production part because it's a lot of code, it would be difficult for you to see it. And if you go through. But there's over 100 lines of these things being used in the github, which i will show you all the data validation, the api routing of one function, one flat function or a micro function. If you like all the batch processing, the secret management. And the more recently, last week, we announced open api integration and data validation in one line as well. All of these things is coming from power tools.

We're using python as the best language in the world. That's why we're here. But you can also use other languages as well. But the idea is for you to not only choose everything at once, but you can use one feature at a time. That's why it's progressive. This is to get the code done. But one question everyone asks is that that's code every time i have a dependency. How about my code start? How do i even figure out that where my code start is coming from?

If you're using py torch, if you're using ml uh pandas or if you're using just normal pent and such, the challenge is for you to know which open source tool can i use before i go to production to make sure it meets my sl a. This is a tiny little tool called tuna. It's just one environment variable that you can use in lambda or just locally. It's called python import time, which python itself will profile how long it takes to import your code, everything that imports will be profiled. It generates a very long file and tuna, it's a file. It's an open source project that visual all of this as an icicle chart.

So you can see where it started and then it goes all the way down. The deeper, the more calls it had to go through this. So most of the time the code starts in python specifically or the languages as well is when you import a new dependency. So just by using tuna, you can easily visualize this, you can call locally, which is what i did here. But you can also use the same in lambda and you get the the stack trace in cloudwatch logs and then visualize your tuna. That's it.

One big notice the big difference you're gonna see is by running this with 100 and 2828 megabytes of memory versus one gigabyte of memory. The amount of difference could be almost as bad as as good as 70% reduction. So visualize it, you cannot improve what you cannot measure. So tuna is the first one you can use, but this is not good enough yet. We wouldn't go to production with this only what if we want to know if we want to change rust or go would actually improve our response times. In this case, it actually wouldn't, not much, at least what you want to learn is we can test through our test how much of our code is being actually used and where we're waiting the most and more importantly in this graph.

Also i icicle chart from psy, another open source project. Great one that you can use in production even on london containers and such, we want to find where the most common code path that has been used. And in here we're testing eventbridge. What you see in purple if you can read it is all of the io the bo tree or aws sdk is just waiting for eventbridge to do something. The tcp connections, the dns resolutions and everything. All of this is actually causing this lowness. If we were to switch to rust or anything, it still need to await from another networking end point to give us a response. So it's not gonna change much.

But what if we want to go deeper into this because this is ok, we can do better. This is another one called pie instrument is my actual favorite. Now, after everything i tried, there's more tools that you can use out there. But pie instrument is the best visually appealing that you can use by instrument. You can have just a context manager to say. Now that i know the most common code path is coming from eventbridge and, and whatnot. You can isolate in your test or in lambda function locally whatever you want. And that would generate a graph to tell you how long it was waiting. The most where exactly was wasting resources and such and you can use this as well.

If you are more pragmatic, even if you have a good test, there's another tool which is not here. But in the link called pi benchmark, pi benchmark takes all these three tools and you automate the benchmarking of all these things so you can use in your test. If you look in the power tools, github, you're gonna see that code there as well.

All right, 100 and 18 slides, right? That's good. When we say about being pragmatic. We also wanted to show you what it is. And we also like to work in public in power tools and run in open source. We wouldn't be in this open source track if we didn't bring you anything in open source, just talking about use case is nice, but actually showing you the code is actually the the piece that gets me more interested as a, as an attendee as well.

We documented, we're still documenting it. To be honest, all of the decisions we made. Why did you choose cdk? Why did we did some hacks in the build process? Because we had to, the tools are not exactly as i wish we would back then as well. And we also documented the integration pieces, the api documentation and even give you the tools that we used under the hood, all these open source projects out there as well.

Don't worry if you missed the keyword code. There's gonna be another one which is our g the treat for you. So don't worry, relax the vegas climate. Ok?

In summary, we showed you five pillars to make it easier to review any application that you have existing or building a new one or something. You're gonna be modernizing it. If you think from a project, it's how do you structure that? So it's easier to onboard new people. So they can see how things are organized, the handler. All it needs to do is to take whatever event and convert into a domain or business logic. Nothing else. If you do more than this, it's very difficult to manage that later. But you don't have to either. It could be just a single file for something simple with you, the domain, all the business logic, the integration like we mentioned and the testing, if you take this into account when reviewing your code is gonna be easier to have a conversation, more pragmatic conversations with your peers and your teams.

If you are interested in knowing everything we we couldn't cover and everything, i wish we could go even deeper into explaining the code and go to a 400 type of level session. Uh this is all the links, but there's also something extra that ron did it for you that i i hope you're gonna love it. Building all these folder structures and all this testing, all these interfaces takes time there's a template right there that you can do in one click or a one cli command line that builds an entire project with all these practices already in for you to be actual proper open source uh for you as well.

And we're also gonna add it tomorrow, uh the slides for this as well. So keep the keyword codes uh close to you. I'll wait for a few people and then i will close it off.

Ok. All right. One person and then we go cool if you want to continue this conversation or if you want to show us uh some of what you have or how do you optimize your code or some of those pieces of, i can type the code for you, but i can definitely tell you don't do this ever uh come find me specifically. Uh maybe ron will be there. I don't know. We have a booth uh at the venetian at the very center where all the open source experts will be there. So i'm bringing my whole team as well, not just python. So typescript people, java.net will be there too if you're interesting to continue that conversation.

And uh please please don't forget to use the survey. This was one of the rare sessions that it took a while to get it approved. We wanted to go to this session for every language and should be more pragmatic on the tools you can use for python for java or for typescript. So please help us with the, with the review and then hopefully next year we can go even deeper into these topics and bring other languages as well.

I was alaa chief architect for power to swed at west london. I hope you enjoyed with my friend bran from cy. Thanks for coming the.