Schedule work with Amazon EventBridge Scheduler

So I'm Michael Gosh, I'm a product manager at AWS for the EventBridge team. And today we're going to talk about schedule work with Amazon EventBridge Schedule. It's literally a tongue twister. I didn't come up with the title, but I do like Scheduler and I hope at the end of the session, you will also like Scheduler.

This is a demo session. But before we jump into the demo, I thought it would be good for those who are new to Scheduler to quickly talk about what Schedule actually is.

Scheduler is a highly available, resilient and easy to use capability as part of EventBridge that lets you automate tasks such as scheduling a Lambda function every 10 minutes or remind a user of an upcoming trip or as the foundation to build highly scalable scheduling services, if for example, you are a service provider.

Yes, as part of the demo, what I will show and it's a very, very simple demo to keep it simple and not distract from the details is we will create a schedule in EventBridge that every minute invokes a Lambda function which then will just dump the event to Amazon CloudWatch so we can see the event. And I also created already the Lambda function and the SQS queue as well as the IAM role so we can focus on the schedule apart and not so much on the other pieces because they're not really interesting here.

Ok, let's jump into the demo.

Ah, now it worked cool. Alright. Let me know if it's too small in the back and I can zoom in if that's needed.

So if you go to the Amazon EventBridge console page, you will see EventBridge is kind of a family if you will, which has different capabilities and services. One is maybe familiar to you Event Buses which allow you to build many to many event driven applications and integrations. Then we have EventBridge Pipes, which is a point to point connection service, for example, sending messages from Amazon SQS to Lambda function point to point. And then we have EventBridge Scheduler which was released last year at re:Invent. And it's, it's become a pretty popular capability in EventBridge.

Ok, so on the left, you see Scheduler here and then we go to Schedules and as part of the console, you will notice that it's pretty slick and slim and we highlight some of the features there which we'll talk about in a second, but all you can do is actually create schedules obviously, but that's it. So we intentionally kept it very simple and easy going.

Scheduler has some really interesting capabilities. For example, what we call templated targets that get you quickly up to speed with building integrations and schedules with common targets like AWS Lambda or EventBridge event buses. We also have universal targets, which is basically the SDK, the AWS SDK that we use internally to call any API in even in AWS services, I will, I'll show that in a second. We also have flexible time windows, which is a neat feature when you don't want the schedule to trigger exactly at the same time. Say you have set up the schedule to trigger at 1am in the morning every Monday, but you don't really care if it's 1am 0000, you would be fine to say, oh, well, as long as it's within 15 minutes or 30 minutes, etcetera. So we allow you to give you a little bit of flexibility, which means we spread the invocations across your targets. And then lastly, we have durability and resiliency capabilities for retries that SQS queues, etc.

Ok, now we create a schedule and I give it just a name pretty common re:Invent 2023. We can also give it a description. I'm leaving this out. This is optional here and then the first thing you will notice is the, oh, actually, should I increase it the text a little bit? It's ok. Then we have the concept of schedule groups. I'll come back to this in a like later and I'll keep the default one, which is the default schedule group, which is always there. It always exists. And then probably the most important part of a schedule capability is the schedule pattern or the schedule pattern. And we offer two types of schedules. One is one off schedules, things that you only want to happen once or one time for. That's for example, this use case of a user signed up to your website and you wanna remind him once in a week that please come and do a look at this promo or you receive the promo like this kind of one off use cases or you have an upcoming trip, a journey coming, which is scheduled to happen at a specific point in time, which is one of the most use cases that we have scheduled for. The other one is recurring schedules and those are the typical cron based ones where you specify a cron syntax for when the schedule is supposed to be invoked. And then we also have to simplify this a little bit. We have rate based schedules if you are just ok, every 12 hours, every minute, for example, so you don't have to go with the cron. This is I would say an easier experience to set up recurring schedules if you don't need the power of cron based schedules.

Let me quickly go back to the one off schedules for one off schedules, you're required to set date and time because that is when the schedule will fire. An important feature of schedules is also time zone support, which means you want to make sure that the users actually reminded or gets the information at their time zone and not UTC or in the at midnight when actually for them, it would be early in the morning. So time zone support is very powerful and useful in scheduling. And then the last one, as I already mentioned is the time window when you want the schedule to be invoked of which is kind of the default is it means please fire at somewhat exactly that point in time with some seconds of deviation from this, the date that you specify in time and then you can go up to four hours when you say I don't really care if it's within that window and we will fire. And again, this helps to if you have millions of schedules invoking a Lambda target, for example, and you want to make sure that the Lambda is not overwhelmed with all those schedules and you want to spread the invocation. So that's I think a very nice feature.

So as part of this demo, what we'll do is we'll do a one minute invocation rate. So we don't have to wait too long and I'm gonna disable the time window just to be sure that the schedule fires during this demo, the time zone by default doesn't really matter for recurring schedules. We don't really care about time zones, but I could optionally also give it a start date and an end date. The start date could be interesting when you say, oh, I know that, for example, in a week or so, I need to invoke this target every day or every 12 hours. Then you say wait with the invocation until this date. And if you specify an end date, which both are optional. If you specify an end date, then the schedule will terminate. It won't invoke any further. And there's also an option to clean up schedules which have expired if you will. So you don't grow with the schedules over time. So we'll leave the default here. We're not gonna do that.

And then a very, a very powerful concept beside the schedules and time zones is the targets. And so we looked at what common targets we have in schedules. And obviously you see lambda here, you can you see key EventBridge. So we have templates that let you easily start with building the target integration. And we'll come back to this in a second. I want to show you the, the powerful universal targets or we call them here all APIs basically and what you see here, you have a filter, you can filter by service. Let's say we want to filter by APIs that we can call and I go into ECS and then I say something service like an API that has service in the name so I can create service update services. Let me click that one here. And then what this will do is we'll open up the input schema or shape that this API expects. And by default, the update service API in ECS only needs a service name. So that's we basically provide you already with the required parameters as part of this. But if you look up the update service API, there are also optional parameters which you can specify here as well. Ok? But again, most of the cases will probably use one of the existing uh the common integrations. For example, in our case, we want to call a Lambda function. So let me look up my Lambda function.

So since I have this done with CDK, I deployed this with CDK, my, the Lambda function name is a little bit long, but it's just the normal Lambda function which writes the event to um CloudWatch to keep it simple. Now, I can give my Lambda function the payload and this is arbitrary, I don't have to specify payload if I just want to invoke the Lambda function to do some business logic. But maybe you also want to give the function, an event context or information about what happened. And in Schedule, let me zoom in a little bit, we have predefined variables that you can also use. You can define your own key values and objects obviously. But we have some predefined variables that allow you to say this was triggered by this arm at that time. And also the number of invocation attempts that we did. For example, if there was a retry, you can use that information as well. Um so if I wanted to do a custom key here, I could go hello and then re:Invent oops. And we have JSON validation here as well. So you see it's a valid JSON object. So I go next and then by default, the scheduled state is enabled. That's what we want. We want the schedule to trigger. And as I said, if we create a schedule that has a termination date, for example, one off schedules, they terminate after they fire or when I specify an end date of a schedule, then I can also say delete the schedule. So I don't grow too much over time to, to stay within the quota. In this case, we don't have an end date. So I don't really care about this one. Then we have the resiliency properties which lets you specify retries and also the maximum time of retries. How far we want to go? I'm gonna skip this for a second because in case something goes wrong. I want to go immediately to my SQS, my SQS that dead letter queue. So let me pick this up from here again, CDK. So I'll paste it in here. This is my queue. I can also create a new queue if you wanted to. But I have an existing queue, I can define custom encryption if that's what I want of my schedules and payloads. And then lastly, I need to define permissions that this schedule has. For example, if it invokes a Lambda function and also needs to go to that dead letter queue. If something goes wrong, this schedule needs the actual IAM policy. Again, I have created this upfront with CDK. So I'll just do this one here up, use existing role, paste it in here. Ok? Again, if you can also let Scheduler automatically create the uh the policy for you and then lastly, it presents me with a review to go through all the second things we have. So we say every minute we want to invoke the Lambda function with this schedule we don't have installed or end date. So it will immediately within a small time window, it will start firing, will go to this Lambda function and then just some information around the permissions and that dead letter queues that, that we have. Ok.

Alright. So if we go back to schedules, we see that the schedule has been created, but it takes some time. The information here is eventually consistent, which means it takes a little bit of time for the schedule to start kicking in for the first time and then it's going every minute. So while we are waiting a little bit, usually 1 to 2 minutes for the initial set up, let me talk about the schedule groups, schedule groups serve two purposes. One is obviously to group your schedules by context. So you cannot assign context to a schedule, but you can assign context to schedule groups. For example, if you have schedules which are related to it, automation, maybe you want to have a schedule group, which is IT ops or automation and you assign context there. Or if you have business applications, you can use the context for the, for the business application. The other like this is for discoverability. The other functionality is that while you can delete a single schedule, you cannot delete batch, delete multiple schedules. If you wanted to delete thousands or millions of your schedules, the way to go would be through the schedule group. So assigning your schedules when you create them to a particular schedule group. And then if you want to have this batch capability to delete stuff, you would do this through the schedule group.

Alright, let's check if our Lambda function has been invoked. So I go to the Lambda target, I could have gone through CloudWatch directly. But let me jump through the Lambda function. So you see what's going on here, then I go to monitor that has this link to CloudWatch logs. And if the demo gods are with me, we should already see an invocation. So it is 10:43. Now we have one invocation 10:43 and zooming in here a little bit. We see that we have our information here, which is the schedule ARN and the schedule time, the execution ID, and the attempt number that we pass to our Lambda function. So those were the context variables that I added to this function.

Ok. That concludes the demo. So let's please switch back to the presentation.

Thank you very much. I have collected some resources for you like blog posts, sample patterns, announcements, API documentation for schedules. So I'm just gonna give you a minute to capture the QR code two sessions related to Scheduler. We have an API 304 tomorrow and today there's another section where we won't necessarily just talk about Scheduler, but Scheduler will be part of the, of the talk.

Alright, just to briefly recap and summarize EventBridge Scheduler. It's one of my favorite capabilities in EventBridge and it allows you to create millions of schedules with, without writing code at scale with resiliency features and it is also easy to use.

Thank you very much for joining the demo. I'm happy to take questions. We have four minutes left roughly.