Scaling connected products and solutions with AWS IoT

Billion. That's how many connected devices are there across the globe. Now, are you ready to learn how to build solutions to manage all of these devices? Are you? Yes, yes, that's what we are going to talk about in today's session.

This is IO 213, Scaling Connected Products and Solutions with AWS.

I, my name is Remia Matthew. I'm a Principal Enterprise Architect with AWS. And today here with me, we have Joe McKinty, Director of Digital Technology and Antennas, Brana IO Lead both from Cric.

Hi, let's start with the agenda.

We'll start with the impact of IO some of the challenges in building and scaling IO solutions. Then we will look at how AWS can help to build and scale your solutions to manage millions of devices. We'll then look at how you can differentiate your solutions with AIML. And then we will hear from Joe and Anton us about Hive their journey to build a smart home solution and scale that to millions of devices and continue to grow. And then we will look at some new releases which will further enable you to scale your solutions.

The market for IOT devices are growing rapidly. I just mentioned we have 15 billion devices. Now that's forecasted to double in the next seven years. That is 30 billion devices by 2030. So you can see the opportunities that you have.

So these are devices like your smart bulbs or your doorbell cameras or your robot, vacuum cleaners or your connector vehicles. How many of you have any of the connected products? How many of you use this? I do as well. We all love our gadgets, right? I'm always on the lookout for new new products. I just replace my robot with a more intelligent version of it. So that's the consumer side of it.

But then you also have IO devices in the industrial and commercial side. It could be in health care, for example, patient monitoring, it could be agriculture. You are uh you have sensors to manage, say in energy and utilities, you have renewable asset management, in transportation, you have logistics tracking and monitoring. So you can see in any industry or any sector you are working with, you're not too far away from an IU product even in our hotel rooms, you have sensors for mini bars, right? So that's IU as well.

So you can see the opportunity and also the complexity which comes from it and the solutions which need to manage these devices at scale in this market.

When you're building a solution, how do you differentiate your solution? You differentiate by adding value by adding value to the customers, your end users and to the business as well. So what are the ways you can add value to end users and your business? There are mainly four areas you can do that.

One is customer experience. You have all your products connected together. You understand the usage of those products, you get telemetry and data from those products so you can improve the customer experience. It also helps you to improve operational efficiency. Say you have a customer is calling up to complain about a product. If you already have enough data coming through from those products, you already understand what is happening without the customer having to go through all of that again. And you can connect them remotely and fix them. You can even do predictive maintenance or you can actually monitor these products and identify any, any anomalies and then fix them before even the customer has to call you. So that's operational efficiency.

The third one is around sustainable innovation because you can build these products and solution in such a way that you can understand how the user behaves and the gaps in terms of any features. You can try out new products. You want to build a solution where you can deploy some of the new features, get feedback from the customers and then decide whether you want to continue with those features or not and thereby creating new revenue streams for your new services. That's how you can add value and differentiate with your IUT solution.

And that's where IWS can help to build such a solution before we go into how to build a solution. Let's look at some of the challenges I mentioned, the number of devices are increasing into billions of numbers. What that means is newer products coming into market and newer products your customers are expecting either it's from a new hardware or software, new features perspective. You need to be able to navigate the breadth and complexity of all these devices, then you need to be able to manage these devices, right? So if you have portfolio that has got millions of devices, how do you make sure they are reliably connected and you can operate them and manage them in at that scale. And the third one is around data. You are getting lots of data from these devices. How can you store and maintain and manage that data in such a way that you create differentiation from that data. And of course, security is key as well. You need to make sure that your devices have continued security posture and last but not least you want to future proof your investments with sustainable innovation. Because when the new products are coming into market, new versions coming into market, you don't want to ditch all your existing solution. You want your solution to be able to scale and evolve with the products and the user experience. And this is where AWS can help so you can mitigate these challenges with AWS.

IOT solutions. AWS gives you the broadest and deepest set of services from devices to the cloud. You have reliable specific services which helps you have reliable connectivity and manage these devices or millions of devices that you can integrate with AWS machine learning data. AIML analytics capabilities to improve and differentiate your solution. You have trusted security from edge to cloud through Amazon specific IOT security services and with the overall Amazon services 200 plus services, you can continuously evolve your solution to keep up with the customer experience or customer requirements and keep up with the new products. I will give a good example of how their solution has been continuously improving and evolving over time.

AWS provides you comprehensive set of tools and services to manage the end to end life cycle of your products. I'm not going through all of that, but you can see there are a number of services specific from devices to connectivity to managing and getting value out of those. So you have to pick and choose the right services for your solution. It could be uh consumer products, it could be industrial products, it be connected vehicles for each of those who have different types of set of IOT services you can pick and choose. But how do you start?

Let's take a consumer product or device solution. For example, where do you start and how do you start to build a solution. Let's look at that. A good place to start is with the devices itself. So broadly speaking, there are two different types of devices. One with a very low processing power like by running microprocessors. These are your smart bulbs or your um washing machines or your home appliances. For these devices. You can use FreeRTOS, FreeRTOS is an open source real time operating system which provides you a responsive and reliable kernel for your hardware, which works with 40 different hardware architecture. So that and FreeRTOS comes with prebuilt software libraries which can connect to AWS IOT services using MQTT MQTT. MQTT is a lightweight connectivity and network protocol which is specifically designed for IOT devices which can handle the intermittent connectivity.

And then you have the second type of devices which has got a bit more processing power. These are things like your security camera or home hubs or your routers which has got a bit more processing and storage power. And for these devices, you can use something called AWS IOT Greengrass, AWS, IOT Greengrass is a edge run time and a cloud solution which helps you to build, deploy and manage your device software. It comes with a number of libraries including running a IL at edge. You can do local processing of data, you can store data as well, not just that with the home hub running Greengrass for example, even if you don't have a number of peripheral devices even even though they are not running FreeRTOS, they can all connect to this hub and they can be managed from this hub with all the peripheral devices. And AWS IO Greengrass also comes with prebuilt libraries which which connects to IO AWS IOT services with MQTT.

So now you connect all your devices to the cloud. Now I mentioned you have a number of services, IOT specific services that you can use. The main one is AWS IOT Core AWS Iot Core helps you connect billions of devices and can manage trillions of messages and you can connect all of these devices together. And I call, lets you connect to all the other IOT services as well. I'll talk about a few more in, in the coming slides as well.

Now you need to build the business logic that your differentiation you want to build. uh what is your solution? what is your business logic? what are the requirement, business requirements or customer requirements you are actually trying to build for that. you build all of that with all the A W services. it could be, you can use compute, you can use machine learning storage databases, you could take the serve options as well and you can manage all of your CD pipelines with the tools. so you build your solutions with all the rest of IWS services and you can build the solutions for end customers, which is customer facing. you have the, it could be a mobile app, it could be a desktop app, a dashboard or connecting to any of the virtual assistants, including your favorite virtual assistant, alexa, for example, i can turn up my heating up or down using asking alexa to do that with a high thermostat. again, you don't have to start from with all of that solutions in one go. you can start small depending on what you want to build and then scale with this pattern. so this gives you a logical view of how you go about building that.

So we talked about building a solution. Now, what about managing those devices which are in the field or with the consumers? How do you actually manage hundreds of thousands or millions or billions of devices? One service which helps you do that is AWS IOT Device Management with AWS IOT Device Management. You can do bulk registration of your devices, say depending on a product type or product version, you can bulk register them, you can organize them depending on product types. It could be different versions of the product, different types of the products and you can monitor each of those devices and also manage them. You can create some jobs configurations to update do over the air update or O update of softwares for all of your devices using AWS IUT Device Management. It actually it helps you to scale to millions of devices and still be able to manage all of them from one place.

Now, everyone's favorite topic. Security, security is complex with IOT it, there is more complexity because you have the solution, site security to deal with. And you also have device site security to deal with. It adds the complexity for security to, to your solution. And that's where again, AWS can help, which can improve the security posture of your solution. And how do we do that? We have some built in security capabilities already with AWS IOT services to start with you have mutual authentication. AWS IOT CO provides you security certificates to connect your devices and you can use Amazon Cognito and Identity and Access Management to manage authorization of your end customers, whether it is to your mobile app or their dashboards and then you can have fine right, authorization using IAM identity and access management so that you are always providing least privilege. You are only giving people the access that they need. Not more than that. And then you have encryption. AWS IOT CO provides you end to end encryption at transit and at rest. So this comes built in with AWS IOT services which help you with that security posture. But when it comes to scale, you need to go beyond that, you need to be able to make, make sure you are managing that security posture continuously as well. And that's where AWS IOT Device Defender can help AWS IOT Device Defender, you can do continuous audit of all your devices to understand or check if the configurations are still secure and you can detect any anomalies

There are two ways to do that. You can write your own rules to understand any anomalies or you can turn on ML Detect which uses machine learning to understand the continuous flow of messages coming from those devices and identify any anomalies. And with those, you can also set up some alerts either to your operational team to manually fix some of these issues or you can automate some of the mitigation. So I talked about the jobs to do some updates. So you can send some security patches depending on what the anomaly is. So again, this is a way to manage millions and hundreds of thousands and millions, even billions of devices, making sure that you have continuous security posture across your devices.

Now, everyone's actual favorite topic, which is AIML. So we have heard a lot about machine learning and AI this week at Reinvent as well. How do you use that to enable differentiation for your IoT solutions? There are different ways to do that to start with. You want to collect all the data and train some models and train your apply some machine learning. You can use Amazon SageMaker for that. It will, it provides you with the capability to to collect data to train models. It comes with some prebuilt models as well, which you can use or you can train your own models, you can label data using Amazon SageMaker. So you build that model and you apply that to your application to provide differentiation.

Another way to do that is apply machine learning at edge in the devices itself for which you can use AWS IoT Greengrass. What you can do is with the SageMaker, you can optimize and fine tune your models to run on the device which is constrained in terms of processing power. So you can optimize those trained models and deploy them on your edge device using AWS IoT Greengrass and then you can run AWS Lambda. You might have heard about AWS Lambda on cloud. You can also run AWS Lambda on your devices with IoT Greengrass and you can trigger those trained models to do inference in real time.

There are certain benefits of doing AIML at edge. It could be cost benefit or performance benefit, cost benefit when you don't have to send all the data back to the cloud, you can do lots of local processing and do the inference at edge. Performance benefit is when you want to do real time or near real time inference without having connectivity back to the cloud. So it can deal with intermittent connection. So depending on your use case, depending on what the what solution you are building, you can use any of these approaches to apply machine learning to differentiate your application.

Let's now look at an example, actual real life example of how, what a solution with the AI edge looks like most of you. I don't know any of you have double cameras or security cameras. Yes, a lot of us have. So this is an example of a security camera which is running AWS IoT Greengrass. And in this scenario, you want this camera get triggered whenever it detects motion in front of it. And you want a machine learning to understand whether it is triggered by a person or an animal, right? So you want to actually add some machine learning capability to it.

So the AWS IoT Greengrass sends all the data to the cloud and with from the camera sensor, you can receive all the camera feeds through Kinesis Data Streams. You can store all of that in Amazon S3, which is Amazon storage service. And then you can also bring in additional data. You might have some external data which has got different animals, pictures or human face pictures, right? So you can collect all of that data and use Amazon SageMaker to label and train those models to identify whether it's a human or an animal.

Once you train that model, you could even try to use some of the prebuilt models as well. Once you train that model, you optimize it to be run on AWS IoT Greengrass and you deploy that to the edge using Greengrass and IoT Core. Once it is running on the edge, you can have a Lambda function which is triggered whenever the camera detects a motion which takes that AWS Lambda function takes that trained model and do the inference in real time.

So you might wonder feature where it actually triggers a notification to the end user whenever there is a human in front of the camera. So they can actually log in or use their mobile app to access that video feed, either in real time or a recorded feed. So this is an example of how you can go about applying machine learning at edge. Again, you don't have to start with all machine learning edge. You could start with doing all the machine learning in the cloud and adding that to the application. But then as and when you understand the behavior of the customers, you could actually see if there are potential advantage in moving your machine learning model and trained model to the edge and move that to the edge.

Hive actually provides a good example of where they started with the cloud and then find some advantage in moving some of the logic to the edge as well. And that's a good place to hand it over to Joan Antanas to talk about their journey with AWS. Thank you.

Thank you very much Remia. Pleasure to be here. Is it just me or has everyone been thinking about this session a lot? Yeah, thank you.

So Antanas and I are here today to talk about Hive's journey with AWS. And I, I stress the word journey from where we were before we adopted managed services to where we are now. We'll take a quick look at the rewards that we had from doing that, but we're going to focus more on the challenges and our thought process to overcome those challenges.

But first who is Hive? Well, if you ask the average person on the street back in the UK, we've got pretty good brand recognition. Who Hive are? They would go? Oh, that's the company that lets you control your heating from your phone to those people. I would like to say correct.

Um Hive is UK's number one smart heating brand with heating installations in over 2 million homes. Through our control insights and coaching that we give to our customers, our differentiators, let's say over the past 10 years, we have managed to save our customers £525 million on their energy bills. That's British pounds and 1.14 billion kg of CO2 reductions.

And, but that's not all we are Hive is part of a wider group, Centrica group, which includes British Gas, an energy tariff company within the UK. Also, it's got the largest UK fleet for installations insurances and so on so how are we going to leverage our wider business to take Hive to the next level? I will talk a little bit about that later in the session, but we're here today to talk about scale.

So let's take a look at the numbers and where we are today, we have 3 million registered users with over 5 million connected devices and we send out about 8 million notifications a day, 72 million API requests a day. Point I'm trying to make is that we're already at scale and we need to concern ourselves with performance cost and robustness.

Um so we will dig into that, but first, I'll hand over to Antanas to talk about where we started.

Thank you, Joan. Hello everyone. What a week it has been right almost there. My name is Antanas and I would like to take you on a quick journey through time with Hive. If we take a step 67 years back, I was running a monolithic architecture. There you go. We were already on AWS cloud running on good old Amazon EC2 instances that you are probably very familiar with.

Our architecture contained a lot of open source solutions that come with their own challenges, like lack of documentation, difficulty to learn or security concerns. And our hubs were using MQTT protocol to talk to the cloud platform, but the hubs themselves had very little business domain logic and most of the heavy lifting was done on the cloud.

Unsurprisingly, the solution we've had brought us to a place where we had to take a step back and re-evaluate our architecture. We had serious scalability concerns. We would have to scale the whole system to accommodate more customers or traffic spikes. Scaling individual components was hard or sometimes impossible. Our operational costs were through the roof as well and our teams were large but required to maintain the architecture we've had and with a business model where a customer pays a single upfront fee for the hardware and we would keep them running and supporting forever. It was not sustainable.

We were constrained by the technologies already used in the monolith. Adopting new technology was difficult and frequently would mean rewriting the whole system. Continuous iterative delivery was hard. We would have to redeploy the whole system to promote smallest updates. Impact of changes was not always well understood, which has led to an extensive manual testing and production incidents happening more frequently than we would have liked.

The architecture became too large and too complex to fully understand, not only has increased our featured delivery lead times but also made it more difficult to hire. Now, none of these challenges were unique to us and you might relate to some of them, what we have done. It's not a magic bullet, but hopefully you'll find some of the ideas useful for your own use cases.

I have to say we did not get you this architecture we did not change our architecture overnight. Over the years, we've been taking baby steps to get to the place where we want to be. This is a representative example highlighting key services used by Hive today. But what you can immediately not that we have updated our hubs on AWS IoT Greengrass service and use MQTT messages over IoT Core to talk to the cloud platform. It has enabled us to move more of the computing to the edge closer to where the data is generated.

We have also been splitting the monolith into multiple services each responsible for its own business function. The microservice approach immediately improved our scalability resilience or continuous iterative delivery. We do not need to redeploy the whole system to promote small updates anymore. In fact, I think we have deployed to production dozens of times today. We can also now easily scale the service that is experiencing traffic spikes and if a single service is struggling, the rest of the system remains unaffected.

Now, of course, that's easier said than done. Most of our micro services were still very tightly coupled with each other, for example, using well defined APIs. So to improve resilience, we have adopted event driven architectures that further reduces coupling between the services.

Another pattern that we have adopted is using serverless solutions and purpose built, fully managed AWS services as much as possible instead of building our own. For example, using Amazon Cognito instead of developing and maintaining our own authentication server. Now, the decision between serverless versus traditional or versus fully managed is always weighted against the business requirements of service behavior. But whenever it made sense, we have chosen to rely on AWS development and support teams to take care of server management, scalability or compliance. Leaving our teams fully focused on the business logic and writing the code itself.

Not only it has reduced our operational costs or team sizes, but also allowed our teams to adopt, you build it, you run it mantra that we strongly advocate for. Having said all of that. There are exceptions to this approach. And later on, we will have a look at an example where building our own solution was a more optimal choice for our business case.

So the takeaway really is system re-architecture can be a very daunting task but do not stagnate on it continuously, re-evaluate your architecture and take baby steps to get to the place where you want to be.

So let's take a look at the rewards of this work. Now, these are numbers that we can obviously be very proud of. This is comparing where we are today to where to where we were with the monolithic architecture. 90% reduction in our average cost per user and 98% reduction in priority one incidents. All while reducing our operational cost through we, we needed less staff. And we're also increased our user base tenfold during this whole process.

Now, I don't want to gloss over these numbers, but this was achieved over many years and it misses that point slightly. So let's focus on the challenges.

So, biggest challenge. When is good, good enough? Like we said, this took many years. If we had stopped 2-3 years ago, would we have the same figures? No, but would it have been good enough? Maybe that's a process that we need to go through with our business continuously as Antanas mentioned.

Um and we will again talk about that more in a minute but continuing to convince your business to invest in this space and continuing to improve is very important operational challenge. You build it, you run it, the dev ops model, upskilling your staff to this mindset. You don't just hand your solutions off to an operational team. You as dev ops staff, you're the subject matter experts, you are best placed to run it in production, learn from the scalability, know the cost, build the right solution and then on to design, talking of solution, there are so many different ways to achieve the same outcome with aws.

So which one is right for you? So I think the answer to these questions are through this approach. Let's start with understanding your system usage, understanding your users, understanding in our example, most of our users, they interact with us in their evenings. That therefore means a serverless approach where we can auto scale really easy is largely a no brainer but not always, we have to do big upfront cost analysis of all the different solutions to find the one that's right for us, we also need to look at the aws managed services and decide when is the right time to adopt.

An example is we were one of the very early adopters of green grass. Now that had a lot of benefits. We got to influence the roadmap, we got feedback straight away. But honestly, on the other side, we've had to migrate from uh green grass version 1 to 2, which is a little bit of a painful exercise. I still think it was worth it though.

Thank you, Joe. So let's say you have decided to change your architecture to address your scalability concerns. But how do you choose your architecture? Maybe you something that your teams are already familiar with, maybe try shining new aw services recently announced. There have been quite a few this week or maybe rely on reference architecture diagrams, hundreds of blog posts because well, they have proven over and over again that those solutions simply work. Some of you are nodding. I'm guilty of all of that on multiple occasions. It is such an easy trap to fall in that without thinking about your customer, your end user first. Over the years, we have had multiple times serious implementations until we found the one that works best for us. And this is a good example where building our own solution was a more optimal choice over fully managed service.

If you have not looked into amazon time stream, please use. So it is a great fully managed time series database that allows you to collect and analyze huge amounts of time series data points. And normally we would have chosen to use it. But after we have taken a look at our customer behavior, one clear pattern immediately became evident. Our customers are quiring time series data much less frequently. When compared to the amounts of data points coming into the system reported by the devices, we have made a conscious choice to store non aggregated data as it comes and only aggregated on the fly when the data is requested. The approach allowed us to build a solution that is very cheap to run, yet still very simple to maintain.

So let's have a quick look at this approach. We have split each data producer to its own. Amazon can uses data stream. In this example, we have hive hubs sending time series data to one stream. Whereas chargers are using a different stream for their own telemetry than we have consumers. Each processing records in batches and writing non aggregated data to amazon dynamo db. The architecture immediately decouples different device types to their own flows allowing us to attribute costs easily and increasing overall resilience. If one consumer is struggling processing the records, the rest remain unaffected while the issue is being resolved. The architecture also allowed us to choose the right technology for each consumer depending on the data flowing through the streams. If you look at it from the distance, it is the same microservices pattern that we have seen hundreds of times before.

So think about your customer one, designing your system architecture, allow your users drive your architecture choices and i promise you you will easily find a solution that is the most optimal for your business case.

So we had a look at refactoring. Now, the point is refactoring is not the only option and sometimes not the most optimal option. When addressing scalability concerns another very lucrative option in the connected devices space will be re hosting on this case specifically, more computing. A very recent example is our heating failure detection feature that we are currently moving from running centrally on the cloud to running on each individual hub. And when I say recent fingers crossed, it's going to be available in the next two weeks.

Heating fail of detection is part of hi heating plus subscription offering. And if i had to describe it in simple terms, the feature tracks your homes target ambient outdoor temperatures and monitors how quickly your home is heating up. If we detect that your home is not heating up as quickly as we would have expected. A heating failure alert is raised.

So up to now, the failure of detection model was running centrally on the cloud for all enabled hubs and the more customers were adopting the future running it centrally became impractical from the cost perspective. For this reason, we are moving the inference model to the edge to the hubs closer to the data and only send alert as they happen to the cloud by the nature of the feature. These are very infrequent. Well, think how frequently heating systems are actually failing, but the result is we have significantly reduced the amount of data going through the cloud that now only needs to act on the alerts raised instead of crunching through huge amounts of telemetry to detect them. In the first place.

We have already started looking at the next iteration of the feature that will be utilizing amazon sage maker to continuously improve failure detection models and run inference on aws green grass. So have a think about your systems, think how much of your cloud logic could be moved to the edge closer to the data, making your devices truly smart and most importantly, enabling you to take that next step required for scale, john.

Thank you. Thank you. Ok. So some key takeaways from us before we take a quick look at what might come next for hive. This is a continuous process cost scale your user behavior changing makes this a continuous process. You need to continue to look at your systems ensure that they are fit for purpose and adopt the right solution.

So onto many solutions there are as antonis has just covered lots of different ways to achieve the same outcome. But what is right for you and what is right for your customers? What is your um resilience threshold? Your um what is the limit you're willing to push to in explore? I mean, you're all here. I'm sure that you've explored a lot this week um and have a look around, see what's on offer time series uh as antanas mentioned.

So i just wanted to take a second to talk about what's next for us and how we are taking on new challenges. So we have already shown that we've got a large fleet of energy consuming connected devices and we have enabled our customers to achieve great savings. But what could we do as a business with that aggregated fleet? How could we push it? We've already started that this, this year, we have uh uh uh an electric vehicle charger that we will automatically take hold of your charging schedule for you to charge at the cheapest time of day. And we've even launched a product free charging for your first year. And this all comes from us being able to take um grid balancing, take the rewards from grid management and pass them on to the end consumer. And what could we do next with solar panels, heat pumps, any electric device? So we're hoping to explore that with aws going forward.

Thank you. Oh, what, what a great story. Thank you joan anton us. Thank you for sharing your journey with us. What a great story, really excited that a w is part of your journey, your net zero transition and continuous growth in terms of products and solutions.

Now, let's have a quick look at two new features which will enable you to take you that journey further to manage millions of devices. The first one is software package catalog with aws device management, with a software package catalog. You can easily manage software versions for all of your devices across your fleet. One of the challenges we see often is when you have large number of devices, you have multiple versions of softwares. How do you manage that? And software package catalog will help. Now you can group your devices using the version meta data in one central location. You can provide create some jobs to update certain versions of the of those group of devices remotely. You can automatically update the device version metadata based on whether they are successfully gone through an update or not. So this will massively uh simplify your device management and software management of your devices.

The next one is very interesting. It's location indexing and geo query with the aws iot device management. This will help you to index location information for all your devices, whether they are already gps enabled, whether they can relay their location or not. You can even use customer user defined location to index those locations with that, you can search and aggregate your devices across a specific location boundary. You can search with that. You can capture insights which are location specific. For example, if you see that certain devices in certain location are all disconnected, it might be a mobile connectivity issue. For example, you can also do some geo targeting for your customers as well. You can do some device location specific updates to your devices and you can have set up configure geo queries which can be run periodically to understand if your devices go outside a geographical boundary you have defined for them. So again, if you have hundreds and thousands of millions of devices which are spread across massive geographies, this feature can help you to monitor manage and maintain them much easily.

With that. We have come to towards the end of the session. Some of the key takeaways as joe was mentioning, we we are aware that we are mentioning takeaways a few times just before lunch, but some of the key takeaways, we talked about the market for io how it is rapidly increasing. We talk. So, so some of the aws iot services which help you to build and scale your solutions. And we saw how you can apply am l to differentiate your solution? And you heard from joe and anton us around how you need to you can add value by continuously improving evolving, scaling and optimizing your solution. Hopefully this was useful for all of you. You learn something new.

If you want to learn more about aws io go to aws.amazon.com/io. You also have some preference architectures there. And hopefully you are all thinking about new ideas to build new io products and solutions. Maybe my next buy will be one of the products you are thinking about. And before we wrap up, please do fill out the survey, we do value your feedback. It helps us to focus our topics in terms of what adds more value to you. What topics are more interesting for you, please fill out the survey. If you have any questions, joe anton and me will be outside to take any questions. And thanks again to joan anton us.

Thank you very much and thank you all for coming and joining us. Enjoy the rest of the day. Thank you. Thank you.