Scale enterprise BI securely with Amazon QuickSight

All right, good morning, everyone. I am uh Tracy Doherty. I'm the General Manager of QuickSight. Uh been with the team about six years.

The beauty of these lights is I can't see any of you, but you get to see me uh as we go through this, but I'll, I'll speak for about 10 minutes or so. And then I'll have Sean and my team come up and go through a lot of the, the details and then we got a great guest from BMW to walk through a, a solution that they've built.

So why don't we get started?

The first thing if you guys have heard me talk before. Um I'm a huge, one of the things that drives my passion is how do I get data to more people in any given organization? Um and, and I've given talks on data driven cultures and how do you go about doing that? And, and a big part of, I guess you'd say our mission at QuickSight is how do we do that? How do we get more and more people in an organization to use data?

It tends to be an incredibly valuable resource um for decision making, but you got to put it to work, right? You gotta make decisions better. There's uh efficiencies you can do. There's a lot of things to actually get value from the data versus just having it. But then even when you do that, the trick is how do you get it to the people that actually need it to make the decisions and data shows us and this number has been debated. Is it 20%? 30%? But uh roughly less than 20% of people in organizations have access to the data. They need, particularly through BI tools.

It turns out when I started doing this stuff years ago, that number was the same percentage. It really hasn't changed that, that much dramatically. Um if you read a talk that we did last year at Re:Invent, we highlighted an organization that got up to 85% of the organization using data and being very, very data driven. So we know it's possible with the right tools and, and going through that. But by and large, we don't see that.

So it kind of tells you that over the history of, of BI, we need a different approach, right? We need to think about this thing differently than, than we have been. And in QuickSight, we kind of take this, I guess you would say this three prong angle to it.

One, we say we want to have a unified BI for all users. And what we mean by that is we do dashboarding, we do reporting and we do gen AI all in the same tool. So the reporting to build reports, you know, actual paget reports versus your dashboards comes through the same authoring tool, authoring experience. They get shared out through the same experience to to the users. As we go through that, we have a specific billing model for end users versus authors to make it much, much more affordable to expand out to the entire organization. So you need that.

The second thing to enable that to go across to everybody, you need the product to be able to scale in scale where performance is the same for every user, no matter what time of day it is and no matter what the activity is. So we as you would expect on AWS do all the auto scaling for you, which is very different than anything that you've experienced when you think of your on premise solutions based on a server and you're limited to that server capacity.

Um the third thing and I'll talk to this a little bit is how we incorporate gen AI. And you'll hear us use the term gen BI specifically for um our team and features that were announced. I don't know how many of you are able to get to the, the keynote from Adam. And then Rashir his was the last part where he talked about Amazon Q um and QuickSight and, and you'll see some of that capabilities that we go through.

And then you see my last point is lowering the cost. Again, the whole mission is how do you get data to more and more people in your organization? I could go on for more and more. How do you get to different device types, different formats, et cetera that they go through along that line.

So, and this is how I, I always like to think about it. If you think of legacy in this space, you typically had a reports product, a dashboarding product, um and embedded analytics product. Sometimes somebody would do two of the three very unlikely. Then NLQ gets added and you go through that. Um in our case, we wanted it all to be in one package, it's easier to maintain, it's easier to manage our architecture allows for it. It's been our vision for several years and how do we keep on going along that path? So it just makes for a great experience.

Probably several of you have the experience of you have a dashboarding application and a reporting application. You have different people that build reports and different people that build dashboards. And you imagine the training that goes across that and then your users have to come into it from two different products. We wanted to eliminate that problem upfront.

The second part was how do we use JAI capabilities which we call JBI with Q in the product. How do we make everything easier and more approachable and more valuable for users? And for those of you that have used QuickSight in the past, you know, that we launched Amazon, I mean QuickSight Q about two years ago, which is purely an NLQ product. You could ask a question, get an answer.

Um we've learned a ton, I kind of wish we would have called it JAI when we announced it two years ago, that was not something that was well known, but that's the under token of what we were doing. And we learned a ton about how users ask questions. How do they want their answers to come back? What are the specifics? And you find out there's a lot of assumptions that, that we made that were wrong, that we corrected, that I see others making that same mistake as we go forward and I'll talk to that a little bit, but we've provided really these three core areas.

You know, how do we accelerate the authoring capability? Two, how do we improve our question and answer and experience of asking questions of the data? And then three super exciting is how do you do storytelling? How do you let your end users tell the story of the data they have access to um which isn't an authoring thing. They're not creating reports and dashboards, but they're telling a story and presenting it to the, the way they want it. And I'll show you a couple of examples of each of these.

The first one. And Adam showed this in the keynote um as well as on building visuals. If you're an author, we've got, you can type it in using Q to say what you want to show. You, ask it a question. It brings up the, the visual that you want, you can modify it from there and then embed it in your dashboard. And you don't have to know anything about where the data was. Um what structures it was, it takes away that, that aspect. So it makes it a lot easier to do um authoring and we find that's incredibly powerful for maybe the more new authors that come to it.

The second one is we use it for building complex calculations and you type in what you want and it builds the complex calculation for you automatically. What we find here is our most advanced users love this feature because this was the most complex thing they did and it made their life easy on that part.

And then the third one, everybody seems to like which is I can go in and refine a visual. I can look at a visual and say, hey, I wanted to have these colors be filtered by xy and z and go through that. And it will change that for you in a really, really quick, fast method. So those are the things to really enhance the authoring side of the, of the house.

The next one was, as I mentioned, um we, we improved our Q and A experience for a lack of a better term. We now have um dashboard summaries. So it's an exact summary that tells you what's changed on it. What's the value, what you should look at some of these dashboards get quite complex and then we've made um the Q and A experience much more valuable.

Uh what would I say? A simple learning for you as we thought, when we first did Q two years ago, people knew what question they had and they wanted a precise answer. It turns out both of those were incorrect. Oftentimes people don't know exactly the question they have, they know approximately the question they have and they inevitably don't want a uh yes, no answer or a specific number. They want a, a multipart answer which says, well, we think you think wanted this, but here's some other things that are related to it. And then there's a follow on question that goes through that and you maintain context.

So this to me is kind of what users really want. So sometimes when I see these chat experiences that say, ask this, get a, ask a get bc, get d. It's not, it's not what users really want. In fact, it's kind of annoying. This is a building a more cohesive experience of asking questions and getting a robust answer that you can continue to go on and then, and then when they get an answer, they can pin that answer so that they have it for later use in case they want to build a story or, or reference it on a regular basis.

And then the third part, let's see, we'll go through this. The third part is around storytelling. Now, authors can do this, but this was really, really designed for the end user, all the what we call the readers that are out there in the organization. And this is this ability to say, hey, I want to tell a story about, you know how sales was this week and it'll go off all the data that they have access to in the system, their dashboards, their visuals and put together a presentation.

And when you go through it, you can have choices if you want it to be a presentation, a document. Um do you want it to be modern looking, light, looking, dark looking? You can say if you want it to be concise, long bulleted, not bulleted and it creates this for you, then you can also add visuals into it and it will build the story for you as well.

The thing that I use as an example for this and this is a true story on my side is andy jassy would used to send me these emails and say, hey, how is uh how's the QuickSight business doing? And there's two things I would not do. I would not send him my dashboard with 60 visuals on it and say, well, here it is. Just look at that, that should tell you all you need to know.

Um two, I wouldn't send him a report that had that either and said I would have to craft an email, put some charts together and describe that with this capability, you could do this really quickly and what we found with customers, this resonates at all levels of, of people in manufacturing, people in medical, everybody wants to tell a story of their data, but they don't want the dashboard is not the, the method for that. That's the method for them to get insights to it. But then they have to tell the story in a unique way so you can do this and um incredibly fast and then modify it and it's secure and safe and you share it with those in your audience.

There is a session tomorrow from a person on my team, Zach that goes into great details of all the gen BI stuff. I would encourage everybody to go. If you're, if you're interested on that, it's really a deep dive into what we're doing there. And it's very exciting, but I highlight this part because these pieces all make in my sense, data, more approachable to your users and getting more broader use across your organization. And that's the, the main thing if I think about going through that.

So I said all of that. But the thing that I'd like to highlight and this is where Sean and By will talk more is that sounds all great. But you got to have a system that is architected to scale is secure.

Um who has the right information, who doesn't have the right information? How do you get data to it? What I would say is the core foundation of a great BI product. And these guys are gonna talk to that in great detail and then specifically how it works for every persona.

So when you're looking at all the things that Byron and Sean are talking to imagine in your own organization, what persona is getting value from that? And that's very much how the team thinks, who, what persona is this for? How do we make it more valuable for them ultimately to get all that data and insights out to their end users?

So I'll, I'll bring Sean up and he'll go through the rest. Thank you.

All right. Thank you, Tracy. Tracy was not kidding about the lights. It is, I cannot see you, but you can see me in that. That works

All right. So uh really quickly, my name is Sean Boone. I lead the product management team at QuickSight that is focused on enterprise BI capabilities. And as Tracy just described, um you know, we have all of these great uh generative AI capabilities that we're bringing to users now and we need to make sure that we do all of this in a secure way and in a way that scales.

Now, fortunately on the scaling part, QuickSight has this all covered for you, right? We're, we're, we're, we're, we're a serverless application service that you can uh you know, as you need more, it just we fire up the resources for you. And so the scaling part is really handled, uh we're gonna talk a little bit today more about the securing aspect.

And again, if we, if we look at the assets that we, that we're talking about. Um these are all of your data assets, right? You have everything from raw and transformed enterprise data to all of the output that your business users and your analysts use. That's based on that data. And you want to make sure that you have that, that, that those capabilities are available, you have the right experiences for each user, which means you have different experiences for each users. Some users will just get a report every Monday in their inbox. Others will look at a dashboard, others will create dashboards, but we want to do that in a secure and governed way.

Now, QuickSight comes with all of these capabilities to access data in particular uh within the QuickSight uh service and with uh capabilities from other services as well. So the first thing is we have this notion of share folders and that allows you to share QuickSight assets uh with groups of stakeholders in your organization.

Now, I really want to highlight um a few of these areas here. We're gonna go into more detail with respect to row and column level security. Uh but we've added capabilities to QuickSight that enable teams to be self service with respect to analytics. And so in some cases, what happens is the data lands with the BI teams and then the BI teams own making sure that it's secure. And so we have all of those capabilities natively built in to QuickSight.

Now we just recently announced a new type of shared folder. It's called a shared restricted folder and with shared restricted folders. What happens is when you put content into that folder, it cannot leave. So if you put data sources and dashboards uh into a restricted folder, they must stay in that folder. And so that's really great for cases where you have projects that people are working on and you need to minimize, uh you're gonna minimize sharing in those cases. It also comes with a new contributor role uh that ensures that the data cannot be shared outside, but that people can work together collectively in that space.

And then we also recently just announced um integration with IAM identity centers, trusted identity propagation. And this is a uh a capability that lets users log in seamlessly and have their identity from their session. And QuickSight transferred all the way down to Redshift where you can then have row and column level permissions applied there. So if you have direct query dashboards that you're using, uh you can use trusted identity propagation to use the row and column level security capabilities there.

All right. So while we're on the topic of data, um QuickSight can connect to data wherever it lives, right? We have support for on premises data. We have connect to data in the cloud and then we also enable you to connect to applications with data as well. And the items here that are highlighted in yellow are recent editions and I just wanted to call them out in case you're using some of these. So we added support for Snowflake and we added support for Starburst Enterprise. Uh we just recently announced uh support for Google BigQuery. So if you're using Google BigQuery, we now support that you can connect to Google BigQuery. And then we added support for Starburst Galaxy which is their cloud offering.

Now in general, QuickSight supports two types of connectivity to data. Uh the first is uh using a technology that we have that's called SPICE, which is our cache, a high performance cache that um that you import data into. And the second is what we call direct query, which means you can connect directly to a data source and query the data live. So you get the most recent data um that's in the database at that moment.

Now, SPICE offers high performance uh and has a super fast calculation engine behind it uh that we manage and scale on your behalf, right? So our goal with SPICE is to make sure that your dashboards render as fast as possible. And so we take care of the of of managing all of the service uh infrastructure needed to make sure that that happens.

Now, we recently uh have a few announcements we want to share with respect to SPICE. The first one is um incredibly important. So today, uh prior to this announcement, um if you wanted to purchase new SPICE capacity, uh you would actually have to go into QuickSight and uh specify the amount of uh capacity that you want to provision that's new. And what we've enabled now is this ability for you to auto purchase SPICE capacity. So it's a simple toggle on the UI, you turn it on and then whenever you need more SPICE capacity, we just purchase the amount that you need based on what your current workload is. So it removes the need for you to log in and do that check. Um you know, when, when you run out of space, uh we've also um with respect to scale uh improved our ingestion performance. So we're up to four times faster now, uh with additional parallelism that we've added uh with respect to uh ingestion.

Now, when it comes to managing data security rules, we have two different ways that you can do this in QuickSight. Now, the first one is you can manage them directly in QuickSight itself as part of the data set definition and that supports both SPICE the SPICE path as well as the direct query capabilities. Now to do this, you simply provide an entitlements table. Uh it can either be a table of data or it could be a flat file of data that specifies the information and you join that to your data and then you can specify your row and column level security from that from that uh methodology.

Now, when you do this right, this enables you to have dashboard readers who then are only going to see the data that they should see. So we check the entitlements table and we make sure that the query to the underlying SPICE engine only returns the data that those users have access to. Uh we also support the ability for unauthenticated users using RLS tags. And so we have customers that use RLS tag based rules uh to, to um restrict access to data as well.

Now, the new capability that we're adding, which is new, uh which is new to QuickSight is this ability to support Amazon Redshift with IAM identity center trusted identity propagation. That is a lot to hold in one in one breath. Um but this allows QuickSight dashboard readers to log in and see only the data that they're entitled to based on the row and column level security rules that are defined in Amazon Redshift and specifically right now, uh in Redshift data tables.

Now, um just to kind of level set here with respect to identity center. Uh this is the recommended AWS service for managing human user access to AWS resources. So this is the single place where you can assign your users and workforce identities uh cause and, and give consistent access to multiple AWS accounts and applications. So you can create and connect your workforce users and centrally manage their access across all their AWS accounts and applications. Uh and this means connecting the workforce users, you already have into AWS uh giving them a simple entry to access their applications and you can see all of the existing identity providers that we uh now support with identity center listed there on the slide.

Now, what's new to identity center? is this trusted identity propagation? This provides a streamlined single sign on experience for users of query tools and BI applications who require access to data in AWS services with no reauthentication required. So you log in once and we basically uh use your identity across all the participating services.

Now, the access management is based on the user's identity. So admins can grant access based on uh the user's existing user and group memberships, uh they can access other events that are recorded in service specific logs and we log everything in CloudTrail in terms of the access events. So then your auditors know what actions the users took in the system, which is really important with respect to governance.

And so with trusted identity propagation, users can sign into an application, they could pass that user's identity and the request to access data in all the AWS analytic services that are available with trusted identity propagation. So what we're specifically announcing here and we announced it, I think it was on Sunday is the ability for you to have direct query dashboards for Amazon Redshift data tables with QuickSight, we're also auditing in CloudTrail for data access events.

I want to be really clear that this initial release that we are supporting here is all about interactive human users working with dashboards. So this is a human is present, they're logged in and they're looking at a dashboard uh in the future, we are looking at being able to support more offline type use cases that you can imagine, right. So uh for example, if you're doing reporting those things were on a schedule, there isn't a human available at that moment. And so we need to support offline identities as well. So that's something to look forward to in the future.

Now, in terms of the identity center uh applications that support trusted identity propagation, you can see them listed here. There are over uh 20 services that integrate with identity center and specifically, and you can see the ones highlighted here in blue that support trusted identity propagation.

All right. So we've covered how you secure access to data within Amazon QuickSight. But there's another half of this story, which is how do we connect people to the data that they need and how do they, how do we connect people to the other to other people? They need to work with around that data that they're working with?

So, Amazon DataZone is our data management service that makes it faster and easier for customers to catalog, discover, share and govern uh and govern data stored across AWS on premises and third party sources. So it's really about connecting people through shared data and tools to help them drive business insights and it automates data and discovery and cataloging with machine learning.

So the types of things that you would do with Amazon DataZone, first and foremost is you need to, if you're looking as you're an analyst, you're looking for the data that you need, you need to get access to it, you need to be able to connect this data to the analysis tools and then you need to work with your team members on that data and those assets that are generated from that data. And you do, you do that in a way that's all governed and meets the governance requirements that you have in your organization.

And so the way that DataZone uh accomplishes this is that there's multiple components of DataZone. One of which is that you have a data catalog that's available to you. And you can search the data catalog based on the metadata that's been applied to the data. So people can, when they bring data to the catalog, they can give it additional information and descriptions about what is in the data that allows people to search for the data that they need.

And then there's a notion of projects inside of DataZone. So when I request access to data, that access has to be approved and then when it's approved, I have access to the data through a data project and there I can work with the data with my colleagues.

Now, we have a lot of exciting integrations between Amazon DataZone and QuickSight. And later today, there is a session that the DataZone team uh is holding where they're gonna talk about everything that's new with DataZone. And there's also a session today with a chalk talk where members of the QuickSight team are gonna be talking about QuickSight and DataZone uh as well. So I encourage you to check those out. Uh if you're interested in data cataloging uh and uh DataZone.

All right. So the next integration that we want to look at is how we're empowering business users to reach out and use more sophisticated machine learning capabilities with Amazon SageMaker Canvas and QuickSight.

Now this is part of a journey that we've already, that we're already on, right. Tracy was just up here talking about how we're using machine learning capabilities and generative BI to help people answer more questions uh to give people more powerful capabilities. And so what we're doing with Amazon SageMaker is we're actually bringing SageMaker to the QuickSight user, right?

And with SageMaker, um I have access to more powerful classification type models as well as prediction capabilities. Um and so, you know, for people today who have questions in the moment where they have data that they're working with that they know and they need an answer. Now, uh they can now use a SageMaker Canvas with QuickSight to get an answer now versus potentially waiting months for the data science team to come back to them um uh with an answer.

So I'm going to walk through a demo here really quickly of this inaction. I can get it to play.

All right. So what I'm doing here is I've got a dashboard and we're going to do some churn analysis. This is based on customer service calls. I've got a dashboard here in QuickSight. I've saved that analysis. I'm now in the analysis experience and I'm gonna filter down to the data that I'm interested in

So I'm going to remove the phone numbers that I don't need. And now I have the data that I'm interested in in the form of a table and I can build a predictive model. So I'm gonna launch the ability to export this data to SageMaker Canvas.

It's now preparing the data again, these, when you're building models, these are not things that are instantaneous. So our first stop is we're going to get the data over to SageMaker Canvas and we're going to build a model to predict churn.

I'm going to create that model. And again, this is going to take a little time to run. I can, I can specify which columns that I want to predict. In this case, it's churn, it's going to validate the data. It's gonna help me understand some of the correlations with respect to this data in terms of which uh fields are, are, are are are potentially the most impactful with respect to defining churn. And again, I'm doing all of this without necessarily being a trained data scientist, right? I know enough about churn. I know what impacts it. And I and I'm able to access these capabilities of SageMaker Canvas.

So now we're building the model. It's telling me 97% accuracy. That's really high accuracy. That's the beauty of demo data. And it's showing me all the column impacts and then we're going to go ahead and do a batch prediction and deploy and we're going to send the results to QuickSight.

And so this is the key thing here. We already have support today. If you want to connect to SageMaker models, you can do that if they're already in your organization. But in this case, we're building one on the fly. And now QuickSight is telling me that it's ready to be used.

So I'm gonna edit a data set, pick the model that we just chose. It's going to ask me for a schema. This is a schema file that's automatically generated on your behalf. We're gonna upload that schema file and then it's just gonna make sure that I'm matching the data set with the fields that are in the model. I'm going to give it a name of churn prediction where you go ahead and prepare the data.

And so now we've published that data set with the model capabilities and then I can go right back into QuickSight and work with the churn prediction, build a dashboard, we can see which uh which customers are predicted to be churned and what the economic impact of that churn potentially is. So that's all new SageMaker capability, SageMaker Canvas capability that we've brought into QuickSight.

All right. Now, the next aspect of scaling now there's a lot of small print on this slide and I'm not going to go into a ton of detail here because our speaker from BMW is gonna walk us through a lot of these capabilities that they're using. But our goal is to help you automate everything with respect to your deployment. And so we have APIs that let you, uh, that let you move things from dev to staging to prod with features like assets as code and access, uh uh uh uh uh uh access as bundles. And then also we have auditing and logging capabilities that help you keep track of your account, uh with AWS and we use CloudTrail for uh for those capabilities.

All right. So now, QuickSight, we're gonna transition now to our customer. Uh now, QuickSight of course has been engineered right from the ground up with an emphasis for speed and performance to help you get insights into your organization as fast as possible. So with that emphasis on speed and performance, it's really only fitting to have a customer like BMW known for producing high performing precision engineered luxury vehicles in our session today.

So it's my pleasure to introduce Ruben, who's the product owner for Cloud Data Hub at BMW, who's leading a largest data platform that drives QuickSight for operational excellence at BMW. So I have Reuben. Thank you.

Thank you Sean for the good introduction. Um I'm Ruben, I'm working in BMW and there, I'm the product owner for Cloud Data Hub. And first of all, I want to give you an idea how it feels like to work with data at BMW.

Imagine a future that is digital and boosts the bond between people and technology, a future where humans and cars are ultimate companions yeah, where we drive things forward together because behind every digital car stands, a digital company, we re invent the way our customers experience, our company, we digitalize our processes and the way we work and collaborate, we rethink the way we use data to unleash our full potential.

Yeah. So um Tracy, you said, how can we make data? How can we bring it to the people? And um Sean you were talking about also data governance and these are both points that we also need to solve a BMW. So this is why we built the Cloud Data, so called Cloud Data Hub, which is our data lake and the biggest uh um big data. Apart from that, we have a BMW to solve these questions, how can we bring the data to the people and how can we make them accessible?

So I want to show you today what is it called Data Hub? Then how, why did we integrate QuickSight into the Cloud Data Hub then? How did we do that?

So first of all, the Cloud Data Hub which I'm responsible for, it's the main data platform that we have for BMW, you can imagine BMW is a very big company, approximately 100 and 50,000 employees. We have many domains like production, we have customers, we have vehicle data. So all this data, how can you bring it to the people? And this is why we build the Cloud Data Hub.

So what is the Cloud Data Hub about? We have the data platform which is fed by over 460 data providers. We call them. It's a huge platform. So we already have 10 petabytes of data. Just imagine all the telemetry data coming in from the cars, the customer that are coming in. So it's very big large scale.

And with the Cloud Data, what we achieved this, we really broke up the data silos in BMW. So we have over 30% reuse of the data. What does it mean production brings data to the Cloud Data? Then the sales departments, they can use it or even production, they need additional information from our R&D departments, they bring it in and then they can share it. So this is why and how we broke up the data silos and BMW the Cloud Data Hub. It's our big data lake built completely on AWS.

Yeah. And so and in the center of this, we also have data governance. So somehow you have to c at the data, you make to have a really data products that we have. So the consumers know, ok, what's the data product about? Can I really use it? And this is what we have in the center of everything of the Cloud Data is the data governance. We really embedded this whole data journey into our organization.

We have 360 data stewards who manage the data sets. They are in the business. So it's not only a t project, it's really also the business involving we have more than 1000 data assets, we call it like the data products and in 17 business objects like retail data, like production data, like R&D data and so on.

And what we also have is where do we create the business value? We have over 20,000 users every year and 6000 monthly active users. We have over 900 use cases working on these data assets and generating already €1.9 billion value for BMW with secure data.

So what is the concept of the Cloud Data? We combine best of both worlds with data mesh and data fabric. So we have to enable our data providers that you can see on the left side and our consumers that you can see on the right side, they should have most flexibility to really create the highest value for BMW. But what we also need is somehow a standardization, we're a big company. So it must be clear that the data products that the consumers can access, that they can really use it.

So can they trust the data? This is why we build our data portal as the main entry point for our consumers and our providers. So for our users to work with the data.

So what does it mean providers? They will ingest the data from different data sources to the Cloud Data Hub and there it will publish to our data catalog. So it's mainly the AWS Glue Catalog customers or users consumers then can search the data. They can see information about the lineage, where does it come from, how was it processed and who else is using it? And they can even preview the data. So they can make a select statement on the data for example, and really get insights into the data so that it's really fit for use data. Can I really use this data for my use case?

And then we have on the consumer side, they will create the business value, they will answer business questions that we have. But the challenge is still on the left side. And on the middle side, we have really good understanding what's happening there, especially on on the middle side, we are controlling it. But what happens on the right side with the consumers use cases, we are still lacking a little bit, little bit of the transparency and we have many um analytics and reporting use cases. They use QuickSight. And this is why then we were integrated QuickSight into our data port into our Cloud Data Hub to get more transparency about what's happening there and share the insights.

So as I just told you what we did is with the Cloud Data, we broke up the data silos, you can provide the data, you can manage data, you can manage the access, you can explore the data and you can consume the data. And with QuickSight, what we want to achieve is we want to get from data sharing really to insight sharing.

So we enable or integrate QuickSight into our data portal where you then can with one click, you can enable QuickSight. In the data portal, you can manage the users, you can manage your dashboards and then you can also share the dashboards.

So how did we do that? Luckily, um Sean also told us we have many APIs. So what you see here is our data portal, some of you perhaps know DataZone. So we are having a similar concept here like with DataZone. So we have one interface, one UI to govern our data and to make it accessible.

So here you can see our data portal, I can log in there, every BMW employee can log in there and explore the data. So you can just have a google like search experience and search for vehicle data. For example, you can see what are all the data assets that we already have? What are the providers, the data providers who provide the data and what are the consumers?

And our goal is now to integrate QuickStart here into this data portal. And where does that happen? It happens in our use cases, we call them use cases. And what you can see here is um the the view of one of the use cases. So it's the vehicle analytics use case. You can see that the name there. You can see like the description, short description on the top, right. You can also see what the meta data score. So is the metadata score already well maintained? You can see how many data assets did they consume and do they also provide data assets? And then there are some tabs like the summary, you have data sets, you have the dashboards and the dashboard is our new capability.

Now here, what you can see here on the bottom of the screen, the user, they can just enable QuickSight. And so then what happens? So what happens when the user clicks on a QuickSight? So we don't want that they have to log in into QuickSight console, click on activate. We don't want that. So we want a seamless end to end user journey.

So what happens is the user logs into the data portal?

First, it's checked - what roles does the user already have? So we built our own identity store.

So then the user clicks on QuickSight and then a CloudFormation stack is getting rolled out.

So what does it do?

First, it subscribes the account, this use case account to QuickSight.

Second, very important we have to remember it for later - it will create namespaces. It will always create a default namespace, but it will also create, we call it a guest namespace. We will need the data for the data for the sharing of the dashboards.

Yeah, it will also create like the SSO for the for the account and the use case account will already have some users. It will also create the QuickSight users for these users in the account and then writes it to the to the identity store.

And then this is how it will look like when the QuickSight site is enabled for this account. So just just showing on the bottom, it's "Open QuickSight Site".

So now the users, they can start and use QuickSight right away.

So let's just imagine now they start using QuickSight, they build dashboards. And then the next thing is we also manage the user access via the data portal.

So what you can see here is how we request the access. So let's assume the account is already signed in, signed up for QuickSight. And now I want to get access.

So what you can see is here our request form, how you can get the access. And we have now two new roles, the QuickSight Reader and the QuickSight Author.

Also for the for this account, you can fill out the form, submit the request and then the admin of the use case account, they can see all the requests here. So you have here like two examples, one Author and one Reader request, they can grant the access or they can also decline it.

But let's assume now they, they grant the access.

Then again, what happens? And luckily, as I said, we have all these APIs like Sean told us or showed us. So what happens is when we, when the access request is granted, we call again, QuickSight API to create a user as it's eventually consistent, we then have to do a get user. So to really check if it was created or not. And then when it's created, we write this user to our identity store and then um the the user access is completed and then comes our main feature.

So this is now really the core of the whole integration of QuickSight into the data portal. How do we manage the the dashboards in in the data portal?

So here, what you can see is that we have again this use case they created now some dashboards and you can see all the dashboards that created in this use case. So this is new now for BMW.

So every employee in BMW can see what are the insights the your colleagues created with this use case? Is it interesting for me or not?

And what you can see is here now the list of all the dashboards they created like the dashboard name, who owns this dashboard? When was it created? And you can see is it public available or is it restricted?

So what does public available mean for us? Every user of the data portal so we are talking about 20,000 people can access this dashboard. So we can really share. Now um the insights that we have and they don't need to know that it's QuickSight or not. They just, they are first interested in these insights that are generated.

So this main feature, how does it work? Now, it's getting a little bit more complicated, but I think we can get through that.

So again, it's all API based with QuickSight. So we, we need to do two things. Now we have to list the dashboards and we have to check does the user have access or not? And the APIs it will be, it will take too long for the APIs to QuickSight.

So this is why we decided to build a dashboard cache. It's what you see at the top and to understand this picture, let's start at the bottom, right at the QuickSight icon.

So the users, they are working in QuickSight, for example, authors they create dashboards, they update dashboards, they delete dashboards, they change some permissions.

Then what happens is it will trigger events. We will capture these events with EventBridge and then we store this to our dashboard cache. So which just says, ok, what's the dashboard, what all the dashboard details and what are the permissions assigned to that?

So that when the user access our use case, it's let's say instantly there, it doesn't have to call the QuickSight API.

So then we have the list of all the dashboards, then we need to know. Ok. Does the user who just use the use case and page? Does it have access or not? And who knows that this is QuickSight.

So here we can then ask QuickSight. So checks the permission of the user. And does the user have access to this dashboard on it?

Then this is why when we are calling then the Quick APIs to get additional information about the permission.

So this is just in general, do we have access? Yes or no.

We also need to determine now what does public mean and what does restrict mean? Because normally when you work with QuickSight, you are working in an account. So you share your dashboards in your account. This is normally in the default namespace and remember we created this guest namespace and this is how we enable this public dashboard.

So at the top, you can see when the user clicks on "Publish Dashboard" in the data portal, this dashboard will get published or will share it with the guest namespace and with a default namespace. And this is that how our users can access dashboard, let's say publicly and they don't have to be part of the account before how we do that later.

And then the the other way is like the normal way, how we are used to that in QuickSight it's like the invite users to groups, uh uh invite users or groups to a dashboard like the normal way inside a QuickSight account, you can grant access to specific users or to groups in this default namespace and this is how it will show for us. Ok? It's a restricted dashboard and either you have access or not.

So either it's here green or yellow. And if you want to get the access, you can manage or you can request the access in the data portal. And then what we just saw before get the access.

So what's really new here is also this this public. Yeah, so that we have the guest namespace, we can share the dashboards there. Um let's say across the across the QuickSight accounts and it's all visible in the data portal.

Then the last thing is that we have, we can also share the dashboard, we can see the dashboards in the data portal. So our users that don't have to log in to the QuickSight console so they can go to one of the dashboards they're interested in.

Let's assume. Now it's a public dashboard, they can click on "Open Dashboard". Then in the data portal, they can see it in the data portal embedded.

And what happens here is the user sees on the use case page, the list of the dashboards, the the user clicks on "Open Dashboard" and then we will ask the QuickSight API again, give me a link back. So this embedded link so that we can then can embed the dashboard into our data portal.

But what is what happens if the user does not yet have access to this account? Yeah, if you want to, to view a QuickSight dashboard, you need to have access to the account, right?

So this is then when how we use the guest namespace or remember the guest namespace.

And so if you click on the public dashboard, we will on the fly, create a user for this user and the QuickSight account and then we will get the embedded link back. So this is how we will manage the access for the user to this QuickSight account.

So this is how we integrate QuickSight at BMW in our cloud data, our main big data platform.

And um I hope it gives you some good ideas how you can also use and leverage the the QuickSight APIs and then also benefit from all these nice features that we saw today.