Inspire the next on AWS through hybrid cloud and data technologies

Good evening everyone and welcome to this session, Inspire the Next on AWS through Hybrid Cloud and Data Technologies. We'll get started here. I know we don't have as many people as I probably expected, but nevertheless, I welcome the people who are here and I'm really looking for a closer dialogue and more interactive session since we have fewer people.

Ok, here's what I'm going to do. I'm going to present with Prem from Hitachi Digital. We also have one of our very important customers, Pensy here. Cran will be joining and of course, we have our AWS partner David here. We'll be doing the session again. I think as we have, I would like to hear your views also at the end.

Now, let's get started with some real life scenarios we see around data, where for the data driven, for those who understand clarity is currency, believe progress requires precision and neutral is not an option - we're for the data driven, the ones who can't tolerate failure, who won't put up with downtime or allow access to just anyone. We're for the data driven who act on insight instead of instinct, bank on privacy instead of probabilities, and rely on resilience instead of reaction.

We see ourselves in the obsessive, the incessive progressive, and the meticulously engineered. We enable the incredible, identify with the analytics, and are synonymous with the mission critical. We know what it means to be data driven because data is in our DNA. We were born industrial and we breathe digital. We speak predictive analytics so you can keep supply chains moving. We bleed in-store and online insights so you can accurately predict customer preferences. We sweat security and digital privacy so you can turn complex regulations into competitive advantage. We break down barriers and eliminate silos so you can go from data rich to data driven because it's clear the future belongs to the data driven.

Ok, so what I'm going to talk about is how data is important when it comes to generative AI. That's going to be the theme of my presentation.

There is absolutely no doubt that a data driven strategy differentiates you from your competition. However, all of us know that there are a lot of challenges when it comes to data. Data is highly distributed, siloed, probably on older legacy infrastructure too. Some of the data is also redundant and people don't know exactly what they have and where they have it. While these are the challenges, with data being in motion, one more thing is that data is doubling every 2 to 3 years. And while it's a challenge, it's also a great opportunity for any enterprise to derive insights from this data and convert it into insights that can drive differentiation.

With all that is going on with AI, I think all of us will agree that the data is more important than ever before - not only the publicly available data, but your proprietary data sets and private data sets. They are so important for you to truly derive value out of gen AI.

So let's see what we noticed over the past few months. AI has been there for the last 50 years. However, ChatGPT - within two months, it reached the number of users that took Netflix 10 years to reach. I think what is so different about it is that ChatGPT exposed the power of AI through a very simple natural language interface and it got popular from kids to grandparents.

I also attribute this success to what has been done over the years. I think it was possible because now we have an enormous amount of data that could produce these kinds of results. Also, all the advancements in transformer models and the parallelism made it possible. And last but not least, the amount of compute we have today made it possible.

As all this happened, one of the things we also saw was a lot of great examples of LLM being used for creative content generation. We saw really impactful use cases for customer support and conversational usage. Apart from this, what amazes me the most is how we can access these LLMs through APIs. I think that's a paradigm shift in itself - we are in a phase where we are going through the democratization of AI. A few months or years back, if someone had to do something with AI, they needed data scientists and a lot more expertise around AI than probably they need today.

While all this is happening, it also got enterprises interested in what can be done with gen AI. In fact, according to McKinsey, gen AI has the potential to pour trillions of dollars into the economy every single year.

So what does it mean for enterprises? That's what I'm going to talk a little more about. I think these LLMs or gen AI have tremendous potential to create content - the generative part is very impressive. However, I also have to say it's one thing for these LLMs to write poems or create creative content, and it's another thing to use it for mission critical enterprise purposes. I think what matters is you have to think about security, explainability, lineage, keeping biases under control, having the right guardrails in place - because you don't want to expose your enterprise to expensive legal battles in case of disputes. It's very important to design these things right from the onset.

Another thing we all know is that training these LLMs is not trivial - it takes a lot of compute, it's very expensive. So how do you do it in a way that cost doesn't become prohibitive later? Being Hitachi, another thing we worry about is sustainability - training these models has a lot of carbon emissions. So how do we do all this in a way that we are also socially responsible?

Here are some of our observations as we are experimenting with it. I would like to hear what you all are seeing, if time permits. One thing we are seeing is that it is more effective to have smaller, fine-tuned models combined with NLP and domain-adaptive pretraining, rather than one large model. We are also seeing that having multiple models is more effective than bringing all data to one place - having virtualized data access wherever it is, on-prem or cloud, is very important. And last but not least, having an overall observable system that can track, monitor all these LLMs, different behaviors, accuracy - I think that's very powerful.

So that's what we are doing at Hitachi Ventures - we are developing an easy to install, deploy and use one-box solution. There are three components:

1. A semantic data layer that provides unified data access for AI applications and gets enrichment from NLP and KGs.

2. Hardware for storage and compute that can be used to train models efficiently with a parallel filesystem combined with object storage and intelligent data optimization.

3. A simple NLP interface to develop companions - think of them as gen AI agents using natural language. It's possible to debug, fine-tune and track them.

Our goal is to target domain experts who may or may not have ML experience.

The underlying architecture has the semantic data plane at the top, providing unified access to data wherever it resides, with enrichment from NLP. What's different is we are complementing NLP with knowledge graphs and relational databases too. For example, for a simple query about inventory levels, we can convert it to SQL and get results through the semantic data plane's virtualized access. It doesn't matter where or in what form the data resides - we can handle objects, files, databases, graph data.

This is complemented at the bottom with our hardware - a high performing filesystem combined with object storage that can handle different workloads and scales. We are intelligently optimizing data movement and cost/control.

Lastly we have Companions - multiple specialized LLMs versus one large model. These can discover each other, figure out specializations, and delegate tasks. This helps address issues like hallucination - we are grounding models to answer specific tasks and teaching them to delegate if they don't know.

Managing many gen AI versus one LLM is complex, so you need systems for discovery, fine-tuning, monitoring.

I hope this transcript is easier to read and comprehend the key points from the presentation. Let me know if you would like me to clarify or expand on any part of it.

So that's, that's the downside of it. But I think it's still it's work. Uh other interesting scenario, think of it this, right, like uh these companies cannot recite, uh all of them cannot decide at one place. I mean, some of these companions need to be on prem some probably makes sense, more sense for them to be in the cloud and sometimes probably makes sense for them to have be near, near uh like they could be iot devices and they are somewhere out there near the edge.

So the fact that they are highly distributed, but they can still talk to each other, communicate with each other and figure out, I think that we are finding and our customers, I actually are finding it very interesting proposition.

Now with that, I think other thing that where we got to start with. Uh Hitachi does have an edge over others in terms of our domain expertise. So we are very uniquely positioned with 100 plus years of data from industrial sector. So we are also using that to, to develop these companions and make it available to our customers for their use.

Now, with that, I will invite Frame, we will talk more about some of our what are some of our customers are doing?

Hello, thanks, be uh and thanks everybody for being here. I'll, what i'll do is the next 10 minutes. Probably i'll walk through some of the customer experiences that we are today working with right in this ecosystem. And i know Bharati took us through Gen AI so i don't want to preach anymore. Ja i all of us know where we are today. So we're in the world of generations. It's generative. But how are customers getting value from ja i, so i wanted to spend a minute on that, right.

There are three key areas that customers are focused on and when we talk to our customers about use cases and i'll show you some use cases that we are actually implementing with our customers. There is the customer experience that they want to improve for their end customers, which is where all the bars board related activity and stuff is happening, right? Where you are enhanced, the customers are enhancing the way their end customers are interacting with their products with their business.

Second is accelerating their own enterprise transformation. So be it software engineering, applying code generation patterns and so on and so forth where they can drive more efficiency within their organization.

Third, and we'll continue to talk and i'll invite a customer of ours shortly to come and speak about this as well as where they are starting to think about their own domains, their data. How can they leverage that to create additional revenue streams? How can you monetize the data that is sitting with you that you have collected and stored for a long number of years, right?

So that those are the three key areas where customers are generating value using ja i. Now, when you think about the design patterns, there are four key patterns that customers use again in the order of complexity and the amount of time you spent prompt engineering, all of us now we can go to open a i and type whatever we want, the better your prompt is the better your answer is nothing to nothing great there. But that's, that's, that's becoming a field that's growing.

Second is R which is a model where 90% of our customers currently fall, which is how do i leverage the data that i have and then leverage an open m or am that uses uh that's already been trained on a huge corpus of data and still get more specific answers.

Third is fine tuning again, this is a use case we will speak about in a bit more depth. Uh fourth is actually building custom domain models and i won't water the slide down. But here is a set of use cases that we're actually working with our customers on be it financial use cases where they're using this for wire fraud analytics, uh logistics, where they're using it for predictive maintenance, fleet awareness repairs in the automotive space in the aerospace, in health care.

So enough examples where people have started to leverage c right? But what's critical in all of this um is there is a pattern to apply this? There's one way we start to think about what do we want to do with this because people are still struggling with, hey, when i apply j a watch my roi, i know it does some fancy stuff, it accelerates. But what does it mean? Am i able to take out wastage from my system? What, how much money am i spending on this? Uh ja i, so that when i, when i recover benefits, what is my roi?

So envision evaluate is around co creation, which is where we spend a lot of time with our customers. And also they spend time with us to go create something that makes sense where we start to then understand the impact of it both from a business standpoint, from an roi standpoint and bar the alluded to using smaller models rather than larger models, all of this is use case specific, right?

Second is execute and then you continue to be nimble and evolve. Now this is a bold statement and i'll explain why i made the statement. I said no ja i without a i probably i think the best light in this deck because it goes against what all of us are doing. The idea is not to say that there is no j a but in all the examples that i showed in the previous life, right? Be it predictive maintenance, be it wire fraud analysis, be it a simple door pricing that we're working with a customer on be it any field, right? Aerospace, where we think about engine uh health at real time, there is an underlying model and that gets way enhanced by applying j a at the topmost layer.

So the underlying model. So that is the way enterprises are really using gen a i to derive value, right? It's not about the bots, bots is just the surface layer, but the actual value is in the model that you build on your own data, that level that helps you create new revenue streams, that helps you unleash the power of your business and good value to your end customers, right?

So that's the idea when i said no ja i without a i. Again, there are multitude of possibilities with this. A lot of existing use cases are getting enhanced with j and one such is the case that we're working with a company called Penske Corporation where we have worked with them for over years, working on their predictive maintenance, uh suite of apps, right? And i'll invite servant from penske onto stage to talk more about what we have done together and how they're thinking j a will be applied to evolve this into the next.

Thank you so much. Good evening, everyone. Hitachi is such a great uh partner and together we have solved many complex problems, uh using a iml. So i'm just very happy and excited to be here. So thank you for inviting. Um, implementing a i is a lot more complex operational a i requires much more than just good data science. But i think that's a topic for some of the day today. We're talking about gen a i and if you are as curious and as excited as i am about this field, i'm thinking you'll be able to get something out of it.

Um if you are an it or a data practitioner, i'm very, very confident that like me, you're getting pinged on a daily basis from business partners. Hey, what are you doing on the, in the, on the gen e i front? Right? And if you are giving that deer in the headlights, look right, then i, i think through my slides, you'll be able to get something meaningful out of it. If you have it already figured out, then please see me after the presentation because i would love to learn from you.

So with that note, um let me start by bringing some business context to what frame was mentioning. So i'll start by talking a little bit about penske. So you may have known us through our yellow trucks that you see on the road, but we are a lot more than that. Uh penske is a closely held diversified group of companies and we are in retail and automotive sector. Uh collectively, we have $35 billion plus just penske automotive, which is in the new and used car sales. It's $25 billion in revenue.

We also have premier truck group that's our truck dealerships, uh penske racing, that is our other very, very visible uh part of our business. So, and this year has been a great year. I mean, joseph new garden won indianapolis 500 ryan blaney won uh nascar cup series championship. So we're all very, very happy, very, very excited about it. And right before covid, penske acquired um indiana motor speedway and that is what became penske entertainment.

I am part of penske transportation solutions group that's uh in the blue box at the bottom there. And we are in truck leasing logistics, uh rental, truck rental, and contract maintenance businesses. Uh collectively, we own a fleet of more than 445,000 trucks and we take care of our customers through our 44,000 employees, very dedicated and more than 900 locations.

So that's a little bit about penske. We started working with hitachi on a i front uh back in 2015 and guided repair. Uh the, the use case prem was talking about was the first use case and we went live in 2017. Then we started working uh on proactive diagnostics. So we have our trucks connected to our data platform and we are processing more than 200 million messages a day.

So we take all those messages, run them through this proactive diagnostics engine. And this tells us when a truck is going to fail, depending upon if that is going to happen in a couple of hours versus a couple of days, we take different actions. Guided repair is all about predicting what's the best repair once a failure has already occurred, once a truck is already in our shop, right?

So these two solutions have been extremely powerful. Let me share a very interesting fact. I don't know if you know this, but in the us, 70% of the freight gets carried by trucks. That means approximately seven out of the 10 things. If you look around in this room have been on a truck at some point of time. So that's the power of these solutions because these solutions are helping us keep our trucks on the road and making sure our customers are happy.

So with gen a i, uh we are looking at that can we expand our guided repair solution so that our technicians can interact with it dynamically, right? And that's where the power lies. Uh let me also share this that we started looking at conversational a i, which is the previous generation of generative a i. Back in 2018, we implemented some solutions back there. Uh like for rental reservations and service tax, those solutions were successful, but we are running into some challenges.

We knew that these smaller models uh they have a lot more growing to do. So when chad gp t came in last year, i know a lot of people in the world were shocked. Some of us were excited and we were like, finally, finally, someone has solved this problem, which we always knew existed. So for us, it's very, very exciting because i think we have the rest of the framework built and for us as soon as we can swap out those smaller models with these larger models, i think we'll have a workable solution.

So without further delay, let me talk about how penske is approaching generative a i. So on the left hand side at the top, you see amazon bedrock. Um so we have a great aws enterprise support team and collectively, we, we did some proof of concepts. Those proof of concepts have turned out to be extremely, extremely successful. The feedback from our sme s has been great. So we settled on amazon bedrock and that is going to be our platform to pull in these external ll ms, which we will not be training will be just using them, right?

So that's first input, then the second input you have is our internal ll ms. So at some point of time, i do believe that we'll end up having either our own llm, which is going to be focusing just on transportation domain or we'll end up picking one. The reason why i'm very excited and very confident about this is what uh bloomberg did with bloomberg gp t, right? Because that's an llm focusing just on the financial domain.

So i do believe that we'll start seeing uh some of the, the ll ms, you know, come in these other domains and definitely in transportation, the third input you see there. And i think uh prim was talking about this is gen a i by itself will not achieve its full potential, it will achieve its full potential when you'll be able to combine gen a i with other a i, right, with other inputs. And that's the third um thing you see um at the bottom and we are planning to build this orchestration platform in the middle using which we will be able to deliver a digital unified gen experience to our knowledge workers.

And right now, we are focusing on intelligent search like the knowledge base as some of our initial use cases, customer service. I think that's a great example. That's a great use case because today we see a lot of emails, customer asking for things using gen i you can interpret and maybe create a response in some cases, maybe send it back automatically. In other cases, maybe have someone review it, right.

So it really cut down on the time to response. Uh code generation and testing is other area. And uh we're building our road map for other gen a i items. So this is my last slide and this is what i we put together at penske that this is how you can build a na i foundation in your enterprise.

So um what you see on the left is this wheel wheel of a truck. Well, we are a transportation company. I think having a, having your gen a i strategy is the wheel of a truck

David: I'm here to tell you that we are at the end of the beginning. Gen AI has no moat. This notion that you can deploy a generative AI solution and have a competitive advantage against your competitors - it doesn't work like that. Essentially we are all on a treadmill to keep moving forward, to deploy this technology.

It's not enough to deploy one model. It's not enough to deploy a few models - we have to keep running. So welcome to the pace of innovation.

From an AWS perspective, we're big believers in democratization of these tools. What we saw in the past was generative AI has been around for a while, right? Gen AI is not new. But it reached a consumer moment when we saw the 100 million users of ChatGPT.

AWS is providing tools at every layer of the stack. In terms of people, you have to upskill your workforce - it's not just data scientists anymore. You need business analysts, marketers, graphic designers all using the tools.

In terms of process, one of the things customers often struggle with is not the technology - the technology isn't that complex. It's the processes to deploy it at a company. You need to think about governance, get sued because of large language models, fit with data practices. Customers say this is confusing, I just want a solution. I'm here to tell you the tooling is here. AWS is coming to the rescue with tools.

We think about it as three layers of the stack:

1. Base layer - Custom silicon like Inferential and Trainium for low cost inference and training. As you think about your own custom LLM, this silicon will matter.

2. Middle layer - Bedrock and SageMaker. Bedrock is a serverless vending machine, put in tokens and get answers. SageMaker gives more control over models. SageMaker gives access to a long tail of models - thousands of models.

3. Top layer - AI apps like CodeWhisper. Simple input and output. In AWS fashion, we provide building blocks and partners help build solutions.

Partners like Hitachi bring hard won experience to help on the journey. Most companies could do it themselves, but take advantage of partner knowledge. There's no compression algorithm for experience. The 10% of implementation distinguishes situations and partners.

Keep eyes open for Swami's keynote. Thank you all for coming and for your time.