Scale and accelerate the impact of generative AI with watsonx (sponsored by IBM) (IBM)

My name is Edward Calves. I'm the Vice President of Product Management for IBM Watson X. I'm going to be joined by Cy Chaudhry for the second half of the presentation from Meta. And of course, we're gonna be talking today about uh JA I but before we do that a little history,

Today's IBM is focused on two things, hybrid cloud and artificial intelligence. Hybrid cloud uh really came to the forefront following our acquisition of Red Hat in 2019. But artificial intelligence uh has been a major part of what we do for a very long time, 25 years ago, perhaps before some of you here were even born. Uh IBM Deep Blue beat Garry Kasparov, widely considered to be the best human chess player of all time.

About 15 years later, IBM Watson became a Jeopardy champion demonstrating impressive natural language understanding and semantic search. If you haven't seen those videos on YouTube, I recommend them. They're very, very exciting.

Four years ago in 2019 IBM's Project Debater, a project from IBM Research in our lab in Haifa, Israel uh debated a human and demonstrated impressive uh speech generation with clarity and purpose and even incorporated humor as appropriate.

And on any given Sunday, if you enjoyed the fan experience at Wimbledon, the Masters, the US Open, ESPN Fantasy Football, you've experienced Watson X delivering uh data driven insights, predictions and AI generated commentary.

So what this history illustrates to me is that the technology has gone from the deeply experimental in an antiseptic and controlled environments like a chess match to obviously the real world and pervasive in the background and a massive scale that we're all basically experiencing almost every day.

So let's dig in. Uh at IBM, we're focused on AI for business. And when you think about the purpose of AI in business, overwhelmingly, it's to drive employee productivity and that's where we're seeing a lot of the early wins in the implementation of the technology.

It's driving employee productivity in a range of functions starting with back office functions like HR or IT support as well as front office functions like generating content for sales and marketing.

Now the technology is also getting put in front of customers directly, right? And we see that in the customer support uh use case, so enhancing the proverbial chatbot to be more relatable, adaptable, flexible, more humanlike and to be able to accommodate more and more questions and answers that customers can self serve. And that drives agent productivity as well as a better support experience directly with customers.

And applying JA I to IT itself has great promise of enhancing developer productivity. Of course, you can never have too many developers uh and also uh really automating a lot of processes today that uh that can be very time consuming and this is just the tip of the iceberg. You know, we see new use cases coming up every single day. I've learned about a few of them uh this week here at Re:Invent.

Um one of our partners developing a digital concierge to basically self-service in the hospitality space or enhancing uh procurement processes by managing RFPs and responses. Uh summarizing them, for example.

And at IBM, we drink our own champagne, which is much better than eating dog food. That's always a good one. Um and we've been using the technology uh internally for a number of years and we've really validated the business value uh of of the technology in certain areas starting with uh HR.

So today at IBM, with almost 300,000 employees around the world, the front door to most of our HR processes is a, an assistant, an automated assistant where we can self-service information about policies and processes as well as execute hundreds of transactions.

Now as a regular user of the tool, as a manager, regular user of the tool, uh I can tell you the experience is excellent and it's really what kind of keeps us coming back and we've already been investing in this for for many years, right?

So the, the productivity enhancements that we've been able to identify 40% improvement in productivity. In HR of course, that means you can do a lot more with, with less while still delivering a great experience to users.

In customer service, we're incorporating uh Watson's Assistant. Watson's X Assistant into that process and it really delivers our uh customer support agents, superhuman knowledge about the thousands of products that IBM offers the documentation uh as well as the history of previous support cases that have been opened and resolved over time.

You can imagine how hard that must be to manage without a technology like this in place. And next year, we will be putting this in front of customers directly. So customers can also self-service and benefit from that superhuman knowledge in customer service.

And in the area of application modernization, we've focused initially on Ansible and Cobalt where we're doing, Cobalt, cogen and Cobalt refactoring to Java. Not only is this a great productivity enhancer, but it really addresses the skills gap that exists in some of these older languages uh in the market.

So, you know, the few skilled resources that are available can obviously be much more productive. When we think about the considerations of JA I in the enterprise AI really in the enterprise. Uh we've developed four guiding principles which really inform our product strategy and our approach to the market.

The first one is open, uh we think that open source is essential to the development of this space, the development of the technology, particularly at the stage that it's in where there's so much community innovation happening.

So not only do you benefit from that community innovation, you also benefit from superior transparency in the technology and superior project governance. Because the direction of the technology, the road maps is being informed, not just by vendors but also a broad and diverse community of users.

What are we talking about that in more detail with me, I in the second half? Targeted, like I said, we are focused on AI for business, right? And there are very specific requirements that businesses will have, particularly as they move away from initial phases of experimentation and into production deployments.

Things for example, like price performance, which may not be really important when you're just doing some science experiments. But when you're starting to deploy the technology at scale throughout the enterprise price performance becomes very important, things like abstractions, right?

So you we know that we're going to have a more heterogeneous hardware landscape in the future. And you know, delivering abstractions to be able to enable customers to optimize across those different hardware platforms is another future consideration.

Governance of course, which we'll be talking about in more detail is another important enterprise consideration where IBM is really raising the bar in terms of how we implement the technology ourselves and the tools that we deliver to our customers that takes me to the third point, which is trusted, trust is the license to operate in business, right?

You cannot operate a business without trust and it takes a long time to build and it can be destroyed instantly. At IBM. we've been earning our customers trusts for decades and we continue to do that every single day. We take it very seriously in the way that we implement the technology and the technology that we provide our customers.

So trust is really built in to the product itself. It starts with the foundation models that we're developing the processes that we take the care that we take in selecting the data and in building those foundation models so that they're explainable and uh transparent.

We're publishing information uh more so than anybody else in the space about the data that goes into training the models and these, these are the ways that we earn the trust uh in in the technology that we're delivering.

Finally empowering. We know that our customers want to make the technology their own, they don't want to be consumers. Uh just, you know, sending API requests to a model, they want to make it their own, with their own data, they want to incorporate it into their own business processes, they want to co locate it with their data and applications. That's what Watson X enables is really allowing customers to become co creators of the technology with us.

And of course, Watson X embodies these principles and provides customers with everything they need to be successful as they, you know, continue down this path to make AI work for them in their business.

So let's look at the platform in a little more detail. It's comprised of three components. Watson X.AI, Watson X.Data and Watson X.Governance.

So Watson X.AI is our studio for AI builders to train customize and deploy uh machine learning and foundation models. Watson X.Data is a fit for purpose lake house. So it is a great repository to unify and enrich data for AI as well as for other analytic uh domains like data engineering and business intelligence.

And Watson X.Governance is our framework for governing the end to end model life cycle for monitoring the quality and performance of models and for ensuring regulatory compliance.

Watson X.Data is available fully managed SaaS on AWS today and we announced this week that Watson X.AI is also certified to run on AWS.

So what's the X.AI? In a little more detail. Uh it is where our customers have access to our foundation models, not just ours but the open source and third party models that we provide. It is the structured environment for prompt engineering effectively.

To experiment with different models, experiment with different prompts, produce different results, assess those results and really tune and optimize the technology or the models to service the particular use cases that they're developing the tuning studio, same thing but with bringing in your own data in addition to prompt engineering and the same platform really addresses both machine learning as well as JA work flows.

Our model library. Uh we believe that there isn't going to be one model to rule them all that customers will use multiple models, large models, small models, uh task specific models uh in their uh in their solutions.

So we have a series of IBM develop models, our Granite series of models that we've developed in our IBM model factory. Something that IBM research uh does for us, we've also added a number of open source models including uh the LAMA two family from, from Meta uh as well as StarCoder for for code generation.

So really starting to diversify the set of models that's available. We're also gonna be adding third party models that we'll be partnering with as well as enable to bring your own model.

So not many customers are training their own models today, but effectively fine tuning is is making them uh their own and then the ability to bring them into the model library uh and uh and consume in them in Watson X.AI,

Watson X.Data like I said is our fit for purpose repository built on a lake house architecture. And if you're not familiar with the lake house, it combines the performance usability functionality of a data warehouse with the flexibility, scalability and cost advantages of a data leak.

Therefore, the data lakehouse it's comprised of three key layers, commodity cloud object storage as your storage tier, the cheapest and most resilient highly available storage, you can get a layer of metadata which is specifically using the Iceberg open table format.

So open table formats bring asset consistency or transactional integrity to what was previously open data formats that were the domain of the data lake. So this is one of the areas where you're getting the best functionality from, from the warehouse, that transactional integrity. But the price benefits of a of a data lake with the file formats on object storage.

And finally, on top a range of open source query engines as well as commercial engines. So we support Presto, we support Spark and we have enhanced IBM's DB2 and the TISA databases to also support Iceberg.

So the exact same underlying architecture, the exact same data at the same time can be queried by multiple query engines and that's how you achieve uh fit for purpose functionality.

So for example, using Presto for interactive querying business intelligence, maybe some data transformation you can use Spark for AI workloads certainly for ingest and data transformation. And then if you have existing applications that are running off of our commercial databases, those will continue to work without having to change the SQL in any way.

I believe I read that this week, Redshift announced support for Iceberg as well. We know that Cloudera, another one of our partners supports Iceberg and even our competitors like Snowflake supports Iceberg.

So there's a really growing and diverse community of analytical engines that are supporting Iceberg as that common table format and that provides customers a lot of flexibility right to select the particular engine that they prefer while sharing the underlying data layer across all of them.

Looking forward, we're going to be enhancing Watson X.Data. These capabilities are already in tech preview today with what we call Semantic Automation, which is infusing JA into Watson X.Data.

So applying it to the data management problem itself, enriching the metadata that's associated with the multiple schemas and tables that can be in a in a lake house and really enabling a conversational experience so that users who are not fluent in SQL or are not experienced with the data in the in the lake house can really now interact with it much more easily, really kind of delivering on the on the long promised idea of democratizing data access and analytics.

We're also enhancing the Presto query engine. Another point of collaboration with Meta to be more performant and deliver warehouse like performance with a technology called Bello.

And finally, we are adding a vector store. So a vector engine open-source vector engine into the lake house so that it will also support any language embeddings, of course, JA workloads as well.

So if you look at data a little bit, a little bit deeper, uh we have this week announced the availability of DB2 OLTP. So for transactional workloads on RDS, so customers who love DB2 customers who love the uh RDS can now run DB2 on RDS fully managed service with all the great advantages that RDS provides.

And DB2 of course, a database that is built for the world's mission critical workloads. And like I think uh one of my colleagues said today, it's one of the silent infrastructure, uh databases of our entire economy and society, right? It, it fuels uh just millions of banking transactions uh and, and many others.

So DB2 on RDS really adds to our analytical repositories that have already been available on AWS. So IBM's entire databases platform is available today on AWS as SaaS.

The integration between DB2 transactional database and DB2 Warehouse provides for zero ETL uh integration so that you've got your data uh that you're using for transactions immediately available to you for analytics as well. And really kind of have the best of those two engine and workload types sitting next to each other.

And like I said before, Watson X.Data available as well, which can start to tear that data into Iceberg really unified data from multiple different sources as well. That's the repository stack, but that's not enough, right? Customers have databases from a lot of different sources. A lot of different vendors fit for purpose, databases across the entire database type.

And for that, you need a data fabric, right? So you need centralized governance, centralized data observable data integration, data virtualization, uh management of data quality. So all these things that are essential to build a strong data foundation for all of your analytical workloads, whether it's AI, BI uh or anything else.

And of course, if you think about this as a supply chain, ultimately, this feeds Watson X itself where all of the AI workloads can be executed with trusted and quality data.

What's next? Our governance is the third pillar of the platform. And like I said, it enables end-to-end lifecycle management of both machine learning and gen AI models. It enables monitoring and alerting of the models. And it also supports regulatory compliance.

And if you want to visualize kind of what this looks like, it's like having a single front door to all of the use cases and applications of the technology. Through that front door, you assess the potential risk. It's the intersection of use cases, industries and the technology itself. You may be selecting between different models. The system will tell you if you're potentially subject to any corporate policies or regulatory policies that apply. Obviously, that kind of elevates the uh the level of care. Uh and the potential risk associated with that particular use case.

The model development process ensues. So you're tracking each and every step in that process. So you have full lineage and explainability. After the fact, once the model is developed, typically there's a handoff in a separation of duties model where somebody then is the one that deploys it. Usually the developer is not the one that should be deploying it or monitoring it. So that process is also being tracked for regulatory compliance purposes afterwards.

Once the model is put into production, you're monitoring the model against the acceptable thresholds that you establish in the development process. So you know, you could be monitoring quality performance data volumes as well as bias risk, instances of potentially objectionable content. All these things are being monitored, recorded and alerted as appropriate.

And ultimately, at the end of this process, what you have is effectively pushbutton compliance, right? Because you've implemented all of the controls and processes necessary to ultimately attest internally to internal audit or externally to a regulator, regulatory authority that you have complied with the processes that you've established.

One last thing I'll say on What's Next Governance - of course, it works very well with Watson XAI and Watson X Data, but it also works with third party AI platforms like for example, SageMaker where we can integrate and capture a lot of those parameters. A lot of those model monitoring metrics and then also incorporate it into that macro level end-to-end lifecycle process.

We have customers that have multiple data science tools that they're using and governing with a single governance tool like Watson Governance.

Alright, moving on to the Assistants. So Watson X also has three fit-for-purpose Assistants, which really accelerate time to value in areas that we've identified are low hanging fruit or opportunities for business impact in the short term.

The first one, Watson X Orchestrate. This is the technology that really emerged from some of those HR use cases that I was mentioning originally and it really drives workflow automation. I think digital labor across the enterprise through a conversational user interface. So any of these back office processes that you want to automate Watson X Orchestrate is a great fit for that.

Watson X Assistant, similar conversational interface, but really oriented towards customers directly. So Watson X Assistant is used today by hundreds of millions of consumers. If you scheduled a vaccine at CVS during COVID, you use Watson X Assistant. There's many other brands that use Watson X Assistant today as their, you know, front end self-service chatbot or digital assistant.

And then Watson X Code Assistant, like I said, focused on answerable playbooks as well as Cobol code generation and refactoring for Java. It can also help with documentation, automating documentation, really assessing the quality of code as well. And this is an area where we see a huge amount of interest from our customers because they are actively, you know, modernizing their IT environments. And this provides obviously a huge amount of developer productivity and addresses that skills gap that I mentioned.

And we're not stopping there. We are embedding and infusing AI into almost all of our application portfolio. Here I've selected a few of the IBM applications that are available today on AWS.

So Turbonomic - with AI, we're able to analyze and identify potential anomalies in cloud resource consumption or any kind of infrastructure resource consumption and suggest potential sources of remediation, potentially even automate some of that resolution very focused on financial governance and optimizing price performance.

With IBM Envizi, we've gone outside of the domain of language and are now incorporating a foundation model that we co-developed with NASA, the Geospatial Foundation Model. Foundation models of course are not just about language. And with this geospatial foundation model and the IBM Envizi application enterprises are able to assess their climate impact as well as manage risk associated with climate changes that can be modeled in this tool with the geospatial foundation model.

IBM Security QRadar - we are capturing information about threats, security threats and we're using gen AI to summarize that as well as to facilitate the access to all the information that's in the cases, right? Previous cases. So summarizing case management, summarizing threat management and automating threat detection and response.

And then IBM Security Guardian very similar to the semantic automation that I mentioned with Watson X Data. The ability to really understand what data you have and are protecting. What is the nature of that data? Is it potentially sensitive for a variety of different reasons like PII or others? This again is all about providing automation and making it easier for more users, maybe less skilled users to be able to leverage the power of these applications.

Stepping back a bit and looking at the overall stack as a summary, like I said, it is built on foundational open source technologies and libraries. In our case, those are sourced, certified and supported by Red Hat. You have your data services, so your databases and your data fabric services governing and ensuring quality and available data for higher consumption.

The AI and data platform itself, which is comprised of the three components - AI, Data and Governance - as well as the Foundation Models and then the various Assistants that leverage the underlying Watson X platform.

And then you've got that second layer from the top, the SDK and API. So effectively the programmatic layer that we use ourselves to embed AI into our own applications and that our community of partners - ISP partners, GSI partners, enterprises - can use to embed the technology within their own applications. Programmatically, we also have IBM Consulting.

So IBM is unique in that we have a sister organization that has the skills and the intimacy with the technology to really help our customers implement it across the enterprise, really focus on business value, experimentation as well as scalability and long term viability.

IBM Consulting of course is very successful with Watson X, but they also work with many AI technologies and can help our customers in that process between the two.

And we've got a number of success stories - customers across multiple different industries that are already reaping the benefits of AI within their business - automating SAP installations, dramatically reducing development and deployment time, for example, or using Watson X Orchestrate in employee onboarding and other related activities.

We're also working with a number of partners. With AWS on AWS, Samsung is using it for employee improving employee productivity as well as client experience. Working very closely with Intel and with Meta to optimize the performance of the Vox query engine in Watson X Data. This is work that we've been doing with Intel for a long time. Basically making sure that our database technology and their hardware technology work very well together, really optimal together.

And then with SAP, we are helping them with their in-product user self-help experience.

Alright. So I'm gonna take a pause there and invite Sai Choudhury to join me on stage and talk a little bit about the IBM medical collaboration.

Thank you, Edward.

Alright, Sai, great to be here with you. You want to introduce yourself real quick?

Sure. Yeah. Hi, everyone. My name is Sai Choudhury. I support the AI partnerships at Meta.

Awesome. So we've been working together for a number of years in this space. Could you maybe start by telling us a little more about Meta's strategy and approach to AI itself and then kind of the role that open source plays in that?

Sure. Absolutely. So, a lot of you saw that last July, we open sourced a pretty important and technology important model called LLaMA LLaMA-2. And the real reason for that was to try to democratize AI. If you think about it, there are very, very few entities in the world, research labs, companies that have access to top AI research scientists, hundreds of thousands of GPUs to be able to do training and experimentation and then actually do all the safety and fine tuning testing.

For example, LLaMA-2 has over 1 million human-related safety tests fix into the actual model itself. These kind of resources from human capital to actually work to actually make these models is huge and so not, you know, very few companies can afford to do that. Companies like Meta do do that. But we, we actually looked at it ourselves and said, look, this technology is really something that many, many companies should be able to enjoy. And so that was the decision we made last July and open source, LLaMA-2.

LLaMA-2 is made up of actually a family of models, right? There's the foundational models so that you can fine tune in and customize it from a very basic state. We have chat models which are more for conversational or summarization. And you can fine tune those as well. And then finally, we've even released more recently a coding set of models to be able to act as code assistance.

And in all of these cases, one of the things that we're really excited about is we're working with companies like IBM on Watson X to bring this to with a much lower touch approach to, to the enterprise, to be able to very easily take these models, be able to fine tune it on your data. Remember your data is your, you know, your gold, so to speak, and then be able to actually then essentially own the derivative models for your use.

Excellent. So we've been working in a, in a variety of different areas together. What makes you most excited about what IBM is doing in the space, IBM and AWS is doing in the space, from, from your perspective?

Yeah. Actually it's, it's funny because, our work together now on, on LLaMA and, um, on Watson X is really a part of a long history of both, what we've been doing with IBM as well as ironically and interestingly, we've been doing with AWS.

And so, I think you've mentioned that our companies, Meta and IBM, have worked for, for years now on technologies like Presto and then more recently on PyTorch. Some of you may know that these models, whether it's LLaMA or these other or many of them generative AI models run on a tool set a framework called an AI framework and that's called PyTorch.

And PyTorch is one of the leading open source frameworks for developing and running these models. It's kind of the engine the operate the, the lower part of the operating system. And last year IBM made some very serious contributions to PyTorch to make it work really well on Ethernet-based networks, which are really prevalent all around the world versus these ultra high perform performance networks that are only in some data centers.

And so that was a great contribution by the way.

Thank you for that. Um, and further to the uh as uh Meta, Meta and Amazon and others had established uh PyTorch Foundation and IBM has joined as a as a board member of the foundation as well. So we're looking forward to a lot more interesting work with between the companies, not only at the model layer, but also even at the tools layer underneath.

Um, I'll also kind of shift and talk about uh a really interesting other kind of uh nuance within this, that the SageMaker environment that, that you're referring to that. Um go that what's the next uh governance uh kind of connects into seamlessly SageMaker is an ML Ops kind of uh de development environment. But what it use uses underneath to actually do the training and then the execution is PyTorch, right? So the worlds do collide in a really interesting way where you have the power of the the Watson X governance tools, you know, using LLama on SageMaker with PyTorch. So this is a, a wonderful place to kind of uh bring this all together and I'm not making this up. This is the way it all works. Absolutely, it's not a coincidence either.

Um, what should enterprises be thinking about uh when they think of a technology like LLama, right? How do they, how should they be thinking about implementing it responsibly and also potentially optimized from a price performance perspective?

Yeah. So one of the things that you know, one of the reasons that we open source LLama is that we benefit from open sourcing LLama. Now, if you think about large language models, LLM’s, there is an analogy 100% analogy to how open source code works. It's a different kind of thing. But it's, there's an analogy what happens in, in open source code, whether it's Linux or like or something like Presto, you have the community adding new features to it. And the so hence the product gets better, you have the community fixing bugs and so hence the technology gets better in LLM’s. There's the exact same analogy, but you don't actually have the community adding new features. What you have the community doing is making new optimizations, how to run the model faster or how to, you know, use tools like the governance tools to really introspect your use of data in the model.

So that's one great example. Secondly, in the same way with open source software, you have a way for the community to report errors and hallucinations that then get fed back into the GitHub repository that Meta can actually fix for the next release of the model. So this community aspect is really important and to, and it's as important for large language models. I would say a matter of fact, maybe even more important for large language models as it is for open source software.

And so to bring this approach to life, what we've been doing at Meta is kind of a three pronged approach to make sure it's available to enterprises and researchers alike everywhere. One is to partner with different companies that can host and enable these models in a very seamless and easy way just like you saw with the Watson X uh studio product. Another is to work with technology providers and lower in the tech stack to make sure the models are optimized and run uh as efficiently as possible. Think about, for example, partners like Nvidia AMD and Intel for the data center on the silicon side and folks like um Qualcomm and MediaTek on the on the mobile side.

And the third very important part as well is to collaborate with s uh system integrators, consulting companies who actually are uh know the best fine tuning techniques on every model architecture. And so, you know, not every company has the ability to actually, you know, devote full resources to use the different and very fast moving fine tuning techniques that are coming out there today. And so uh this is another place where actually we're working with the consulting group IBM Consulting to make sure that their team members are knowledgeable about how the nuances of the LLama two model family to be able to work with um enterprises to actually do the additional fine tuning and deployment.

Um, you know, in, in a good set of time. Awesome. What, what's next for, what's next for the LLama family? Yeah. So you know, one of the things that we're very serious about in any open open source project that we do and this is no different for a LLama too, is a commitment to continue to bring out innovations to the community. And so, while I'm not really preannouncing anything here, there are definitely some trends that we're seeing um with, even in what's happening on to, you know, iterations of LLama two out there that we hope to be able to incorporate.

And we, our, our research teams are, you know, uh working on this right now as we think about what could be LLama three or LLama four, et cetera. I do know a couple of trends that I'd like to share with you that we're seeing that are looking very promising, one is bigger is not always better. I think we've already shown with the LLama 27B and 13B model that there's, you know, three models, there's a 70B model that they're really, really effective in what they do in being able to, you know, do summarization or content creation, etcetera.

I think you're gonna start to see a trend and we are definitely gonna lead the way that to make the smaller models even more effective, even the 70B model to be even more effective. And I say 70B is small because some of these language models are much bigger than that. So that's one trend you'll see even as early as 2024 I think in parallel, more in a research stage is the, the move to or the, the move to bring out multimodal functionality in the models where you can actually feed the model a picture, an audio stream, maybe even a video and have it be able to not only understand that work on that, but also maybe feed you out a picture, an audio stream or a video.

And so you know, can you do that from a science perspective theoretically, you can but to do it efficiently is is the important part right? To do it where it doesn't, where you, where as an enterprise, you can actually run that same unified multimodal model um at only maybe a slightly higher cost but get a lot of functionality. And so that's an area of research for the, for many of the research labs including uh Meta's fundamental AI research lab, which uh I think turned 10 years old uh this week which is yay happy birthday.

But um that is some, the i i mentioned this to you because in addition to just the large language model innovation you're gonna see in 2024 you're gonna see a lot more of that in uh in uh beyond that with multimodal uh models.

Uh we were talking earlier and uh you were sharing with me a little bit of how uh Meta is using the models and actually delivering the experiences uh to, to users, i actually uh played around with uh with Instagram uh over the last couple of days. If you haven't done that. It's actually quite fascinating. It's right embedded in the uh in the chat, uh function chat with AI right.

Uh can you tell us a little bit more about what's happening behind the scenes there? Yeah, so we, you know, um we drink our own champagne. Is that the way you say we eat our own dog food, et cetera. Um you know, the, the one thing that um we are really proud of is the same LLama, two models that we open source are the LLama, two models that we use for our own use, right? We're, we're, we're doing this because we need to use, we need to bring generative AI experiences um both for our kind of our consumer surfaces as well as we're actually using that for internal uses for things like you could think of like, you know, uh productivity gains for our developers.

Um if any one of you, uh if you haven't used it, I would encourage you to try some of our new AI’s as we call them. If you go into our messaging apps, WhatsApp, Instagram, direct or messenger, you can actually pick to chat with an AI, you can even include it in your group chats. And it's a really good example, the, the, you know, so you can talk to it. There's one, there's then there's a few that are, you know, more curated for and actually essentially fine tuned for as a travel expert. There's some that are can be your health coach, et cetera. It's kind of a, you don't even and there's a generic one as well. But it's a really good example, these, the way when we develop this a really good example that really, um there is a fair amount of system engineering you have to do to really have a very seamless experience.

As an example. When you put a prompt to that AI, you, you would think that it just goes to one model and outcomes. The answer that's that couldn't be further from the truth. We actually have multiple models changed together to actually work together and make a better system. So first, uh as an example, the your prompt goes first to an intent model that actually tries to understand what is the uh the question, what's the context maybe time or date some of the additional metadata, what the user is doing? Do I have to do it? Uh uh more of a real time search for some additional context to, to actually be even have the um the query uh be richer.

So that intent model then passes all that to another LAO model. Luckily, these are all LMO models uh which actually is a safety model and the safety model of course is looking for. And it's been tuned to be more of a guardian, let's say, and to actually look for, you know, malicious intent, et cetera that then goes the output of that, then goes to the actual, what you would think of the response model that actually, you know, actually brings together all of this to develop a response.

And then that goes to 1/4 model, 1/4 lambo model which actually then actually curates that um and concatenates that down to a much more, you know, um more concise kind of response. Because in a chat type of interface, you actually want things to be pretty concise. So, so all of this is going on behind the scenes and what it actually uh really demonstrate is to have a very seamless and really pleasant and interesting user experience. There is a level of system engineering that's involved. It's not just, you know, you can use an LLM and you know, check your, your uh your solution is right there.

And so I would encourage all of you as you work with uh your development team or IBM consulting firm. And for that matter to think about the overall system and, you know, you could start by prototyping uh small and simply, but there are benefits in doing things like chaining together different models that are fine tuned for different purposes and, and uh different sizes by the way as well. Excellent. Thank you.

Ok. Anything else you want to share? No, no, it's good. All right. Thank you. Thanks. I, all right. So just kind of wrapping up some of the highlights from uh from that discussion, uh open source essential, right, not just about the community, innovation also about transparency also about benefiting from uh security, right, security enhancements, safety enhancements, the more people are using and contributing to an open source technology, the more secure that will be as well as from a governance perspective, that model that that technology roadmap being driven by a community of diverse users and vendors.

We're working with Meta n AWS on three specific projects that we've been discussing. PyTorch for that kind of low level uh foundational uh libraries and capabilities. That's gonna give us, for example, the hardware abstractions that I mentioned earlier as the infrastructure landscape becomes more diverse. Uh and you have Intel and AMD as well as Nvidia and others. Uh enterprises will basically be abstracted from that complexity. And we'll be able to really kind of optimize and allocate those workloads, schedule those workloads across the various platforms and optimize the price performance of their solutions hybrid cloud.

Of course, is a huge part of what makes IBM different. Uh and some of these technologies enable uh deployments within, within data centers, right? Not just on hyper scalar infrastructure. Uh for example, like PyTorch, of course, uh in our case, Red Hat OpenShift in all of its forms, whether it's self managed software or fully managed services on AWS is how we achieve that hybrid cloud flexibility and that workload portability that customers really benefit from.

And finally, the idea that this is going to be a multimodal world. And there's great value in that uh you get the benefit of tasks specific fit for purpose models chained together really to deliver an optimal outcome, of course, price performance optimization. Another important attribute of trying to use the smallest model possible right to achieve the performance that you're looking for.

IBM's partnership with AWS is, is really special and I'd say it's really been over the last three years that it's that it's taken off. We have a vast uh army of, of resources and consultants that have achieved the highest levels of certifications that AWS provides. Uh and beyond, the uh consulting skills are IBM technology products. Uh 65 plus available on the AWS marketplace, almost 30 SaaS services delivered by IBM on AWS, really meeting our customers where they are and delivering the best of AWS with the best of IBM.

And I will close with a final thought uh before inviting you to join us uh at our booth for I believe the happy hour that already started um in this whirlwind of exciting innovation. Um, you know, it's easy to get wrapped up in the excitement and all the different technologies that are, that are coming out and evolving rapidly. But those core principles that I mentioned earlier, right, openness, transparency trust, they're not going anywhere, right? They will endure and they will make the difference in terms of how the technology is adopted successfully uh within enterprise. And IBM of course, is here to assist you uh in that journey to put AI to work within your business.

So, thank you very much. Please do join us in the booth.