[EN] re:Invent 2023 - Dr. Swami Sivasubramanian Keynote

Keyword: Amazon Web Services re:Invent 2023, Amazon Bedrock, Data, Ai, Generative Ai, Foundation Models

Words: 4000, Reading time: 20 minutes
 

Video

Introduction

Swami Sivasubramanian, Vice President of Data and AI at AWS explores the powerful relationship between humans, data, and AI, unfolding right before us. Generative AI is augmenting our productivity and creativity in new ways, while also being fueled by massive amounts of enterprise data and human intelligence. Swami discovers how you can use your company data to build differentiated generative AI applications and accelerate productivity for employees across your organization. You will also hear from customer speakers with real-world examples of how they’ve used their data to support their generative AI use cases and create new experiences for their customers.
 

Highlights of the Speech

The following is a summary of the highlights from this speech, around 3,800 words, estimated reading time is 19 minutes. If you would like to further understand the speech content or watch the full speech, please view the complete speech video or refer to the original speech text below.

As Dr. Swami Sivasubramanian took the stage, he welcomed the audience to re:Invent 2023, expressing his excitement to share AWS's vision in this era where a powerful relationship between humans and technology is unfolding. He emphasized that generative AI is augmenting human productivity in many unexpected ways while fueling human intelligence and creativity.

Drawing parallels to symbiotic relationships observed in nature, such as the mutually beneficial coexistence between whale sharks and remora fish, or even symbiotic stars that share gases and heat, occasionally producing a beautiful explosion of energy called a supernova, Swami highlighted the rife possibilities presented by the partnership between humans and AI in forming new innovations.

Swami traced the exploration of this symbiotic relationship back over the last 200 years, acknowledging the dedication of mathematicians, computer scientists, and visionaries who have invented technologies to reduce manual labor and automate complex human tasks. These innovations range from new machines for mathematical computation and big data processing to new architectures, algorithms for recognizing patterns and making predictions, and new programming languages that significantly eased working with data.

Paying homage to one such visionary, Ada Lovelace, known as the world's first computer programmer, Swami recounted her work with Charles Babbage on the Analytical Engine, one of the earliest versions of a computer in the 1830s. Like Babbage, Ada believed that computers could remove the heavy lifting from many computational and analytical tasks. However, what set Ada apart was her ability to recognize the potential of computers beyond just number crunching.

Ada discovered that computers could be programmed to read symbols and perform logical sequences of operations, marking a shift towards using computers for complex tasks. Remarkably, she even speculated that these computers could understand musical notations and create music in the future – an early ode to the potential of generative AI.

Despite her belief in their potential, Ada made it clear that these machines could not originate anything; they could only generate outputs or perform tasks that humans were capable of ordering them to do. True creativity and intelligence, she argued, only originate from humans. In this way, humans and computers would have a mutually beneficial relationship, each contributing their unique strengths.

Swami found Ada's analysis of computing inspiring, not only because her work stood the test of time but also because he has a young daughter named Ada, who considers Ada Lovelace one of her favorite people from the book "Goodnight Stories for Rebel Girls."

Swami emphasized that Ada's contributions are gaining even more relevance as the world of generative AI unfolds. While ML and AI have spurred rapid innovation over the past couple of decades, he believes this new era of human-led creativity with generative AI will shift the relationship with technology towards one of intelligence augmentation – a core evolution he finds incredibly exciting.

Swami highlighted another symbiotic relationship happening under the hood, which is key to driving progress: the relationship between data and generative AI. The massive explosion of data has enabled the foundational models that power generative AI to finally form, accelerating new innovations everywhere. With these advancements, customers can harness data to build and customize AI models faster than ever before, while also using generative AI to make it easier to work with and organize that data.

Swami likened the current situation to a beneficial relationship among three things – data, generative AI, and humans – all unleashing their creativity together, akin to a symbiosis typically observed between two different entities.

To build an AI, Swami outlined the essentials: access to a variety of large language models (LLMs) and foundation models to meet diverse needs, a secure and private environment to customize these models with data, the right tools to build and deploy new applications on top of these models, and a powerful infrastructure to run these data-intensive and ML-intensive applications.

AWS, according to Swami, has a long history of providing customers with a comprehensive set of AI/ML data and compute stack for these requirements. In fact, according to Hedge Insights, more machine learning workloads run on AWS than any other cloud provider.

Swami described AWS's AI/ML offerings as a three-layer stack:

1. The infrastructure tier, where customers can cost-effectively train foundational models and deploy them at scale, including AWS's hardware chips and GPUs. This layer also includes Amazon SageMaker, a service that enables ML practitioners to easily build, train, and deploy ML models, including LLMs.

2. The middle layer, Amazon Bedrock, which provides access to leading LLMs and other foundation models to build and scale generative AI applications.

3. The top layer, comprising applications that help customers take advantage of generative AI without specialized knowledge or coding, such as Amazon Q, AWS's new generative AI-powered assistant tailored to businesses.

Swami emphasized that each layer builds on the other, and customers may need some or all of these capabilities at different points in their generative AI journey.

Focusing on Amazon Bedrock, Swami highlighted its broad set of capabilities for building and scaling generative AI applications with the highest levels of privacy and security. One of the main reasons customers gravitate towards Bedrock is the ability to select from a wide range of leading foundation models that support their unique needs. Swami stressed that customer choice is paramount because no one model will rule the world, as the field is still in its early days, with models continuing to evolve at unprecedented speeds.

To underscore this point, Swami announced Bedrock's support for a diverse array of models from leading providers:

- Anthropic's Claude 2.1, renowned for its industry-leading 200k context window, 50% fewer hallucinations, 2x reduction in false statements in open-ended conversations, improved system prompts for better user experience and 25% cost reduction for prompts and completions.
- Stability AI's Stable Diffusion model for generating images, art, and design.
- Cohere's Command for tasks like copywriting and dialogue, Cohere Embed for search and personalization, Command Light, Embed English, and Embed Multilingual.
- Meta's LAMA-2 for high performance and relatively low cost.
- LAMA-213B, optimized for a variety of small-range use cases.
- Stable Diffusion's Stable XL, one of Stability AI's advanced text-image models, now generally available.

Swami also highlighted AWS's own innovations, born from the company's 25-year investment in AI technologies. Many of AWS's customer-facing applications are driven by ML models, including foundation models that power various facets of the enterprise business and supply chain.

One such innovation is Titan Multimodal, which enables customers to create richer multimodal search and recommendation options by generating, storing, and retrieving embeddings for accurate and contextually relevant multimodal search. Companies like OfferUp and Alamy are leveraging Titan Multimodal embeddings to revolutionize their search experiences.

Additionally, Swami announced the general availability of Titan Text Light and Titan Text Express. Text Light is a small, cost-effective model that supports use cases like chatbot Q&A and text summarization, offering a highly customizable option through fine-tuning. Text Express can be used for a wider range of tasks, such as open-ended text generation and conversational chat, providing a sweet spot for cost and performance compared to larger models.

In an exciting development, Swami introduced Titan Image Generator and Radar, now available in preview. This model enables customers to produce high-quality, realistic images or enhance existing ones using simple natural language prompts. Customers can customize these images using their own data to create content that better reflects their industry or brand.

Titan Image Generator is trained on diverse data sets to enable more accurate outputs and includes built-in mitigations for toxicity and bias through human evaluation. Swami highlighted that Titan Image Generator has higher scores than many other leading models. Importantly, to combat the spread of misinformation, all Titan-generated images come with an invisible watermark designed to help identify AI-generated images, making AWS among the first model providers to widely release built-in invisible watermarks integrated into image outputs.

Swami demonstrated the model's editing features, including outpainting to seamlessly swap out backgrounds while retaining the main subject, generating variations of the subject and background, and changing the orientation of the image using natural language prompts.

Swami emphasized that each Titan model has its unique strengths across capabilities, price, and performance, providing customers with the flexibility to select the best models for their requirements as their needs grow and change.

Since the launch of Bedrock, more than 10,000 customers are rapidly developing generative AI-powered applications for use cases like self-service, customer support, text analysis, and forecasting trends. This includes customers like SAP, using Bedrock to automate the creation of trip requests with SAP Concur, saving employees hours of time; Georgia Pacific, powering a chat system that helps employees quickly retrieve critical factory data and answer questions about their machines; and United Airlines, helping employees access up-to-date information and summaries on delays using natural language, resolving operational and customer issues faster.

To showcase how to put generative AI into action, Swami welcomed Nan Jiang, VP of AI from Intuit, to the stage. Nan emphasized Intuit's mission to power prosperity around the world for 100 million consumers and small business customers, a mission that hits close to home as half of her 10 siblings are small business owners.

Intuit has been on an incredible transformation journey over the past five years, declaring in 2019 its commitment to becoming an AI-driven expert platform. By combining cutting-edge AI with tax and human expertise, Intuit is delivering unparalleled experiences for its customers today.

Running all of its data capabilities, data lake, and machine learning platform on AWS, with SageMaker as a foundational capability, Intuit achieves remarkable scale, making 65 billion machine learning predictions per day, utilizing over half a million data points for small business customers, 60,000 for consumers, and driving over 810 million customer-backed AI interactions per year.

In the era of generative AI, Intuit is well-positioned to change the game thanks to its multi-year investment in ensuring clean data, strong data governance, and responsible AI principles. Intuit has built a proprietary generative AI operating system called GenOS, comprising four primary components:

1. GenStudio, where anyone at the company can build, prototype, and test new generative AI experiences.
2. GenRuntime, with connectivity to multiple LLMs and access to the right underlying data for personalized and accurate experiences, scalable as needed.
3. GenUX, a design system for consistency across products when customers interact with generative AI.
4. A series of financial large language models, including third-party LLMs and custom-trained models specialized in tax, accounting, marketing, and personal finance.

Nan emphasized that three things remain constant in the generative AI world: accuracy, latency, and cost. Intuit's ability to use smaller, faster models while hosting them on SageMaker allows the company to realize significant latency gains and manage costs effectively, optimizing for all three constants.

Intuit also leverages third-party LLMs from Bedrock, as the goal is to build the best customer experience possible, which requires using best-in-class solutions.

Intuit's collaboration with AWS over the past decade has enabled the company to grow into a global financial technology platform, leveraging a range of AWS services.

Swami then introduced Arvin Shrinivas, CEO and co-founder of Perplexity, a company striving to be the world's leading conversational answer engine that directly answers questions with provided citations.

Perplexity's Copilot is an interactive search companion that starts with a general question, digs deeper to clarify needs, and provides a great answer after a few interactions – the first global publicly deployed example of generative user interfaces that reduces the need for prompt engineering.

To run this complex product, Perplexity decided to go all-in on AWS. They started by testing frontier models like Anthropic's Claude 2 on AWS Bedrock, which enabled quick testing and deployment of Claude 2 to improve general question-answering capabilities by providing more natural-sounding answers. Claude 2 also helped inject new capabilities like the ability to upload multiple large files and ask questions about their contents, positioning Perplexity as a leading research assistant.

However, Perplexity is not just a wrapper on top of closed-source large language models. Instead, they orchestrate several different models, including those they have trained themselves. Building on open-source models like LAMA 2 and Mistral, and fine-tuning them to be accurate and live with no knowledge cut-off by grounding them with web search data using cutting-edge RAG techniques, Perplexity started working with the AWS Startups team on an Amazon SageMaker HyperPod POC.

SageMaker HyperPod made it easier to debug large model training and handle distributed capacity efficiently. Perplexity obtained AWS EC2 P4DE capacity for training and, once they moved to HyperPod and enabled AWS Elastic Fabric Adapter, observed a significant increase in training throughput by a factor of 2x.

AWS has also helped Perplexity with customized services to support their inference needs, especially on P4D and P5 instances, enabling them to build top-of-the-market APIs for their open-source and in-house fine-tuned models.

Perplexity announced the general availability of these models in the form of APIs, including the first-of-its-kind live LM APIs with no knowledge cut-off, fully hosted on AWS and plugged into their search index.

Arvin Shrinivas expressed excitement about the future of generative AI, stating that while it is still in its nascent stages, Perplexity strives to be the "earth's most knowledge-centric company," working with AWS to ensure that no one needs to go back to the traditional 10 blue link search engine.

Swami then welcomed Rob Francis, SVP and CTO of booking.com, to share insights from the travel industry. Rob reflected on the arc of technology throughout his career, recalling his first attempts at human-like interaction with Eliza, a program developed by Joseph Weizenbaum in the mid-1960s, which proved frustrating due to its limitations.

However, Rob expressed excitement about the possibilities enabled by the emergence of generative AI for booking.com's customers. Providing an overview of booking.com's vast scale, Rob highlighted the company's presence in accommodations, flights, rental cars, and attractions worldwide, with over 28 million accommodation listings, flights in 54 countries, and 52,000 rental car locations.

To tackle the immense data challenges posed by this scale, booking.com partnered with AWS, resulting in remarkable improvements. Data scientists can now train 3x more jobs concurrently, with a 2x decrease in failed jobs and a 5x reduction in job training time, some of which involve over 3 billion examples.

Leveraging AWS's offerings, including hosting the LAMA 2 model on SageMaker for intent detection, booking.com has built an AI trip planner that makes it easier to book trips through conversational interactions. The planner incorporates moderation techniques, privacy protection, and review data integration through RAG implementation, providing personalized recommendations and a seamless booking experience.

Rob demonstrated the AI trip planner in action, showcasing its ability to understand natural language inputs, moderate conversations, protect user privacy, integrate review data, and provide personalized recommendations through a recommendation engine powered by machine learning.

To illustrate the arc of technology, Rob asked Titan, AWS's generative AI model, to summarize what he should have said during his presentation. Titan's response highlighted key points, including the large market for travel booking, the difficulty of personalizing the booking experience until now, the use of generative AI to create personalized hotel recommendations tailored to each user's needs and preferences, and an invitation to visit booking.com.

Swami reiterated the critical role of data as the fuel for generative AI, emphasizing the symbiotic relationship between data and generative AI. While ML and AI are typically seen as outcomes of data, Swami highlighted how generative AI can also transform the way data is managed, enhancing the very data foundation that fuels it.

AWS has infused AI and ML across many of its data services to remove the heavy lifting from data management and analytics. However, Swami acknowledged that managing and extracting value from data can still be challenging, prompting some customers to explore leapfrogging their data strategy with generative AI.

One area where AI can be applied is optimizing the performance of data storage and querying systems, such as data warehouses. Swami highlighted the recently launched AI-driven scaling and optimizations for Amazon Redshift, enabling proactive scaling across multiple dimensions while managing price and performance trade-offs. This AI-driven approach helps customers like Honda deliver better price performance and gain actionable insights from millions of vehicle data points without manual intervention.

Swami also discussed the role of Amazon Q, AWS's generative AI-powered assistant, in supporting data management. Q connects to customer data, providing context about roles, internal processes, and governance policies, and supports virtually every area of a business through natural language interactions. Q can remove the heavy lifting from repetitive tasks like looking up documentation, building and testing new features, maintenance and upgrades, and troubleshooting.

In an exciting announcement, Swami introduced the preview of Amazon Q integration with Amazon Redshift, enabling customers to turn natural language prompts into customized SQL query recommendations. Q analyzes the schema and produces SQL query recommendations in seconds, leveraging query history and specific user access without compromising data privacy.

Swami demonstrated Q's capabilities in action, showcasing how it can generate SQL queries based on natural language prompts, course-correct based on feedback, and provide more accurate and relevant recommendations by enabling query history access.

Q's integration with Amazon Redshift extends beyond data querying. Swami announced that customers will soon be able to use Q for creating data integration pipelines using natural language. With this new integration, customers can build data integration jobs faster, troubleshoot with a simple chat interface, and get instant data integration help, all powered by Agents for Amazon Bedrock under the hood.

Swami then welcomed Shannon Kalisky, a senior product manager at AWS, to demonstrate how AWS's data foundation and generative AI capabilities can spur new innovations.

Shannon painted a scenario where customers face two common challenges: getting the data they need in the desired format and using that data to tell a compelling story. She illustrated how AWS's offerings can address these challenges by walking through a hypothetical example of creating a new feature for a travel booking application.

In the scenario, customers want to enable easy flight rebooking without stress, requiring the integration of data from various sources, including S3, Redshift, Aurora, and Amazon Kinesis. Shannon demonstrated how zero-ETL integrations across AWS services, such as Redshift Serverless, data sharing, and Redshift streaming ingestion, can bring all the necessary data together without building pipelines or writing code.

With the data in place, Shannon showcased how Amazon QuickSight can be used to measure success by creating an executive summary dashboard using natural language prompts with Amazon Q. She then demonstrated how Q can generate a data story based on the business data, complete with visuals and customizable formatting, enabling secure sharing and better decision-making throughout the organization.

Swami revisited the symbiotic relationship between data, generative AI, and humans, emphasizing the role of humans as facilitators in creating a powerful cycle of reinforcement over time. He drew an analogy from the Panama rainforest, where the Virola tree, the toucan, and the agouti mammal engage in a mutually beneficial relationship, each contributing to the longevity and sustainability of the ecosystem.

Similarly, Swami highlighted that humans provide a unique set of benefits that create more efficient, responsible, and differentiated AI. Humans generate the data that makes AI technology possible, identify use cases and lead the development of AI applications tailored to business needs, and provide valuable feedback to maximize the efficiency and output of these applications.

One common way to integrate human feedback into an AI strategy is through the model evaluation and selection process. However, Swami acknowledged that model evaluation requires deep expertise in data science and can be a tedious, time-consuming process involving the creation of benchmarking datasets, metrics, algorithms, human evaluation workflows, and benchmarking tools.

To address this challenge, Swami announced the preview of Model Evaluation on Amazon Bedrock, a new capability that allows customers to quickly evaluate, compare, and select the best foundation model for their use case. With automatic and human-based evaluations, curated or custom datasets, and comprehensive reports on model performance, Bedrock streamlines the model evaluation process, ensuring that human inputs remain a critical component of the development process.

As the adoption of AI technologies continues to accelerate, with the World Economic Forum predicting that nearly 75% of companies will adopt AI by 2027, Swami emphasized the importance of supporting the workforce of tomorrow. While some tasks may become obsolete, AI will also create entirely new roles, products, and services, necessitating the development of new skill sets.

In addition to technical skills in ML and AI, soft skills like creativity, ethics, and adaptability will grow increasingly critical in this "reskilling revolution." AWS is supporting this transition through initiatives like the AWS AI Scholarship with Udacity, providing over $12 million in value in scholarships to over 50,000 high school and university students, as well as the AWS AIM Scholarship program.

AWS also offers more than 100 AI and ML courses and low-cost trainings, enabling professionals to build new skills and get started with AI. Additionally, Swami introduced new ways to experiment and learn AI through BedRock, an Amazon Bedrock playground that provides an easy and accessible way to learn about AI with a hands-on, code-free app builder.

Swami demonstrated how BedRock enables users to create AI applications using powerful foundation models from leading AI companies, including the newly added Titan Text Express and Light models. With social logins and no AWS account required, BedRock encourages experimentation, sharing, and remixing of applications within the community.

In conclusion, Swami emphasized the powerful symbiotic relationship between data, AI, and humans, accelerating the ability to create new innovations and differentiated experiences. He encouraged attendees to unlock this transformative technology using AWS's comprehensive offerings, including a secure place to customize foundation models with customer data, AI-powered services to strengthen the data foundation, tools with AI built-in to augment employee productivity, and mechanisms for implementing human feedback.

With tools like BedRock to kickstart the AI journey, Swami expressed his excitement for the innovations that attendees will create next with AWS.
 

Here are some exciting moments of the speech:

The leader welcomes the audience to re:Invent 2023 and shares excitement about standing at the edge of a new technological era of powerful human-technology relationships.

The collaboration with AWS over the past 10 years has helped the company grow into a global financial technology platform.


Announcing the general availability of agents for Amazon Bedrock that enables GenAI applications to execute complex tasks by dynamically invoking APIs and connecting to internal systems.


The leader discusses how a gen-powered assistant built on Amazon Bedrock can help with complex DIY projects by providing step-by-step guidance.


Anthropic is launching a new innovation center custom model program that will enable customers to work with experts to customize powerful AI models for their business needs using their own data.


AI can optimize data warehouse performance by managing dimensions like data variability, concurrent users, and query complexity - balancing price and performance.


The leader encourages developers to use AWS tools like BedRock to kickstart their AI journey and create innovative solutions.

Summary

Dr. Sivasubramanian opened by describing the symbiotic relationship emerging between humans and AI, comparing it to mutually beneficial partnerships found in nature. He highlighted Amazon's long history developing AI/ML infrastructure and announced new AI services like Amazon Q.

Next, he emphasized the importance of data in training AI models. He announced vector search capabilities in databases like DynamoDB and previewed ML-powered data integrations with Amazon Q.

Dr. Sivasubramanian then discussed how AI can augment human abilities, mentioning cancer detection by Huron AI. He announced new AI-powered developer tools like CodeWhisperer Customization and previewed Model Evaluation in Amazon Bedrock.

Throughout, Dr. Sivasubramanian emphasized that humans remain essential to innovating with AI. He announced an easy, code-free AI playground called Bedrock to make AI more accessible. Overall, his keynote highlighted the symbiotic relationship between data, AI, and humans in driving ongoing innovation.