bigquery_为什么BigQuery Omni很重要

bigquery

Google Cloud’s bet on an open platform is starting to materialize with Anthos and BigQuery Omni.

在开放平台上，Google Cloud的赌注开始与Anthos和BigQuery Omni结合。

Three years ago, I started (and sadly never finished) a series called Platform Wars (Part I, Part II), evaluating the tech giants and their strategies in the age of artificial intelligence. In my piece on Google, I explained Google’s shift to an AI-first company, and why Kubernetes was a crucial part of Google’s strategy to compete in the enterprise cloud market. Fast forward two years, Google Cloud reported meaningful growth, but still stood a distant third to AWS and Azure. Thomas Kurian, a former Oracle exec, was brought in to replace Diane Greene, carrying with him a vision for a multi-cloud strategy. Then, a year ago, Google Cloud introduced Anthos, a Kubernetes-based, open platform to extend Google’s cloud services to hybrid (i.e. on-prem, multi-cloud) environments. It materialized Kurian’s vision into a product, which Ben Thompson noted in his post: “Google Cloud Next, Athos, Google Cloud and Open Source.”

三年前，我开始(可悲的是从未完成)称为平台战争的系列( 第一部分 ， 第二部分 )，评估了人工智能时代的技术巨头及其战略。 在关于Google的文章中，我解释了Google向AI领先公司的转变，以及Kubernetes为什么是Google竞争企业云市场战略的关键部分。 快进了两年，Google Cloud取得了可观的增长，但仍然仅次于AWS和Azure。 Oracle公司前高管托马斯·库里安(Thomas Kurian)被任命接替黛安·格林(Diane Greene)，带着对多云战略的愿景 。 然后，在一年前， Google Cloud引入了Anthos ，这是一个基于Kubernetes的开放平台，用于将Google的云服务扩展到混合(即本地，多云)环境。 这使Kurian的愿景变成了一种产品，Ben Thompson在他的帖子中指出：“ Google Cloud Next，Athos，Google Cloud和Open Source” 。

Just last week, Google announced BigQuery Omni, a multi-cloud analytics solution to run BigQuery across Google Cloud, AWS, and Azure (coming soon). Now customers can use the same BigQuery UI or API to run SQL queries and build BigQuery ML models regardless of where the data is stored. More importantly, BigQuery Omni runs on Anthos and reveals Kurian’s — and Google Cloud’s — strategy to grow its addressable market. Initially, Anthos was “simply” a hybrid and multi-cloud application platform, leveraging its strong Kubernetes backbone to migrate on-prem and existing AWS/Azure applications onto GCP. With BigQuery Omni, Google is attempting to commoditize cloud infrastructure as a whole and use Anthos as a middleware to win market share. In essence, Google is betting on its superior container and AI/ML technology to compete in a growing multi-cloud world as it did with search to commoditize the underlying OS in the Internet era.

就在上周， Google宣布了BigQuery Omni ，这是一种多云分析解决方案，可以在Google Cloud，AWS和Azure上运行BigQuery(即将推出)。 现在，无论数据存储在何处，客户都可以使用相同的BigQuery UI或API来运行SQL查询并构建BigQuery ML模型。 更重要的是，BigQuery Omni在Anthos上运行，并揭示了Kurian和Google Cloud的战略，以扩大其可寻址市场。 最初，Anthos是“简单”的混合和多云应用程序平台，利用其强大的Kubernetes主干网将本地和现有AWS / Azure应用程序迁移到GCP上。 Google希望通过BigQuery Omni将整个云基础设施商品化，并使用Anthos作为中间件来赢得市场份额。 从本质上讲，谷歌将赌注押在其卓越的容器和AI / ML技术上，以在不断发展的多云世界中竞争，就像它通过搜索使互联网时代的基础操作系统商品化一样。

BigQuery Omni深潜 (BigQuery Omni Deep-dive)

Before jumping to the strategic implications of BigQuery Omni, let’s take a deeper look at how BigQuery Omni works. Unlike AWS Redshift, BigQuery decouples storage and compute (similar to how Snowflake works), taking advantage of cheaper storage costs and charging users separately for the processed data. This architectural decision makes BigQuery Omni a natural extension of that concept.

在探讨BigQuery Omni的战略含义之前，让我们深入了解BigQuery Omni的工作原理。 与AWS Redshift不同，BigQuery将存储和计算分离(类似于Snowflake的工作方式)，利用便宜的存储成本并分别向用户收费以处理数据。 这种结构上的决定使BigQuery Omni成为该概念的自然扩展。

Previously, BigQuery was limited to data stored in Google Cloud. Although Google acquired cloud migration startups like Velostrata and Alooma over the years to facilitate moving data from other cloud platforms, for most enterprises the switching costs were still too high to justify using BigQuery over AWS Redshift or Azure Data Warehouse regardless of the developer experience, ease of use, or additional features.

以前，BigQuery仅限于存储在Google Cloud中的数据。 尽管Google 多年来收购了诸如Velostrata和Alooma之类的云迁移初创公司以促进从其他云平台迁移数据，但对于大多数企业而言，转换成本仍然很高，以至于不管开发人员的经验如何，都可以通过AWS Redshift或Azure Data Warehouse使用BigQuery来证明其合理性，使用或其他功能。

Now BigQuery Omni runs on Anthos inside AWS to directly access data in S3 and other databases. Since BigQuery separated compute and processing with storage to start, it can now treat S3 as if it’s like data stored in GCS and run analytics on multiple clouds. The big advantage here is reduced cost in network egress charges and the removal of data migration burden to use BigQuery in the first place. With Anthos, BigQuery Omni acts almost as an AWS Marketplace solution running analytics inside AWS natively.

现在，BigQuery Omni在AWS内部的Anthos上运行，以直接访问S3和其他数据库中的数据。 由于BigQuery从存储开始就将计算和处理分离开来，因此现在可以将S3像存储在GCS中的数据一样对待，并在多个云上运行分析。 此处的最大优势是降低了网络出口费用的成本，并且消除了首先使用BigQuery的数据迁移负担。 借助Anthos，BigQuery Omni几乎充当了AWS本地解决方案，在AWS内部运行分析。

Google Cloud Blog Google Cloud Blog

战略意义 (Strategic Implications)

AWS continues to lead the $100 billion cloud market by a wide margin with Microsoft Azure carving out its space with the acquisition of Github to appeal to developers and winning the $10 billion Pentagon JEDI cloud contract. This has squeezed Google Cloud into a distant third place, even given a growing cloud market. The bright side for Google — and perhaps the justification to the Anthos and BigQuery Omni strategy — is that more companies are embracing multi- and hybrid cloud solutions.

AWS继续遥遥领先于1000亿美元的云市场 ，微软Azure通过收购Github扩大了自己的市场 ，以吸引开发人员并赢得了价值100亿美元的五角大楼JEDI云合同 。 即使在云市场不断增长的情况下，这也将Google Cloud挤到了遥远的第三位。 对于Google来说，好的一面(也许是Anthos和BigQuery Omni策略的理由)是，越来越多的公司开始采用多云和混合云解决方案。

A recent Gartner research survey on cloud adoption revealed that more than 80% of respondents using the public cloud were using more than one cloud service provider.

Gartner最近对云采用情况进行的一项调查显示，使用公共云的受访者中有80％以上使用了不止一个云服务提供商。

- Gartner, The Future of Cloud Data Management Is Multicloud

-Gartner，云数据管理的未来是多云

Market leaders like AWS and Azure have no motivation to pursue a multi-cloud product. Their goal is to gobble up as much of the market and lock them into their cloud platform. Google, on the other hand, has no choice and is better positioned to play the middleware card and move up the stack. Although Google owns massive infrastructure of its own to power search, maps, and email for billions of users, rather than attempting to steal enterprise customers away from the incumbent giants, it has decided that offering multi-cloud services and treating existing data siloes as potential data sources for its products would be more profitable and competitive.

像AWS和Azure这样的市场领导者没有动机去追求多云产品。 他们的目标是吞噬尽可能多的市场并将其锁定在他们的云平台中。 另一方面，谷歌别无选择，处于更好的位置，可以发挥中间件卡并向上发展。 尽管Google拥有自己的庞大基础设施，可以为数十亿用户提供搜索，地图和电子邮件服务，而不是试图从企业巨头手中抢走企业客户，但Google决定提供多云服务并将现有的数据孤岛视为潜在机会其产品的数据源将更具盈利性和竞争力。

statista statista

From this perspective, Google Cloud’s competition may really be IBM rather than AWS and Azure. IBM acquired Red Hat for $34 billion in 2019, betting on the same open, hybrid cloud strategy with OpenShift, a popular enterprise Kubernetes platform. Google clearly has an advantage in Kubernetes as the creators (not to mention its 15+ years experience of running Borg, Google’s original container management system that Kubernetes is based on) and continues to widen its lead with active contributions to Kubernetes, Istio, and container technology. Combined with Kurian’s experience running the Fusion Middleware product at Oracle, Google seems well-positioned to grow Anthos and BigQuery Omni as the next massive scale middleware product in the cloud.

从这个角度来看，Google Cloud的竞争对手可能实际上是IBM，而不是AWS和Azure。 IBM在2019年以340亿美元的价格收购了Red Hat，押注与流行的企业Kubernetes平台OpenShift相同的开放式混合云战略。 Google作为创建者显然在Kubernetes中具有优势(更不用说它在运行Borg方面的15年以上的经验了，Borg是Kubernetes所基于的Google原始容器管理系统)，并且继续通过对Kubernetes，Istio和容器的积极贡献来扩大其领先地位。技术。 结合Kurian在Oracle上运行融合中间件产品的经验，Google似乎有条件将Anthos和BigQuery Omni成为云中的下一个大规模中间件产品。

So what’s next for Anthos? The obvious answer is extending the product line to support other databases: Cloud SQL, Dataproc, BigTable, and Spanner. Personally, I’m more interested in how Google uses Looker to entice users looking for an alternative solution to AWS Quicksight or Azure PowerBI. The other interesting avenue is extending Firebase for mobile development and leveraging the existing ecosystem to expand the “middleware” market. Finally, the big question is whether or not this strategy will also accelerate the widespread adoption of AI/ML technologies. Google is widely regarded as a leader in this space, and integrating BigQuery Omni with its existing AI Platform products (i.e. kubeflow, TensorFlow, AI hub/managed Jupyter notebooks, and Kaggle) may be the final piece needed to help enterprise companies adopt AI/ML.

那么Anthos的下一步是什么？ 显而易见的答案是扩展产品线以支持其他数据库：Cloud SQL，Dataproc，BigTable和Spanner。 就个人而言，我对Google如何使用Looker来吸引用户寻找AWS Quicksight或Azure PowerBI的替代解决方案感兴趣。 另一个有趣的途径是将Firebase扩展到移动开发，并利用现有的生态系统来扩展“中间件”市场。 最后，最大的问题是该策略是否还会加速AI / ML技术的广泛采用。 Google被广泛视为这一领域的领导者，将BigQuery Omni与现有的AI平台产品(即kubeflow ， TensorFlow ， AI集线器/托管的Jupyter笔记本和Kaggle )相集成可能是帮助企业采用AI / ML。

This isn’t to say that Google Cloud is without competition. AWS Outposts and Azure Stacks provide similar functionality to run their respective infrastructure on hybrid environments. I also wrote about how that same narrative is playing out in the IoT space with AWS Wavelength, Azure Edge Zone, and Anthos for Telecom. Finally, we also have SaaS companies like Snowflake and MongoDB focusing on multi-cloud database technology with true no vendor lock-in. Only time will tell if Google’s strategy to move up the stack will be successful or be marked as yet another futile attempt to dethrone AWS.

这并不是说Google Cloud没有竞争。 AWS Outposts和Azure Stack提供了类似的功能，可在混合环境中运行各自的基础架构。 我还写了关于 AWS Wavelength ， Azure Edge Zone和Anthos for Telecom 在IoT领域如何发挥同样的叙事的文章 。 最后，我们还拥有Snowalake和MongoDB等SaaS公司，他们专注于真正没有供应商锁定的多云数据库技术。 只有时间能证明Google的提升策略成功或被标记为再次废除AWS的徒劳尝试。

翻译自: https://medium.com/dataseries/why-bigquery-omni-is-a-big-deal-e7e696b4cd60

bigquery