Virtual Kubernetes clusters (vClusters) have the potential to bring Kubernetes adoption to the next level. They run in a physical Kubernetes cluster and can be used in the same way as normal clusters, but they are still just a virtual construct. For a detailed definition and description of virtual Kubernetes clusters, take a look at this article.
虚拟Kubernetes集群(vClusters)有潜力将Kubernetes的采用推向新的高度。 它们在物理Kubernetes集群中运行,并且可以与普通集群相同的方式使用,但它们仍然只是虚拟结构。 有关虚拟Kubernetes集群的详细定义和描述,请参阅本文 。
Similarly to virtual machines that revolutionized the use of physical servers, virtual Kubernetes clusters have some benefits compared to physical clusters that make them particularly useful for some scenarios.
与彻底改变物理服务器使用方式的虚拟机类似,虚拟Kubernetes集群与物理集群相比具有一些优势,这使其在某些情况下特别有用。
In this article, I will describe the benefits of virtual Kubernetes clusters and provide some use cases in which vClusters are more advantageous than other solutions such as many individual clusters or namespace-based multi-tenancy.
在本文中,我将描述虚拟Kubernetes集群的好处,并提供一些用例,其中vClusters比其他解决方案(例如许多单独的集群或基于命名空间的多租户)更具优势。
虚拟Kubernetes集群的好处 (Benefits of Virtual Kubernetes Clusters)
The benefits of virtual clusters for Kubernetes are mainly based on two characteristics:
虚拟集群对Kubernetes的好处主要基于两个特征:
- Sharing of a physical cluster 共享物理集群
- Isolation 隔离
集群共享/多租户 (Cluster sharing/multi-tenancy)
Since vClusters are a virtual abstraction within Kubernetes, it is possible to run many vClusters on just a single physical cluster, which has the following advantages:
由于vClusters是Kubernetes中的虚拟抽象,因此可以在单个物理群集上运行许多vClusters,这具有以下优点:
- Reduced management effort. Since there is only one physical cluster to maintain, the administrative effort is significantly reduced by virtual clusters. This becomes especially clear when comparing it to another alternative that would lead to a similar outcome from a user perspective: Instead of running virtual clusters in one physical cluster, it is possible to run many physical clusters that all have to be maintained, which can become infeasible pretty fast even in only mid-sized teams. Additionally, the physical cluster can be configured in a pretty “basic” way without extensive additional installations, as most of this will happen on the level of the vCluster. 减少管理工作量。 由于只有一个物理群集要维护,因此虚拟群集可以显着减少管理工作。 当将它与另一个替代方案进行比较时,这一点尤其明显,从用户的角度来看,它会导致类似的结果:代替在一个物理集群中运行虚拟集群,可以运行许多必须维护的物理集群,这可能成为现实。即使仅在中型团队中,这也是不可行的。 此外,物理群集可以以相当“基本”的方式进行配置,而无需进行大量额外的安装,因为大多数操作将在vCluster级别上进行。
- Reduced cost. Besides the reduced management effort that also results in cost reductions, virtual clusters are also more efficient in utilizing computing resources because the resources are shared by the tenants. Again, similar efficiency improvements were gained by introducing virtual machines to a physical server infrastructure. This cost efficiency is further increased by the disposable nature of virtual clusters. They can be “thrown away” (shut off) when they are not needed. Alternatively, they can be “put to sleep” (scaled down), a process that can even be automized (e.g. with a sleep mode). 降低成本。 除了减少管理工作量(还可以降低成本)之外,虚拟集群在利用计算资源方面也更加有效,因为这些资源由租户共享。 同样,通过将虚拟机引入物理服务器基础架构也获得了类似的效率提升。 虚拟集群的可抛弃性进一步提高了这种成本效率。 不需要它们时,可以将它们“扔掉”(关闭)。 或者,它们可以“进入睡眠”(按比例缩小),该过程甚至可以自动化(例如,使用睡眠模式)。
隔离/硬多租户 (Isolation/hard multi-tenancy)
Since virtual Kubernetes clusters provide a harder form of multi-tenancy (i.e. users are strictly isolated from each other), they have some additional benefits — especially in comparison to a namespace-based multi-tenancy approach:
由于虚拟Kubernetes集群提供了更难的多租户形式(即用户彼此严格隔离),因此它们具有一些额外的好处,尤其是与基于命名空间的多租户方法相比:
- Stable system. Using virtual Kubernetes clusters does not compromise the stability of the system. Even if a virtual cluster fails, the underlying physical cluster is usually not affected by this. Only in some extreme cases, the failure of a virtual cluster would lead to a failure of the physical cluster, which is again similar to a VM that can rarely lead to a break in the underlying physical machine. This is regardless of the source of the error, which can come from within the cluster (e.g. an engineer accidentally breaks something) or from outside it (e.g. a malicious attack on the system). This makes the whole system more resilient and helps to implement a reasonable microservice architecture with a true separation of concerns for the individual services. 系统稳定。 使用虚拟Kubernetes集群不会损害系统的稳定性。 即使虚拟群集出现故障,通常也不会影响基础物理群集。 仅在某些极端情况下,虚拟群集的故障才会导致物理群集的故障,这又类似于VM,它很少会导致基础物理计算机的中断。 这与错误的来源无关,它可能来自群集内部(例如,工程师意外破坏了某些东西),也可能来自群集外部(例如,对系统的恶意攻击)。 这使整个系统更具弹性,并有助于实现合理的微服务体系结构,并将各个服务的关注点真正分开。
- Full flexibility. While every engineer works or every microservice runs on the same physical cluster, their virtual clusters are completely independent. This allows the vClusters to also be configured in very different ways. For example, the virtual clusters can have different Kubernetes versions or different API server configurations. This, again, enables engineers to work freely and to use whatever is best for their use case without having to consider other requirements or the underlying physical Kubernetes cluster. 充分的灵活性。 当每个工程师工作或每个微服务都在同一物理群集上运行时,他们的虚拟群集是完全独立的。 这允许vClusters也可以以非常不同的方式进行配置。 例如,虚拟集群可以具有不同的Kubernetes版本或不同的API服务器配置。 同样,这使工程师可以自由工作,并使用适合其用例的最佳方法,而不必考虑其他要求或基础的物理Kubernetes集群。
Overall, virtual Kubernetes clusters are the only approach to combine efficiency with a stable, flexible system due to its hard multi-tenancy with Kubernetes-native resources. The alternative of using many individual clusters is only able to resolve the isolation issue but creates a huge cost burden, while the approach of namespace-based soft multi-tenancy keeps cost reasonable but can only provide limited stability and flexibility. Virtual clusters are thus so far the only option for companies that want to have the best of both worlds.
总体而言,虚拟Kubernetes集群是将效率与稳定,灵活的系统相结合的唯一方法,这是因为其具有Kubernetes本地资源的硬多租户功能。 使用许多单个群集的替代方案只能解决隔离问题,但会带来巨大的成本负担,而基于命名空间的软多租户方法使成本合理,但只能提供有限的稳定性和灵活性。 因此,到目前为止,虚拟集群是想要兼顾两全其美的公司的唯一选择。
用例 (Use Cases)
Since virtual clusters mostly behave like regular Kubernetes clusters, their scope of application is similarly broad. For this, I will concentrate on three main use cases that are not covered well by alternative solutions with namespaces or a multitude of physical clusters.
由于虚拟集群的行为与常规Kubernetes集群大致相同,因此它们的应用范围同样广泛。 为此,我将集中讨论三个主要用例,这些用例在具有名称空间或大量物理集群的替代解决方案中没有很好地涵盖。
CI / CD和测试 (CI/CD & testing)
- Requirements: For CI/CD and testing scenarios in a Kubernetes environment, the engineers need to have access to Kubernetes whenever they need it. To keep costs low, the environment should only be running if it is actually being used. Engineers should also be able to modify the Kubernetes configuration to get more realistic tests. 要求:对于Kubernetes环境中的CI / CD和测试场景,工程师需要在需要时有权访问Kubernetes。 为了降低成本,只有在实际使用环境的情况下,环境才应该运行。 工程师还应该能够修改Kubernetes配置以获得更实际的测试。
- Problem: Creating a Kubernetes cluster takes some time, even in public clouds. Since computing resources are costly and many CI/CD pipelines are billed by the minute, it is often not feasible to wait 30 minutes until a cluster is started. This would also slow down the feedback loop for the engineer’s actions and interrupt their workflow. Therefore, a cluster is often simply shared by engineers for testing, which again can lead to waiting times if a colleague is using it at the same time. It is also highly inefficient, as such a cluster is always running and costing money even if no tests or pipelines are running. 问题:即使在公共云中,创建Kubernetes集群也需要一些时间。 由于计算资源昂贵,并且许多CI / CD管道按分钟计费,因此,等待30分钟直到启动群集通常是不可行的。 这也会减慢工程师动作的反馈循环并中断他们的工作流程。 因此,群集通常由工程师简单共享以进行测试,如果同事同时使用它,则又可能导致等待时间。 它也是非常低效的,因为即使没有测试或管道在运行,这样的集群也总是在运行,并且要花钱。
- Virtual cluster solution: Virtual clusters have the advantage of being able to start in a few seconds, so it is possible to create them on-demand by the engineers. Since they are full-fledged clusters that are only used by one engineer or application at a time, they can be freely configured and adapted to the individual situation. After the completion of a test or CI/CD process, they can also be thrown away without any issues, which keeps the cost for computing resources as low as possible. As such, virtual Kubernetes clusters are a perfect fit for CI/CD and testing scenarios. 虚拟集群解决方案:虚拟集群的优势在于能够在几秒钟内启动,因此可以由工程师按需创建它们。 由于它们是成熟的集群,一次只能由一位工程师或应用程序使用,因此可以自由配置它们并使其适应各种情况。 在完成测试或CI / CD流程后,也可以将它们扔掉而没有任何问题,从而使计算资源的成本尽可能低。 因此,虚拟Kubernetes集群非常适合CI / CD和测试方案。
云原生开发 (Cloud-native development)
Requirements: More and more companies want to give developers direct access to Kubernetes during development. For this, the engineers need to have Kubernetes access they can work and experiment with while keeping the costs and management efforts for these environments low.
要求:越来越多的公司希望在开发过程中为开发人员提供直接访问Kubernetes的权限 。 为此,工程师需要具有Kubernetes访问权限,以便他们可以工作和进行试验,同时保持这些环境的成本和管理工作量较低。
- Problem: Individual clusters for every engineer are often not feasible, as this is an expensive solution that requires some Kubernetes knowledge and leads to high maintenance efforts. For this, namespace-based multi-tenancy is already used sometimes. However, the lack of strong isolation leaves the engineers with some risk of breaking the whole system. They usually do not have admin access to the cluster either, so they are limited in terms of configuration options. 问题:对于每个工程师来说,单独的集群通常是不可行的,因为这是一个昂贵的解决方案,需要一定的Kubernetes知识并需要大量维护。 为此,有时已经使用了基于名称空间的多租户。 但是,缺乏强大的隔离性使工程师有破坏整个系统的风险。 他们通常也没有对群集的管理员访问权限,因此在配置选项方面受到限制。
- Virtual cluster solution: Again, virtual Kubernetes clusters can be started by the engineers on-demand and without much prior Kubernetes knowledge. The engineers are then isolated very well from each other and work in secure dev sandboxes without the fear of affecting others. At the same time, this approach is very cost-efficient because only a low management effort is needed with a single physical cluster, the resources can be shared, and the virtual clusters can be easily deleted. Instead of deleting the virtual clusters, it is also possible to put them to sleep with a sleep mode. This allows developers to resume their work where they ended while no computing costs are created in the meantime. 虚拟集群解决方案:同样,虚拟Kubernetes集群可以由工程师按需启动,而无需太多的Kubernetes知识。 这样,工程师彼此之间就可以很好地隔离开来,并且可以在安全的开发沙箱中工作,而不必担心会影响其他人。 同时,这种方法具有很高的成本效益,因为单个物理群集只需要很少的管理工作,可以共享资源,并且可以轻松删除虚拟群集。 除了删除虚拟集群之外,还可以通过睡眠模式使它们进入睡眠状态。 这使开发人员可以在他们结束的地方继续工作,而同时不会产生任何计算成本。
AI + ML实验 (AI+ML experiments)
- Requirements: For artificial intelligence and machine learning applications, a lot of computing resources are often needed. To run experiments, engineers thus need access to a cloud environment with a lot of power. They also need to be able to easily replicate these environments to rerun the experiment with different parameters or to run them in parallel. 要求:对于人工智能和机器学习应用程序,通常需要大量的计算资源。 因此,为了进行实验,工程师需要访问功能强大的云环境。 他们还需要能够轻松复制这些环境,以使用不同的参数重新运行实验或并行运行它们。
- Problem: Due to the high resource requirements, cost is a major issue for AI and ML experiments. It is thus not reasonable to have an experimentation environment always running, as this would create high costs even if it is not used. From a workflow perspective, a shared environment leads to waiting times for engineers if another experiment is still running. The same goes for individual clusters that take long to start and therefore also interrupt the engineers’ workflows. 问题:由于高资源需求,成本是AI和ML实验的主要问题。 因此,始终运行实验环境是不合理的,因为即使不使用它也会产生高昂的成本。 从工作流的角度来看,如果另一个实验仍在运行,则共享环境会导致工程师等待时间。 对于需要很长时间启动的单个集群也是如此,因此也会中断工程师的工作流程。
- Virtual cluster solution: Virtual clusters are a great alternative for artificial intelligence and machine learning engineers who want to run experiments. They can be created within seconds when they are needed and are very replicable, so it is even possible to run experiments in parallel, which can accelerate many workflows. Even though vClusters are almost instantly available, they do not run idle, as they can be easily shut off after the experiment is over. The costs for these expensive experiments are thus minimized. 虚拟集群解决方案:虚拟集群是想要运行实验的人工智能和机器学习工程师的理想选择。 它们可以在需要时在几秒钟内创建,并且具有很高的可复制性,因此甚至可以并行运行实验,从而可以加速许多工作流程。 即使vCluster几乎立即可用,它们也不会空闲,因为在实验结束后它们很容易关闭。 这些昂贵的实验的成本因此被最小化。
存在哪些虚拟集群解决方案 (Which Virtual Cluster Solutions Exist)
Virtual Kubernetes clusters are still a very new topic, but they are already available today. The multi-tenancy working group has presented an experimental solution in this area. k3v by Darren Shepherd is further proof of concept implementation in the open-source community.
虚拟Kubernetes集群仍然是一个非常新的话题,但是今天它们已经可用。 多租户工作组提出了这一领域的实验解决方案。 Darren Shepherd的k3v进一步证明了开源社区中概念的实现。
结论 (Conclusion)
The concept of virtual Kubernetes clusters is still very new but could be a big advancement for Kubernetes, and similarities to the groundbreaking move from physical servers to virtual machines are striking. The combination of a shared cluster with strong isolation merges efficiency with resilience and flexibility. This also makes Kubernetes more attractive for use cases where it is currently still hard to implement, such as CI/CD, testing, cloud-native development, and AI/ML experimentation. Therefore, it could spur Kubernetes adoption even further.
虚拟Kubernetes集群的概念仍然很新,但是对于Kubernetes来说可能是一个很大的进步,并且与从物理服务器到虚拟机的突破性发展相似。 共享群集与强隔离性的结合提高了效率,弹性和灵活性。 这也使Kubernetes对于目前仍难以实现的用例更具吸引力,例如CI / CD,测试,云原生开发和AI / ML实验。 因此,这可能会进一步推动Kubernetes的采用。