文章目录
总目录
- 读我
- 关于CM
CM介绍:这篇文档介绍ChaosMesh的概念,用例,核心优势和架构
基本功能:这篇文档描述了CM的基本特性,包括错误注入,混沌工作流,可视化操作和安全保证 - 安装&部署
快速开始:这篇文章介绍如何在测试或本地环境开始CM
使用Helm安装:这篇文章讲如何在生产环境安装CM
离线安装:这份文档讲如何离线安装CM
卸载CM:这篇文档讲如何卸载CM,包括使用helm卸载或手工卸载 - 管理用户许可:这篇文章讲如何在CM中管理用户许可,包括创建不同角色的用户账号,绑定许可,管理令牌,启用或停用许可授权
- 配置混沌实验命名空间:这个章节带你在指定的命名空间配置混沌实验,从而避免其他没有指定的命名空间里注入错误
- 运行一个混沌实验
定义混沌实验范围:这篇文档讲如何定义混沌实验的范围从而精确的控制混沌错误爆炸范围
定义调度规则:这篇文档讲如何使用CM创建调度任务,从而在一个固定时间创建混沌实验
运行混沌实验:这篇文档讲如何在CM中创建,运行,观察,暂停,更新和删除混沌实验
查看混沌实验结果:这篇文档讲如何使用CM查看混沌实验的运行状态和结果 - 编排多个混沌实验
创建CM工作流:
并行&串行实验
向工作流发送HTTP请求:类似K8s Job,CM工作流对任何任务都支持任务节点,为了让用户体验更方便,CM在web界面提供一个基于Task的模板创建HTTP请求
检查工作流状态: - 混沌实验类型
- Kubernetes
- Pod错误:这篇文档讲如何使用CM将错误注入pod中从而模拟pod或容器错误
- 网络错误:这篇文章讲如何使用NetworkChaos模拟网络错误
- 压力场景:CM提供StressChaos在容器内模拟压力场景,这篇文章将如何创建StressChaos实验和准备相应配置文件
- 文件IO错误:这篇文档讲如何使用CM创建IO混沌
- DNS错误:这篇文档讲如何在CM中创建DNS混沌实验从而模拟DNS错误
- 时间错误:CM提供TimeChaos实验类型,你可以使用这个类型模拟时间偏移场景,这篇文章讲如何创建一个TimeChaos实验及相关配置文件
- JVM应用错误:
- Linux内核错误:这篇文章讲如何使用KernelChaos模拟内核错误,这个特性使用BPF在内核相关路径注入IO相关,内存相关错误
- HTTP错误:这篇文章讲如何通过CM创建HTTPChaos来模拟HTTP错误
- 物理节点
- Chaosd介绍:Chaosd是CM提供的混沌引擎测试工具
- 物理机错误:这篇文档讲如何通过CM制造物理机混沌来在物理机或虚拟机上模拟网络,硬盘,时间,jvm等错误
- 进程错误:这篇文章讲如何使用chaosd模拟进程错误,这个错误使用go语言提供的杀死指令来模拟进程被杀死或停止的场景
- 网络错误:这篇文章讲如何使用chaosd模拟网络错误,这个模拟可以使用 iptables, ipsets, tc等通过修改路由和流量控制来完成
- 宿主机错误:这篇文章讲如何使用chaosd模拟宿主机关闭的错误
- 压力场景:这篇文章讲如何使用chaosd模拟压力场景,这个特性使用stress-ng在宿主机上产生CPU和内存压力
- 硬盘错误:这篇文章讲如何通过Chaosd来模拟硬盘错误,这个特性帮助你模拟硬盘读写负载(通过dd)和硬盘填充(通过dd或者fallocate)
- JVM应用错误:chaosd使用Byteman模拟jvm错误
- 时间错误:这篇文章描述如何使用chaosd模拟时间偏移的场景.你可以通过命令行模式或者服务模式创建该场景
- 查找和恢复Chaosd实验:你可以通过条件来查找实验并通过Chaosd使用UID来恢复实验.这篇文档讲如何通过chaosd查找和恢复实验并提供相关例子
- Kubernetes
正文
原文 | 翻译 |
---|---|
This document introduces how to use Chaosd to simulate network faults. The simulations can be completed by modifying network routing and traffic flow control using iptables, ipsets, tc, etc. | 这篇文章讲如何使用chaosd模拟网络错误,这个模拟可以使用 iptables, ipsets, tc等通过修改路由和流量控制来完成 |
note Make sure the NET_SCH_NETEM module is installed in the Linux kernel. If you are using CentOS, you can install the module through the kernel-modules-extra package. Most other Linux distributions have installed it already by default. | 注意 确认Linux内核安装了NET_SCH_NETEM模块.如果你使用CentOS,你可以通过kernel-modules-extra来安装此模块.大多数其他Linux发行版默认已安装 |
Create network fault experiments using command-line mode 使用命令行创建网络故障实验
原文 | 翻译 |
This section introduces how to create network fault experiments using command-line mode. | 这一节介绍如何通过命令行模式创建网络失败实验 |
Before creating an experiment, you can run the following command to check the types of network faults supported by Chaosd: | 在创建实验前,你可以运行下面命令来检查Chaosd支持的网络故障类型 |
chaosd attack network --help | |
The output is as follows: | 输出如下 |
Network attack related commands Usage: chaosd attack network [command] Available Commands: corrupt corrupt network packet delay delay network duplicate duplicate network packet loss loss network packet Flags: -h, --help help for network Global Flags: --log-level string the log level of chaosd, the value can be 'debug', 'info', 'warn' and 'error' Use "chaosd attack network [command] --help" for more information about a command. | |
Currently, you can simulate four experimental scenarios using Chaosd: network corruption, network latency, network duplication, and network loss. | 现在,你可以通过Chaosd模拟四种实验:网络损坏,网络延时,网络重复和网络丢失 |
Network corruption网络损坏
原文 | 翻译 |
---|---|
You can run the command below to see the configuration of simulated network corruption using Chaosd. | 你可以运行下面指令来查看使用Chaosd模拟网络损坏的配置 |
The command for network corruption网络损坏的指令
原文 | 翻译 |
---|---|
The command is as follows: | 指令如下 |
chaosd attack network corrupt --help | |
The output is as follows: | 输出如下 |
corrupt network packet Usage: chaosd attack network corrupt [flags] Flags: -c, --correlation string correlation is percentage (10 is 10%) (default "0") -d, --device string the network interface to impact -e, --egress-port string only impact egress traffic to these destination ports, use a ',' to separate or to indicate the range, such as 80, 8001:8010. It can only be used in conjunction with -p tcp or -p udp -h, --help help for corrupt -H, --hostname string only impact traffic to these hostnames -i, --ip string only impact egress traffic to these IP addresses --percent string percentage of packets to corrupt (10 is 10%) (default "1") -p, --protocol string only impact traffic using this IP protocol, supported: tcp, udp, icmp, all -s, --source-port string only impact egress traffic from these source ports, use a ',' to separate or to indicate the range, such as 80, 8001:8010. It can only be used in conjunction with -p tcp or -p udp Global Flags: --log-level string the log level of chaosd, the value can be 'debug', 'info', 'warn' and 'error' |
Configuration items related to network corruption网络损坏相关配置项
The related configuration items are described as follows:
相关配置项如下描述:
Configuration item | 配置项 | Abbreviation/缩写 | Description | 描述 | Value | 值 |
---|---|---|---|---|---|---|
correlation | 相关性 | c | The correlation between the percentage of current corrupt occurrence and the previous occurrence. | 当前包错误与前一次相关百分比 | Int. It is a percentage ranging from 0 to 100 (10 is 10%) (“0” by default ). | 整型.它是一个从0到100的百分比范围(10是10%,默认是0) |
device | 设备 | d | Name of the impacted network interface card. | 受冲击的网卡名称 | String, such as “eth0”. The value is required. | 字符串,例如eth0,这个值是必填项 |
egress-port | 出端口 | e | The egress traffic that only impacts specific destination ports. It can only be configured when the protocol is TCP or UDP. | 出口流量仅仅影响特定目的端口.它只有在协议是TCP或UDP时配置 | String. You need to use a ‘,’ to separate the specific port or to indicate the range of the port, such as “80,8001:8010”. | 字符串.你需要使用’,'来分隔指定的端口或者指明端口范围,例如80,8001:8010 |
hostname | 主机名 | H | The host name impacted by traffic. | 受流量影响的主机名 | String, such as “chaos-mesh.org”. | 字符串,例如chaos-mesh.org |
ip | ip | i | The IP address impacted by egress traffic. | 受出口流量影响的IP地址 | String, such as “123.123.123.123”. | 字符串,例如123.123.123.123 |
protocol | 协议 | p | The IP protocol impacted by traffic. | 受流量影响的IP协议 | String. Supported protocols: tcp, udp, icmp, all (all network protocols). | 字符串,支持的协议:tcp,udp,icmp,all(所有网络协议) |
source-port | 源端口 | s | The egress traffic which only impact specific source ports. It can only be configured when the protocol is tcp or udp. | 出口流量仅仅影响特定目的端口.它只有在协议是TCP或UDP时配置 | String. Use a ‘,’ to delimit the specific port or to indicate the range of the ports, such as “80,8001:8010”. | 字符串.你需要使用’,'来分隔指定的端口或者指明端口范围,例如80,8001:8010 |
An example of network corruption一个包损坏的例子
Run the following command to simulate network corruption:
运行下面命令来模拟网络包损坏
chaosd attack network corrupt -d eth0 -i 172.16.4.4 --percent 50
If the command runs successfully, the output is as follows:
如果这个指令成功执行,有如下输出
Attack network successfully, uid: 4eab1e62-8d60-45cb-ac85-3c17b8ac4825
Network latency网络延迟
You can run the command below to see the configuration of simulated network latency using Chaosd.
你可以运行下列指令来查看使用Chaosd来模拟网络延迟时的配置项
The command for network latency网络延迟指令
The command is as follows:
指令如下
chaosd attack network delay --help
The output is as follows:
输出如下
delay network
Usage:
chaosd attack network delay [flags]
Flags:
-c, --correlation string correlation is percentage (10 is 10%) (default "0")
-d, --device string the network interface to impact
-e, --egress-port string only impact egress traffic to these destination ports, use a ',' to separate or to indicate the range, such as 80, 8001:8010. It can only be used in conjunction with -p tcp or -p udp
-h, --help help for delay
-H, --hostname string only impact traffic to these hostnames
-i, --ip string only impact egress traffic to these IP addresses
-j, --jitter string jitter time, time units: ns, us (or µs), ms, s, m, h.
-l, --latency string delay egress time, time units: ns, us (or µs), ms, s, m, h.
-p, --protocol string only impact traffic using this IP protocol, supported: tcp, udp, icmp, all
-s, --source-port string only impact egress traffic from these source ports, use a ',' to separate or to indicate the range, such as 80, 8001:8010. It can only be used in conjunction with -p tcp or -p udp
Global Flags:
--log-level string the log level of chaosd, the value can be 'debug', 'info', 'warn' and 'error'
Configuration items related to network latency网络延迟相关配置项
The related configuration items are described as follows:
相关配置项如下描述:
Configuration item | 配置项 | Abbreviation/缩写 | Description | 描述 | Value | 值 |
---|---|---|---|---|---|---|
correlation | 相关性 | c | The correlation between the current latency and the previous one. | Int. It is a percentage ranging from 0 to 100 (10 is 10%) (“0” by default). | ||
device | d | Name of the impacted network interface card. | String, such as “eth0”. The value is required. | |||
egress-port | e | The egress traffic which only impact specific destination ports. It can only be configured when the protocol is TCP or UDP. | String. You need to use a ‘,’ to separate the specific port or to indicate the range of the port, such as “80,8001:8010”. | |||
hostname | H | The host name impacted by traffic. | String, such as “chaos-mesh.org”. | |||
ip | i | The IP address impacted by egress traffic. | String, such as “123.123.123.123”. | |||
jitter | 抖动 | j | Range of the length of network delay time. | 网络延时长度范围 | String. The time units can be: ns, us (µs), ms, s, m, h, such as “1ms”. | 字符串,时间单位可以是ns,us(µs),ms,s,m,h例如1ms |
latency | l | Length of network delay time. | String. The time units can be: ns, us (μs), ms, s, m, h, such as “1ms”. | |||
protocol | p | The IP protocol impacted by traffic. | String. It supports the following protocol types: tcp, udp, icmp, all (all network protocols). | |||
source-port | s | The egress traffic that only impacts specified source ports. It can only be configured when the protocol is TCP or UDP. | String. You need to use a ‘,’ to separate the specific port or to indicate the range of the port, such as “80,8001:8010”. |
An example of network latency一个网络延时的例子
Run the following command to simulate network latency:
运行下列指令来模拟网络延迟
chaosd attack network delay -d eth0 -i 172.16.4.4 -l 10ms
If the command runs successfully, the output is as follows:
如果指令成功运行,输出如下
Attack network successfully, uid: 4b23a0b5-e193-4b27-90a7-3e04235f32ab
Network duplication网络包重复
You can run the command below to see the configuration of simulated network duplication using Chaosd:
你可以使用Chaosd运行下列指令来查看模拟网络包重复的配置项
The command for network duplication
The command is as follows:
命令如下:
chaosd attack network duplicate --help
The output is as follows:
输出如下:
duplicate network packet
Usage:
chaosd attack network duplicate [flags]
Flags:
-c, --correlation string correlation is percentage (10 is 10%) (default "0")
-d, --device string the network interface to impact
-e, --egress-port string only impact egress traffic to these destination ports, use a ',' to separate or to indicate the range, such as 80, 8001:8010. It can only be used in conjunction with -p tcp or -p udp
-h, --help help for duplicate
-H, --hostname string only impact traffic to these hostnames
-i, --ip string only impact egress traffic to these IP addresses
--percent string percentage of packets to duplicate (10 is 10%) (default "1")
-p, --protocol string only impact traffic using this IP protocol, supported: tcp, udp, icmp, all
-s, --source-port string only impact egress traffic from these source ports, use a ',' to separate or to indicate the range, such as 80, 8001:8010. It can only be used in conjunction with -p tcp or -p udp
Global Flags:
--log-level string the log level of chaosd, the value can be 'debug', 'info', 'warn' and 'error'
Configuration items related to network duplication网络包重复相关配置项
The related configuration items are described as follows:
相关配置项如下描述:
Configuration item | 配置项 | Abbreviation/缩写 | Description | 描述 | Value | 值 |
---|---|---|---|---|---|---|
correlation | c | The correlation between the percentage of current duplication occurrence and the previous one. | Int. It is a percentage which range is 0 to 100 (10 is 10%) (default “0”). | |||
device | d | Name of the impacted network interface card. | String, such as “eth0”. The value is required. | |||
egress-port | e | The egress traffic that only impacts specified destination ports. It can only be configured when the protocol is TCP or UDP. | String. You need to use a ‘,’ to separate the specific port or to indicate the range of the port, such as “80,8001:8010”. | |||
hostname | H | The host name impacted by traffic. | String, such as “chaos-mesh.org”. | |||
ip | i | The IP address impacted by egress traffic. | String, such as “123.123.123.123”. | |||
percent | 百分比 | none | Ratio of network packet duplicate. | 网络包重复比例 | Int. It is a percentage which range is 0 to 100 (10 is 10%) (default “1”). | 整型.它是一个范围从0到100的百分比(10表示10%,默认1) |
protocol | p | The IP protocol impacted by traffic. | String. It supports the following protocol types: tcp, udp, icmp, all (all network protocols). | |||
source-port | s | The egress traffic which only impact specific source ports. It can only be configured when the protocol is tcp or udp. | String. You need to use a ‘,’ to separate the specific port or to indicate the range of the port, such as “80,8001:8010”. |
An example of network duplication网络包重复的例子
Run the following command to simulate network duplication:
运行下列指令来模拟网络包重复
chaosd attack network duplicate -d eth0 -i 172.16.4.4 --percent 50
If the command runs successfully, the output is as follows:
如果指令成功运行,输出如下:
Attack network successfully, uid: 7bcb74ee-9101-4ae4-82f0-e44c8a7f113c
Network loss网络包丢失
You can run the command below to see the configuration of simulated network loss using Chaosd:
你可以运行下面指令来查看通过Chaosd模拟网络包丢失的配置项
The command for network loss网络包丢失的指令
The command is as follows:
命令如下:
chaosd attack network loss --help
The output is as follows:
输出如下
loss network packet
Usage:
chaosd attack network loss [flags]
Flags:
-c, --correlation string correlation is percentage (10 is 10%) (default "0")
-d, --device string the network interface to impact
-e, --egress-port string only impact egress traffic to these destination ports, use a ',' to separate or to indicate the range, such as 80, 8001:8010. It can only be used in conjunction with -p tcp or -p udp
-h, --help help for loss
-H, --hostname string only impact traffic to these hostnames
-i, --ip string only impact egress traffic to these IP addresses
--percent string percentage of packets to drop (10 is 10%) (default "1")
-p, --protocol string only impact traffic using this IP protocol, supported: tcp, udp, icmp, all
-s, --source-port string only impact egress traffic from these source ports, use a ',' to separate or to indicate the range, such as 80, 8001:8010. It can only be used in conjunction with -p tcp or -p udp
Global Flags:
--log-level string the log level of chaosd, the value can be 'debug', 'info', 'warn' and 'error'
Configuration items related to network loss网络包丢失相关配置项
The related configuration items are described as follows:
相关配置项如下描述
Configuration item | 配置项 | Abbreviation/缩写 | Description | 描述 | Value | 值 |
---|---|---|---|---|---|---|
correlation | c | The correlation between the percentage of the current network loss and the previous one. | Int. It is a percentage which range is 0 to 100 (10 is 10%) (default “0”). | |||
device | d | Name of the impacted network interface card. | String, such as “eth0”. The value is required. | |||
egress-port | e | The egress traffic that only impacts specified destination ports. It can only be configured when the protocol is TCP or UDP. | String. You need to use a ‘,’ to separate the specific port or to indicate the range of the port, such as “80,8001:8010”. | |||
hostname | H | The host name impacted by traffic. | String, such as “chaos-mesh.org”. | |||
ip | i | The IP address impacted by egress traffic. | String, such as “123.123.123.123”. | |||
percent | none | Ratio of network packet loss. | Int. It is a percentage which range is 0 to 100 (10 is 10%) (default “1”). | |||
protocol | p | Only impact traffic using this IP protocol. | String. It supports the following protocol types: tcp, udp, icmp, all (all network protocols). | |||
source-port | s | The egress traffic which only impact specific source ports. It can only be configured when the protocol is tcp or udp. | String. You need to use a ‘,’ to separate the specific port or to indicate the range of the port, such as “80,8001:8010”. |
An example of network loss一个网络包丢失的例子
Run the following command to simulate network loss:
运行下列指令模拟网络包丢失
chaosd attack network loss -d eth0 -i 172.16.4.4 --percent 50
If the command runs successfully, the output is as follows:
如果指令成功运行,输出如下
Attack network successfully, uid: 1e818adf-3942-4de4-949b-c8499f120265
Create network fault experiments using service mode使用服务模式创建网络故障
(To be added)
(待添加)