记一次k8s集群表征6443:connect refused,无法使用kubectl get pod的问题解决

10 篇文章 0 订阅
2 篇文章 0 订阅

集群概况(就一个master节点):
master 192.168.1.1
发现问题:(在一个看似平凡的上午,突然集群出现了这样反应)
问题表征图

发现问题后的一个思路:
解决问题的思路
步骤:

  1. 开始查看防火墙的6443是否打开,(需要打开—zone=internal方向的防火墙),检查后没问题。
  2. swap是否关闭,本集群检查后没问题。
    关闭方式:
    ① 注释 /etc/fstab 的swap注释(永久关闭swap)。
    ② swappoff -a (暂时关闭swap)。
    ③ 查看是否关闭 free -m
[root@km1 ~]# free -m
              total        used        free      shared  buff/cache   available
Mem:         257117        8944      239641          20        8530      247290
Swap:             0           0           0

swap现在是关闭了。
3.既然kube-apiserver经常重启,logs又看不出什么异常,是否是docker本身的问题,查看docker的/etc/docker/daemon.json发现这条“native.cgroupdriver=systemd”没有了,加上它:

{
 "registry-mirrors": [
    "https://dockerhub.azk8s.cn",
    "https://reg-mirror.qiniu.com",
    "https://quay-mirror.qiniu.com"
  ],
  "exec-opts": [ "native.cgroupdriver=systemd" ]
}

然后重启docker:

systemctl daemon-reload //重新加载docker服务的配置文件
systemctl restart docker //重启docker

哦吼!失败!重启失败!好吧,看来docker有问题,重装!(我承认我的方法比较低端,哈哈哈)
重装之路:
①我们安装k8s是1.18.0所以通过查询其配套的docker版本是

k8s-1.18对应docker版本 1.13.1, 17.03, 17.06, 17.09, 18.06, 18.09

我们安装18.09版本。
安装命令:

yum install -y docker-ce-18.09.9 docker-ce-cli-18.09.9 containerd.io

设置阿里源:

yum-config-manager \
    --add-repo \
    http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

设置/etc/docker/daemon.json,修改cgroupdriver:

{
 "registry-mirrors": [
    "https://dockerhub.azk8s.cn",
    "https://reg-mirror.qiniu.com",
    "https://quay-mirror.qiniu.com"
  ],
  "exec-opts": [ "native.cgroupdriver=systemd" ]
}

重启docker:

systemctl daemon-reload //重新加载docker服务的配置文件
systemctl restart docker //重启docker

然后就可以通过docker info | grep Cgroup来查看修改后的 docker cgroup 状态,发现变为systemd即为修改成功。
4.用journalctl -fu kubelet查看kubelet的日志,会发现大量的这样报错:
kubelet报错截图
node master 找不到?WTF?就一个节点还有这幺蛾子?防火墙还防自己?索性直接iptables -F刷了iptables规则(要非常小心执行这个,最好先备份,因为我是测试节点,而且就一个节点,刷也就刷了),还是不行!WTF?难道?难道是?对!去看/etc/hosts文件:
果然有一些端倪,大家上眼:
/etc/hosts文件
有可能是对192.168.1.1的双重定义导致的无法找到master节点,我们作以下处理,注释掉192.168.1.1 km1的字段:
注释掉192.168.1.1 km1的字段

保存退出,然后

reboot

一定要reboot,哈哈哈哈哈
一定要记得
reboot
reboot
reboot

最后,成功!!!Got it!
总结
排查思路:
①防火墙端口。
②swap关闭。
③docker是否正常cgroupdriver=systemd。
④修改了/etc/hosts文件。

  • 8
    点赞
  • 12
    收藏
    觉得还不错? 一键收藏
  • 1
    评论
TABLE OF CONTENTS PART I: SOFTWARE ENGINEERING AND TESTING Chapter 1. Introduction to Software Engineering 3 1.1 Introduction to Software 3 1.2 Types of Software 5 1.3 Classes of Software 8 1.4 Introduction to Software Engineering 9 1.5 Software Components 11 1.6 Software Characteristics 12 1.7 Software Crisis 13 1.8 Software Myths 15 1.9 Software Applications 15 1.10 Software-Engineering Processes 18 1.11 Evolution of Software 20 1.12 Comparison of Software Engineering and Related Fields 22 1.13 Some Terminologies 25 1.14 Programs Versus Software Products 26 Chapter 2. Software-Development Life-Cycle Models 29 2.1 Software-Development Life-Cycle 29 2.2 Waterfall Model 36 2.3 Prototyping Model 41 2.4 Spiral Model 44 2.5 Evolutionary Development Model 46 2.6 Iterative-Enhancement Model 47 2.7 RAD Model 49 2.8 Comparison of Various Process Models 50 v vi TABLE OF CONTENTS Chapter 3. Introduction to Software Requirements Specification 53 3.1 Requirement Engineering 53 3.2 Process of Requirements Engineering 55 3.3 Information Modeling 61 3.4 Data-Flow Diagrams 62 3.5 Decision Tables 67 3.6 SRS Document 70 3.7 IEEE Standards for SRS Documents 73 3.8 SRS Validation 75 3.9 Components of SRS 75 3.10 Characteristics of SRS 78 3.11 Entity-Relationship Diagram 79 Chapter 4. Software Reliability and Quality Assurance 85 4.1 Verification and Validation 85 4.2 Software Quality Assurance 87 4.3 Software Quality 89 4.4 Capability Maturity Model (SEI-CMM) 96 4.5 International Standard Organization (ISO) 99 4.6 Comparison of ISO-9000 Certification and the SEI-CMM 106 4.7 Reliability Issues 107 4.8 Reliability Metrics 110 4.9 Reliability Growth Modeling 112 4.10 Reliability Assessment 115 Chapter 5. System Design 117 5.1 System/Software Design 117 5.2 Architectural Design 123 5.3 Low-Level Design 125 5.4 Coupling and Cohesion 136 5.5 Functional-Oriented Versus The Object-Oriented Approach 143 5.6 Design Specifications 144 5.7 Verification for Design 145 5.8 Monitoring and Control for Design 146 TABLE OF CONTENTS vii Chapter 6. Software Measurement and Metrics 149 6.1 Software Metrics 149 6.2 Halstead’s Software Science 151 6.3 Function-Point Based Measures 154 6.4 Cyclomatic Complexity 157 Chapter 7. Software Testing 161 7.1 Introduction to Testing 161 7.2 Testing Principles 162 7.3 Testing Objectives 163 7.4 Test Oracles 164 7.5 Levels of Testing 165 7.6 White-Box Testing/Structural Testing 173 7.7 Functional/Black-Box Testing 175 7.8 Test Plan 178 7.9 Test-Case Design 179 Chapter 8. Software-Testing Strategies 181 8.1 Static-Testing Strategies 181 8.2 Debugging 186 8.3 Error, Fault, and Failure 189 Chapter 9. Software Maintenance and Project Management 193 9.1 Software as an Evolution Entity 193 9.2 Software-Configuration Management Activities 193 9.3 Change-Control Process 197 9.4 Software-Version Control 199 9.5 Software-Configuration Management 200 9.6 Need for Maintenance 202 9.7 Categories of Maintenance 203 9.8 Maintenance Costs 204 9.9 Software-Project Estimation 207 9.10 Constructive Cost Model (COCOMO) 211 9.11 Software-Risk Analysis and Management 215 viii TABLE OF CONTENTS Chapter 10. Computer-Aided Software Engineering 223 10.1 Case and its Scope 223 10.2 Levels of Case 224 10.3 Architecture of Case Environment 224 10.4 Building Blocks for Case 226 10.5 Case Support in Software Life-Cycle 227 10.6 Objectives of Case 228 10.7 Case Repository 229 10.8 Characteristics of Case Tools 231 10.9 Case Classification 231 10.10 Categories of Case Tools 233 10.11 Advantages of Case Tools 234 10.12 Disadvantages of Case Tools 235 10.13 Reverse Software Engineering 235 10.14 Software Re-Engineering 240 Chapter 11. Coding 247 11.1 Information Hiding 247 11.2 Programming Style 248 11.3 Internal Documentation 250 11.4 Monitoring and Control for Coding 251 11.5 Structured Programming 252 11.6 Fourth-Generation Techniques 255 PART II: SOFTWARE DEVELOPMENT AND APPLICATIONS Chapter 12. Introduction to Software Development 261 12.1 Program Phase 262 12.2 How to Write a Good Program 262 12.3 Programming Tools 263 Chapter 13. Visual Basic 6.0 265 13.1 Hardware and Software Requirements for Visual Basic 266 13.2 Application Types 266 13.3 Compilation in Visual Basic 268 TABLE OF CONTENTS ix 13.4 Visual Basic Terminology 268 13.5 Integrated Development Environment (IDE) 269 Chapter 14. Controls in Visual Basic 273 14.1 Tool-Box Controls 276 Chapter 15. Variables and Operators in Visual Basic 297 15.1 Variable Naming Conventions 297 15.2 Variable Declaration 297 15.3 Scope of Variables 298 15.4 Logical Operators 298 15.5 Logical Operators 299 15.6 If-Else Statement 301 15.7 Do While …. Statement 301 15.8 For Loop 302 15.9 With–End With Statement 302 Chapter 16. Functions in Visual Basic 303 Chapter 17. Introduction to Databases 315 17.1 Tables 316 17.2 Structure of a Database 317 17.3 Keys 317 17.4 Data Integrity 318 Chapter 18. MS Access 2000 319 18.1 Creating a Database in MS Access 2000 319 18.2 Data Types 324 18.3 Field Properties 324 18.4 Saving the Table 327 18.5 Modifying the Table 328 18.6 Importing the Table 328 Chapter 19. Oracle 329 19.1 Starting with Oracle 8 329 19.2 How to Create a New User 331 x TABLE OF CONTENTS 19.3 User Creation by Navigator 332 19.4 Data Types in Oracle 335 19.5 Syntax and Query in Oracle 336 19.6 Functions 344 19.7 Primary Keys 345 19.8 Data Export 346 19.9 Data Import 347 Chapter 20. SQL Server 2000 349 20.1 What’s New in Microsoft SQL Server 2000? 349 20.2 Starting Microsoft SQL Server 2000 349 20.3 Installation of SQL Server 2000 351 20.4 Creating a Database 353 20.5 How to Create a Database Using Enterprise Manager 354 20.6 Create a Database Using the Create Database Wizard in Enterprise Manager 358 20.7 Creating a New Table 358 20.8 Data Types 361 20.9 Query Analyzer 368 20.10 How to use Query Analyzer 368 20.11 Generating an SQL Script 370 20.12 How to use the Script 374 20.13 Attaching a Database 376 20.14 Detaching a Database 378 20.15 Copy Database Wizard 380 20.16 Importing and Exporting a Database 380 20.17 SQL Server Service Manager 386 Chapter 21. Programming in Visual Basic with MS Access 2000 391 21.1 Saving Projects and Forms 393 21.2 Database Designing 399 21.3 Use of App.Path 412 Chapter 22. Programming with Oracle and SQL Server 2000 413 22.1 Table Creation 413 TABLE OF CONTENTS xi 22.2 Data Links 414 22.3 Working with the Project 418 22.4 Data Export at Runtime 420 22.5 Working in a Project with an SQL Server 2000 Database 420 Chapter 23. Graphs 421 Chapter 24. Data Reports 425 24.1 Data Report Creation 425 24.2 Data Environment and the Connection 425 24.3 Data Report Designing 430 24.4 Data Report Controls 433 24.5 Calling a Report 436 24.6 Retrieval of Selected Data in the Data Report 436 24.7 Index Number of Data Report Section 439 24.8 Grouping in Data Reports 440 Chapter 25. Crystal Reports 447 25.1 Advantages over Visual Basic Data Reports 448 25.2 Starting with Crystal Report 8.5 448 25.3 Creating Reports Using DSN of the SQL Server 2000 Database 451 25.4 Creating Connection Using DSN 456 Chapter 26. Error Handling 465 26.1 Key Handling 465 26.2 Key Locking at Key Press Event 469 26.3 Other Error-Handling Methods 470 26.4 Some Common Errors 471 26.5 Precautions 479 Chapter 27. Creating the Setup Package 481 27.1 How to Create a Setup 481 Index 493

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值