多节点OpenStack Charms 部署指南0.0.1.dev--44--访问openstack中vm所需要的配置,访问部署在openstack上的kubernetes

56 篇文章 4 订阅
16 篇文章 1 订阅

参考文档: ubuntu20.04下使用juju+maas环境部署k8s-8-基本操作-1-访问 kubernetes dashboard

在openstack中部署完charmed kubernetes,按 ubuntu20.04下使用juju+maas环境部署k8s-8-基本操作-1-访问 kubernetes dashboard方法,准备同步k8s配置数据。

juju scp kubernetes-master/1:config ~/.kube/config

输出是类似:

can't connect 192.168.0.43

首先查看openstack的拓扑图:

在这里插入图片描述
从拓扑图上看,路由没什么问题。

开始觉得应该是从本机10.0.0.3应该有一条静态路由到192.168.0.0/24网段,因为能ping通192.168.0.1这个网关。就没有在本机显示路由。

在openstack的vm上ping测试

登陆到openstack最开始创建的测试机在这里插入图片描述
能通所有vm的内网地址,看来内网所在的vlan互联互通是没有问题的。

查看安全组

发现安全组中没有允许icmp和ssh。

遂在安全组中增加原先在测试时配置的default组。
在这里插入图片描述

在这里插入图片描述
在这里插入图片描述

增加浮动ip

又发现juju新建的vm们,没有浮动ip

又将所有的vm,增加了浮动ip。开始在图形界面增加,实在太慢了。遂在命令行中增加:

显示实例

openstack server list
+--------------------------------------+--------------------------+--------+-----------------------------------+-------------+----------+
| ID                                   | Name                     | Status | Networks                          | Image       | Flavor   |
+--------------------------------------+--------------------------+--------+-----------------------------------+-------------+----------+
| 069f8ee3-2eb1-4e10-babc-0de25faedec0 | juju-d3562c-k8s-13       | ACTIVE | int_net=10.0.9.97, 192.168.0.145  | focal-amd64 | r16G     |
| bd591201-506e-4078-bd37-cc728538e199 | juju-d3562c-k8s-12       | ACTIVE | int_net=10.0.9.55, 192.168.0.85   | focal-amd64 | r16G     |
| 52233035-1c94-4c06-89f0-f84cf5451319 | juju-d3562c-k8s-11       | ACTIVE | int_net=10.0.9.65, 192.168.0.40   | focal-amd64 | v2M4R16  |
| c2fa2e24-b21a-40f6-8c80-f01dca4043fd | juju-d3562c-k8s-7        | ACTIVE | int_net=10.0.9.100, 192.168.0.166 | focal-amd64 | v4m4r16  |
| dabf4ed4-6a1e-4976-9559-bbad8d110cdc | juju-d3562c-k8s-10       | ACTIVE | int_net=10.0.9.64, 192.168.0.161  | focal-amd64 | r16G     |
| 173b815a-6682-4dd5-83fb-77d62d7a5f5f | juju-d3562c-k8s-5        | ACTIVE | int_net=192.168.0.47              | focal-amd64 | v2M4R16  |
| f2c57ad7-6d46-4943-832f-9da343a7c211 | juju-d3562c-k8s-6        | ACTIVE | int_net=192.168.0.43              | focal-amd64 | v2M4R16  |
| 13ac8f35-9a8d-48e7-83b5-0aea3971ba4a | juju-d3562c-k8s-8        | ACTIVE | int_net=192.168.0.49              | focal-amd64 | v4m4r16  |
| f7840404-2b1e-47cc-84bb-8c3181a8253d | juju-d3562c-k8s-4        | ACTIVE | int_net=192.168.0.157             | focal-amd64 | m4r16    |
| baca1e8b-fdd4-4818-94b1-3c3ba9dea7d3 | juju-d3562c-k8s-9        | ACTIVE | int_net=10.0.9.110, 192.168.0.136 | focal-amd64 | v4m4r16  |
| fe9c5a0c-e36d-4cd7-a829-2e07967ec55e | juju-d3562c-k8s-3        | ACTIVE | int_net=192.168.0.110             | focal-amd64 | r16G     |
| 306fc000-cfcd-4440-b475-e51bb017e0a2 | juju-d3562c-k8s-2        | ACTIVE | int_net=192.168.0.175             | focal-amd64 | r16G     |
| f64b471d-76f7-4bd5-9eaf-417b27e19401 | juju-d3562c-k8s-1        | ACTIVE | int_net=192.168.0.15              | focal-amd64 | r16G     |
| 93ae9def-432a-4f26-89ad-dba737cfb567 | juju-d3562c-k8s-0        | ACTIVE | int_net=10.0.9.102, 192.168.0.107 | focal-amd64 | r16G     |
| 47b81b26-b5d0-4759-b4da-06aaaddf76ba | juju-5a7fd0-controller-0 | ACTIVE | int_net=10.0.9.175, 192.168.0.52  | focal-amd64 | v3584M   |
| 2c803a1a-19a4-4e58-b90f-3138629b39ba | focal-1                  | ACTIVE | int_net=10.0.9.121, 192.168.0.101 | focal-amd64 | m1.small |
+--------------------------------------+--------------------------+--------+-----------------------------------+-------------+----------+

给某一实例附加浮动ip

openstack floating ip create -f value -c floating_ip_address ext_net
10.0.9.162
openstack server add floating ip juju-d3562c-k8s-1 10.0.9.162

也可以用这两个命令附加:

 FIP=$(openstack floating ip create -f value -c floating_ip_address ext_net)
root@vivodo-3:~# openstack server add floating ip juju-d3562c-k8s-4 $FIP

最后的结果:

 openstack server list
+--------------------------------------+--------------------------+--------+-----------------------------------+-------------+----------+
| ID                                   | Name                     | Status | Networks                          | Image       | Flavor   |
+--------------------------------------+--------------------------+--------+-----------------------------------+-------------+----------+
| 069f8ee3-2eb1-4e10-babc-0de25faedec0 | juju-d3562c-k8s-13       | ACTIVE | int_net=10.0.9.97, 192.168.0.145  | focal-amd64 | r16G     |
| bd591201-506e-4078-bd37-cc728538e199 | juju-d3562c-k8s-12       | ACTIVE | int_net=10.0.9.55, 192.168.0.85   | focal-amd64 | r16G     |
| 52233035-1c94-4c06-89f0-f84cf5451319 | juju-d3562c-k8s-11       | ACTIVE | int_net=10.0.9.65, 192.168.0.40   | focal-amd64 | v2M4R16  |
| c2fa2e24-b21a-40f6-8c80-f01dca4043fd | juju-d3562c-k8s-7        | ACTIVE | int_net=10.0.9.100, 192.168.0.166 | focal-amd64 | v4m4r16  |
| dabf4ed4-6a1e-4976-9559-bbad8d110cdc | juju-d3562c-k8s-10       | ACTIVE | int_net=10.0.9.64, 192.168.0.161  | focal-amd64 | r16G     |
| 173b815a-6682-4dd5-83fb-77d62d7a5f5f | juju-d3562c-k8s-5        | ACTIVE | int_net=10.0.9.146, 192.168.0.47  | focal-amd64 | v2M4R16  |
| f2c57ad7-6d46-4943-832f-9da343a7c211 | juju-d3562c-k8s-6        | ACTIVE | int_net=10.0.9.26, 192.168.0.43   | focal-amd64 | v2M4R16  |
| 13ac8f35-9a8d-48e7-83b5-0aea3971ba4a | juju-d3562c-k8s-8        | ACTIVE | int_net=10.0.9.174, 192.168.0.49  | focal-amd64 | v4m4r16  |
| f7840404-2b1e-47cc-84bb-8c3181a8253d | juju-d3562c-k8s-4        | ACTIVE | int_net=10.0.9.156, 192.168.0.157 | focal-amd64 | m4r16    |
| baca1e8b-fdd4-4818-94b1-3c3ba9dea7d3 | juju-d3562c-k8s-9        | ACTIVE | int_net=10.0.9.110, 192.168.0.136 | focal-amd64 | v4m4r16  |
| fe9c5a0c-e36d-4cd7-a829-2e07967ec55e | juju-d3562c-k8s-3        | ACTIVE | int_net=10.0.9.33, 192.168.0.110  | focal-amd64 | r16G     |
| 306fc000-cfcd-4440-b475-e51bb017e0a2 | juju-d3562c-k8s-2        | ACTIVE | int_net=10.0.9.168, 192.168.0.175 | focal-amd64 | r16G     |
| f64b471d-76f7-4bd5-9eaf-417b27e19401 | juju-d3562c-k8s-1        | ACTIVE | int_net=10.0.9.162, 192.168.0.15  | focal-amd64 | r16G     |
| 93ae9def-432a-4f26-89ad-dba737cfb567 | juju-d3562c-k8s-0        | ACTIVE | int_net=10.0.9.102, 192.168.0.107 | focal-amd64 | r16G     |
| 47b81b26-b5d0-4759-b4da-06aaaddf76ba | juju-5a7fd0-controller-0 | ACTIVE | int_net=10.0.9.175, 192.168.0.52  | focal-amd64 | v3584M   |
| 2c803a1a-19a4-4e58-b90f-3138629b39ba | focal-1                  | ACTIVE | int_net=10.0.9.121, 192.168.0.101 | focal-amd64 | m1.small |
+--------------------------------------+--------------------------+--------+-----------------------------------+-------------+----------+
juju status
Model  Controller                 Cloud/Region               Version  SLA          Timestamp
k8s    openstack-cloud-regionone  openstack-cloud/RegionOne  2.9.18   unsupported  22:23:12+08:00

App                    Version   Status  Scale  Charm                  Store       Channel   Rev  OS      Message
containerd             go1.13.8  active      5  containerd             charmstore  stable    178  ubuntu  Container runtime available
easyrsa                3.0.1     active      1  easyrsa                charmstore  stable    420  ubuntu  Certificate Authority connected.
etcd                   3.4.5     active      3  etcd                   charmstore  stable    634  ubuntu  Healthy with 3 known peers
flannel                0.11.0    active      5  flannel                charmstore  stable    597  ubuntu  Flannel subnet 10.1.31.1/24
kubeapi-load-balancer  1.18.0    active      1  kubeapi-load-balancer  charmstore  stable    844  ubuntu  Loadbalancer ready.
kubernetes-master      1.22.4    active      2  kubernetes-master      charmstore  stable   1078  ubuntu  Kubernetes master running.
kubernetes-worker      1.22.4    active      3  kubernetes-worker      charmstore  stable    816  ubuntu  Kubernetes worker running.
openstack-integrator   xena      active      1  openstack-integrator   charmstore  stable    182  ubuntu  Ready

Unit                      Workload  Agent  Machine  Public address  Ports             Message
easyrsa/0*                active    idle   0        10.0.9.102                        Certificate Authority connected.
etcd/0*                   active    idle   1        10.0.9.162      2379/tcp          Healthy with 3 known peers
etcd/1                    active    idle   2        10.0.9.168      2379/tcp          Healthy with 3 known peers
etcd/2                    active    idle   3        10.0.9.33       2379/tcp          Healthy with 3 known peers
kubeapi-load-balancer/0*  active    idle   4        10.0.9.156      443/tcp,6443/tcp  Loadbalancer ready.
kubernetes-master/1*      active    idle   6        10.0.9.26       6443/tcp          Kubernetes master running.
  containerd/3            active    idle            10.0.9.26                         Container runtime available
  flannel/3               active    idle            10.0.9.26                         Flannel subnet 10.1.69.1/24
kubernetes-master/2       active    idle   11       10.0.9.65       6443/tcp          Kubernetes master running.
  containerd/4            active    idle            10.0.9.65                         Container runtime available
  flannel/4               active    idle            10.0.9.65                         Flannel subnet 10.1.4.1/24
kubernetes-worker/0       active    idle   7        10.0.9.100      80/tcp,443/tcp    Kubernetes worker running.
  containerd/2            active    idle            10.0.9.100                        Container runtime available
  flannel/2               active    idle            10.0.9.100                        Flannel subnet 10.1.16.1/24
kubernetes-worker/1*      active    idle   8        10.0.9.174      80/tcp,443/tcp    Kubernetes worker running.
  containerd/0*           active    idle            10.0.9.174                        Container runtime available
  flannel/0*              active    idle            10.0.9.174                        Flannel subnet 10.1.31.1/24
kubernetes-worker/2       active    idle   9        10.0.9.110      80/tcp,443/tcp    Kubernetes worker running.
  containerd/1            active    idle            10.0.9.110                        Container runtime available
  flannel/1               active    idle            10.0.9.110                        Flannel subnet 10.1.20.1/24
openstack-integrator/2*   active    idle   13       10.0.9.97                         Ready

Machine  State    DNS         Inst id                               Series  AZ    Message
0        started  10.0.9.102  93ae9def-432a-4f26-89ad-dba737cfb567  focal   nova  ACTIVE
1        started  10.0.9.162  f64b471d-76f7-4bd5-9eaf-417b27e19401  focal   nova  ACTIVE
2        started  10.0.9.168  306fc000-cfcd-4440-b475-e51bb017e0a2  focal   nova  ACTIVE
3        started  10.0.9.33   fe9c5a0c-e36d-4cd7-a829-2e07967ec55e  focal   nova  ACTIVE
4        started  10.0.9.156  f7840404-2b1e-47cc-84bb-8c3181a8253d  focal   nova  ACTIVE
6        started  10.0.9.26   f2c57ad7-6d46-4943-832f-9da343a7c211  focal   nova  ACTIVE
7        started  10.0.9.100  c2fa2e24-b21a-40f6-8c80-f01dca4043fd  focal   nova  ACTIVE
8        started  10.0.9.174  13ac8f35-9a8d-48e7-83b5-0aea3971ba4a  focal   nova  ACTIVE
9        started  10.0.9.110  baca1e8b-fdd4-4818-94b1-3c3ba9dea7d3  focal   nova  ACTIVE
11       started  10.0.9.65   52233035-1c94-4c06-89f0-f84cf5451319  focal   nova  ACTIVE
13       started  10.0.9.97   069f8ee3-2eb1-4e10-babc-0de25faedec0  focal   nova  ACTIVE

现在machine们的地址变成浮动ip地址了。

在10.0.0.3增加本地路由到192.168.0.0/24

但是这个还是不能解答本机10.0.0.3为什么不能ping通除192.168.0.1外,其他vm的192.168.0.1的地址均不能ping通。

突然想到,maas所在的本机,是不在openstack范围内的,能够ping通192.168.0.1这个网关,是因为192.168.0.1/10.0.9.193是个路由器,恰好证明从10.0.0.3这台机器缺乏一条到192.168.0.0/24这个网段的静态路由。

显示了下本机路由:
route -v
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         _gateway        0.0.0.0         UG    0      0        0 eno1
10.0.0.0        0.0.0.0         255.255.240.0   U     0      0        0 eno1
10.20.153.0     0.0.0.0         255.255.255.0   U     0      0        0 lxdbr0
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0

果然没有到192.168.0.0/24的静态路由。

增加到192.168.0.0/24的静态路由:
 route add -net 192.168.0.0/24 gw 10.0.9.193 

注:10.0.9.193为虚拟路由器地址。

显示路由:
 route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         _gateway        0.0.0.0         UG    0      0        0 eno1
10.0.0.0        0.0.0.0         255.255.240.0   U     0      0        0 eno1
10.20.153.0     0.0.0.0         255.255.255.0   U     0      0        0 lxdbr0
192.168.0.0     10.0.9.193      255.255.255.0   UG    0      0        0 eno1
192.168.122.0   0.0.0.0         255.255.255.0   U     0      0        0 virbr0
ping 测试:
 ping 192.168.0.145
PING 192.168.0.145 (192.168.0.145) 56(84) bytes of data.
64 bytes from 192.168.0.145: icmp_seq=1 ttl=63 time=4.10 ms
64 bytes from 192.168.0.145: icmp_seq=2 ttl=63 time=1.88 ms
^C
--- 192.168.0.145 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 1.876/2.990/4.104/1.114 ms
root@vivodo-3:~# ping 192.168.0.85
PING 192.168.0.85 (192.168.0.85) 56(84) bytes of data.
64 bytes from 192.168.0.85: icmp_seq=1 ttl=63 time=4.57 ms
64 bytes from 192.168.0.85: icmp_seq=2 ttl=63 time=2.01 ms
^C
--- 192.168.0.85 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 2.006/3.287/4.568/1.281 ms
root@vivodo-3:~# ping 192.168.0.40
PING 192.168.0.40 (192.168.0.40) 56(84) bytes of data.
64 bytes from 192.168.0.40: icmp_seq=1 ttl=63 time=2.83 ms
64 bytes from 192.168.0.40: icmp_seq=2 ttl=63 time=1.39 ms
^C
--- 192.168.0.40 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 1.391/2.112/2.833/0.721 ms
root@vivodo-3:~# ping 192.168.0.166
PING 192.168.0.166 (192.168.0.166) 56(84) bytes of data.
64 bytes from 192.168.0.166: icmp_seq=1 ttl=63 time=3.59 ms
64 bytes from 192.168.0.166: icmp_seq=2 ttl=63 time=1.81 ms
^C
--- 192.168.0.166 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 1.814/2.700/3.587/0.886 ms
root@vivodo-3:~# ping 192.168.0.161
PING 192.168.0.161 (192.168.0.161) 56(84) bytes of data.
64 bytes from 192.168.0.161: icmp_seq=1 ttl=63 time=4.54 ms
64 bytes from 192.168.0.161: icmp_seq=2 ttl=63 time=1.85 ms
64 bytes from 192.168.0.161: icmp_seq=3 ttl=63 time=1.19 ms
^C
--- 192.168.0.161 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 1.185/2.523/4.539/1.450 ms
root@vivodo-3:~# ping 192.168.0.47
PING 192.168.0.47 (192.168.0.47) 56(84) bytes of data.
64 bytes from 192.168.0.47: icmp_seq=1 ttl=63 time=4.44 ms
64 bytes from 192.168.0.47: icmp_seq=2 ttl=63 time=1.82 ms
^C
--- 192.168.0.47 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 1.824/3.131/4.438/1.307 ms
root@vivodo-3:~# ping 192.168.0.43
PING 192.168.0.43 (192.168.0.43) 56(84) bytes of data.
64 bytes from 192.168.0.43: icmp_seq=1 ttl=63 time=2.81 ms
^C
--- 192.168.0.43 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.808/2.808/2.808/0.000 ms
root@vivodo-3:~# ping 192.168.0.49
PING 192.168.0.49 (192.168.0.49) 56(84) bytes of data.
64 bytes from 192.168.0.49: icmp_seq=1 ttl=63 time=4.73 ms
64 bytes from 192.168.0.49: icmp_seq=2 ttl=63 time=1.37 ms
^C
--- 192.168.0.49 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 1.371/3.050/4.730/1.679 ms
root@vivodo-3:~# ping 192.168.0.157
PING 192.168.0.157 (192.168.0.157) 56(84) bytes of data.
64 bytes from 192.168.0.157: icmp_seq=1 ttl=63 time=3.91 ms
^C
--- 192.168.0.157 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 3.906/3.906/3.906/0.000 ms
root@vivodo-3:~# ping 192.168.0.136
PING 192.168.0.136 (192.168.0.136) 56(84) bytes of data.
64 bytes from 192.168.0.136: icmp_seq=1 ttl=63 time=3.88 ms
64 bytes from 192.168.0.136: icmp_seq=2 ttl=63 time=1.87 ms
^C
--- 192.168.0.136 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 1.871/2.875/3.880/1.004 ms
root@vivodo-3:~# ping 192.168.0.110
PING 192.168.0.110 (192.168.0.110) 56(84) bytes of data.
64 bytes from 192.168.0.110: icmp_seq=1 ttl=63 time=2.55 ms
^C
--- 192.168.0.110 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 2.545/2.545/2.545/0.000 ms
root@vivodo-3:~# ping 192.168.0.175
PING 192.168.0.175 (192.168.0.175) 56(84) bytes of data.
64 bytes from 192.168.0.175: icmp_seq=1 ttl=63 time=3.66 ms
^C
--- 192.168.0.175 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 3.657/3.657/3.657/0.000 ms
root@vivodo-3:~# ping 192.168.0.15
PING 192.168.0.15 (192.168.0.15) 56(84) bytes of data.
64 bytes from 192.168.0.15: icmp_seq=1 ttl=63 time=2.60 ms
64 bytes from 192.168.0.15: icmp_seq=2 ttl=63 time=1.41 ms
^C
--- 192.168.0.15 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 1.413/2.006/2.599/0.593 ms
root@vivodo-3:~# ping 192.168.0.107
PING 192.168.0.107 (192.168.0.107) 56(84) bytes of data.
64 bytes from 192.168.0.107: icmp_seq=1 ttl=63 time=4.26 ms
64 bytes from 192.168.0.107: icmp_seq=2 ttl=63 time=1.96 ms
^C
--- 192.168.0.107 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 1.960/3.109/4.258/1.149 ms
root@vivodo-3:~# ping 192.168.0.52
PING 192.168.0.52 (192.168.0.52) 56(84) bytes of data.
64 bytes from 192.168.0.52: icmp_seq=1 ttl=63 time=2.88 ms
64 bytes from 192.168.0.52: icmp_seq=2 ttl=63 time=1.54 ms
^C
--- 192.168.0.52 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 1.542/2.210/2.878/0.668 ms
root@vivodo-3:~# ping 192.168.0.101
PING 192.168.0.101 (192.168.0.101) 56(84) bytes of data.
64 bytes from 192.168.0.101: icmp_seq=1 ttl=63 time=4.64 ms
64 bytes from 192.168.0.101: icmp_seq=2 ttl=63 time=2.02 ms
^C
--- 192.168.0.101 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1002ms
rtt min/avg/max/mdev = 2.017/3.328/4.640/1.311 ms

均能ping通。

再次同步kubernetes配置:

juju scp kubernetes-master/1:config ~/.kube/config

验证配置:

kubectl cluster-info
Kubernetes control plane is running at https://10.0.9.156:443
CoreDNS is running at https://10.0.9.156:443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://10.0.9.156:443/api/v1/namespaces/kube-system/services/https:metrics-server:/proxy

总结:

1 首先要确定vm们是否有权限进行icmp和ssh。
2 浮动ip应该不是要全部加,只需要在需要暴露给外界的vm上添加浮动ip。
3 和不用juju和maas配置的openstack相比,命令行是在maas所在服务器上执行,需要添加一条maas到内网的静态路由。

  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值