- 博客(26)
- 资源 (11)
- 收藏
- 关注
RSS订阅原创 Kubernetes 集群安装
环境准备To disable SELinux, run the following command: setenforce 0But this just disables it temporarily (until the next reboot). To disable it permanently, edit the/etc/selinux/config file and chang...
2019-04-30 18:21:34
3085
原创 Kubernetes inter-pod networking
no network address translation (NAT)Each pod gets its own unique IP address and can communicate with all other pods through a flat, NAT-less network.When pod A connects to(sends a network packet to)...
2019-04-25 20:40:47
288
原创 Pod 再探究
进入pod# kubectl exec kubia-p465w -it bashroot@kubia-p465w:/# root@kubia-p465w:/# root@kubia-p465w:/# pwd/root@kubia-p465w:/# ls -ltotal 68-rw-r--r-- 1 root root 354 Apr 17 07:36 app.jsdrwxr...
2019-04-19 18:01:23
231
原创 spark start-thriftserver.sh & Kubernetes
启动命令sh sbin/start-thriftserver.sh –master k8s://https://192.168.99.108:8443 –name spark-thriftserver –conf spark.executor.instances=1 –conf spark.kubernetes.container.image=zhixingheyitian/spark...
2019-04-19 14:05:40
1441
1
原创 Creating pods from YAML or JSON descriptors
Examining a YAML descriptor of an existing pod#kubectl get pod kubia-p465w -o yamlapiVersion: v1kind: Podmetadata: creationTimestamp: "2019-04-18T07:04:25Z" generateName: kubia- labels: ...
2019-04-19 09:04:37
231
原创 Kubernetes pods
pod & containersA pod is a collocated group of containers and represents the basic building block in Kubernetes.It is actually very common for pods to only contain a single container. The key t...
2019-04-18 17:20:59
351
原创 Overview of Kubernetes
overviewSetting up a full-fledged（完整的） multi-node Kubernetes cluster is not a simple task, especially if you’re not very well versed in Linux and networking administration. A proper Kubernetes instal...
2019-04-18 15:43:30
219
原创 Docker 安装
下载docker-engine-1.13.1-1.el7.centos.x86_64.rpm 或 在线安装 wget https://yum.dockerproject.org/repo/main/centos/7/Packages/docker-engine-1.13.1-1.el7.centos.x86_64.rpm
2019-04-18 11:13:42
472
原创 Docker
重启dockersystemctl restart dockerrun busybox第一次运行docker run busybox echo “Hello world”Unable to find image 'busybox:latest' locallyTrying to pull repository docker.io/library/busybox ... latest:...
2019-04-17 20:51:18
1115
原创 The Kubernetes Operator for Apache Spark (spark-on-k8s-operator)
Kubernetes Operator for Apache Spark DesignIntroductionIn Spark 2.3, Kubernetes becomes an official scheduler backend for Spark, additionally to the standalone scheduler, Mesos, and Yarn. Compared w...
2019-04-16 20:23:02
777
原创 Spark源码分析之SparkSubmit.scala
clusterManager, deployMode//Spark 2.3.2 SparkSubmit.scalaprivate def doPrepareSubmitEnvironment( args: SparkSubmitArguments, conf: Option[HadoopConfiguration] = None) : (Seq[Stri...
2019-04-16 15:36:53
378
1
原创 Introducing Kubernetes
Kubernetes’s originsAfter having kept Borg and Omega secret for a whole decade (整整十年), in 2014, Google introduced Kubernetes, an open-source system based on the experiences gathered through Borg, Ome...
2019-04-16 11:54:33
228
原创 Container Technologies
A process running in a containerA process running in a container is actually running inside the host’s operating system, just like all the other processes (unlike VMs, where processes run in separate...
2019-04-15 21:18:09
282
原创 implement a spark-sql case of separating computation and storage using Kubernetes
PrerequisitesSet up a single-node Kubernetes（minikube）with --cpus 8 --memory 8192Build and push the spark2.4.1 imagePut hive-site.xml in the conf dirRunbin/spark-sql \ --master k8s://https:...
2019-04-12 16:57:47
126
原创 spark2.4.1 on Kubernetes
修改 dockerfile参见 https://blog.csdn.net/zhixingheyi_tian/article/details/89028989编译部署./bin/docker-image-tool.sh -r zhixingheyitian -t spark2.4.1 build./bin/docker-image-tool.sh -r zhixingheyitian -t...
2019-04-11 10:57:02
775
原创 Spark StandAlone 集群搭建
启动master节点./sbin/start-master.sh登陆到UIhttp://10.239.10.11:8080/
2019-04-10 19:15:20
230
原创 spark 、hadoop、yarn 集群那些事
hadoop 主节点上jps 后，有这样几个进程hadoop：SecondaryNameNodeNameNodeyarn：ResourceManagerSpark的Job History服务启动 sbin/start-history-server.shHistoryServerHiveRunJar...
2019-04-09 17:07:29
137
原创 helm 安装
可参考 https://helm.sh/docs/using_helm/#installing-helm客户端安装在github 上下载最新 Binary包 helm-v2.13.1-linux-amd64.tar.gz安装helmtar -zxvf helm-v2.13.1-linux-amd64.tar.gzmv linux-amd64/helm /usr/local/bin/hel...
2019-04-09 16:12:23
1799
原创 Spark PI demo with spark-on-k8s-operator
安装 spark-on-k8s-operatorhelm repo add incubator http://storage.googleapis.com/kubernetes-charts-incubatorhelm install incubator/sparkoperator --namespace spark-operatorNAME: eponymous-fleeLAST D...
2019-04-09 16:00:34
1090
4
原创 kubectl一些命令
kubectl get pods --namespace kube-system[root@bdpe822n2 helmPackage]# kubectl get pods --namespace kube-systemNAME READY STATUS RESTARTS AGEcoredns-fb8b8dccf-bq...
2019-04-09 13:56:33
488
原创 run demo on minikube
kubectl run hello-minikube --image=k8s.gcr.io/echoserver:1.10 --port=8080kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=...
2019-04-08 21:57:27
217
原创 Spark on kubernetes
Dockerfile 文件里添加代理vim kubernetes/dockerfiles/spark/Dockerfile在 FROM ARG 中间添加ENV http_proxy http://xxx:portENV https_proxy http://xxx:portENV no_proxy localhost,127.0.0.1,10.96.0.0/12,192.168.99.0...
2019-04-04 17:06:47
1340
原创 安装 Minikube
检查kvm模块是否安装[root@localhost ~]# lsmod |grep kvmkvm_intel 55496 32kvm 337772 1 kvm_intel安装kubectl添加 repo配置文件，/etc/yum.repos.d/kubernetes.repo[kubernetes]name=Kubernetesbaseurl=https://packages....
2019-04-02 14:27:37
2413
1
原创 Kubernetes 二三
KubernetesKubernetes is a portable, extensible open-source platform for managing containerized workloads and services, that facilitates both declarative configuration and automation. It has a large, ...
2019-04-02 09:13:22
131
原创 MAC 开发环境安装
安装java从 ORACLE 官网上下载 jdk-8u112-macosx-x64.dmg，然后双击安装下载jdk 8 ，方法一从java 下载页面进去https://www.oracle.com/technetwork/java/javase/downloads/index.html上面呈现的是最新版。到该页面最下方“Java Archive”点击“Download” 按钮，进入...
2019-04-01 15:50:24
1056
1
原创 Spark 之 InternalRow
InternalRow — Abstract Binary Row FormatInternalRow is also called Catalyst row or Spark SQL row.abstract class InternalRow extends SpecializedGetters with Serializable {}UnsafeRowUnsafeRow is a...
2019-04-01 14:32:18
858
Spring Boot in Action
2017-11-23
从PAXOS到ZOOKEEPER 国人写的技术实践书 分布式
2017-09-22
机器学习实战 英文版 Machine Learning in Action 书中所有代码和数据集
2017-09-13
机器学习实战 英文版 Machine Learning in Action
2017-09-13
数据库实现英文第二版 Database System Implementation
2017-09-11
yammer metrics-2.2.0 源码
2017-09-06
空空如也
TA创建的收藏夹 TA关注的收藏夹
TA关注的人