NYC's Blog 今天继续接着前文：
继续来部署Kubernetes。今天主要来部署Dashboard和KubeDNS，主要是后者。因为Kubernetes在每个Pod内都会起一个很小的“系统”容器pause-amd64:3.0来实现Pod内的网络，而这个容器默认会去从Google的Registry拉，但国内如果没有梯子访问不了。所以我pull了一个，传到了国内的镜像仓库，我们改下每个Node节点上的kubelet的配置，增加--pod_infra_container_image选项，这个选项可以指定从哪里pull这个基础镜像，我修改后的配置如下：
KUBELET_ARGS="--api-servers=http://192.168.56.101:8080 --cluster-dns=169.169.0.2 --cluster-domain=cluster.local --hostname-override=192.168.56.101 --logtostderr=false --log-dir=/var/log/kubernetes --v=2 --pod_infra_container_image=hub.c.163.com/allan1991/pause-amd64:3.0"然后重启Kubelet：systemctl restart kubelet.service.另外，
除了这个镜像外，还有很多镜像也会有这个问题，所以我都传到了国内的镜像仓库，本文介绍时都使用国内我自己传的。大家没有梯子的话，直接使用我用的这个就可以了。OK，下面开始今天的主题。
部署Dashboard
Kubernetes提供了一个基础的图形化界面，就是这个Dashboard。基本上使用kubectl可以实现的功能现在都可以通过界面去做了，本质上它俩都是调用Kubernetes API。而部署这个Dashboard也非常简答，就是创建一个Pod和一个Service。最新的创建dashboard的yaml文件可以去查看官方：https://rawgit.com/kubernetes/dashboard/master/src/deploy/kubernetes-dashboard.yaml。我的yaml文档如下：
# Copyright 2015 Google Inc. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Configuration to deploy release version of the Dashboard UI.
#
# Example usage: kubectl create -f <this_file>
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
labels:
app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
spec:
replicas: 1
selector:
matchLabels:
app: kubernetes-dashboard
template:
metadata:
labels:
app: kubernetes-dashboard
# Comment the following annotation if Dashboard must not be deployed on master
annotations:
scheduler.alpha.kubernetes.io/tolerations: |
[
{
"key": "dedicated",
"operator": "Equal",
"value": "master",
"effect": "NoSchedule"
}
]
spec:
containers:
- name: kubernetes-dashboard
image: hub.c.163.com/allan1991/kubernetes-dashboard-amd64:v1.5.1
imagePullPolicy: IfNotPresent
ports:
- containerPort: 9090
protocol: TCP
args:
# Uncomment the following line to manually specify Kubernetes API server Host
# If not specified, Dashboard will attempt to auto discover the API server and connect
# to it. Uncomment only if the default does not work.
- --apiserver-host=http://192.168.56.101:8080
livenessProbe:
httpGet:
path: /
port: 9090
initialDelaySeconds: 30
timeoutSeconds: 30
---
kind: Service
apiVersion: v1
metadata:
labels:
app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
spec:
type: NodePort
ports:
- port: 80
targetPort: 9090
selector:
app: kubernetes-dashboard这里需要注意的是--apiserver-host默认是注释掉的，我有次部署时虽然找到了APIServer，但是连接被决绝，但是显式的去掉这个注释，写上APIServer的地址后就好了，所以我推荐还是写上比较好。然后使用kubectl create -f <this_file>就可以创建了。待这个Deployment和Service创建成功后，执行kubectl --namespace=kube-system get svc命令查看映射的端口号，当然我们也可以在yaml文件里面使用nodePort指定为我们想要的端口，这里我没有指定，让系统自己选择，因为选择好以后就会固定，不会再变了。我的创建好有如下：
Master➜ kube-system kubectl --namespace=kube-system get svc
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes-dashboard 169.169.162.93 <nodes> 80:10977/TCP 3d这里需要注意执行命令时要加上--namespace=kube-system。前面我们也已经介绍过了，Kubernetes部署好后默认有两个namespace：default和kube-system。用户自己的默认在default下，kubectl默认也使用这个namespace；Kubernetes自己的系统模块在kube-system下面。这样我们的Dashboard已经创建好了，通过APIServer的IP加上面的端口号10977就可以访问界面了：
界面还是非常的清爽的，使用起来也很方便和简单，所以这里就不详细介绍了，可以参考官方图文教程：https://kubernetes.io/docs/user-guide/ui/。
部署KubeDNS
在这之前我们先要搞清楚为什么需要DNS？是为了实现服务发现。而且DNS其实是Kubernetes 1.3开始才内置的功能，在这之前是利用Linux环境的方式来做的服务发现。这里我们举例说明一下利用环境变量做服务发现的过程：比如我们部署了几个服务，这些服务需要找到彼此。利用环境变量的方式就是在创建Pod的时候将Service的一些信息（主要是IP+端口）以环境变量的方式（环境变量按照一定的规则命名）注入到Pod上的容器内，这样就做到了服务发现。这种方式虽然简单，但显然有两个不足：
- 随着服务越来越多，环境变量会越来越多。
- Pod必须在Service之后创建。
所以后来就出现了DNS，利用这种方式去做服务发现就不会有上面所说的限制了。那DNS又是怎么做服务发现的呢，再举个例子：比如我们在Kubernetes的叫bar的namespace下面创建了一个Service叫foo。那么在bar下运行的Pod只需要做一个foo DNS查询便可获取到这个服务的IP和Port。如果是运行在其他namespace下的Pod想要查询foo这个服务，则只需要查询foo.bar即可。当然一般还会有一个域名（domain）。更详细的信息请参阅https://kubernetes.io/docs/admin/dns/。
起初，Kubernetes的DNS是使用SkyDNS来实现的，网上现在的教程也基本是基于此的。但是新版本的Kubernetes已经不需要SkyDNS了，它有了自己的DNS模块——KubeDNS。这也是本文要介绍的。SkyDNS依赖于etcd，而KubeDNS不需要etcd。KubeDNS的部署也很简单，官方已经提供了基础的yaml文件：https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/dns。下面是我的yaml文件：
kubedns-deployment.yaml:
# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# TODO - At some point, we need to rename all skydns-*.yaml.* files to kubedns-*.yaml.*
# Should keep target in cluster/addons/dns-horizontal-autoscaler/dns-horizontal-autoscaler.yaml
# in sync with this file.
# Warning: This is a file generated from the base underscore template file: skydns-rc.yaml.base
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: kube-dns
namespace: kube-system
labels:
k8s-app: kube-dns
kubernetes.io/cluster-service: "true"
spec:
# replicas: not specified here:
# 1. In order to make Addon Manager do not reconcile this replicas parameter.
# 2. Default is 1.
# 3. Will be tuned in real time if DNS horizontal auto-scaling is turned on.
strategy:
rollingUpdate:
maxSurge: 10%
maxUnavailable: 0
selector:
matchLabels:
k8s-app: kube-dns
template:
metadata:
labels:
k8s-app: kube-dns
annotations:
scheduler.alpha.kubernetes.io/critical-pod: ''
scheduler.alpha.kubernetes.io/tolerations: '[{"key":"CriticalAddonsOnly", "operator":"Exists"}]'
spec:
containers:
- name: kubedns
image: hub.c.163.com/allan1991/kubedns-amd64:1.9
resources:
# TODO: Set memory limits when we've profiled the container for large
# clusters, then set request = limit to keep this container in
# guaranteed class. Currently, this container falls into the
# "burstable" category so the kubelet doesn't backoff from restarting it.
limits:
memory: 170Mi
requests:
cpu: 100m
memory: 70Mi
livenessProbe:
httpGet:
path: /healthz-kubedns
port: 8080
scheme: HTTP
initialDelaySeconds: 60
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 5
readinessProbe:
httpGet:
path: /readiness
port: 8081
scheme: HTTP
# we poll on pod startup for the Kubernetes master service and
# only setup the /readiness HTTP server once that's available.
initialDelaySeconds: 3
timeoutSeconds: 5
args:
- --kube-master-url=http://192.168.56.101:8080
- --domain=cluster.local.
- --dns-port=10053
- --config-map=kube-dns
# This should be set to v=2 only after the new image (cut from 1.5) has
# been released, otherwise we will flood the logs.
- --v=0
# {{ pillar['federations_domain_map'] }}
env:
- name: PROMETHEUS_PORT
value: "10055"
ports:
- containerPort: 10053
name: dns-local
protocol: UDP
- containerPort: 10053
name: dns-tcp-local
protocol: TCP
- containerPort: 10055
name: metrics
protocol: TCP
- name: dnsmasq
image: hub.c.163.com/allan1991/kube-dnsmasq-amd64:1.4
livenessProbe:
httpGet:
path: /healthz-dnsmasq
port: 8080
scheme: HTTP
initialDelaySeconds: 60
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 5
args:
- --cache-size=1000
- --no-resolv
- --server=127.0.0.1#10053
- --log-facility=-
ports:
- containerPort: 53
name: dns
protocol: UDP
- containerPort: 53
name: dns-tcp
protocol: TCP
# see: https://github.com/kubernetes/kubernetes/issues/29055 for details
resources:
requests:
cpu: 150m
memory: 10Mi
# - name: dnsmasq-metrics
# image: gcr.io/google_containers/dnsmasq-metrics-amd64:1.0
# livenessProbe:
# httpGet:
# path: /metrics
# port: 10054
# scheme: HTTP
# initialDelaySeconds: 60
# timeoutSeconds: 5
# successThreshold: 1
# failureThreshold: 5
# args:
# - --v=2
# - --logtostderr
# ports:
# - containerPort: 10054
# name: metrics
# protocol: TCP
# resources:
# requests:
# memory: 10Mi
- name: healthz
image: hub.c.163.com/allan1991/exechealthz-amd64:1.2
resources:
limits:
memory: 50Mi
requests:
cpu: 10m
# Note that this container shouldn't really need 50Mi of memory. The
# limits are set higher than expected pending investigation on #29688.
# The extra memory was stolen from the kubedns container to keep the
# net memory requested by the pod constant.
memory: 50Mi
args:
- --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1 >/dev/null
- --url=/healthz-dnsmasq
- --cmd=nslookup kubernetes.default.svc.cluster.local 127.0.0.1:10053 >/dev/null
- --url=/healthz-kubedns
- --port=8080
- --quiet
ports:
- containerPort: 8080
protocol: TCP
dnsPolicy: Default # Don't use cluster DNS.这里我把dnsmasq-metrics这个container注释掉了，因为我没把这个镜像下载下来...不过没了这个也没什么影响，这个也是后来才新增的，原来是没有的。
kubedns-svc.yaml:
# Copyright 2016 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This file should be kept in sync with cluster/images/hyperkube/dns-svc.yaml
# TODO - At some point, we need to rename all skydns-*.yaml.* files to kubedns-*.yaml.*
# Warning: This is a file generated from the base underscore template file: skydns-svc.yaml.base
apiVersion: v1
kind: Service
metadata:
name: kube-dns
namespace: kube-system
labels:
k8s-app: kube-dns
kubernetes.io/cluster-service: "true"
kubernetes.io/name: "KubeDNS"
spec:
selector:
k8s-app: kube-dns
clusterIP: 169.169.0.2
ports:
- name: dns
port: 53
protocol: UDP
- name: dns-tcp
port: 53
protocol: TCP这里需要注意下clusterIP，你可以不设置，让系统自己去选择。如果显式的设置的话，指定的IP必须在APIServer的--service-cluster-ip-range参数指定的网段内。我们可以用以下命令去检查是否创建成功：
Master➜ ~ kubectl --namespace=kube-system get deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
kube-dns 1 1 1 1 1d
kubernetes-dashboard 1 1 1 1 3d
Master➜ ~ kubectl --namespace=kube-system get pod
NAME READY STATUS RESTARTS AGE
kube-dns-2528035289-r7vrq 3/3 Running 0 19h
kubernetes-dashboard-1077846755-lj7sc 1/1 Running 0 19h
Master➜ ~ kubectl --namespace=kube-system get svc
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns 169.169.0.2 <none> 53/UDP,53/TCP 1d
kubernetes-dashboard 169.169.162.93 <nodes> 80:10977/TCP 3dService、Deployment都OK后，我们需要在每个Node的Kubelet配置中加入DNS的信息，主要是--cluster-dns和--cluster-domain，比如我的配置如下：
KUBELET_ARGS="--api-servers=http://192.168.56.101:8080 --cluster-dns=169.169.0.2 --cluster-domain=cluster.local --hostname-override=192.168.56.101 --logtostderr=false --log-dir=/var/log/kubernetes --v=2 --pod_infra_container_image=hub.c.163.com/allan1991/pause-amd64:3.0"这里我们使用的是默认的Domain cluster.local，当然你可以在kubedns-deployment.yaml将所有的cluster.local换为你想要的，比如time-track.cn。
最后我们来验证一下，首先创建一个Pod，里面运行一个busybox容器，主要是为了使用里面的nslookup命令：
busybox.yaml
apiVersion: v1
kind: Pod
metadata:
name: busybox
namespace: default
spec:
containers:
- image: hub.c.163.com/library/busybox
command:
- sleep
- "3600"
imagePullPolicy: IfNotPresent
name: busybox
restartPolicy: Always然后我们再创建一个服务，比如mysql吧：
mysql.yaml
apiVersion: v1
kind: Service
metadata:
name: mysql
spec:
ports:
- port: 3306
selector:
app: mysql然后我们在先创建的busybox里面通过DNS查询后创建的mysql服务，看是否可以查到：
Master➜ ~ kubectl exec busybox -c busybox -- nslookup mysql
Server: 169.169.0.2
Address 1: 169.169.0.2 kube-dns.kube-system.svc.cluster.local
Name: mysql
Address 1: 169.169.96.10 mysql.default.svc.cluster.local
Master➜ ~ kubectl describe svc mysql
Name: mysql
Namespace: default
Labels: <none>
Selector: app=mysql
Type: ClusterIP
IP: 169.169.96.10
Port: <unset> 3306/TCP
Endpoints: <none>
Session Affinity: None
No events.可以看到，成功的查到了。这样通过DNS变做到了服务发现，而且没有先后顺序的限制了。
评论已关闭