目的在安装了win10操作系统个人笔记本电脑上搭建Kubernetes集群实验环境做为Kubernetes学习之用。虚拟环境Oracle VM VirtualBox”建立三台虚拟机虚拟机安装CentOshttp://mirrors.aliyun.com/centos/7.9.2009/isos/x86_64/CentOS-7-x86_64-DVD-2009.iso每个虚拟机设置2块网卡一块网卡通过NAT连接外网另一块用于三台虚拟机之间通信。软件版本CentOSLinux version 3.10.0-957.el7.x86_64Kubernetes1.23.16一、配置虚拟机与安装所需要的软件1.1 配置VirtualBox的NAT网络VirtualBox 菜单管理-全局设定-网络-增加一新的NAT网络1.2 配置VM的网卡每台VM设两块网卡一块为NAT网络一块为仅主机如下图1.3 安装操作系统centos选择最小化安装主机名设为master.k8s1.4 安装dockerce(Install Docker Engine on CentOS | Docker Documentation 和Kubernetes更改repoyum install -y yum-utilsyum-config-manage --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo解决containerd.io-1.6.33-3.1.el7.x86_64: [Errno 256] No more mirrors to try.解决 centos7源不再维护的问题http://mirrors.aliyun.com/repo/Centos-7.repo安装完成后登录禁用两个安全功能SELinux以及防火墙禁用SELinux执行以下命令修改 /etc/selinux/config文件把SELINUXenforcing行改为SELINUXdisabled禁用防火墙# systemctl disable firewalld systemctl stop firewalld加入kubernetes阿里云的源# vi /etc/yum.repos.d/kubernetes.repo [kubernetes] nameKubernetes baseurlhttps://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled1 gpgcheck0安装DOCKER, KUBELET, KUBEADM, KUBECTL, AND KUBERNETES-CNI# yum install -y kubelet-1.23.16-0 kubeadm-1.23.16-0 kubectl-1.23.16-0 kubernetes-cni-1.1.1-0 --nogpgcheck # systemctl enable docker systemctl start docker # systemctl enable kubelet # curl -sSL https://get.daocloud.io/daotools/set_mirror.sh | sh -s http://f1361db2.m.daocloud.io启用net.bridge.bridge-nf-call-iptables内核选项# sysctl -w net.bridge.bridge-nf-call-iptables1 # echo net.bridge.bridge-nf-call-iptables1 /etc/sysctl.d/k8s.conf # echo 1 /proc/sys/net/ipv4/ip_forward禁用 SWAP# swapoff -a sed -i / swap / s/^/#/ /etc/fstabdocker 加入systemd ,解决Failed connect to localhost:10248DockerHub国内镜像源列表2024年6月18日 亲测可用 - 资源荟萃 - LINUX DO docker hub国内镜像vi /etc/docker/daemon.json 加入 { exec-opts: [native.cgroupdriversystemd] } systemctl daemon-reload systemctl restart docker systemctl restart kubelet执行以下操作解决[ERROR CRI]: container runtime is not running: output: E0606rm -rf /etc/containerd/config.toml systemctl restart containerd1.5 复制VM关闭VM# shutdown nowVirtualBox 界面选择复制复制2个虚拟机node1.k8s,node2.k8s二、配置三台虚拟机环境启动 master.k8s,node1.k8s,node2.k83设置node1.k8s, node2.k8s的主机名# hostnamectl --static set-hostname node1.k8s # hostnamectl --static set-hostname node2.k8s3. 修改三台虚拟机网络配置三台虚拟机的内网配置如下主机名IPmaster.k8s192.168.56.101node1.k8s192.168.56.102node2.k83192.168.56.103# cat /etc/sysconfig/network-scripts/ifcfg-enp0s8 TYPEEthernet PROXY_METHODnone BROWSER_ONLYno BOOTPROTOnone DEFROUTEyes IPV4_FAILURE_FATALno IPV6INITyes IPV6_AUTOCONFyes IPV6_DEFROUTEyes IPV6_FAILURE_FATALno IPV6_ADDR_GEN_MODEstable-privacy NAMEenp0s8 UUID9993cb88-7de1-4038-bd36-f64e681f4d54 DEVICEenp0s8 ADDRES192.168.56.101 IPADDR192.168.56.101 ONBOOTyes4. 设置三台VM的hosts文件# cat /etc/hosts 192.168.56.101 master.k8s 192.168.56.102 node1.k8s 192.168.56.103 node2.k8s三、配置Kubernetes3.1 拉取docker images 编写脚本拉取所需要的image,用kubeadm config images list 查看当前kube版本# vi pullimages.sh #!/bin/bash images( kube-apiserver:v1.23.16 kube-controller-manager:v1.23.16 kube-scheduler:v1.23.16 kube-proxy:v1.23.16 pause:3.6 etcd:3.5.6-0 coredns:v1.8.6 ) for imageName in ${images[]}; do docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName registry.k8s.io/$imageName docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName done3.3 在虚拟机master.k8s用kubeadm配置master# sh pullimages.sh # kubeadm init --kubernetes-version v1.23.16 --apiserver-advertise-address 1 92.168.56.101 --pod-network-cidr 10.244.0.0/16 [init] Using Kubernetes version: v1.14.0 [preflight] Running pre-flight checks [WARNING IsDockerSystemdCheck]: detected cgroupfs as the Docker cgroup driver. The recommended driver is systemd. Please follow the guide at https://kubernetes.io/docs/setup/cri/ [preflight] Pulling images required for setting up a Kubernetes cluster [preflight] This might take a minute or two, depending on the speed of your internet connection [preflight] You can also perform this action in beforehand using kubeadm config images pull [kubelet-start] Writing kubelet environment file with flags to file /var/lib/kubelet/kubeadm-flags.env [kubelet-start] Writing kubelet configuration to file /var/lib/kubelet/config.yaml [kubelet-start] Activating the kubelet service Your Kubernetes control-plane has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run kubectl apply -f [podnetwork].yaml with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ Then you can join any number of worker nodes by running the following on each as root: kubeadm join 192.168.56.101:6443 --token pllbqq.at7ky0jtv4exkt1h \ --discovery-token-ca-cert-hash sha256:aeef41f9631853a270a7a4640e6332cdad3ba1f5fefef5e158502e9509375b47建立和修改所需要的目录# mkdir -p $HOME/.kube # sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config # sudo chown $(id -u):$(id -g) $HOME/.kube/config3.4 配置虚拟机node1.k8s,与node2.k8s# vi pullimages.sh #!/bin/bash images( kube-proxy:v1.23.16 pause:3.6 ) for imageName in ${images[]}; do docker pull registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName docker tag registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName registry.k8s.io/$imageName docker rmi registry.cn-hangzhou.aliyuncs.com/google_containers/$imageName done加入集群# kubeadm join 192.168.56.101:6443 --token pllbqq.at7ky0jtv4exkt1h \ --discovery-token-ca-cert-hash sha256:aeef41f9631853a270a7a4640e6332cdad3ba1f5fefef5e158502e9509375b47四、配置Kubernetes网络4.1 master主机安装Weave Net container networking pluginkubectl apply -f https://github.com/weaveworks/weave/releases/download/v2.8.1/weave-daemonset-k8s.yaml4.2 查询节点情况[rootmaster ~]# kubectl get node NAME STATUS ROLES AGE VERSION master.k8s Ready master 178m v1.14.0 node1.k8s Ready none 162m v1.14.0 node2.k8s Ready none 113m v1.14.0 [rootmaster ~]# kubectl get pods -n kube-system -l nameweave-net NAME READY STATUS RESTARTS AGE weave-net-9vcqs 2/2 Running 1 156m weave-net-c5gp2 2/2 Running 2 156m weave-net-dh48f 2/2 Running 0 108m五、解决token过期master节点初始化生成的token有有效期过期后要加入新的节点要重新生成。5.1 生成一条永久有效的token# kubeadm token create --ttl 0 # kubeadm token list5. 2 获取ca证书sha256编码hash值# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2/dev/null | openssl dgst -sha256 -hex | sed s/^.* //5.3 加入节点# kubeadm join 10.167.11.153:6443 --token o4avtg.65ji6b778nyacw68 --discovery-token-ca-cert-hash sha256:2cc3029123db737f234186636330e87b5510c173c669f513a9c0e0da395515b0
