단순하게 한대만 구성 한다고 하면 kubeadm 으로 kubenetes 를 설치 해도 좋지만 여러대를 구성 한다거나 타이핑 치는게 귀찮을때가 가끔 있습니다. -_-
그럴땐 bastion-host 구성후 kubespray 로 구성 하는것이 굉장히 편합니다.
onpremise 구성시 필요한 metallb , helm 등등을 한번에 구성후 자동으로 설치 할수 있습니다.
bastion-host /etc/hosts 파일 설정
$ sudo vi /etc/hosts
192.168.0.10 k8s-master
192.168.0.11 k8s-node01
192.168.0.12 k8s-node02
192.168.0.20 bastion-hostsystem-upgrade 및 system-rebooting
$ sudo apt update
$ sudo apt list --upgradable
$ sudo apt upgrade -y
$ sudo init 6bastion 호스트 에서 작업 합니다.
편의상 /usr/bin/python 에 심볼릭 링크를 생성 합니다.
$ ls -al /usr/bin/python3.8
-rwxr-xr-x 1 root root 5490448 3월 15 21:22 /usr/bin/python3.8
$ sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.8 1
$ sudo update-alternatives --config python
$ python --version
Python 3.8.10python3-pip 설치
$ sudo apt -y install python3-pip
$ sudo pip3 install --upgrade pipssh-key 를 생성 합니다.
$ ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/home/ubuntu/.ssh/id_rsa): Created directory '/home/ubuntu/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/ubuntu/.ssh/id_rsa
Your public key has been saved in /home/ubuntu/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:CnFH44R4dBkP8UMmpCxsAkvQDsZpjB1fCAxzJs0QAI8 ubuntu@bastion-host
The key's randomart image is:
+---[RSA 3072]----+
|^@*o +oo@+o |
|+/*oooo*.O |
|Eoo *.+ o + |
| .o + . . |
| . S |
| . . |
| . |
| |
| |
+----[SHA256]-----+
# bastion host 에서 생성한 ssh-key 를 복사 합니다.
$ ssh-copy-id ubuntu@k8s-master
$ ssh-copy-id ubuntu@k8s-node01
$ ssh-copy-id ubuntu@k8s-node02
$ ssh-copy-id ubuntu@192.168.0.20사전작업
kubespray 로 kubernetes 설치전 node 에서 작업을 진행 합니다
$ sudo vi /etc/hosts
192.168.30.90 k8s-master
192.168.30.91 k8s-node01
192.168.30.92 k8s-node02
$ sudo vi /etc/fstab
~
#/swap.img none swap sw 0 0
$ sudo swapoff -a
$ sudo cat <<EOF | sudo tee /etc/sysctl.d/kubernetes.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF
$ sudo sysctl --system
$ echo 'ubuntu ALL=NOPASSWD: ALL' | sudo tee -a /etc/sudoerskubespray 특정 버전 설치시 git checkout 으로 버전을 지정 할수 있습니다.
kubernetes 1.21.6 을 설치 하기 위하여 v2.17.1 을 체크아웃 합니다.
$ git clone https://github.com/kubernetes-sigs/kubespray.git
$ cd kubespray
$ git tag
$ git checkout v2.17.1
HEAD is now at eeeca4a1 [2.17] Update kubernetes version to 1.21.6 (#8142)
$ pip install -r requirements.txtpath 추가
$ vi ~/.bashrc
PATH=$PATH:/home/ubuntu/.local/bin
$ source ~/.bashrcsample 디렉토리를 복사후 inventory.ini 파일을 수정 합니다.
$ cp -rpf inventory/sample inventory/k8s-cluster
$ vi inventory/k8s-cluster/inventory.ini
[all]
k8s-master ansible_host=192.168.0.10 ip=192.168.0.10 etcd_member_name=etcd1
k8s-node01 ansible_host=192.168.0.11 ip=192.168.0.11
k8s-node02 ansible_host=192.168.0.12 ip=192.168.0.12
# bastion host 사용시 주석 해제 / ssh-copy-id ubuntu@192.168.0.20 bastion host ssh-key 복사
[bastion]
bastion ansible_host=192.168.0.20 ansible_user=ubuntu
[kube_control_plane]
k8s-master
# node1
# node2
# node3
[etcd]
k8s-master
# node1
# node2
# node3
[kube_node]
k8s-node01
k8s-node02
# node2
# node3
# node4
# node5
# node6
[calico_rr]
[k8s_cluster:children]
kube_control_plane
kube_node
calico_rraddons.yml 파일을 수정 하여 metalLB 를 설정 합니다.
$ vi inventory/k8s-cluster/group_vars/k8s_cluster/addons.yml
---
metallb_enabled: true
metallb_speaker_enabled: true
metallb_ip_range:
- "192.168.0.30-192.168.0.50"
metallb_protocol: "layer2"
metallb_pool_name: "loadbalanced"
---helm deployment
$ vi inventory/k8s-cluster/group_vars/k8s_cluster/addons.yml
helm_enabled: truek8s-cluster.yaml 수정
$ vi inventory/k8s-cluster/group_vars/k8s_cluster/k8s-cluster.yml
kube_proxy_strict_arp: true설치
–become-user=root 로 지정 합니다.
$ ansible-playbook -i ~/kubespray/inventory/k8s-cluster/inventory.ini --become --become-user=root cluster.yml배포 완료 메세지
PLAY RECAP *********************************************************************************************************************************************************************************************
bastion : ok=14 changed=1 unreachable=0 failed=0 skipped=19 rescued=0 ignored=0
k8s-master : ok=589 changed=127 unreachable=0 failed=0 skipped=1147 rescued=0 ignored=2
k8s-node01 : ok=366 changed=73 unreachable=0 failed=0 skipped=642 rescued=0 ignored=1
k8s-node02 : ok=366 changed=74 unreachable=0 failed=0 skipped=641 rescued=0 ignored=1
localhost : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Thursday 07 July 2022 06:50:38 +0000 (0:00:00.065) 0:09:02.372 *********
===============================================================================master 에서 작업시
# bastion-host 에서 master 로 접속 합니다.
$ ssh k8s-master
$ mkdir -p $HOME/.kube
$ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$ sudo chown $(id -u):$(id -g) $HOME/.kube/configbastion-host 설정
bastion-host 를 사용하지 않을시 이과정은 스킵 합니다.
$ mkdir ~/.kube
$ scp k8s-master:/home/ubuntu/.kube/config ~/.kube
$ vi .kube/config
~
server: https://k8s-master:6443
$ curl -LO https://dl.k8s.io/release/v1.21.6/bin/linux/amd64/kubectl
$ sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectlbastion-host 에서 kubernetes-cluster 를 관리 할수 있습니다.
$ kubectl get pod -A
NAMESPACE NAME READY STATUS RESTARTS AGE
kube-system calico-kube-controllers-8575b76f66-l6ccx 1/1 Running 0 6m13s
kube-system calico-node-7dj7f 1/1 Running 0 6m32s
kube-system calico-node-7rxdf 1/1 Running 0 6m32s
kube-system calico-node-l2mmr 1/1 Running 0 6m32s
kube-system coredns-8474476ff8-5zvkp 1/1 Running 0 6m
kube-system coredns-8474476ff8-jsmzc 1/1 Running 0 6m3s
kube-system dns-autoscaler-7df78bfcfb-hwzpw 1/1 Running 0 6m1s
kube-system kube-apiserver-k8s-master 1/1 Running 0 7m38s
kube-system kube-controller-manager-k8s-master 1/1 Running 1 7m38s
kube-system kube-proxy-4xxpd 1/1 Running 0 6m45s
kube-system kube-proxy-grbk7 1/1 Running 0 6m45s
kube-system kube-proxy-q8bzd 1/1 Running 0 6m45s
kube-system kube-scheduler-k8s-master 1/1 Running 1 7m38s
kube-system nginx-proxy-k8s-node01 1/1 Running 0 6m51s
kube-system nginx-proxy-k8s-node02 1/1 Running 0 6m50s
kube-system nodelocaldns-djzsl 1/1 Running 0 6m
kube-system nodelocaldns-jc5c9 1/1 Running 0 6m
kube-system nodelocaldns-jnvtj 1/1 Running 0 6m
metallb-system controller-6b78bff7d9-vwzp2 1/1 Running 0 5m49s
metallb-system speaker-bbxj2 1/1 Running 0 5m49s
metallb-system speaker-f2jwg 1/1 Running 0 5m49s
metallb-system speaker-kzzkz 1/1 Running 0 5m49skubectl-completion-bash
$ echo 'source <(kubectl completion bash)' >>~/.bashrc
$ echo 'alias k=kubectl' >>~/.bashrc
$ echo 'complete -F __start_kubectl k' >>~/.bashrc
$ source .bashrc
Comments