Ansible by Example
This posting explains the basic concepts of Ansible at the hand of scripts booting up a K8s cluster to ease novices into Ansible IAC at the hand of an example.
Join the DZone community and get the full member experience.
Join For FreeIn my previous posting, I explained how to run Ansible scripts using a Linux virtual machine on Windows Hyper-V. This article aims to ease novices into Ansible IAC at the hand of an example. The example being booting one's own out-of-cloud Kubernetes cluster. As such, the intricacies of the steps required to boot a local k8s cluster are beyond the scope of this article. The steps can, however, be studied at the GitHub repo, where the Ansible scripts are checked in.
The scripts were tested on Ubuntu20, running virtually on Windows Hyper-V. Network connectivity was established via an external virtual network switch on an ethernet adaptor shared between virtual machines but not with Windows. Dynamic memory was switched off from the Hyper-V UI. An SSH service daemon was pre-installed to allow Ansible a tty terminal to run commands from.
Bootstrapping the Ansible User
Repeatability through automation is a large part of DevOps. It cuts down on human error, after all. Ansible, therefore, requires a standard way to establish a terminal for the various machines under its control. This can be achieved using a public/private key pairing for SSH authentication. The keys can be generated for an Elliptic Curve Algorithm as follows:
ssh-keygen -f ansible -t ecdsa -b 521
The Ansible script to create and match an account to the keys is:
---
- name: Bootstrap ansible
hosts: all
become: true
tasks:
- name: Add ansible user
ansible.builtin.user:
name: ansible
shell: /bin/bash
become: true
- name: Add SSH key for ansible
ansible.posix.authorized_key:
user: ansible
key: "{{ lookup('file', 'ansible.pub') }}"
state: present
exclusive: true # to allow revocation
# Join the key options with comma (no space) to lock down the account:
key_options: "{{ ','.join([
'no-agent-forwarding',
'no-port-forwarding',
'no-user-rc',
'no-x11-forwarding'
]) }}" # noqa jinja[spacing]
become: true
- name: Configure sudoers
community.general.sudoers:
name: ansible
user: ansible
state: present
commands: ALL
nopassword: true
runas: ALL # ansible user should be able to impersonate someone else
become: trueAnsible is declarative, and this snippet depicts a series of tasks that ensure that:
- The Ansible user exists;
- The keys are added for SSH authentication and
- The Ansible user can execute with elevated privilege using sudo
Towards the top is something very important, and it might go unnoticed under a cursory gaze:
hosts: all
What does this mean? The answer to this puzzle can be easily explained at the hand of the Ansible inventory file:
masters:
hosts:
host1:
ansible_host: "192.168.68.116"
ansible_connection: ssh
ansible_user: atmin
ansible_ssh_common_args: "-o ControlMaster=no -o ControlPath=none"
ansible_ssh_private_key_file: ./bootstrap/ansible
comasters:
hosts:
co-master_vivobook:
ansible_connection: ssh
ansible_host: "192.168.68.109"
ansible_user: atmin
ansible_ssh_common_args: "-o ControlMaster=no -o ControlPath=none"
ansible_ssh_private_key_file: ./bootstrap/ansible
workers:
hosts:
client1:
ansible_connection: ssh
ansible_host: "192.168.68.115"
ansible_user: atmin
ansible_ssh_common_args: "-o ControlMaster=no -o ControlPath=none"
ansible_ssh_private_key_file: ./bootstrap/ansible
client2:
ansible_connection: ssh
ansible_host: "192.168.68.130"
ansible_user: atmin
ansible_ssh_common_args: "-o ControlMaster=no -o ControlPath=none"
ansible_ssh_private_key_file: ./bootstrap/ansible It is the register of all machines the Ansible project is responsible for. Since our example project concerns a high availability K8s cluster, it consists of sections for the master, co-masters, and workers. Each section can contain more than one machine. The root-enabled account atmin on display here was created by Ubuntu during installation.
The answer to the question should now be clear — the host key above specifies that every machine in the cluster will have an account called Ansible created according to the specification of the YAML.
The command to run the script is:
ansible-playbook --ask-pass bootstrap/bootstrap.yml -i atomika/atomika_inventory.yml -K
The locations of the user bootstrapping YAML and the inventory files are specified. The command, furthermore, requests password authentication for the user from the inventory file. The -K switch, on its turn, asks that the superuser password be prompted. It is required by tasks that are specified to be run as root. It can be omitted should the script run from the root.
Upon successful completion, one should be able to login to the machines using the private key of the ansible user:
ssh ansible@172.28.110.233 -i ansible
Note that since this account is not for human use, the bash shell is not enabled. Nevertheless, one can access the home of root (/root) using 'sudo ls /root'
The user account can now be changed to ansible and the location of the private key added for each machine in the inventory file:
host1:
ansible_host: "192.168.68.116"
ansible_connection: ssh
ansible_user: ansible
ansible_ssh_common_args: "-o ControlMaster=no -o ControlPath=none"
ansible_ssh_private_key_file: ./bootstrap/ansibleOne Master To Rule Them All
We are now ready to boot the K8s master:
ansible-playbook atomika/k8s_master_init.yml -i atomika/atomika_inventory.yml --extra-vars='kubectl_user=atmin' --extra-vars='control_plane_ep=192.168.68.119'
The content of atomika/k8s_master_init.yml is:
# k8s_master_init.yml
- hosts: masters
become: yes
become_method: sudo
become_user: root
gather_facts: yes
connection: ssh
roles:
- atomika_base
vars_prompt:
- name: "control_plane_ep"
prompt: "Enter the DNS name of the control plane load balancer?"
private: no
- name: "kubectl_user"
prompt: "Enter the name of the existing user that will execute kubectl commands?"
private: no
tasks:
- name: Initializing Kubernetes Cluster
become: yes
# command: kubeadm init --pod-network-cidr 10.244.0.0/16 --control-plane-endpoint "{{ ansible_eno1.ipv4.address }}:6443" --upload-certs
command: kubeadm init --pod-network-cidr 10.244.0.0/16 --control-plane-endpoint "{{ control_plane_ep }}:6443" --upload-certs
#command: kubeadm init --pod-network-cidr 10.244.0.0/16 --upload-certs
run_once: true
#delegate_to: "{{ k8s_master_ip }}"
- pause: seconds=30
- name: Create directory for kube config of {{ ansible_user }}.
become: yes
file:
path: /home/{{ ansible_user }}/.kube
state: directory
owner: "{{ ansible_user }}"
group: "{{ ansible_user }}"
mode: 0755
- name: Copy /etc/kubernetes/admin.conf to user home directory /home/{{ ansible_user }}/.kube/config.
copy:
src: /etc/kubernetes/admin.conf
dest: /home/{{ ansible_user }}/.kube/config
remote_src: yes
owner: "{{ ansible_user }}"
group: "{{ ansible_user }}"
mode: '0640'
- pause: seconds=30
- name: Remove the cache directory.
file:
path: /home/{{ ansible_user }}/.kube/cache
state: absent
- name: Create directory for kube config of {{ kubectl_user }}.
become: yes
file:
path: /home/{{ kubectl_user }}/.kube
state: directory
owner: "{{ kubectl_user }}"
group: "{{ kubectl_user }}"
mode: 0755
- name: Copy /etc/kubernetes/admin.conf to user home directory /home/{{ kubectl_user }}/.kube/config.
copy:
src: /etc/kubernetes/admin.conf
dest: /home/{{ kubectl_user }}/.kube/config
remote_src: yes
owner: "{{ kubectl_user }}"
group: "{{ kubectl_user }}"
mode: '0640'
- pause: seconds=30
- name: Remove the cache directory.
file:
path: /home/{{ kubectl_user }}/.kube/cache
state: absent
- name: Create Pod Network & RBAC.
become_user: "{{ ansible_user }}"
become_method: sudo
become: yes
command: "{{ item }}"
with_items:
kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml
- pause: seconds=30
- name: Configure kubectl command auto-completion for {{ ansible_user }}.
lineinfile:
dest: /home/{{ ansible_user }}/.bashrc
line: 'source <(kubectl completion bash)'
insertafter: EOF
- name: Configure kubectl command auto-completion for {{ kubectl_user }}.
lineinfile:
dest: /home/{{ kubectl_user }}/.bashrc
line: 'source <(kubectl completion bash)'
insertafter: EOF
...
From the host keyword, one can see these tasks are only enforced on the master node. However, two things are worth explaining.
The Way Ansible Roles
The first is the inclusion of the atomika_role towards the top:
roles:
- atomika_baseThe official Ansible documentation states that: "Roles let you automatically load related vars, files, tasks, handlers, and other Ansible artifacts based on a known file structure."
The atomika_base role is included in all three of the Ansible YAML scripts that maintain the master, co-masters, and workers of the cluster. Its purpose is to lay the base by making sure that tasks common to all three member types have been executed.
As stated above, an ansible role follows a specific directory structure that can contain file templates, tasks, and variable declaration, amongst other things. The Kubernetes and ContainerD versions are, for example, declared in the YAML of variables:
k8s_version: 1.28.2-00
containerd_version: 1.6.24-1In short, therefore, development can be fast-tracked through the use of roles developed by the Ansible community that open-sourced it at Ansible Galaxy.
Dealing the Difference
The second thing of interest is that although variables can be passed in from the command line using the --extra-vars switch, as can be seen, higher up, Ansible can also be programmed to prompt when a value is not set:
vars_prompt:
- name: "control_plane_ep"
prompt: "Enter the DNS name of the control plane load balancer?"
private: no
- name: "kubectl_user"
prompt: "Enter the name of the existing user that will execute kubectl commands?"
private: noHere, prompts are specified to ask for the user that should have kubectl access and the IP address of the control plane.
Should the script execute without error, the state of the cluster should be:
atmin@kxsmaster2:~$ kubectl get pods -o wide -A
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-flannel kube-flannel-ds-mg8mr 1/1 Running 0 114s 192.168.68.111 kxsmaster2 <none> <none>
kube-system coredns-5dd5756b68-bkzgd 1/1 Running 0 3m31s 10.244.0.6 kxsmaster2 <none> <none>
kube-system coredns-5dd5756b68-vzkw2 1/1 Running 0 3m31s 10.244.0.7 kxsmaster2 <none> <none>
kube-system etcd-kxsmaster2 1/1 Running 0 3m45s 192.168.68.111 kxsmaster2 <none> <none>
kube-system kube-apiserver-kxsmaster2 1/1 Running 0 3m45s 192.168.68.111 kxsmaster2 <none> <none>
kube-system kube-controller-manager-kxsmaster2 1/1 Running 7 3m45s 192.168.68.111 kxsmaster2 <none> <none>
kube-system kube-proxy-69cqq 1/1 Running 0 3m32s 192.168.68.111 kxsmaster2 <none> <none>
kube-system kube-scheduler-kxsmaster2 1/1 Running 7 3m45s 192.168.68.111 kxsmaster2 <none> <none>All the pods required to make up the control plane run on the one master node. Should you wish to run a single-node cluster for development purposes, do not forget to remove the taint that prevents scheduling on the master node(s).
kubectl taint node --all node-role.kubernetes.io/control-plane:NoSchedule-However, a cluster consisting of one machine is not a true cluster. This will be addressed next.
Kubelets of the Cluster, Unite!
Kubernetes, as an orchestration automaton, needs to be resilient by definition. Consequently, developers and a buggy CI/CD pipeline should not touch the master nodes by scheduling load on it. Therefore, Kubernetes increases resilience by expecting multiple worker nodes to join the cluster and carry the load:
ansible-playbook atomika/k8s_workers.yml -i atomika/atomika_inventory.yml
The content of k8x_workers.yml is:
# k8s_workers.yml
---
- hosts: workers, vmworkers
remote_user: "{{ ansible_user }}"
become: yes
become_method: sudo
gather_facts: yes
connection: ssh
roles:
- atomika_base
- hosts: masters
tasks:
- name: Get the token for joining the nodes with Kuberenetes master.
become_user: "{{ ansible_user }}"
shell: kubeadm token create --print-join-command
register: kubernetes_join_command
- name: Generate the secret for joining the nodes with Kuberenetes master.
become: yes
shell: kubeadm init phase upload-certs --upload-certs
register: kubernetes_join_secret
- name: Copy join command to local file.
become: false
local_action: copy content="{{ kubernetes_join_command.stdout_lines[0] }} --certificate-key {{ kubernetes_join_secret.stdout_lines[2] }}" dest="/tmp/kubernetes_join_command" mode=0700
- hosts: workers, vmworkers
#remote_user: k8s5gc
#become: yes
#become_metihod: sudo
become_user: root
gather_facts: yes
connection: ssh
tasks:
- name: Copy join command to worker nodes.
become: yes
become_method: sudo
become_user: root
copy:
src: /tmp/kubernetes_join_command
dest: /tmp/kubernetes_join_command
mode: 0700
- name: Join the Worker nodes with the master.
become: yes
become_method: sudo
become_user: root
command: sh /tmp/kubernetes_join_command
register: joined_or_not
- debug:
msg: "{{ joined_or_not.stdout }}"
...There are two blocks of tasks — one with tasks to be executed on the master and one with tasks for the workers.
This ability of Ansible to direct blocks of tasks to different member types is vital for cluster formation. The first block extracts and augments the join command from the master, while the second block executes it on the worker nodes.
The top and bottom portions from the console output can be seen here:
janrb@dquick:~/atomika$ ansible-playbook atomika/k8s_workers.yml -i atomika/atomika_inventory.yml
[WARNING]: Could not match supplied host pattern, ignoring: vmworkers
PLAY [workers, vmworkers] *********************************************************************************************************************************************************************
TASK [Gathering Facts] ************************************************************************************************************************************************************************ok: [client1]
ok: [client2]
...........................................................................
TASK [debug] **********************************************************************************************************************************************************************************ok: [client1] => {
"msg": "[preflight] Running pre-flight checks\n[preflight] Reading configuration from the cluster...\n[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'\n[kubelet-start] Writing kubelet configuration to file \"/var/lib/kubelet/config.yaml\"\n[kubelet-start] Writing kubelet environment file with flags to file \"/var/lib/kubelet/kubeadm-flags.env\"\n[kubelet-start] Starting the kubelet\n[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...\n\nThis node has joined the cluster:\n* Certificate signing request was sent to apiserver and a response was received.\n* The Kubelet was informed of the new secure connection details.\n\nRun 'kubectl get nodes' on the control-plane to see this node join the cluster."
}
ok: [client2] => {
"msg": "[preflight] Running pre-flight checks\n[preflight] Reading configuration from the cluster...\n[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'\n[kubelet-start] Writing kubelet configuration to file \"/var/lib/kubelet/config.yaml\"\n[kubelet-start] Writing kubelet environment file with flags to file \"/var/lib/kubelet/kubeadm-flags.env\"\n[kubelet-start] Starting the kubelet\n[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...\n\nThis node has joined the cluster:\n* Certificate signing request was sent to apiserver and a response was received.\n* The Kubelet was informed of the new secure connection details.\n\nRun 'kubectl get nodes' on the control-plane to see this node join the cluster."
}
PLAY RECAP ************************************************************************************************************************************************************************************client1 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
client1 : ok=23 changed=6 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
client2 : ok=23 changed=6 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
host1 : ok=4 changed=3 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0Four tasks were executed on the master node to determine the join command, while 23 commands ran on each of the two clients to ensure they were joined to the cluster. The tasks from the atomika-base role accounts for most of the worker tasks.
The cluster now consists of the following nodes, with the master hosting the pods making up the control plane:
atmin@kxsmaster2:~$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k8xclient1 Ready <none> 23m v1.28.2 192.168.68.116 <none> Ubuntu 20.04.6 LTS 5.4.0-163-generic containerd://1.6.24
kxsclient2 Ready <none> 23m v1.28.2 192.168.68.113 <none> Ubuntu 20.04.6 LTS 5.4.0-163-generic containerd://1.6.24
kxsmaster2 Ready control-plane 34m v1.28.2 192.168.68.111 <none> Ubuntu 20.04.6 LTS 5.4.0-163-generic containerd://1.6.24With Nginx deployed, the following pods will be running on the various members of the cluster:
atmin@kxsmaster2:~$ kubectl get pods -A -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
default nginx-7854ff8877-g8lvh 1/1 Running 0 20s 10.244.1.2 kxsclient2 <none> <none>
kube-flannel kube-flannel-ds-4dgs5 1/1 Running 1 (8m58s ago) 26m 192.168.68.116 k8xclient1 <none> <none>
kube-flannel kube-flannel-ds-c7vlb 1/1 Running 1 (8m59s ago) 26m 192.168.68.113 kxsclient2 <none> <none>
kube-flannel kube-flannel-ds-qrwnk 1/1 Running 0 35m 192.168.68.111 kxsmaster2 <none> <none>
kube-system coredns-5dd5756b68-pqp2s 1/1 Running 0 37m 10.244.0.9 kxsmaster2 <none> <none>
kube-system coredns-5dd5756b68-rh577 1/1 Running 0 37m 10.244.0.8 kxsmaster2 <none> <none>
kube-system etcd-kxsmaster2 1/1 Running 1 37m 192.168.68.111 kxsmaster2 <none> <none>
kube-system kube-apiserver-kxsmaster2 1/1 Running 1 37m 192.168.68.111 kxsmaster2 <none> <none>
kube-system kube-controller-manager-kxsmaster2 1/1 Running 8 37m 192.168.68.111 kxsmaster2 <none> <none>
kube-system kube-proxy-bdzlv 1/1 Running 1 (8m58s ago) 26m 192.168.68.116 k8xclient1 <none> <none>
kube-system kube-proxy-ln4fx 1/1 Running 1 (8m59s ago) 26m 192.168.68.113 kxsclient2 <none> <none>
kube-system kube-proxy-ndj7w 1/1 Running 0 37m 192.168.68.111 kxsmaster2 <none> <none>
kube-system kube-scheduler-kxsmaster2 1/1 Running 8 37m 192.168.68.111 kxsmaster2 <none> <none>
All that remains is to expose the Nginx pod using an instance of NodePort, LoadBalancer, or Ingress to the outside world. Maybe more on that in another article...
Conclusion
This posting explained the basic concepts of Ansible at the hand of scripts booting up a K8s cluster. The reader should now grasp enough concepts to understand tutorials and search engine results and to make a start at using Ansible to set up infrastructure using code.
Opinions expressed by DZone contributors are their own.
Comments