Ansible by Example

This posting explains the basic concepts of Ansible at the hand of scripts booting up a K8s cluster to ease novices into Ansible IAC at the hand of an example.

Oct. 04, 23 · Tutorial

Likes (8)

Comment

Save

7.6K Views

In my previous posting, I explained how to run Ansible scripts using a Linux virtual machine on Windows Hyper-V. This article aims to ease novices into Ansible IAC at the hand of an example. The example being booting one's own out-of-cloud Kubernetes cluster. As such, the intricacies of the steps required to boot a local k8s cluster are beyond the scope of this article. The steps can, however, be studied at the GitHub repo, where the Ansible scripts are checked in.

The scripts were tested on Ubuntu20, running virtually on Windows Hyper-V. Network connectivity was established via an external virtual network switch on an ethernet adaptor shared between virtual machines but not with Windows. Dynamic memory was switched off from the Hyper-V UI. An SSH service daemon was pre-installed to allow Ansible a tty terminal to run commands from.

Bootstrapping the Ansible User

Repeatability through automation is a large part of DevOps. It cuts down on human error, after all. Ansible, therefore, requires a standard way to establish a terminal for the various machines under its control. This can be achieved using a public/private key pairing for SSH authentication. The keys can be generated for an Elliptic Curve Algorithm as follows:

ssh-keygen -f ansible -t ecdsa -b 521

The Ansible script to create and match an account to the keys is:

    YAML
   
 

   ---

- name: Bootstrap ansible
  hosts: all
  become: true
  tasks:
  - name: Add ansible user
    ansible.builtin.user:
      name: ansible
      shell: /bin/bash
    become: true

  - name: Add SSH key for ansible
    ansible.posix.authorized_key:
      user: ansible
      key: "{{ lookup('file', 'ansible.pub') }}"
      state: present
      exclusive: true  # to allow revocation
      # Join the key options with comma (no space) to lock down the account:
      key_options: "{{ ','.join([
          'no-agent-forwarding',
          'no-port-forwarding',
          'no-user-rc',
          'no-x11-forwarding'
        ]) }}"  # noqa jinja[spacing]
    become: true

  - name: Configure sudoers
    community.general.sudoers:
      name: ansible
      user: ansible
      state: present
      commands: ALL
      nopassword: true
      runas: ALL  # ansible user should be able to impersonate someone else
    become: true
  

Ansible is declarative, and this snippet depicts a series of tasks that ensure that:

The Ansible user exists;
The keys are added for SSH authentication and
The Ansible user can execute with elevated privilege using sudo

Towards the top is something very important, and it might go unnoticed under a cursory gaze:

hosts: all

What does this mean? The answer to this puzzle can be easily explained at the hand of the Ansible inventory file:

    YAML
   
masters:

  hosts:

    host1:

      ansible_host: "192.168.68.116"

      ansible_connection: ssh

      ansible_user: atmin

      ansible_ssh_common_args: "-o ControlMaster=no -o ControlPath=none"

      ansible_ssh_private_key_file: ./bootstrap/ansible

comasters:

  hosts:

    co-master_vivobook:

      ansible_connection: ssh

      ansible_host: "192.168.68.109"

      ansible_user: atmin

      ansible_ssh_common_args: "-o ControlMaster=no -o ControlPath=none"

      ansible_ssh_private_key_file: ./bootstrap/ansible

workers:

  hosts:

    client1:
      ansible_connection: ssh

      ansible_host: "192.168.68.115"

      ansible_user: atmin

      ansible_ssh_common_args: "-o ControlMaster=no -o ControlPath=none"

      ansible_ssh_private_key_file: ./bootstrap/ansible
    client2:
      ansible_connection: ssh

      ansible_host: "192.168.68.130"

      ansible_user: atmin

      ansible_ssh_common_args: "-o ControlMaster=no -o ControlPath=none"

      ansible_ssh_private_key_file: ./bootstrap/ansible

It is the register of all machines the Ansible project is responsible for. Since our example project concerns a high availability K8s cluster, it consists of sections for the master, co-masters, and workers. Each section can contain more than one machine. The root-enabled account atmin on display here was created by Ubuntu during installation.

The answer to the question should now be clear — the host key above specifies that every machine in the cluster will have an account called Ansible created according to the specification of the YAML.

The command to run the script is:

ansible-playbook --ask-pass bootstrap/bootstrap.yml -i atomika/atomika_inventory.yml -K

The locations of the user bootstrapping YAML and the inventory files are specified. The command, furthermore, requests password authentication for the user from the inventory file. The -K switch, on its turn, asks that the superuser password be prompted. It is required by tasks that are specified to be run as root. It can be omitted should the script run from the root.

Upon successful completion, one should be able to login to the machines using the private key of the ansible user:

ssh ansible@172.28.110.233 -i ansible

Note that since this account is not for human use, the bash shell is not enabled. Nevertheless, one can access the home of root (/root) using 'sudo ls /root'

The user account can now be changed to ansible and the location of the private key added for each machine in the inventory file:

    YAML
   
    host1:
      ansible_host: "192.168.68.116"
      ansible_connection: ssh
      ansible_user: ansible
      ansible_ssh_common_args: "-o ControlMaster=no -o ControlPath=none"
      ansible_ssh_private_key_file: ./bootstrap/ansible

One Master To Rule Them All

We are now ready to boot the K8s master:

ansible-playbook atomika/k8s_master_init.yml -i atomika/atomika_inventory.yml --extra-vars='kubectl_user=atmin' --extra-vars='control_plane_ep=192.168.68.119'

The content of atomika/k8s_master_init.yml is:

    YAML
   
 

   # k8s_master_init.yml

- hosts: masters
  become: yes
  become_method: sudo
  become_user: root
  gather_facts: yes
  connection: ssh
  roles:
    - atomika_base

  vars_prompt:
    - name: "control_plane_ep"
      prompt: "Enter the DNS name of the control plane load balancer?"
      private: no
    - name: "kubectl_user"
      prompt: "Enter the name of the existing user that will execute kubectl commands?"
      private: no

  tasks:
   - name: Initializing Kubernetes Cluster
     become: yes
     #     command: kubeadm init --pod-network-cidr 10.244.0.0/16 --control-plane-endpoint "{{ ansible_eno1.ipv4.address }}:6443" --upload-certs
     command: kubeadm init --pod-network-cidr 10.244.0.0/16 --control-plane-endpoint "{{ control_plane_ep }}:6443" --upload-certs
     #command: kubeadm init --pod-network-cidr 10.244.0.0/16 --upload-certs
     run_once: true
     #delegate_to: "{{ k8s_master_ip }}"

   - pause: seconds=30

   - name: Create directory for kube config of {{ ansible_user }}.
     become: yes
     file:
       path: /home/{{ ansible_user }}/.kube
       state: directory
       owner: "{{ ansible_user }}"
       group: "{{ ansible_user }}"
       mode: 0755

   - name: Copy /etc/kubernetes/admin.conf to user home directory /home/{{ ansible_user }}/.kube/config.
     copy:
       src: /etc/kubernetes/admin.conf
       dest: /home/{{ ansible_user }}/.kube/config
       remote_src: yes
       owner: "{{ ansible_user }}"
       group: "{{ ansible_user }}"
       mode: '0640'

   - pause: seconds=30

   - name: Remove the cache directory.
     file:
       path: /home/{{ ansible_user }}/.kube/cache
       state: absent

   - name: Create directory for kube config of {{ kubectl_user }}.
     become: yes
     file:
       path: /home/{{ kubectl_user }}/.kube
       state: directory
       owner: "{{ kubectl_user }}"
       group: "{{ kubectl_user }}"
       mode: 0755

   - name: Copy /etc/kubernetes/admin.conf to user home directory /home/{{ kubectl_user }}/.kube/config.
     copy:
       src: /etc/kubernetes/admin.conf
       dest: /home/{{ kubectl_user }}/.kube/config
       remote_src: yes
       owner: "{{ kubectl_user }}"
       group: "{{ kubectl_user }}"
       mode: '0640'

   - pause: seconds=30

   - name: Remove the cache directory.
     file:
       path: /home/{{ kubectl_user }}/.kube/cache
       state: absent

   - name: Create Pod Network & RBAC.
     become_user: "{{ ansible_user }}"
     become_method: sudo
     become: yes
     command: "{{ item }}"
     with_items:
       kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml

   - pause: seconds=30

   - name: Configure kubectl command auto-completion for {{ ansible_user }}.
     lineinfile:
       dest: /home/{{ ansible_user }}/.bashrc
       line: 'source <(kubectl completion bash)'
       insertafter: EOF


   - name: Configure kubectl command auto-completion for {{ kubectl_user }}.
     lineinfile:
       dest: /home/{{ kubectl_user }}/.bashrc
       line: 'source <(kubectl completion bash)'
       insertafter: EOF
...

  

From the host keyword, one can see these tasks are only enforced on the master node. However, two things are worth explaining.

The Way Ansible Roles

The first is the inclusion of the atomika_role towards the top:

    YAML
   
   roles:
  - atomika_base

The official Ansible documentation states that: "Roles let you automatically load related vars, files, tasks, handlers, and other Ansible artifacts based on a known file structure."

The atomika_base role is included in all three of the Ansible YAML scripts that maintain the master, co-masters, and workers of the cluster. Its purpose is to lay the base by making sure that tasks common to all three member types have been executed.

As stated above, an ansible role follows a specific directory structure that can contain file templates, tasks, and variable declaration, amongst other things. The Kubernetes and ContainerD versions are, for example, declared in the YAML of variables:

    YAML
   
   k8s_version: 1.28.2-00
containerd_version: 1.6.24-1

In short, therefore, development can be fast-tracked through the use of roles developed by the Ansible community that open-sourced it at Ansible Galaxy.

Dealing the Difference

The second thing of interest is that although variables can be passed in from the command line using the --extra-vars switch, as can be seen, higher up, Ansible can also be programmed to prompt when a value is not set:

    YAML
   
 

     vars_prompt:
    - name: "control_plane_ep"
      prompt: "Enter the DNS name of the control plane load balancer?"
      private: no
    - name: "kubectl_user"
      prompt: "Enter the name of the existing user that will execute kubectl commands?"
      private: no
  

Here, prompts are specified to ask for the user that should have kubectl access and the IP address of the control plane.

Should the script execute without error, the state of the cluster should be:

    
   atmin@kxsmaster2:~$ kubectl get pods -o wide -A

NAMESPACE      NAME                                 READY   STATUS    RESTARTS   AGE     IP               NODE         NOMINATED NODE   READINESS GATES

kube-flannel   kube-flannel-ds-mg8mr                1/1     Running   0          114s    192.168.68.111   kxsmaster2   <none>           <none>

kube-system    coredns-5dd5756b68-bkzgd             1/1     Running   0          3m31s   10.244.0.6       kxsmaster2   <none>           <none>

kube-system    coredns-5dd5756b68-vzkw2             1/1     Running   0          3m31s   10.244.0.7       kxsmaster2   <none>           <none>

kube-system    etcd-kxsmaster2                      1/1     Running   0          3m45s   192.168.68.111   kxsmaster2   <none>           <none>

kube-system    kube-apiserver-kxsmaster2            1/1     Running   0          3m45s   192.168.68.111   kxsmaster2   <none>           <none>

kube-system    kube-controller-manager-kxsmaster2   1/1     Running   7          3m45s   192.168.68.111   kxsmaster2   <none>           <none>

kube-system    kube-proxy-69cqq                     1/1     Running   0          3m32s   192.168.68.111   kxsmaster2   <none>           <none>

kube-system    kube-scheduler-kxsmaster2            1/1     Running   7          3m45s   192.168.68.111   kxsmaster2   <none>           <none>

All the pods required to make up the control plane run on the one master node. Should you wish to run a single-node cluster for development purposes, do not forget to remove the taint that prevents scheduling on the master node(s).

kubectl taint node --all  node-role.kubernetes.io/control-plane:NoSchedule-

However, a cluster consisting of one machine is not a true cluster. This will be addressed next.

Kubelets of the Cluster, Unite!

Kubernetes, as an orchestration automaton, needs to be resilient by definition. Consequently, developers and a buggy CI/CD pipeline should not touch the master nodes by scheduling load on it. Therefore, Kubernetes increases resilience by expecting multiple worker nodes to join the cluster and carry the load:

ansible-playbook atomika/k8s_workers.yml -i atomika/atomika_inventory.yml

The content of k8x_workers.yml is:

    YAML
   
 

   # k8s_workers.yml
---
- hosts: workers, vmworkers
  remote_user: "{{ ansible_user }}"
  become: yes
  become_method: sudo
  gather_facts: yes
  connection: ssh
  roles:
    - atomika_base

- hosts: masters
  tasks:
   - name: Get the token for joining the nodes with Kuberenetes master.
     become_user: "{{ ansible_user }}"
     shell: kubeadm token create  --print-join-command
     register: kubernetes_join_command

   - name: Generate the secret for joining the nodes with Kuberenetes master.
     become: yes
     shell: kubeadm init phase upload-certs --upload-certs
     register: kubernetes_join_secret

   - name: Copy join command to local file.
     become: false
     local_action: copy content="{{ kubernetes_join_command.stdout_lines[0] }} --certificate-key {{ kubernetes_join_secret.stdout_lines[2] }}" dest="/tmp/kubernetes_join_command" mode=0700

- hosts: workers, vmworkers
  #remote_user: k8s5gc
  #become: yes
  #become_metihod: sudo
  become_user: root
  gather_facts: yes
  connection: ssh

  tasks:

   - name: Copy join command to worker nodes.
     become: yes
     become_method: sudo
     become_user: root
     copy:
       src: /tmp/kubernetes_join_command
       dest: /tmp/kubernetes_join_command
       mode: 0700

   - name: Join the Worker nodes with the master.
     become: yes
     become_method: sudo
     become_user: root
     command: sh /tmp/kubernetes_join_command
     register: joined_or_not

   - debug:
       msg: "{{ joined_or_not.stdout }}"
...
  

There are two blocks of tasks — one with tasks to be executed on the master and one with tasks for the workers.

This ability of Ansible to direct blocks of tasks to different member types is vital for cluster formation. The first block extracts and augments the join command from the master, while the second block executes it on the worker nodes.

The top and bottom portions from the console output can be seen here:

    YAML
   
   janrb@dquick:~/atomika$ ansible-playbook atomika/k8s_workers.yml -i atomika/atomika_inventory.yml

[WARNING]: Could not match supplied host pattern, ignoring: vmworkers

PLAY [workers, vmworkers] *********************************************************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************************************************************ok: [client1]

ok: [client2]

...........................................................................

TASK [debug] **********************************************************************************************************************************************************************************ok: [client1] => {
    "msg": "[preflight] Running pre-flight checks\n[preflight] Reading configuration from the cluster...\n[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'\n[kubelet-start] Writing kubelet configuration to file \"/var/lib/kubelet/config.yaml\"\n[kubelet-start] Writing kubelet environment file with flags to file \"/var/lib/kubelet/kubeadm-flags.env\"\n[kubelet-start] Starting the kubelet\n[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...\n\nThis node has joined the cluster:\n* Certificate signing request was sent to apiserver and a response was received.\n* The Kubelet was informed of the new secure connection details.\n\nRun 'kubectl get nodes' on the control-plane to see this node join the cluster."
}
ok: [client2] => {
    "msg": "[preflight] Running pre-flight checks\n[preflight] Reading configuration from the cluster...\n[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'\n[kubelet-start] Writing kubelet configuration to file \"/var/lib/kubelet/config.yaml\"\n[kubelet-start] Writing kubelet environment file with flags to file \"/var/lib/kubelet/kubeadm-flags.env\"\n[kubelet-start] Starting the kubelet\n[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...\n\nThis node has joined the cluster:\n* Certificate signing request was sent to apiserver and a response was received.\n* The Kubelet was informed of the new secure connection details.\n\nRun 'kubectl get nodes' on the control-plane to see this node join the cluster."
}

PLAY RECAP ************************************************************************************************************************************************************************************client1                    : ok=3    changed=1    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0
client1                    : ok=23   changed=6    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0
client2                    : ok=23   changed=6    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0
host1                      : ok=4    changed=3    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Four tasks were executed on the master node to determine the join command, while 23 commands ran on each of the two clients to ensure they were joined to the cluster. The tasks from the atomika-base role accounts for most of the worker tasks.

The cluster now consists of the following nodes, with the master hosting the pods making up the control plane:

    
   atmin@kxsmaster2:~$ kubectl get nodes -o wide

NAME         STATUS   ROLES           AGE   VERSION   INTERNAL-IP      EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION      CONTAINER-RUNTIME

k8xclient1   Ready    <none>          23m   v1.28.2   192.168.68.116   <none>        Ubuntu 20.04.6 LTS   5.4.0-163-generic   containerd://1.6.24

kxsclient2   Ready    <none>          23m   v1.28.2   192.168.68.113   <none>        Ubuntu 20.04.6 LTS   5.4.0-163-generic   containerd://1.6.24

kxsmaster2   Ready    control-plane   34m   v1.28.2   192.168.68.111   <none>        Ubuntu 20.04.6 LTS   5.4.0-163-generic   containerd://1.6.24

With Nginx deployed, the following pods will be running on the various members of the cluster:

    
   atmin@kxsmaster2:~$ kubectl get pods -A -o wide

NAMESPACE      NAME                                 READY   STATUS    RESTARTS        AGE   IP               NODE         NOMINATED NODE   READINESS GATES

default        nginx-7854ff8877-g8lvh               1/1     Running   0               20s   10.244.1.2       kxsclient2   <none>           <none>

kube-flannel   kube-flannel-ds-4dgs5                1/1     Running   1 (8m58s ago)   26m   192.168.68.116   k8xclient1   <none>           <none>

kube-flannel   kube-flannel-ds-c7vlb                1/1     Running   1 (8m59s ago)   26m   192.168.68.113   kxsclient2   <none>           <none>

kube-flannel   kube-flannel-ds-qrwnk                1/1     Running   0               35m   192.168.68.111   kxsmaster2   <none>           <none>

kube-system    coredns-5dd5756b68-pqp2s             1/1     Running   0               37m   10.244.0.9       kxsmaster2   <none>           <none>

kube-system    coredns-5dd5756b68-rh577             1/1     Running   0               37m   10.244.0.8       kxsmaster2   <none>           <none>

kube-system    etcd-kxsmaster2                      1/1     Running   1               37m   192.168.68.111   kxsmaster2   <none>           <none>

kube-system    kube-apiserver-kxsmaster2            1/1     Running   1               37m   192.168.68.111   kxsmaster2   <none>           <none>

kube-system    kube-controller-manager-kxsmaster2   1/1     Running   8               37m   192.168.68.111   kxsmaster2   <none>           <none>

kube-system    kube-proxy-bdzlv                     1/1     Running   1 (8m58s ago)   26m   192.168.68.116   k8xclient1   <none>           <none>

kube-system    kube-proxy-ln4fx                     1/1     Running   1 (8m59s ago)   26m   192.168.68.113   kxsclient2   <none>           <none>

kube-system    kube-proxy-ndj7w                     1/1     Running   0               37m   192.168.68.111   kxsmaster2   <none>           <none>

kube-system    kube-scheduler-kxsmaster2            1/1     Running   8               37m   192.168.68.111   kxsmaster2   <none>           <none>

All that remains is to expose the Nginx pod using an instance of NodePort, LoadBalancer, or Ingress to the outside world. Maybe more on that in another article...

Conclusion

This posting explained the basic concepts of Ansible at the hand of scripts booting up a K8s cluster. The reader should now grasp enough concepts to understand tutorials and search engine results and to make a start at using Ansible to set up infrastructure using code.

Kubernetes Ansible (software) cluster Infrastructure as code k8s

Opinions expressed by DZone contributors are their own.

Related

Trending