Ansible limiting node count on demand

Running ansible playbooks on all nodes in a group is a easy task. Sometimes it is needed to spontaneously pick one node of a group or a group of nodes, for tests. This can be achived many ways. Here it is explained using the --limit option, without changing anything in the files.

How to limit ansible playbook execution without changing a single file i and be able to run following tests:

  • pick exactly one node
  • pick one particular, well known node
  • a small group 3 nodes out of many
  • fibonaci number deployment on hudge group of nodes

Let's assume ansible inventory has 100 nodes:

user % more hosts

[a99k] a9k1.cgn a9k1.fra a9k1.dus a9k1.muc a9k1.ham a9k1.ber ... a9k1.igs

Limit ansible playbook to only only one node, first entry: 0. The limit is in the brackets [], it starts with a zero and ends with an zero, Zero is the first element. Both zeros are separated by a colon :

user % ansible-playbook show-version.yml --limit[0:0]

Limit ansible playbook to one particular node: a9k1.ber

user % ansible-playbook show-version.yml --limit a9k1.ber

Limit ansible playbook to 3 nodes in a row, 4-th to 7-th node chosen:

user % ansible-playbook show-version.yml --limit[4:7]

Limit ansible playbook to the fibonacci sequence:

0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181, 6765, 10946, 17711, 28657, 46368, 75025, 121393, 196418, 317811, ...

Ansible fibonacci sequence style deployment example, statically

ansible-playbook show-version.yml --limit[0:0]
ansible-playbook show-version.yml --limit[1:1]
ansible-playbook show-version.yml --limit[2:3]
ansible-playbook show-version.yml --limit[4:7]
ansible-playbook show-version.yml --limit[8:13]
ansible-playbook show-version.yml --limit[14:22]
ansible-playbook show-version.yml --limit[23:36]
ansible-playbook show-version.yml --limit[37:58]
ansible-playbook show-version.yml --limit[59:93]
ansible-playbook show-version.yml --limit[94:149]
ansible-playbook show-version.yml --limit[150:239]
ansible-playbook show-version.yml --limit[240:384]
ansible-playbook show-version.yml --limit[385:618]
ansible-playbook show-version.yml --limit[619:996]
ansible-playbook show-version.yml --limit[997:1607]
ansible-playbook show-version.yml --limit[1608:2595]

The fibonacci style deployment is an idea from a blog networking blog run by networkingnerd - automating change with help from fibonacci. Networkingnerd wrote a great article about the fibonacci sequence deployments, in the context of dealing with big node count. Try it out, just to find out.

Running dropbear as fallback SSH daemon

Most linux distributions default choice SSH daemon is the OpenSSH implementaton. But managing and updating the openssh server results sometimes in connection loss to the target node. A nasty task to solve, especially if there is no out of band management available f.e. : via serial connection or KVM console or power management via PoE. This ends in a visit on site to reset the failed hardware, in good hope the node will survive the reboot, and come up in a manageable state.

This specific kind of management loss can be solved easy by running 2 different SSH daemons on the target node. Both SSH deamons listening on different TCP port. Linux distributions offer sometimes a additional SSH daemon called dropbear. Both SSH implementations OpenSSH and dropbear will run at the same time on the target node. This helps out during occasional updates on the system. By having a separate, redundant SSH installation that is not affected by current SSH server update. It works vice-versa. By updating the dropbear SSH server, the running OpenSSH server is the fallback solution.

The assumption is target node runs OpenSSH server already:

Verify first using the command ss:

user % ss -tulpen | grep ssh
tcp   LISTEN 0      128                              0.0.0.0:22        0.0.0.0:*    users:(("sshd",pid=26925,fd=3)) ino:2884866 sk:5 <->

Install dropbear server:

root # emerge -va dropbear

Configure dropbear to run listening on port 22222, not allowing root login. Display dropbear help to get an overview of available daemon running options:

user % dropbear -h

Edit the /etc/conf.d/dropbear config file, and set -w and -p option:

# see dropbear -h for more information
# -w disables root logins
# -p changes the port number to listen on
DROPBEAR_OPTS="-w -p 0.0.0.0:22222"

Start the dropbear daemon:

root # /etc/init.d/dropbear start
root # service dropbear start

Verify active TCP sockets:

root # ss -tulpen | egrep 'ssh|bear'
tcp   LISTEN 0      0                                    0.0.0.0:22        0.0.0.0:*    users:(("sshd",pid=4322,fd=3)) ino:5231627 sk:12a93723
tcp   LISTEN 0      0                                    0.0.0.0:22222     0.0.0.0:*    users:(("dropbear",pid=2324,fd=4)) ino:5232378 sk:2c1684223

On nodes the entry 0.0.0.0 means, the daemon is listening on all local available IP interfaces. No explicit interfaces given, catchall option.

Test local connectivity and availability by using the dropbear SSH client dbclient:

user % dbclient localhost/22222
user % dbclient localhost

Do the same with OpenSSH SSH client software:

user % ssh localhost -p 222222
user % ssh localhost

If you use PuTTy by any chance then plink is the CLI client software:

user % plink localhost -P 22222
user % plink localhost

This is a solution solving a single point of failure for a node, where no physical management access is given to the target. It solves this specific problem, by adding another, redundant, independent, working SSH daemon.

I used to be an adventurer like you, then I took an arrow in the knee.
Ansible export facts to simple csv file

Generating anible CSV reports of your Cisco hardware.

For firmware updates, one either has a monitoring tool that offers an overview of the currently used firmware versions on the networking hardware, or one can create a report using ansible to an comma separated value file, known as .csv and then import the data into a spreadsheet program like librecalc or gnumeric or MS Excel. This is only the reporting side of the process. Nothing fancy so far, but gives all the interesting information one would need for decision making on how, if ... etc.

The ansible playbook below will generate a CSV file with basic hardware facts. Actually no more data is needed. This ansible playbook is very basic, and can be used for small enterprise switches much like f.e. the Catalyst 1000, Catalyst 2000 and Catalyst 3000 series. This will not work with Catalyst 4500 and bigger modular switches, only because of the hardcoded flash name in the script. Modular switches mostly have bootflash instead of flash. You would need to rewrite that script for all the other switch series, be it a switch stack or be it a modular chassis switch. Honestly I wanted the script to be as easy as it gets, and did not want to cover all the corner cases. It should be simple and expandable at first. Feel free to take this as a basis for further own expansions.

More it is a simplified and rewritten version of the Brendan Choi gather script who seems to be an networking engineer in Australia. Thanks Brendan.

This script does following:

  • Generate CSV filename using date
  • Put in a header int the output file
  • Get facts from netowrking hardware and write it to a temporary file called csv_tmp
  • Write data into the final export file
  • Remove blank lines from csv
---
# ansible v2.11
- name: Write ios_facts into a csv file
  hosts: c29k
  connection: network_cli
  gather_facts: yes
  vars_files:
    - vault.yml

  vars:
    output_path: "./reports/"
    filename: "device_report_{{ date }}.csv"

  tasks:
  - name: CSV - Generate output filename
    set_fact: date="{{lookup('pipe','date +%Y%m%d')}}"
    run_once: true

  - name: CSV - Create file and set the header
    lineinfile:
      dest: "{{ output_path }}/{{ filename }}"
      line:
        hostname,image,iostype,model,serialnum,system,version,spacefree_kb,spacetotal_kb
      create: yes
      state: present

  - name: CSV - Get IOS devices facts
    set_fact:
      csv_tmp: >
        {{ ansible_net_hostname }},{{ ansible_net_image }},{{ ansible_net_iostype }},{{ ansible_net_model }},{{ ansible_net_serialnum }},{{ ansible_net_system }},{{ ansible_net_version }},{{ ansible_net_filesystems_info['flash:']['spacetotal_kb'] }},{{ ansible_net_filesystems_info['flash:']['spacefree_kb'] }}

  - name: CSV - Write information into .csv file
      lineinfile:
    insertafter: EOF
    dest: "{{ output_path }}/{{ filename }}"
    line: "{{ csv_tmp }}"

  - name: CSV - Blank lines removal
    lineinfile:
      path: "./{{ output_path }}/{{ filename }}"
      state: absent
      regex: '^\s*$'

For further information look up the ansible line in file documentation. This works with ansible version 2.11, check the header of the playbook. Since the ansible development is rather quick, the header makes it easier to identify compatible ansible versions. Much like the cisco version header in the configuration file of your networking gear. Wishing happy automating.