Using PowerShell with Ansible AWX: Part 2

Now that you have a containerized deployment of AWX with PowerShell (see part 1), you can start to leverage the additional capabilities it provides within your Ansible Playbooks.

PowerShell in Ansible

Over the last year or so, I’ve found that there are 2 main ways to run PowerShell code from within a Playbook, not counting the more-traditional approach of using a Windows Bridge Host to execute PowerShell code on a Windows-based machine. Each method allows for varying levels of complexity and control, depending on the desired outcome.

Single, direct commands

There are some simple tasks that need to occur within your Ansible Playbook where a module either does not exist or is missing required functionality. I’ve often found in these cases that a PowerShell cmdlet for the same operation does exist. For example, Ansible’s ec2_instance module does not support changing the size of an EC2 instance (check out this 4 year old issue on GitHub). Many examples use the AWS CLI, but because we already have PowerShell installed, we can simply do the following using Ansible’s shell module.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

- name: Change the EC2 instance type
    shell: >
      Import-Module AWSPowerShell.NetCore; Edit-EC2InstanceAttribute
      -InstanceId {{ instance_id }}
      -InstanceType {{ instance_type }}
      -Region {{ region }}
      -AccessKey {{ access_key | quote }}
      -SecretKey {{ secret_key | quote }}      
    args:
      executable: /usr/bin/pwsh
    no_log: true
    register: psout

Breaking it down

So, how does this all work? Well, we’re using PowerShell as an alternative shell within our CentOS-based AWX docker container to execute a command. This is the same way that executing an awscli command would work when using the container’s bash shell. Let’s go step by step.

1. Use the Ansible shell module to execute a command on the Ansible control node. The > character is used within YAML syntax to denote a folded block scalar, which folds newline characters to spaces in order to make long commands easier to read.
2. Use the PowerShell cmdlet Import-Module to add the AWS cmdlets (contained within the module AWSPowerShell.NetCore) to the current session. PowerShell on Linux does not import modules into sessions automatically in the way that Windows PowerShell does, so to execute commands you have to explicitly load any modules you’ll need beforehand.
3. Use the Edit-EC2InstanceAttribute cmdlet to update the EC2 instance type. The instance ID, new instance type, and AWS connection information for region and authentication are added as PowerShell parameters, their values provided by Ansible variables (denoted by the {{}} jinja2 expression syntax). Note that the AWS authentication parameters should be quoted using the Ansible | quote filter to ensure that special characters aren’t interpreted by the PowerShell engine.
4. The module uses the /usr/bin/pwsh shell as defined by the executable attribute to execute the command rather than the default /bin/bash shell.
5. Ansible’s no_log attribute is set to true for the Task to prevent AWS authentication secrets being exposed in the AWX job log.
6. An Ansible variable named psout is created to store the various data streams generated by the PowerShell command.

Complex PowerShell scripts

Ansible, despite all its awesome capabilities such as when and loop, has a hard time with complex conditional logic and nested loops. This is of course partially by design, as Ansible was never intended to be a full programming language that supports these functions. The underlying Python brain that Ansible was built on top of is much more suited to solve these problems, but PowerShell fills the same gap.

Over the last year, I’ve had great success managing massive, long-running PowerShell scripts (some almost 1000 lines long) from within Ansible. PowerShell scripts are also great for compacting multiple similar operations that would take huge blocks of code within YAML (due to the way conditional expressions are handled) to a single switch statement. Here’s how to run a PowerShell script using the Ansible shell module:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

- name: Take snapshot of each VMware server
  shell: >
    ./New-Snapshot.ps1
    -vCenterUser {{ vc_user }}
    -vCenterPassword {{ vc_pass | quote }}
    -ServiceNowAuth {{ servicenow_auth | quote }}
    -Servers {{ vm_names | join(', ') }}
    -TowerJobId {{ tower_job_id }}    
  args:
    chdir: "{{ playbook_dir }}/../powershell"
    executable: /usr/bin/pwsh
  no_log: true
  register: psout

Similar to how a single command is executed, the YAML folded block scalar is used to execute a dot-sourced script, rather than an imported cmdlet. Within the Servers parameter, I use the Ansible | join filter to flatten the Ansible list variable vm_names into a comma-separated string that can be interpreted by PowerShell. I am also including the Ansible magic variable tower_job_id as a parameter when executing the PowerShell script, so PowerShell is aware of which AWX job it was spawned from. This can be useful for logging, user notification, or error handling within the PowerShell script.

An additional difference to executing a single command is the addition of the chdir attribute to the shell module. This value indicates where the PowerShell script file is located relative to the Ansible Playbook that executes it (whose filesystem location is stored in the Ansible magic variable playbook_dir).

PowerShell output parsing and error handling

Now that you know how to execute PowerShell commands and scripts within Ansible, it’s important to understand how to parse and then use the various PowerShell output streams (stdout, stderr) that are returned to Ansible within the psout variable that was registered.

Output

Any data that is written to PowerShell’s stdout stream is returned to Ansible within the stdout and stdout_lines attributes of the psout variable. To view the output in an AWX job log, it needs to be passed to Ansible’s debug module. This is also why we have the no_log parameter applied on the PowerShell command or script invocation, so the cmd attribute of the psout variable (which contains secrets) is not written to the AWX job log along with the output.

1
2
3
4

- name: Output results of PowerShell script
  debug:
    msg: "{{ psout.stdout_lines }}"
  when: psout.stdout_lines is defined

The Ansible conditional attribute when is used to ensure that the Task is only executed when the PowerShell command or script returned data to the stdout stream. If there was no PowerShell output generated, the debug Task will fail due to an undeclared variable attribute. The stdout_lines attribute is used to display each entry of the stdout stream on a separate line.

Error handling

Any data that is written to PowerShell’s error stream is returned to Ansible within the stderr and stderr_lines attributes of the psout variable.

1
2
3
4
5

- name: Write errors of PowerShell script
  debug:
    msg: '{{ psout.stderr_lines | regex_replace("\\x1b|\\x1B|[0-9]{1,2}m|[ |~]{2,}","") }}'
  failed_when: true
  when: psout.stderr_lines is defined and psout.stderr_lines | length > 0

Unlike the output from psout.stdout_lines, the output from psout.stderr_lines needs to be formatted slightly for the sake of readability. PowerShell 7 introduced new, colorful error views which wreak havoc on plaintext output streams like those found in Ansible. These new error views cause all the ANSI color codes to be written to the AWX job log along with the error messages, making troubleshooting and debugging a massive pain. Thankfully, regex came to the rescue in the form of the Ansible | regex_replace filter, which removes the ANSI codes to provide more readable error information.

You might have also noticed the failed_when attribute that is applied to this Task. This is to ensure that any user viewing the AWX job log is aware that the text being output by the Task is error output rather than standard output. Ansible will always report debug tasks as OK unless otherwise specified using either the changed_when or failed_when attributes. These attributes can take conditional expressions similar to the when attribute, but for this use case I want to always indicate that the output, regardless of what it is, should be defined as error output (and therefore fail the job).

The last thing to make note of is the addition of a second conditional in the when attribute for this Task. I’ve found that sometimes Ansible will initialize the stderr_lines attribute of the psout variable even when PowerShell does not return any errors. Therefore, we have to ensure that error output is only written when the attribute is not an empty string by using the Ansible | length filter.

If you’re now left thinking

Ansible already has a massive library of modules

There are some cases (more than I care to get into) where Ansible modules lack specific functionality of a cloud provider or virtualization platform, but PowerShell has a cmdlet. I understand the disparities in maturity and level of adoption between PowerShell and Ansible, but I was shocked at how many basic things were missing, broken, or just inefficient from some Ansible module collections.

Use the URI module and call the APIs directly

While utilizing the RESTful web APIs of cloud providers directly rather that pre-packaged Ansible modules is a viable option to access edge or preview features, building query parameters and handling authentication using Ansible’s limited syntax and logic support can sometimes become an exercise in futility, where you end up with a 200+ line Playbook that could have been achieved with a 50 line (or less!) script.

Write your own Python modules

I could of course write my own module(s) to provide missing functionality, and it could be argued that many of the cases were purely due to the fact that I was trying to use Ansible for things it was never designed. However, part of the fun of open-source tooling is being able to extend it, bolt it together, and leverage a network of capabilities rather than tools in isolation to achieve your ultimate business or technical goals. I happen to be very experienced and proficient with PowerShell, while my knowledge is practitioner level with Python, so by embedding PowerShell into Ansible rather than custom Python, I am able to leverage my existing skill set and reduce the time to deliver solutions.

Wrapping up

Now that we are out of the weeds and past all that code, I hope that you are starting to see some value in allowing PowerShell to support and extend the functionality of Ansible far beyond its out-of-box capabilities.

There are a couple nuances to understand before you begin developing, and valid arguments exist against implementing this solution at all. However, I think this is a perfect example of using your existing skills and the tools at your disposal to create something amazing and unique, while still providing production-level reliability and performance that enterprises have come to expect from tools such as Ansible, AWX, and PowerShell.