Ever Wondered Why Ansible Feels Slow? Using Border0 Session Log Analysis

The other day I was troubleshooting a simple Ansible connectivity issue and decided to peek under the hood. Using Border0's session recording feature I went down a bit of a rabbit hole. What I found was… a bit more chatty than expected.

Here’s what happens when you run just this:

 ansible all -i my-server, -m command -a "uptime"

You’d naively expect a single SSH connection, maybe one exec, and done… But in reality, here’s what Border0 recorded: 8 SSH sessions, multiple exec calls, and a full SFTP transfer ... just to run uptime.

🔍 What Happens Behind the Scenes

Border0's SSH session logs provide detailed visibility into every SSH event, the screenshot below shows the various sessions channels and types!

If you're wondering why Ansible sometimes feels slow, well… this is why! Lots to dig into.

SSH Session record for Ansible

Session 1 – Find Home Directory

 /bin/sh -c 'echo ~ && sleep 0'

Ansible needs to know where to drop temp files. In this case the output was /root. It will use that for the next few steps.

Border0 captures the result in STDOUT

Session 2 – Create Temporary Working Dir

 /bin/sh -c '( umask 77 && mkdir -p "` echo /root/.ansible/tmp `"&& mkdir "` echo /root/.ansible/tmp/ansible-tmp-1743568662.9087949-90053-53568272910543 `" && echo ansible-tmp-1743568662.9087949-90053-53568272910543="` echo /root/.ansible/tmp/ansible-tmp-1743568662.9087949-90053-53568272910543 `" ) && sleep 0' 

It prepares a temp directory for module execution.

Session 3 – Discover Python

/bin/sh -c 'echo PLATFORM; uname; echo FOUND; command -v '"'"'python3.13'"'"'; command -v '"'"'python3.12'"'"'; command -v '"'"'python3.11'"'"'; command -v '"'"'python3.10'"'"'; command -v '"'"'python3.9'"'"'; command -v '"'"'python3.8'"'"'; command -v '"'"'/usr/bin/python3'"'"'; command -v '"'"'python3'"'"'; echo ENDFOUND && sleep 0'

It tries to figure out which versions of Python are available.

Session 4 – Confirm Python Works

/bin/sh -c '/usr/bin/python3.12 && sleep 0

Just making sure the interpreter doesn’t blow up.

Session 5 – Upload Module via SFTP

Ansible now uploads its payload (AnsiballZ_command.py) over SFTP. No command, just file transfer.

SFTP request was captured by Border0

Session 6 – Make the Script Executable

/bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1743568662.9087949-90053-53568272910543/ /root/.ansible/tmp/ansible-tmp-1743568662.9087949-90053-53568272910543/AnsiballZ_command.py && sleep 0'

Session 7 – Run the Payload

/bin/sh -c '/usr/bin/python3.12 /root/.ansible/tmp/ansible-tmp-1743568662.9087949-90053-53568272910543/AnsiballZ_command.py && sleep 0'

This is where the actual "uptime" gets run.

Border0 captures STDOUT, so we can see for each shell or exec, what happened. Here you'll recognize the familiar ansible output, notice that it comes straight from the server.

STDOUT for each exec available in Border0 portal

Session 8 – Cleanup

'rm -f -r /root/.ansible/tmp/ansible-tmp-1743568662.9087949-90053-53568272910543/ > /dev/null 2>&1 && sleep 0'

Deletes all the temporary files it just created.

🐢 And That’s Why It Feels Slow

Ansible isn’t just SSH-ing in and running a command. It’s:

• doing discovery
• uploading Python scripts
• setting permissions
• running temporary code
• cleaning up after itself

All good practices… but it adds a lot of overhead.

⚡️ Pro Tip: Use Raw Mode

If you just want to run a command like uptime, use raw mode:

ansible all -i my-server, -m raw -a "uptime"

Raw mode skips the module upload and just executes your command directly: one SSH session, one exec, done

🤔 So Why Was I Looking at This?

This all started while helping a customer run Ansible over Border0, using AWS SSM as the upstream protocol.

Turns out there's a catch: Ansible can work over SSM, but only in raw mode, where it just issues a single exec.

The default mode, with all its file uploads and session juggling, doesn’t work well with SSM. That’s because the SSM API isn’t a real interactive shell, you can’t stream input or output. You just send a command and get back a response later. Kind of limiting, honestly 😅

The good news? You can skip all that and just run Ansible directly over Border0, no SSM required.

For what seemed like a “simple” issue… this turned into a surprisingly deep dive. Reminded me a bit of this scene  👇

Sometimes it's just a lightbulb.

Other times, it's… a whole thing. 😄 Luckily, with the right tools everything can be fixed!

Ready to level up
your security?