Python is a popular choice for shell scripting and task automation that even Google uses Python in its online courses for Automation. It is also popular in system administration because it can execute shell commands using only its default libraries. In this tutorial, you will learn just that – to run Linux shell commands with Python using a Raspberry Pi as a host computer.
Things you need:
- Basic knowledge in using CLI (Command-Line Interface)
If you don’t know what CLI is or know it, but you need a refresher, kindly visit this tutorial first.
Automation is a buzzword hot as your processor running at 100% in a room with no AC. Home management, data extraction, those omniscient talking boxes in the kitchen, and even self-driving cars! They are all products of automation. Well, it makes sense since we’re living in the fourth industrial revolution. Heard of it? It is where humans and machines try to work as one, and it is happening now.
Using a Raspberry Pi board opens a lot of doors in automation. With its GPIO, you can interface the credit-card sized computer into almost anything that throws or receives digital data. And since it is a fully-fledged Linux computer, you can automate your computer tasks as well. What’s more, Python makes it unimaginably easier! There are two ways to run Linux commands with Python: using the os module and using the subprocess module. Read further and see what you prefer.
Using the os module
First, is the os module using its
system() method. According to official documentation, the os module provides a portable way of using operating system dependent functionality. Ain’t that convenient? To use it to run a Linux command, you must write the code like this:
Sample Code using system()
import os os.system('pwd') os.system('cd ~') os.system('ls -la')
This 4-liner checks your current directory, change location to your home directory, and lists all the contents in detail. It’s a pretty straightforward implementation, but there’s a downside. With system(), you are not allowed to store the resulting output as a variable.
Alternatively, you can use the
popen() method – still under the os module. It opens a pipe from or to the command line. A pipe connects a command’s output to another command’s input. This makes it accessible within Python. To use
popen() to store a as a variable, it should be similar to this:
Sample code using popen()
import os stream = os.popen('ls -la') output = stream.readlines()
If you print the stream variable, you will see its return data. This consists of the actual commands executed, the mode, and the address. Furthermore, if you want to get the whole output as one string, change
Using the subprocess module
The second way to run Linux commands with Python is by using the newer
subprocess module. This module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. It was created to replace both os.system() and os.popen() functions.The only method that matters in subprocess is
run(). With it, you can do everything we’ve done above and more using different arguments. Use the following code as reference:
Writing a simple command using subprocess
import subprocess subprocess.run('ls')
Using the method like this will execute the command ls in your terminal. Unlike
os.system(), it doesn’t work when you add a switch and enter it fully like
subprocess.run('ls -la'). This feature allows the method to take care of quoting and escaping problems hence preventing errors from formatting. To execute
ls -la, pass the command as a list:
subprocess.run(['ls','-la'). Alternatively, you can make the shell argument True to pass the whole thing as a string. Just take note that this can be a security risk if you’re using untrusted input.
Writing a command with switches
import subprocess x = subprocess.run(['ls', '-la'])
import subprocess subprocess.run(['ls -la'], shell=True)
Next, to store the command output inside a variable, simply do it just like any other data. The result won’t be what you’re expecting, however. Since the main purpose of run is to execute the shell command within python, the result won’t be the output you see in the terminal. It will be the return data just like in
os.open. You can check it using the code below.
Storing the command output to a variable
import subprocess x = subprocess.run(['ls', '-la']) print(x) print(x.args) print(x.returncode) print(x.stdout) print(x.stderr)
This sketch dissects the return data of your command using the method’s arguments. Here are some of the frequently used ones:
- args – returns the actual commands executed.
- returncode – returns the return code of the output. 0 means no error.
- stdout – captured stdout from the child process.
- stderr – captured stderr stream from the child process.
Since we did not capture the output in the previous code, we will get None with both stdout and stderr arguments. To enable the capture output argument, refer to the following code:
import subprocess x = subprocess.run(['ls', '-la'], capture_output=True)
Now, if you print x, you will get the list of items in your current directory of type bytes. Convert it to a string by writing
x.stdout.decode(). Alternatively, you can pass the argument
text=True with the main function. The output now should look exactly the same as what you have in the terminal.
Ultimately, we will run Linux commands with Python and save the output you see in the terminal into a text file – a simple task with subprocess. You just need to redirect the stdout stream to your text file using the argument
Saving the command output to a text file
import subprocess with open('list.txt', 'w') as f: subprocess.run(['ls','-la'], stdout=f)
That’s it for this tutorial! To summarize, for quick execution of a short script, consider using
os.popen() methods. But if you have a more comprehensive script to run, you might want to use the
subprocess module instead.