close

[Solved] File not found when launching a subprocess containing piped commands

Hello Guys, How are you all? Hope You all Are Fine. Today I get the following error File not found when launching a subprocess containing piped commands in python. So Here I am Explain to you all the possible solutions here.

Without wasting your time, Let’s start This Article to Solve This Error.

How File not found when launching a subprocess containing piped commands Error Occurs?

Today I get the following error File not found when launching a subprocess containing piped commands in python.

How To Solve File not found when launching a subprocess containing piped commands Error ?

  1. How To Solve File not found when launching a subprocess containing piped commands Error?

    To Solve File not found when launching a subprocess containing piped commands Error If you need a pipeline or other shell features, the simple fix is to add shell=True:

  2. File not found when launching a subprocess containing piped commands

    To Solve File not found when launching a subprocess containing piped commands Error If you need a pipeline or other shell features, the simple fix is to add shell=True:

Solution 1


You have to add shell=True to execute a shell command. check_output is trying to find an executable called: date | grep -o -w '"+tz+"'' | wc -w and he cannot find it. (no idea why you removed the essential information from the error message).

See the difference between:

>>> subprocess.check_output('date | grep 1')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.4/subprocess.py", line 603, in check_output
    with Popen(*popenargs, stdout=PIPE, **kwargs) as process:
  File "/usr/lib/python3.4/subprocess.py", line 848, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.4/subprocess.py", line 1446, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'date | grep 1'

And:

>>> subprocess.check_output('date | grep 1', shell=True)
b'gio 19 giu 2014, 14.15.35, CEST\n'

Read the documentation about the Frequently Used Arguments for more information about the shell argument and how it changes the interpretation of the other arguments.


Note that you should try to avoid using shell=True since spawning a shell can be a security hazard (even if you do not execute untrusted input attacks like Shellshock can still be performed!).

The documentation for the subprocess module has a little section about replacing the shell pipeline. You can do so by spawning the two processes in python and use subprocess.PIPE:

date_proc = subprocess.Popen(['date'], stdout=subprocess.PIPE)
grep_proc = subprocess.check_output(['grep', '1'], stdin=date_proc.stdout, stdout=subprocess.PIPE)
date_proc.stdout.close()
output = grep_proc.communicate()[0]

You can write some simple wrapper function to easily define pipelines:

import subprocess
from shlex import split
from collections import namedtuple
from functools import reduce

proc_output = namedtuple('proc_output', 'stdout stderr')


def pipeline(starter_command, *commands):
    if not commands:
        try:
            starter_command, *commands = starter_command.split('|')
        except AttributeError:
            pass
    starter_command = _parse(starter_command)
    starter = subprocess.Popen(starter_command, stdout=subprocess.PIPE)
    last_proc = reduce(_create_pipe, map(_parse, commands), starter)
    return proc_output(*last_proc.communicate())

def _create_pipe(previous, command):
    proc = subprocess.Popen(command, stdin=previous.stdout, stdout=subprocess.PIPE)
    previous.stdout.close()
    return proc

def _parse(cmd):
    try:
        return split(cmd)
    except Exception:
        return cmd

With this in place you can write pipeline('date | grep 1') or pipeline('date', 'grep 1') or pipeline(['date'], ['grep', '1'])

Solution 2

The most common cause of FileNotFound with subprocess, in my experience, is the use of spaces in your command. If you have just a single command (not a pipeline, and no redirection, wildcards, etc), use a list instead.

# Wrong, even with a valid command string
subprocess.run(['grep -o -w "+tz+"'])

# Fixed; notice also 
subprocess.run(["grep", "-o", "-w", '"+tz+"'])

This change results in no more FileNotFound errors, and is a nice solution if you got here searching for that exception with a simpler command.

If you need a pipeline or other shell features, the simple fix is to add shell=True:

subprocess.run(
    '''date | grep -o -w '"+tz+"'' | wc -w''',
    shell=True)

However, if you are using python 3.5 or greater, try using this approach:

import subprocess

a = subprocess.run(["date"], stdout=subprocess.PIPE)
print(a.stdout.decode('utf-8'))

b = subprocess.run(["grep", "-o", "-w", '"+tz+"'],
                   input=a.stdout, stdout=subprocess.PIPE)
print(b.stdout.decode('utf-8'))

c = subprocess.run(["wc", "-w"],
                   input=b.stdout, stdout=subprocess.PIPE)
print(c.stdout.decode('utf-8'))

You should see how one command’s output becomes another’s input just like using a shell pipe, but you can easily debug each step of the process in python. Using subprocess.run is recommended for python > 3.5, but not available in prior versions.

Summery

It’s all About this issue. Hope all solution helped you a lot. Comment below Your thoughts and your queries. Also, Comment below which solution worked for you? Thank You.

Also, Read