Quantcast
Channel: Active questions tagged python - Stack Overflow
Viewing all articles
Browse latest Browse all 13951

python subprocess.run with awk failing [duplicate]

$
0
0

My function:

def run(cmd = ""):    sp = subprocess.run(shlex.split(cmd), shell = False, capture_output = True)    pprint(sp)    print(sp.stdout.decode("utf-8", 'ignore'))

This function was working normally. Then I suddenly came across awk and it gave me an error.

Command passed to the function:

run(cmd = "ssh myserver \"tail -10 server.log | awk '{print $7}' | sort | uniq -c | sort -nr | head\"")

To simplify this further,

run(cmd = "ssh myserver \"tail -10 server.log | awk '{print $7}'\"")

Mark the ssh above. All the commands passed to ssh needs to run on the remote and not on local machine. Hence multiple subprocess is not gonna help. The error thrown for both of the above commands is

awk: line 2: missing } near end of file

Both the above commands work fine from terminal.

I suspected, pipe (|) or single quotes ('') might be the culprit. But the below code works fine.

run(cmd = "ssh myserver \"tail -10 server.log | grep something\"")run(cmd = "ssh myserver \"tail -10 server.log | grep 'something'\"")

Can anyone help me out with awk in the above?

Edit 1

The associated question mentioned here is not related and won't help in my case as the moment tail -10 server.log changes to tail -10000 server.log or cat server.log, multiple subprocess will try to download the whole content of the server.log from the remote server which might be in GBs and then process the output while the whole command of

ssh myserver cat server.log | awk '{print $7}' | sort | uniq -c | sort -nr | head

might be 15-20 lines.
Since it's used with ssh which as mentioned in the comments: concatenates multiple arguments, intentionally multiple subprocess haven't been used.

Edit 2

Multiple tried snippets after suggestions from comment which doesn't work and throw same or similar error as mentioned above. Both the snippet in the question and suggestions work for normal scenarios and only fail for awk with the above mentioned error.

Try 1:

cmd = "ssh myserver \"tail -10 server.log | awk '{print $7}'\""sp = subprocess.run(shlex.split(cmd), shell = False, capture_output = True)pprint(sp)

Try 2:

cmd = ["ssh", "myserver", "tail -10 server.log | awk '{print $7}'"]sp = subprocess.run(cmd, shell = False, capture_output = True)pprint(sp)

Try 3:
(Exact suggestion from the comment)

cmd = ['ssh', 'myserver', '''tail -10 server.log | awk '{print $7}'''']sp = subprocess.run(cmd, shell = False, capture_output = True)pprint(sp)

Try 4:

cmd = ["ssh", "myserver", """tail -10 server.log | awk '{print $7}'"""]sp = subprocess.run(cmd, shell = False, capture_output = True)pprint(sp)

Viewing all articles
Browse latest Browse all 13951

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>