My function:
def run(cmd = ""): sp = subprocess.run(shlex.split(cmd), shell = False, capture_output = True) pprint(sp) print(sp.stdout.decode("utf-8", 'ignore'))
This function was working normally. Then I suddenly came across awk
and it gave me an error.
Command passed to the function:
run(cmd = "ssh myserver \"tail -10 server.log | awk '{print $7}' | sort | uniq -c | sort -nr | head\"")
To simplify this further,
run(cmd = "ssh myserver \"tail -10 server.log | awk '{print $7}'\"")
Mark the ssh above. All the commands passed to ssh needs to run on the remote and not on local machine. Hence multiple subprocess is not gonna help. The error thrown for both of the above commands is
awk: line 2: missing } near end of file
Both the above commands work fine from terminal.
I suspected, pipe (|) or single quotes ('') might be the culprit. But the below code works fine.
run(cmd = "ssh myserver \"tail -10 server.log | grep something\"")run(cmd = "ssh myserver \"tail -10 server.log | grep 'something'\"")
Can anyone help me out with awk
in the above?
Edit 1
The associated question mentioned here is not related and won't help in my case as the moment tail -10 server.log
changes to tail -10000 server.log
or cat server.log
, multiple subprocess will try to download the whole content of the server.log
from the remote server which might be in GBs and then process the output while the whole command of
ssh myserver cat server.log | awk '{print $7}' | sort | uniq -c | sort -nr | head
might be 15-20 lines.
Since it's used with ssh which as mentioned in the comments: concatenates multiple arguments, intentionally multiple subprocess haven't been used.
Edit 2
Multiple tried snippets after suggestions from comment which doesn't work and throw same or similar error as mentioned above. Both the snippet in the question and suggestions work for normal scenarios and only fail for awk
with the above mentioned error.
Try 1:
cmd = "ssh myserver \"tail -10 server.log | awk '{print $7}'\""sp = subprocess.run(shlex.split(cmd), shell = False, capture_output = True)pprint(sp)
Try 2:
cmd = ["ssh", "myserver", "tail -10 server.log | awk '{print $7}'"]sp = subprocess.run(cmd, shell = False, capture_output = True)pprint(sp)
Try 3:
(Exact suggestion from the comment)
cmd = ['ssh', 'myserver', '''tail -10 server.log | awk '{print $7}'''']sp = subprocess.run(cmd, shell = False, capture_output = True)pprint(sp)
Try 4:
cmd = ["ssh", "myserver", """tail -10 server.log | awk '{print $7}'"""]sp = subprocess.run(cmd, shell = False, capture_output = True)pprint(sp)