I want to execute a shell script while handling stdout and stderr output. Currently I execute commands using Process.run, with shell=false and three pipes for stdin, stdout and stderr. I spawn fibers to read from stdout and stderr and log (or otherwise process) the output. This works pretty well for individual commands, but fails horribly for scripts.
I could simply set shell=true when calling Process.run, but looking at the Crystal source it seems that merely prepends "sh" to the commandline. I've tried prepending "bash" and it didn't help.
Things like redirection (>file) and pipes (e.g. curl something | bash) don't seem to work with Process.run
For example, to download a shell script and execute it, I tried:
cmd = %{bash -c "curl http://dist.crystal-lang.org/apt/setup.sh" | bash}
Process.run(cmd, ...)
The initial bash was added in the hope that it would enable the pipe operator. It doesn't seem to help. I also tried executing each command separately:
script.split("\n").reject(/^#/, "").each { Process.run(...) }
But of course, that still fails when a command uses redirection or pipes. For example, the command echo "deb http://dist.crystal-lang.org/apt crystal main" >/etc/apt/sources.list.d/crystal.list simply outputs:
"deb http://dist.crystal-lang.org/apt crystal main" >/etc/apt/sources.list.d/crystal.list`
It might work if I used the `` backticks method of execution instead; but then I wouldn't be able to capture the output in real time.
The problem is a UNIX problem. The parent process must be capable to access the STDOUT of the child process. Using a pipe you must start a shell process that will run the whole command, including the | bash and not just curl $URL. In Crystal this is:
command = "curl http://dist.crystal-lang.org/apt/setup.sh | bash"
io = MemoryIO.new
Process.run(command, shell: true, output: io)
output = io.to_s
Or if you want to duplicate what Crystal does for you:
Process.run("sh", {"-c", command}, output: io)
I'm basing my understanding on reading the source code of the run.cr file. The behaviour is very similar to other languages in how it deals with commands and arguments.
Without shell=true, the default behaviour of Process.run is to use the command as the executable to run. This means that the string needs to be a program name, without any arguments, e.g. uname would be a valid name as there's a program on my system called uname in /usr/bin.
If you ever got behaviour of successfully using %{bash -c "echo hello world"} with shell=false, then something is wrong - the default behaviour should have been to try to run a program called bash -c "echo hello world", which is unlikely to exist on any system.
Once you pass in 'shell=true', then it does sh -c <command>, which will allow strings like echo hello world as a command to work; this will also allow redirections and pipelines to work.
The shell=true behaviour can generally be interpreted as doing the following:
cmd = "sh"
args = [] of String
args << "-c" << "curl http://dist.crystal-lang.org/apt/setup.sh | bash"
Process.run(cmd, args, …)
Note that I'm using an array of arguments here - without the array of arguments, you don't have any control over how the arguments are passed into the shell.
The reason why the first version, with or without
shell=truedoesn't work is because the pipeline is outside the-c, which is the command you're sending to bash.
or if you want to call a shell script and get the output i just tried with crystal 0.23.1 and it's work !
def screen
output = IO::Memory.new
Process.run("bash", args: {"lib/bash_scripts/installation.sh"}, output: output)
output.close
output.to_s
end