I am running an Apache (Version 2.4.29) server on Ubuntu 18.04. I am trying to scan a folder of a large amount of data (stored txt files) for a keyword that the user enters on the website. I have tried running Grep on a .sh file (I have enabled the use of shell files and all that in my .htaccess file) but it gives a code 500 error - internal error.
Please can someone give me some ideas of how I can accomplish this? Many thanks!
Edit:
Here is my .htaccess code
Options -Indexes
Options +ExecCGI
order deny,allow
deny from all
allow from 192.168.0.
AddHandler cgi-script .sh
(the CGI part allows the .sh files to run and I know that they can run because I ran a simple script that returned the date and time and that worked)
Here is the script I am trying to run
ack "userinputdata" --files-with-matches
or
grep "userinputdata" /path/to/file
(ack is something I downloaded but I believe grep is a part of ubuntu) Also, to get the user's input I planned to use PHP to write a new file replacing "userinputdata" with whatever they wanted and then running that and deleting it after.
Here is a snippet of the error in the error.log file
[Tue Mar 31 12:32:39.614059 2020] [cgi:error] [pid 26249] [client
192.168.0.56:58830] AH01215: : /var/www/html/test.sh
[Tue Mar 31 12:32:39.614075 2020] [cgi:error] [pid 26249] [client
192.168.0.56:58830] AH01215: : /var/www/html/test.sh
[Tue Mar 31 12:32:39.614091 2020] [cgi:error] [pid 26249] [client
192.168.0.56:58830] AH01215: : /var/www/html/test.sh
[Tue Mar 31 12:32:39.614107 2020] [cgi:error] [pid 26249] [client
192.168.0.56:58830] AH01215: : /var/www/html/test.sh
[Tue Mar 31 12:32:39.614123 2020] [cgi:error] [pid 26249] [client
192.168.0.56:58830] AH01215: : /var/www/html/test.sh
You have multiple problems.
Your script has to start out with line that tells the web server what program to invoke to execute script. In your case that would be #!/bin/bash.
You have to send a "Content-type" header before the content -- a string followed by two newline characters. So that would be
echo "Content-type: text/html"
echo ""
Next, you need to access the user input as variable data to your program. In your snippet, the string "userinputdata" means just that: the literal string "userinputdata", with no relationship to the form data (or query string) submitted by the user.
Assuming your form's method is GET (which is what is should be in this case, as opposed to POST), your script has the query string available to it in the variable $QUERY_STRING, but it is url-encoded and needs to be parsed somehow, e.g., by using awk.
You may also have other configuration and/or permissions issues despite the steps you've taken. You might want to put the script in your server's cgi-bin directory.
I would suggest a couple of things: (1) have a good look at https://blog.eduonix.com/shell-scripting/learn-cgi-scripting-using-bash-in-linux-shell-scripting/; (2) consider using a different language, such as PHP. Parsing input parameters manually is a great learning exercise, but it is also a wheel that has been invented for you already in a multitude of languages.
Finally, opening and closing a large number of files -- modern operating systems are really good at that, but it's not the most efficient way to search a lot of data for a text string. So grep may not be the ideal tool. There are databases for stuff like that. But that's getting a bit ahead of ourselves.
Hope this helps -- and have fun!