Let's say I want to edit all the files in a folder, changing every header. I'm using this script:
for thing in $(ls $1); do sed -i '1c\SNP A2 A1 beta N P' $thing done
The problem is that it takes a lot of time. So, I'd like to find a way to dedicate more RAM for this script, in order to do it quickly. Is it possible?
I'd like to find a way to dedicate more RAM for this script, in order to do it quickly. Is it possible?
No. The tools you use, bash to enumerate your files and sed to edit them, take the RAM they need to do their work. They can't use more RAM even if they had a way to give it to them.
You can run your sed operations in parallel. This uses more cores on your machine and may finish faster. Put an & character after each command in the loop. Something like this.
for thing in $(ls $1); do
sed -i '1c\SNP A2 A1 beta N P' $thing &
done
With GNU Parallel you can control how many are run in parallel:
doit() {
sed -i '1c\SNP A2 A1 beta N P' "$1"
}
export -f doit
ls $1 | parallel -j5 doit
Adjust -j5 until you find the number that gives you most throughput.