I have a bunch of code files I've selected with a find command.
I need to count lines of code but filtering out lines I don't care about – namely all the import statements.
How can I do this in bash?
Here's what I have so far:
function codecount { find . -name "$@" | grep -v test | xargs wc -l; }
So if I run codecount *.java it will find all my Java files, eliminating any that are test code, then count the lines. I want to further refine this to filter out the "import" lines on the remaining files before the line count.
In Bash, you can use globstar together with extglob:
shopt -s extglob
shopt -s globstar
codecount() {
grep -v ^import **/!(test*).java | wc -l
}
Explaination:
grep -v ^import <file-list> prints all lines from <file-list> except those starting with import.**/!(test*).java can be decomposed into three parts:
** is used to match all files in the current directory and in subdirectories;!(test*) means: everything except those files starting with test;.java: everything that ends with .javaSo, in the end, it'll match all Java files except those starting with test.
wc -l counts lines.Note that it counts blank lines too. If you want to exclude blank lines, then use:
grep -v -e "^import" -e "^$" **/!(test*).java | wc -l
I suggest consider using ag instead of grep.
It's created for this usage, and it's faster than grep, since it automatically ignores binary files and .git folders etc. It also have lots of useful extra options.
List file paths with import and the matching line count in each file
ag import --java --count
Detailed summary
ag import --java --stats
You can of course also search by regex. I think this will give the result you are looking for, by using invert-match to only show non-matching lines, and nogroup to not add gaps and headers for each file.
ag '(^import|test)' --invert-match --java --nogroup | wc -l
It is almost possible using only grep:
grep -r --include='*.java' --exclude='*test*' -vch '^import' *
This does the following:
-r: recursively search through all subdirectories--include='*.java': only grep files ending with .java--exclude-'*test*': ... but exclude files where the name contains test-v '^import': invert match, exclude lines starting with import-c: count matches instead of returning them (counts per line)-h: suppress output of file names, just print number of matchesThis returns something like
2
3
5
and we just want the total. Instead of piping to | paste -s -d '+' | bc or even | awk '{sum += $1} END { print sum }', we can go one step back, have the matching lines printed and then count them with wc:
grep -r --include='*.java' --exclude='*test*' -vh '^import' * | wc -l
This can be turned into a function where the parameter determines what should be included, we just have be careful with quoting:
codecount () {
grep -r --include="$1" --exclude='*test*' -vh '^import' * | wc -l
}
This has to be called like
codecount '*.java'
to avoid expansion of the * before being used in the function.
Minimal change to OP's solution:
If you really want to use find, grep and xargs, you could modify your try like this:
codecount () {
find . -name "$@" | grep -v test | xargs grep -v '^import' | wc -l
}
I've just added one grep -v step to filter lines starting with import (and replaced function codecount with codecount () for improved portability).
Notice that this still needs to be called with quotes on the command line:
codecount '*.java'
find -exec cat and grep:
A last solution: using find to filter file names containing test instead of xargs grep -v, then cat so grep doesn't see the file names:
find -type f -name '*.java' -not -name '*test*' -exec cat {} \; |
grep -vc '^import'
This can be made into a function just like the other two examples, and the quoting remark still applies.