I'm using awk to strip useful infos from log files. This is what my log file looks like:
2016-02-19 20:18:46,861 115971,100126017,524,523,1,[144115198332971054]
2016-02-19 20:18:46,874 95496,100126019,5,5,0,[]
2016-02-19 20:18:46,883 115974,100126025,57,57,0,[]
2016-02-19 20:18:46,891 115975,100126026,4,4,0,[]
2016-02-19 20:18:46,918 115976,100126027,122,122,0,[]
2016-02-19 20:18:47,688 115978,100126029,11656,11641,15,[144115198334490817,144115197319238988,144115197291063350,144115198332904743,144115197318718547,144115197319714394,144115197306930902,144115197250548791,144115198320676757,14411519
7253880518,144115197289305237,144115198083289344,144115197319697491,144115198273784435,144115198081583082]
2016-02-19 20:18:47,731 99590,100126032,12,12,0,[]
2016-02-19 20:18:47,832 115982,100126034,1397,1396,1,[144115198273784435]
2016-02-19 20:18:47,849 106705,100126035,31,31,0,[]
2016-02-19 20:18:47,860 107469,100126036,16,16,0,[]
2016-02-19 20:18:47,927 115983,100126037,824,824,0,[]
2016-02-19 20:18:47,985 115985,100126039,564,564,0,[]
2016-02-19 20:18:48,048 115986,100126040,338,338,0,[]
2016-02-19 20:18:48,108 115987,100126041,259,259,0,[]
2016-02-19 20:18:48,187 115989,100126043,693,692,1,[144115198273784435]
I use "," for the FS variable; I need the complete content between the [] square brackets, so I tried to set RS as "]":
awk 'BEGIN { FS=","; RS="]";} { print $2 ,$3, $6 ,$7}' removed-apply.log.2016-02-19
but the result is wrong:
861 115971 100126017 1 [144115198332971054]
874 95496 100126019 0 []
883 115974 100126025 0 []
891 115975 100126026 0 []
918 115976 100126027 0 []
688 115978 100126029 15 [144115198334490817
731 99590 100126032 0 []
832 115982 100126034 1 [144115198273784435]
849 106705 100126035 0 []
860 107469 100126036 0 []
927 115983 100126037 0 []
985 115985 100126039 0 []
048 115986 100126040 0 []
108 115987 100126041 0 []
187 115989 100126043 1 [144115198273784435]
Seems the RS is still the line break.
Update: On second thought, you can get away with a single input field-separator regex (specified via option -F, which translates to variable FS):
awk -F ',\\[?|\\]' '{ print $2 ,$3, $6 ,$7 }' removed-apply.log.2016-02-19
Note the need to double \ instances to produce the character following them as a literal in the context of a regex. E.g, \\[ is turned into literal \[ by awk's initial string parsing, which regex parsing then sees as \[, resulting in interpretation as literal [. In short: string ,\\[?|\\] results in regex ,\[?|\].
Original answer (as accepted):
Your input is still clearly line-oriented, so there's no reason to change RS, the input-record separator.
Instead, parse each line in two steps:
FS to initially break the input into 2 fields: before [, and between [...].
[][], is a character set( [...]) containing 2 literal characters, ] and [; it is conceptually equivalent to \[|\].split() to split each resulting field by , into subfields stored in arrays:awk '
BEGIN { FS="[][]" } # split into $1 (before "[") and $2 (between "[...]")
{
split($1, fa1, ",") # split $1 into subfields by "," and store in array fa1
split($2, fa2, ",") # split $2 into subfields by "," and store in array fa2
# Output fields of interest
print fa1[2], fa1[3], fa2[1], fa2[2]
}' removed-apply.log.2016-02-19
Using "]" as the RS seems to be a step in the wrong direction. (How would you tell the difference between an input file consisting of "[abc]" and one consisting of "[abc"?)
Assuming the text you want is between square brackets on the same line, the following would be able to handle the type of input you describe:
grep '\[.*\]' | sed -e 's/^[^[]*\[\(.*\)\].*/\1/'
You might need to tweak this depending on the details of your requirements. It would also be easy to translate this into awk.
If your requirements are more complex than the above can handle, then please elaborate them.