I have a bunch of text files (I'm calling them 'index files' here) in a directory each containing a list of files separated by newlines.
In my NodeJS script I then want to iterate over these index files and make a call to tar using the index file as an input via the -T argument. For this I'm using spawnSync
What should happen is that tar then archives all of the files listed in the index file.
Instead what is happening is I get a completely empty archive, and no output.
Here is the relevant part of my script:
console.log("Processing index files");
process.chdir(sourcePath);
for(const key in indexFiles) {
let index = indexFiles[key];
console.log("Processing: "+index);
let commandLine = "tar acf "+outputPath+'/'+key+".tar.bz2 -T "+index;
console.log(process.cwd());
console.log(commandLine);
let tar = spawnSync("tar", ["acf", outputPath+"/"+key+".tar.bz2", "-T", index], {cwd:sourcePath, stdio:"inherit"});
}
In this script sourcePath is the location of the files listed within the index file. I'm setting that as the CWD since that is the way tar would work when I call it from the command line.
What's odd is that, as you can see, I am logging out both the sourcePath and the equivalent command line for my spawnSync call. So the output looks like this.
Processing index files
/media/chmo/NewLinux/scan/2022-03-22/temp
Processing: /media/chmo/NewLinux/wrangled/d01d36c8-698a-4791-9075-73fa4c0af881_0.txt
/media/chmo/NewLinux/scan/2022-03-22/temp
tar acf /media/chmo/NewLinux/wrangled/0.tar.bz2 -T /media/chmo/NewLinux/wrangled/d01d36c8-698a-4791-9075-73fa4c0af881_0.txt
I can literally take the second from last line, which should be the CWD for my call to spawnSync, cd into that directory, and then run the command as it appears in the last line, and that works perfectly. Yet when I do what should be the exact same operation but called from within NodeJS it simply creates an empty tar file.
What's up with that? Not sure how else to explain it. It seems like something very basic that just isn't working, so I'm hoping that I've just done something wrong and someone can point out what that is.
you dont need to changecwd like that, just pass it to 3rd arg of spawnSync.
spawnSync('tar', [args], { cwd: sourcePath })
I recommend using execSync over spawnSync in your situation, since you don't need any info other than output of tar command (which is empty on success).
I made little script that is close to your requirement, hope this helps you.
const fs = require('fs/promises')
const { execSync } = require('child_process')
const path = require('path')
const [, , indexfilePath] = process.argv
async function main() {
const indexFilesDir = path.resolve(indexfilePath)
const files = await fs.readdir(indexFilesDir, { withFileTypes: true })
for (const f of files) {
if (!f.isFile()) continue
console.log('processing index file', f.name)
const output = execSync(`tar -acf /tmp/node-${f.name}.tar.bz2 -T ${f.name}`, {
cwd: indexFilesDir,
// encoding: 'utf8',
})
console.log(`------> ${f.name} <------`)
process.stdout.write(output)
console.log(`------> end <------`)
}
}
main()
.then(() => {
console.log('completed main')
})
.catch(console.error)
above script tags index files dir and saves tar file in /tmp dir.
$ tree ./
.
├── cli.js
└── test
├── contents
│ ├── en.txt
│ └── fr.txt
└── index-1.txt
file content of index-1.txt index file.
$ cat test/index-1.txt
./contents/en.txt
./contents/fr.txt
usage
$ node cli.js ./test/
processing index file index-1.txt
------> index-1.txt <------
------> end <------
completed main
$ ls /tmp/
node-index-1.txt.tar.bz2
$ tar --list -f /tmp/node-index-1.txt.tar.bz2
./contents/en.txt
./contents/fr.txt
note: if you are using this inside of a webserver or other, use either exec or spawn with Promise wrap. Because sync operations blocks event loop of nodejs.