I'm in the process of rewriting a PHP app for node.js.
One of the components involves parsing a live webserver log, using the information then writing some of it back to the log.
I lock the file to avoid overwrites (with up to 100+ entries/second in production and parsing every hour or so, they do happen quite often without the lock) like this:
$fh = fopen('./my.log', 'r');
flock($fh, LOCK_EX);
// parse the content, remove old entries, rewrite back the newer ones
flock($fh, LOCK_UN);
fclose($fh);
With node's fs, I get that there's no native way to lock the files. Surely there are libraries, but my first drive was to try something like this:
fs.renameSync('./my.log','./my.log_') // rename the file to something else
fs.writeFileSync('./my.log_', info_to_keep, {flag: 'w+'}) // safely(!?) replace the content
fs.renameSync('./my.log_','./my.log') // rename back
// (would be done async in production, this is for quick testing only)
It seems to get the job done. Even if the original my.log is recreated in those few miliseconds, it'd be nicely replaced (I don't quite care about the bits of lost data! - it's all transient and used for sampling purposes).
I didn't get to test this live though, so I'm not sure if it'd work reliably. Is there any risk involved, like collisions even during the renaming process, or anything like this?
Well, according to some nginx doc (see the "Log Rotation" section), nginx keeps the log file open so when you rename it, nginx just keeps writing to the same file descriptor it had which is now the renamed file. Until you signal it, it won't make a new file. So renaming the log file won't do anything to nginx. According to that, it will just keep writing to the renamed file.
So, I guess the answer to "Is renaming a safe way of locking files?" for nginx is "No". It doesn't lock anything.
In simple words: Linux process doesn't care about file name after the file has been opened by the process. The file handler would retain an address to the file on the disk and use it for IO ops.
Now, if something happens with the name while file being opened it doesn't impact the processes that already opened the handler.
What's more, even physical space is not de-allocated until all the processes close the handles to file. Meaning you can even delete the file using rm but as long as program reads/writes to the file, the data stays healthy.
That is the basis of NGinx log rotation mechanism btw.
Here's a good article about basics of Linux filesystem operation and Stale File Handles: https://www.baeldung.com/linux/stale-file-handles