The problem is that you're changing static files, but not their filenames.

By default apache/nginx/etc serve static content with headers that say "cache this for a very long time" because it's static content, why would you not?

Tacking on random trash to the URL like you're doing with you JS is a kludge that permanently breaks all caching and ensures that users will repeatedly download the exact same static file every time they request a page. You can make the trash less random to break the cache less frequently, but it's still an inefficient kludge. [Albeit a popular one, to my immense annoyance.]

Ideally for resource bundles like JS and CSS, you make a new resource bundle file every time you change it, eg: somefile-v1234.js or somefile-20211007.js and update the reference in your HTML source. This has the side-benefit of ensuring that the versions of your resource bundles always match.

The same goes for any other static file: images, CSV, etc.

The trouble you're having now is that you've updated some HTML pages and the only way to break the cache is to have the user perform an action, like hitting CTRL+F5 to force a refresh.

There are a couple ways around this:

1. Change the Apache/Nginx/etc settings to set shorter expiries for static file cache headers. You may be able to target specific files like index.html, but YMMV.
2. Serve the content with PHP. Anything served via a PHP script will not have any cache headers set by default, as the assumption is that the output of a script is dynamic. You can also issue the caching headers yourself in PHP to control what gets cached for how long.

Lastly, you cannot solve this problem retroactively. If a user has a cached version of the HTML page that has not yet reached its expiration, the user MUST take action to break that cache. There's nothing that can be done server side because the valid cache tells the client that it doesn't have to ask the server.

Once you get to the point of your application being popular enough to warrant putting a CDN in front of it this problem gets much worse as now there's a cache in the middle that the user doesn't have control of, and it's potentially an expensive problem because some CDN providers charge a fee for forcing CDN cache invalidations.