How can I check if the current GIF from an API is different from the previous GIF saved from the same API endpoint?
There is an API that gives out a GIF file of visual weather forecast data. This is the API: https://api.met.gov.my/static/images/swirl-latest.gif. This static GIF is updated from time to time.
The Situation: I'm trying to fetch this GIF and save it in my DB, so that I can have a series of GIFs that shows the progression of the weather over time. My current stack is Nodejs, MeteorJs, MongoDB, and React.
The Problem: I don't know how frequent the GIF is updated to schedule a cron job to fetch the new updated GIF. I can set a cron job for every nth minutes/hours to fetch the GIF from the API, but how to determine whether the latest fetched GIF is the same or different from the previously fetched GIF(s) in the DB, thus avoiding a duplicate?
Potential Solution: Since GIFs are basically a series of images, there might be a way to compare two GIFs in terms of their similarities. But I'm not sure how to go about it. Found an npm package gifuct-js that can basically parse a GIF and return some information about the GIF, for example (taken from the package's GitHub page):
{
// The color table lookup index for each pixel
pixels: [...],
// the dimensions of the gif frame (see disposal method)
dims: {
top: 0,
left: 10,
width: 100,
height: 50
},
// the time in milliseconds that this frame should be shown
delay: 50,
// the disposal method (see below)
disposalType: 1,
// an array of colors that the pixel data points to
colorTable: [...],
// An optional color index that represents transparency (see below)
transparentIndex: 33,
// Uint8ClampedArray color converted patch information for drawing
patch: [...]
}
Since the GIFs from the aforementioned API may have the same dimensions, I'm thinking of using either pixels or colorTable as a comparison point between the two GIFs. But I am not sure whether this would yield the result that I want i.e. to determine if the two GIFs are identical or not.
Any help will be much appreciated!
Make a hash from the previous gif and the currently fetched and you can see if there's any difference. You can also check theis file size, but in my opinion checking with hash is more accurate.
Robi Hahn 's idea is more generic and will work.
But in this case there are some other approaches which will work, too. I have checked the given URL and the following two ideas would work.
You can check the response-header's last-modified-field - is it different from the timestamp you got with your last request x minutes ago then save the new image otherwise discard it (as it is the same as your already stored image).
Or you can request for the .gif with the If-Modified-Since-Header containing the timestamp of the latest .gif stored in your database. Then the server either returns a 304 - not modified (in which case there is no newer .gif available) or returns the updated picture.