My discord bot is supposed to remove the last reaction if a user clicks a second reaction. I tried fetching the message, reaction and reaction users. Whatever I do, if my bot was restarted after it reacted, reaction.me is always false. How can I fix this?

This is my current code:

//trying to detect a 2nd user reaction gives expected result
console.log('Reaction.me: ', reaction.me) //always false no matter how many times I click the reaction
if(user.bot) return;
if(reaction.partial) await reaction.fetch().catch(console.log)
if(reaction.message.partial) await reaction.message.fetch().catch(console.log);
await reaction.users.fetch()
await reaction.message.reactions.cache.find(r => r.emoji.name === (reaction.emoji.name === '⬆️') ? '⬇️' : '⬆️').users.fetch()
if(!reaction.me) return; //always returns early here

Note: after further testing, it seems that the client user is found in the cache, but reaction.me remains false. It does fix my problem, but I would like to understand why I have this problem with reaction.me.