Have You tried using the toggle() method? I think that it might be what You're looking for

Give it a shot!

https://www.w3schools.com/howto/tryit.asp?filename=tryhow_js_toggle_class

Another example that might be helpful

https://codepen.io/StrengthandFreedom/pen/ZOGVLg

I've created an Asp.NET Core application to test/debug your code with a fresh html page and javascript with your code snippet.

If this code block below is the only code in your Javascript file,

let fetchNumbers = document.querySelectorAll(`.cardnumber`);

for (let index = 0; index < fetchNumbers.length; index++) {

if (fetchNumbers[index].classList.contains('unselected') === true) {
    fetchNumbers[index].addEventListener(`click`, function () {
        fetchNumbers[index].classList.add(`selected`);
        fetchNumbers[index].classList.remove(`unselected`);
        fetchNumbers = document.querySelectorAll(`.cardnumber`);
        return fetchNumbers;
    }); 
} else {
    fetchNumbers[index].addEventListener(`click`, function () {
        fetchNumbers[index].classList.add(`unselected`);
        fetchNumbers[index].classList.remove(`selected`);
        fetchNumbers = document.querySelectorAll(`.cardnumber`)
        return fetchNumbers;
    }); 
}

Info

Javascript files are loaded statically, which means they are ran immediately after the html page is loaded into the DOM, so this code is run immediately after the html page is loaded.

Problem

Based upon the classes defined by your html elements (in this case - class="cardnumber unselected"), once the page loads, your code block above would hit the first if statement, then assign the click event to all cardnumber elements, then proceed to change the class from unselected to selected. When you go to click your html element the second time, the click event from the if block in your if/else block will NOT be ran as the page has already loaded. This means the else block will never execute. This is the reason you are not seeing any functionality on the second click.

note

Javascript events can only be defined once, so you'd never be able to bind both the click events in the if block AND else block.

Solution

Define the click event for your cardnumber. Move your if/else logic into your new click event like below:

let fetchNumbers = document.querySelectorAll('.cardNumber');

for (let index = 0; index < fetchNumbers.length; index++) {

    fetchNumbers[index].addEventListener('click', function () {
        if (fetchNumbers[index].classList.contains('unselected') === true) {
            fetchNumbers[index].classList.add('selected');
            fetchNumbers[index].classList.remove('unselected');
            fetchNumbers = document.querySelectorAll('.cardNumber');
            return fetchNumbers;
        } else {
            fetchNumbers[index].classList.add('unselected');
            fetchNumbers[index].classList.remove('selected');
            fetchNumbers = document.querySelectorAll('.cardNumber')
            return fetchNumbers;
        }
    });
}