you actually don't have to add another class list for your div

you can just use CSS to make the element goes red whenever you hover them like this

.pixel:hover {
background-color : red,
}

and that's it

also if you want to make an animation and delay for the hovered div you can use animation property

pixel:hover {
animation : color,
animation-delay : 5s
}

@keyframes color {
0% { 
background-color : white
}
100% {
background-color : red
}
}

that's all you need with CSS, no need for JS

There are a few issues with the original code that prevent it from working as expected:

1. boxes is empty because at the time of running querySelectorAll('div'), no divs exist. Unless your canvas is also a div, in which case it will only find the canvas div element instead. Otherwise, empty.

2. By using addEventListener, you're invoking window.addEventListener (i.e. the top level addEventListener), which is acting on the window itself. You want to target each element, so you should be using pixel.addEventListener (but more on this in a bit).

So modifying your current solution slightly by moving the search for pixel class elements to AFTER the creation loop + doing pixel.addEventListener, which attaches the event listener to each pixel div, the following snippet works. (But keep reading after the snippet).

canvas = document.getElementById('canvas');
const DELAY = 0;

for (i = 0; i <= 20; i++) {
  const box = document.createElement('div')
  box.classList.add('pixel')
  canvas.appendChild(box)
}

boxes = Array.from(document.getElementsByClassName('pixel'));

boxes.forEach((pixel) => {
  pixel.addEventListener('mouseover', function() {
    setTimeout(() => pixel.classList.add('red'), DELAY);
  })

  pixel.addEventListener('mouseout', function() {
    setTimeout(() => pixel.classList.remove('red'), DELAY);
  })
});

.pixel {
  width: 32px;
  height: 32px;
  background-color: black;
  margin: 4px;
}

.red {
  background-color: red;
}

<section id="canvas">
</section>

However, in the above example that's only 20 event listeners. It also doesn't work if you dynamically add more pixel elements - they wont have the associated event listeners. In your original code, you want 2000 elements. That's 2000 event listeners on the page (4000 if you did my example by adding a mouseout event as well to remove the added class).

A better way to scale this for dynamic + large numbers of elements is by just having 1 single event listener on the page and checking to see what element is being targeted:

document.addEventListener('click',function(e){
    if(e.target && e.target.classes.includes('pixel')){
          e.target.classList.add('red');
     }
 });

This creates a single, dynamic event listener on the whole document that checks for the target it's hovering on. This way whether you have 2000 or 2 million elements, you only have 1 event listener effectively checking your hover state.

Note: If all you want to do is just change the color, you can use the other answer and do this via CSS without any event handling. But if you have a reason to do this within javascript (i.e. dynamic color, maybe you want some other javascript code to run when something is hovered over), the above approach is ideal.