I think you may just want Array.prototype.filter(). It pretty much works just like map but it filters out values based on your callback. (The callback in this case always returns a boolean)

The filter() method creates a new array with all elements that pass the test implemented by the provided function.

The example on MDN shows filtering based on length but you can easily get rid of undefined values by changing the callback to something like:

const result = words.filter(word => !word);

filter is really handy and I use it alone and in conjunction with map all the time. Since it returns a new array, you can just string map right on to it. Try this:

{carousel.cards.filter(img => !img).map(
    image => (
        <img
            src={image.fields.carouselHome.fields.file.url}
            alt="Image d'acceuil"
        />;
    )
})}

Note: I also changed that src property to an expression, your example will just render "image.fields.carouselHome.fields.file.url" as a string :)

You're missing a return so it doesn't return a node to render.

And if it's empty just return a document fragment or null.

        <div className="embla__container-home">
          {carousel.cards.map((image) => {
            if (image.fields.carouselHome !== undefined)
              return <img
                src="image.fields.carouselHome.fields.file.url"
                alt="Image d'acceuil"
              />;

            return <></> // or null;
          })}
        </div>

EDIT: brainfart; a cleaner solution would be to use filter:

carousel.cards.filter((image) => image.fields.carouselHome))...