The key bit is where you said "unchanged." There's no point in having a value in state that never changes. The purpose of state is to hold information that, when changed, will cause a rendering update. If you never change it, there's no reason to put it in state.

But, don't use useMemo for this. From the documentation:

You may rely on useMemo as a performance optimization, not as a semantic guarantee. In the future, React may choose to “forget” some previously memoized values and recalculate them on next render, e.g. to free memory for offscreen components. Write your code so that it still works without useMemo — and then add it to optimize performance.

(their emphasis)

If you have information you want to hold statically for the entire lifetime of your component, use useRef, not useMemo:

function Example() {
    const refMesh = useRef(null);
    if (!refMesh.current) {
        // One-time initialization
        refMesh.current = new Mesh();
    }
    const mesh = refMesh.current;
    // ...use `mesh` from here on out...
}

Unlike useMemo, you can be certain that only a single Mesh instance is created during the lifecycle of the component. (This also has the other slight advantage that you aren't constantly creating a function to pass useMemo on each render.)

I sometimes use a single ref storing an object to hold multiple pieces of instance information that isn't appropriate for state (including stable callback functions for child components). It's quite handy.

Re your edit:

I know useRef(someValue) is more fit for that but I find it "undeclaratively" to write Mesh.current anywhere

You don't. See the const mesh = refMesh.current; above, you'd use mesh, not refMesh.current, throughout the remainder of the component.

If you find the if awkward, you can wrap this in a hook that provides the semantic guarantee that useMemo doesn't provide, at the cost of creating a function on every render (as with useMemo, useCallback, etc.):

const useData = (instanceInitializer) => {
    const ref = useRef(null);
    if (!ref.current) {
        ref.current = instanceInitializer();
    }
    return ref.current;
};

Then it would be:

function Example() {
    const mesh = useData(() => new Mesh());
    // ...use `mesh` from here on out...
}

That has the (very) slight extra cost of the function creation, but better ergonomics.