Instead of naming your function getPublic, consider instead usePublic. Or even better, generalize it so you can use different paths.

But first, we need to look at the definition of onSnapshot() (a CollectionReference extends from Query):

export declare function onSnapshot<T>(query: Query<T>, observer: {
    next?: (snapshot: QuerySnapshot<T>) => void;
    error?: (error: FirestoreError) => void;
    complete?: () => void;
}): Unsubscribe;

As you can see here, the messages aren't returned from this function, but an Unsubscribe function is (a () => void). So to update a messages array, you'll need to use useState and because you are using realtime listeners, you should use useEffect to manage the listener lifecycle. You also should handle the intermediate states such as loading, errored and fetched data. This results in:

import { useEffect, useState } from 'react';
import { getFirestore, collection, onSnapshot } from "firebase/firestore";

function useMessageFeed(feed = "public", firestore = getFirestore()) { // use default firestore instance unless told otherwise
  // set up somewhere to store the data
  const [ messagesInfo, setMessagesInfo ] = useState(/* default messagesInfo: */ {
    status: "loading",
    messages: null,
    error: null
  });
  
  // attach and manage the listener
  useEffect(() => {
    const unsubscribe = onSnapshot( // unsubscribe is a () => void
      collection(/* firestore instance: */ firestore, /* collection path: */ feed),
      {
        next: querySnapshot => setMessagesInfo({
          status: "loaded",
          messages: querySnapshot.docs(), // consider querySnapshot.docs().map(doc => ({ id: doc.id, ...doc.data() }))
          error: null
        }),
        error: err => setMessagesInfo({
          status: "error",
          messages: null,
          error: err
        })
      }
    );

    return unsubscribe;
  }, [firestore, feed]); // <-- if these change, destroy and recreate the listener
  
  return messagesInfo; // return the data to the caller
}

Elsewhere in your code, you would use it like this:

const SomeComponent = (props) => {
  const { status, messages, error: messagesError } = useMessageFeed("public");
  
  switch (status) {
    case "loading":
      return null; // hides component
    case "error":
      return (
        <div class="error">
          Failed to retrieve data: {messagesError.message}
        </div>
      );
  }

  // render messages
  return (
    /* ... */
  );
}