I fixed all of your problems as you said.

Problem #1 -> solved: you should use white-space:nowrap. if your text is large so in end of your box it will break to the next line. white-space:nowrap don't let that. (I don't recommend that cause this is not good for too long title text.)

Problem #2 -> solved: happens when you click some item and in middle of action (opening box) click on another one. this is because of display: flex and flexGrow: 1.

you used flex box and justifyContent: "center". so when you click and click on another one your wrapper getting smaller and all the cards seem to get closer to each other. flexGrow: 1 will break them. so flexGrow: 5 is solution.

Problem #3 -> solved: wrong place to set onClick. you should set onClick event on your box not on your text. and a condition for cursor is what you want. (item is selected so cursor must be default otherwise must be poiner).

Bonus :) if you set a small width to your wrapper so your rotate box will be prettier. it rotates at start of text and bingo.

import { useState } from "react";

const dataset = [
  { name: "A section" },
  { name: "B section" },
  { name: "C section with long title" },
  { name: "D section" },
  { name: "E section" },
];

function Page() {
  const [openSection, setOpenSection] = useState(1);

  return (
    <div
      style={{
        display: "flex",
        justifyContent: "center",
        alignItems: "center",
        height: "100vh",
      }}
    >
      {dataset.map((datum, i) => {
        const { name } = datum;
        const isOpen = i === openSection;
        return (
          <div
            key={name}
            onClick={() => setOpenSection(i)}
            style={{
              height: "100%",
              backgroundColor: isOpen ? "white" : "lightgray",
              border: `1px solid ${isOpen ? "white" : "black"}`,
              padding: "20px 30px",
              flex: 1,
              flexGrow: isOpen ? 5 : 0,
              transition: "all 1s linear",
              boxSizing: "border-box",
              cursor: openSection !== i ? 'pointer' : 'default',

              "&:first-child": {
                left: 0,
              },

              "&:last-child": {
                right: 0,
              },
            }}
          >
            <div
              style={{
                transform: `rotate(${isOpen ? "0" : "90"}deg)`,
                transition: "all 1s linear",
                width: 1,
                whiteSpace: "nowrap",
              }}
            >
              {name}
            </div>
          </div>
        );
      })}
    </div>
  );
}

export default Page;

1. Smooth transition: add white-space:nowrap so that the text wont shrink when size reduces. Make title div's position absolute so that it's width won't mess with layout and transition.
2. click bug: Couldn't reproduce. Sometimes the codesandbox doesn't update properly. You need to refresh the output page and test again. Check with my updated code if it happens there too.
3. make gray area clickable: move the mouse pointer and the click handler to parent with isOpen condition.

Modified codesandbox:

import { useState } from "react";

const dataset = [
  { name: "A section" },
  { name: "B section" },
  { name: "C section with long title" },
  { name: "D section" },
  { name: "E section" }
];

export default function App() {
  return <Page />;
}

function Page({}) {
  const [openSection, setOpenSection] = useState(0);

  return (
    <div
      style={{
        display: "flex",
        justifyContent: "center",
        alignItems: "center",
        height: "100vh"
      }}
    >
      {dataset.map((datum, i) => {
        const { name } = datum;
        const isOpen = i === openSection;

        return (
          <div
            key={name}
            style={{
              height: "100%",
              backgroundColor: isOpen ? "white" : "lightgray",
              border: `1px solid ${isOpen ? "white" : "black"}`,
              flex: 1,
              flexGrow: isOpen ? 1 : 0,
              transition: "all 2s ease",

              //my changes
              padding: 0,
              flexBasis: "1.2rem",
              cursor: !isOpen ? "pointer" : "auto",
              position: "relative"
            }}
            onClick={!isOpen ? () => setOpenSection(i) : null}
          >
            <div
              style={{
                transition: "all 2s ease",

                //my changes
                transform: `rotate(${isOpen ? "0" : "90"}deg) 
                translateX(${isOpen ? "0" : "50"}%)`,
                whiteSpace: "nowrap",
                width: isOpen ? "100%" : "1rem",
                position: "absolute",
                top: isOpen ? "1rem" : "0",
                left: isOpen ? "1rem" : "0",
                fontWeight: "bold"
              }}
            >
              {name}
            </div>
          </div>
        );
      })}
    </div>
  );
}

I will follow the same order in which you introduced your problems.

1. writing-mode which you're using on the titles, cannot be animated. You could try rotating the text instead using the transform property
2. If you want to make the whole grey area clickable, you should move both the onClick function and the cursor: "pointer" property to the parent div of that exact element which is the title.