Issues

1. The useEffect to start an interval has no dependency array so a new interval was started each time the component rendered. This is what led to the bigger and bigger jumps.
2. The progressTimer is redeclared each render cycle so there's no way to clear it.
3. The check of the progress state is closed over in callback scope when passed to the setInterval callback. You're only ever looking at the initial state value. In other words, it's a stale enclosure.
4. Using prevState => (prevState += 10) in the functional state update actually mutates the previous state. All state mutations should be avoided.

A Solution

1. Add a dependency array to the useEffect so it runs once on component mount. Move the handlePlay logic into the effect callback so there are no external dependencies when mounting. Don't forget to return a cleanup function to clear any running intervals when the component unmounts.
2. Store the progressTimer as a React ref so it's a stable reference.
3. Unmix the state update with the side-effect of clearing the timer. Use a second useEffect hook to check when the current progress value reaches 100.
4. In the state updater just return the previous state plus 10, prevState => prevState + 10 as the next state value.

Code

function App() {
  const [progress, setProgress] = useState(0);

  const progressTimer = useRef();

  function handleTime() {
    setProgress((prevState) => prevState + 10);
  }

  useEffect(() => {
    console.log("Progress: " + progress);
    if (progress >= 100) clearInterval(progressTimer.current);
  }, [progress]);

  useEffect(() => {
    console.log("Timer start");
    progressTimer.current = setInterval(handleTime, 1000);

    return () => clearInterval(progressTimer.current);
  }, []);

  return (
    <div className="App">
      <h1>Hello CodeSandbox</h1>
      {progress}
      <h2>Start editing to see some magic happen!</h2>
    </div>
  );
}

setInterval with React, when called on the first render, will result in the interval callback having a stale closure of the stateful variable(s) after the first render.

I'd use setTimeout instead, so that whenever the callback runs, it'll have scope of the most up-to-date state.

const { useState, useEffect } = React;

function App() {
  const [progress, setProgress] = useState(0);
  function handleTime() {
    if (progress <= 100) {
      console.log("Progress: " + progress);
      setProgress((prevState) => (prevState += 10));
    } else {
      console.log("greater");
    }
  }

  useEffect(() => {
    const timerId = setTimeout(handleTime, 1000);
    return () => clearTimeout(timerId);
  });

  return (
    <div className="App">
      <h1>Hello CodeSandbox</h1>
      {progress}
      <h2>Start editing to see some magic happen!</h2>
    </div>
  );
}


ReactDOM.render(<App />, document.querySelector('.react'));

<script crossorigin src="https://unpkg.com/react@16/umd/react.development.js"></script>
<script crossorigin src="https://unpkg.com/react-dom@16/umd/react-dom.development.js"></script>
<div class='react'></div>