Within the same state machine execution

You can use a Map state to run these tasks in parallel, and use the maximum concurrency setting to reduce excessive lambda executions.

The Map state ("Type": "Map") can be used to run a set of steps for each element of an input array. While the Parallel state executes multiple branches of steps using the same input, a Map state will execute the same steps for multiple entries of an array in the state input.

• MaxConcurrency (Optional) The MaxConcurrency field’s value is an integer that provides an upper bound on how many invocations of the Iterator may run in parallel. For instance, a MaxConcurrency value of 10 will limit your Map state to 10 concurrent iterations running at one time.

This should reduce the likelihood of issues. That said, you would still benefit from adding a retry statement for these cases. Here's an example:

{
  "Retry": [ {
    "ErrorEquals": ["Lambda.TooManyRequestsException", "Lambda.ServiceException"],
    "IntervalSeconds": 2,
    "MaxAttempts": 6,
    "BackoffRate": 2
  } ]
}

Across different executions

If you want to control this concurrency across different executions, you'll have to implement some kind of separate control yourself. One way to prepare your state machine for that is to request the data you need and then using an Activity to wait for a response.

You can use the lambda concurrency you mentioned but then add a retry clause to your step function so that when you hit the concurrency limit, step functions manages the retry of that task that failed.

https://docs.aws.amazon.com/step-functions/latest/dg/concepts-error-handling.html#error-handling-examples

There’s a limit to the number of retries, but you get to define it.

Alternatively , if you want to retry without limit, you could use catch to move to a Wait state when that concurrency is thrown. You can read about catch in the link above too. Here’s a wait state doc.

https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-wait-state.html

You just have wait state transition back to the task state after it completes its wait.