I struggled with the same problem and the answer for me was to use NumberOfMessagesSent instead. Then I could set my criteria for new messages that came in during my configured period of time. Here is what worked for me in CloudFormation.

Note that individual alarms do not occur if the alarm stays in an alarm state from constant failure. You can setup another alarm to catch those. ie: Alarm when 100 errors occur in 1 hour using the same method.

Updated: Because the metrics for NumberOfMessagesReceived and NumberOfMessagesSent are dependent on how the message is queued, I have devised a new solutions for our needs using the metric ApproximateNumberOfMessagesDelayed after adding a delay to the dlq settings. If you are adding the messages to the queue manually then NumberOfMessagesReceived will work. Otherwise use ApproximateNumberOfMessagesDelayed after setting up a delay.

MyDeadLetterQueue:
    Type: AWS::SQS::Queue
    Properties:
      MessageRetentionPeriod: 1209600  # 14 days
      DelaySeconds: 60 #for alarms

DLQthresholdAlarm:
 Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: "Alarm dlq messages when we have 1 or more failed messages in 10 minutes"
      Namespace: "AWS/SQS"
      MetricName: "ApproximateNumberOfMessagesDelayed"
      Dimensions:
        - Name: "QueueName"
          Value:
            Fn::GetAtt:
              - "MyDeadLetterQueue"
              - "QueueName"
      Statistic: "Sum"
      Period: 300  
      DatapointsToAlarm: 1 
      EvaluationPeriods: 2       
      Threshold: 1
      ComparisonOperator: "GreaterThanOrEqualToThreshold"
      AlarmActions:
        - !Ref MyAlarmTopic

We had the same issue and solved it by using 2 metrics and creating an math expression.

    ConsentQueue:
        Type: AWS::SQS::Queue
        Properties:
            QueueName: "queue"
            RedrivePolicy:
                deadLetterTargetArn:
                    Fn::GetAtt:
                        - "DLQ"
                        - "Arn"
                maxReceiveCount: 3 # after 3 tries the event will go to DLQ
             VisibilityTimeout: 65
    DLQ:
        Type: AWS::SQS::Queue
        Properties:
            QueueName: "DLQ"

    DLQAlarm:
        Type: AWS::CloudWatch::Alarm
        Properties:
            AlarmDescription: "SQS failed"
            AlarmName: "SQSAlarm"
            Metrics:
            - Expression: "m2-m1"
              Id: "e1"
              Label: "ChangeInAmountVisible"
              ReturnData: true
            - Id: "m1"
              Label: "MessagesVisibleMin"
              MetricStat:
                  Metric:
                      Dimensions:
                      - Name: QueueName
                        Value: !GetAtt DLQ.QueueName
                      MetricName: ApproximateNumberOfMessagesVisible
                      Namespace: "AWS/SQS"
                  Period: 300 # evaluate maximum over period of 5 min
                  Stat: Minimum
                  Unit: Count
              ReturnData: false
            - Id: "m2"
              Label: "MessagesVisibleMax"
              MetricStat:
                  Metric:
                      Dimensions:
                      - Name: QueueName
                        Value: !GetAtt DLQ.QueueName
                      MetricName: ApproximateNumberOfMessagesVisible
                      Namespace: "AWS/SQS"
                  Period: 300 # evaluate maximum over period of 5 min
                  Stat: Maximum
                  Unit: Count
              ReturnData: false
            ComparisonOperator: GreaterThanOrEqualToThreshold
            Threshold: 1
            DatapointsToAlarm: 1
            EvaluationPeriods: 1

The period is important so the minimum and maximum are evaluated over a longer period.

I used metric math function RATE to trigger an alarm whenever a message arrives in the dead letter queue.

Select two metrics ApproximateNumberOfMessagesVisible and ApproximateNumberOfMessagesNotVisible for your dead letter queue.

Configure the metric expression as RATE(m1+m2), set the threshold to 0 and select the comparison operator as GreaterThanThreshold.

m1+m2 is the total number of messages in the queue at a given time. Whenever a new message arrives in the queue the rate of this expression will go above then zero. That's how it works.