Previously I was adding an AVCaptureMovieFileOutput to my AVCaptureSession and enabling cinematic preferredVideoStabilizationMode, receiving great, expected results.
I just switched over to an AVCaptureVideoDataOutput and the cinematic stabilization is causing extremely noticeable lag — about a full second delay before the image preview updates.
Setup:
// Video data output
let outputSettings: [String: Any] = [kCVPixelBufferPixelFormatTypeKey as String: kCVPixelFormatType_32BGRA]
videoDataOutput = AVCaptureVideoDataOutput()
videoDataOutput?.videoSettings = outputSettings
let videoSessionQueue = DispatchQueue(label: "video_capture_session_queue")
videoDataOutput?.setSampleBufferDelegate(self, queue: videoSessionQueue)
videoDataOutput?.alwaysDiscardsLateVideoFrames = true
// Audio data output
let audioDataOutput = AVCaptureAudioDataOutput()
let audioSessionQueue = DispatchQueue(label: "audio_capture_session_queue")
audioDataOutput.setSampleBufferDelegate(self, queue: audioSessionQueue)
// Configure session
captureSession.beginConfiguration()
let videoInput = try AVCaptureDeviceInput(device: currentCameraDevice)
captureSession.addInput(videoInput)
captureSession.addOutput(videoDataOutput!)
if let videoDataOutputConnection = videoDataOutput?.connection(with: .video), videoDataOutputConnection.isVideoStabilizationSupported {
videoDataOutputConnection.preferredVideoStabilizationMode = .cinematic
}
if let audioInput = AVCaptureDevice.default(for: .audio) {
try captureSession.addInput(AVCaptureDeviceInput(device: audioInput))
captureSession.addOutput(audioDataOutput)
}
captureSession.commitConfiguration()
captureSession.startRunning()
Does anyone have any insight or a solution? Thanks in advance.
To anyone interested, here's what I've learned after a few more days of digging:
The lag can't be avoided when rendering to a non-AVCaptureVideoPreviewLayer — which is essentially any time you're running an AVCaptureVideoDataOutput to process individual frames (and true for both CI and Metal setups).
Stabilization mode .cinematic leads to the most noticeable lag — ~1 second. Switching to .standard cuts that down to ~1/10 of a second, and produces better results than no stabilization at all.