I am trying to live-stream video from my Raspberry Pi 4 using the h264_v4l2m2m codec (HWA). At the receiving end, I am using the Broadway decoder. For lower resolutions (320x320 @ 60fps) the decoded video quality is fine. However, when I increase the resolution I start to notice a strange tessellation effect as in the following captures:
Tessellation effect on higher resolution (960x960)
I tried other codecs and players (mpeg1video and jsmpeg as a decoder), but at higher resolutions only the hardware accelerated H.264 reaches an acceptable frame rate. For my particular application I do not need audio.
I am running the following command to launch ffmpeg on a separate process:
ffmpeg -hide_banner -fflags nobuffer -f rawvideo -pixel_format yuv420p -s 960x960 \
-i tcp://127.0.0.1:15000?listen=1 -f h264 -codec:v h264_v4l2m2m -s 960x960 -profile:v 66 \
-level 4.2 -coder 0 -bf 0 -an -b:v 4M tcp://0.0.0.0:15001?listen=1
I then send the H.264 NAL Units using a WebSocket.
The following snippet is my adjusted example application for decoding the received frames.
<!DOCTYPE html>
<html>
<head>
<meta charset='utf-8'>
<title>PiCamera H264 Streaming</title>
</head>
<body>
<h1>PiCamera H264 Streaming</h1>
<div id='viewer'></div>
<script src='Decoder.js'></script>
<script src='YUVCanvas.js'></script>
<script src='Player.js'></script>
<script>
// player
window.player = new Player({ useWorker: true, webgl: false })
var playerElement = document.getElementById('viewer')
playerElement.appendChild(window.player.canvas)
// Websocket
var wsUri = window.location.protocol.replace(/http/, 'ws') + '//' + window.location.hostname + ':20000'
var ws = new WebSocket(wsUri)
ws.binaryType = 'arraybuffer'
ws.onopen = function (e) {
console.log('Client connected')
ws.onmessage = function (msg) {
// decode stream
let data = new Uint8Array(msg.data);
console.log(data);
window.player.decode(data);
}
}
ws.onclose = function (e) {
console.log('Client disconnected')
}
</script>
</body>
</html>
What am I missing? Is it some option on ffmpeg, or could it be, that the decoder is not able to fully process the received frames. I have tried multiple bitrate settings to no avail.