As the title suggests, the aim is to convert Direct3DCaptureFrame from the Windows.Graphics.Capture API to a matrix object available in OpenCvSharp (Mat) so that the captured frame may be used with the popular Computer Vision Library.
There's an in-depth issue available on GitHub which gives reference to current attempts and findings. It also includes sample of what the result should look like and what it does as of now.
I personally believe my way isn't the right way and there may be an easier way to handle this, but I just decided to map the memory and copy it to a byte[]. OpenCvSharp::Mat has constructor overloads which allow one to reference data.
If there is a better (hopefully as performant) and easy way to do this, please do suggest it. I would love some help here.
This issue was recently fixed by myself.
The missing thing was padding when referencing the byte[]. It is required to set the step parameter of Mat's overloaded constructor to width * (bytes / pixel). In my case, bytes / pixel was 4.