I'm trying to record and replay hand animations on the Hololens 2. I managed to record the tracked Transforms of Joints and use the recordings to animate given hand rigs. Now I'm trying to also record the tracked hand mesh. I'm aware of OnHandMeshUpdated in the IMixedRealityHandMeshHandler interface. Also, the following post guided me in this direction (very helpful):
How to get hand mesh data from Hololens2 without turning on Hand Mesh Visualization option
My question is: Is there a simple way to simulate hand mesh data in the Unity Editor? At the moment I don't have access to my team's Hololens, so I'm trying to figure out how to develop this feature directly in Unity. AFAIK the OnHandMeshUpdated event is only called when there is actual mesh data on the Hololens, but not in the Editor where there are only the simulated joints of the controller, but not the hand mesh.
Any suggestions are welcome!
To simulate hand mesh input, you can use the RiggedHandVisualizer to control a SkinnedMesh built with hand joints data to visualize the hands, and it can work with InputSimulation in the Unity editor. You can find an example in the RiggedHandVisualizer scene under: MRTK/Examples/Experimental/RiggedHandVisualizer/Scenes, and more detail please seeRigged Hand Visualizer [Experimental]