Version 2: Part 1
Last Updated: Dec 1, 2020
Previous: Part 6
It’s time to see how our model performs with a webcam. Prerecorded videos are great for testing, but most real-world applications will likely use a live video feed. We’ll set up our video feed for a front-facing camera that mirrors the user.
We can add the option to use a webcam feed by making some modifications to the
PoseNet script and create a new public
bool variable. Name the variable
useWebcam and set the default value to
false. This will create a checkbox in the
Inspector tab that we can use to enable and disable the webcam.
We’ll use a
WebCamTexture variable to store the live video input from our webcam. Name the variable
Set Up Webcam Feed
We’ll prepare the webcam feed at the top of the
Start() method. You can find the completed code below.
First, initialize the
webcamTexture. We’ll use the first video input device Unity finds. If you have more than one webcam attached, you’ll need to specify the device name.
Next, we need to adjust the rotation and scale of the
VideoScreen object. The webcam feed doesn’t mirror the user by default. For example, the user’s right arm appears on the left side of the screen. This can be disorienting when looking at the generated pose skeleton. We’ll flip the
VideoScreen to compensate.
Start the Camera
We’ll use the
webcamTexture.Play() method to start the camera.
Deactivate the Video Player
Finally, we’ll deactivate the
Video Player as it’s not being used.
We’ll use the
Graphics.Blit() method to update the
videoTexture with the data from
webcamTexture. Add the following code at the top of the
Flip Key Point Locations
VideoScreen does not flip the
videoTexture itself. Therefore, the output of the model will not be flipped either. We can fix this by mirroring the
xPos values for the calculated key point locations.
Set Inspector Variable
Now we can enable and disable the webcam from the
Note: Don’t toggle the
useWebcam parameter during runtime with the code as it is.
We can now perform pose estimation using either prerecorded or live video feeds. We’ll further increase our flexibility for input sources in the next post by adding the ability to handle input with different aspect ratios.