In-Browser Hand Gesture Recognition for Unity with Fastai and TensorFlow.js Pt. 3

fastai
unity
tensorflow
webgl
Build a Unity project as a WebGL application and host it using GitHub Pages.
Published

October 6, 2022

Overview

Part 2 covered implementing a hand gesture classifier in a Unity project as a jslib plugin. In this post, we’ll host the Unity project as a shareable web demo using GitHub Pages. The demo will run locally in the user’s web browser.

Create GitHub Pages Repository

We first need to create a new GitHub repository to store the WebGL build. We can do this on GitHub or locally using Git, GitHub Desktop, or another tool.

Open the Settings tab for the new repository on GitHub.

Open the Pages section. Switch the source branch to main and click the Save button to start the automated build process.

We can check the GitHub Pages build progress under the Actions tab for the repository.

The web page will be accessible once the pages build and deployment workflow completes. Although, we don’t have any web pages at the moment.

GitHub will provide a URL for accessing the web demo in the GitHub Pages section once it finishes building.

Add Unity Build Files

Now let’s add the WebGL build from the previous post. Open the Build folder from part 2.

Copy the folder content to the repository for the GitHub Pages demo.

We can push the local changes to GitHub, which will automatically trigger the pages build and deployment workflow.

Test Live Demo

We can test the web demo at the URL provided by GitHub once the build workflow completes.

Summary

In this tutorial series, we trained a hand gesture classifier with the fastai library, created a TensorFlow.js plugin for Unity to perform inference in a web browser, built the Unity project as a shareable web demo, and hosted it for free using GitHub Pages.

Now that we have a general approach for using TensorFlow.js in Unity applications, we could extend the work in this tutorial by instead using one of the several pretrained models TensorFlow.js provides.

Previous: In-Browser Hand Gesture Recognition for Unity with Fastai and TensorFlow.js Pt. 2

Project Resources: GitHub Repository