Code Walkthrough: Unity Media Display Package
Introduction
The Unity Media Display package simplifies the creation and management of demo screens for displaying test images, videos, and webcam streams in Unity projects. As many of my tutorials use a demo screen, this package makes that shared functionality more modular and reusable, allowing me to streamline my tutorial content. Check out the demo video below to see this package in action.
In this post, I’ll walk through the package code, providing a solid understanding of its components and their roles.
Package Overview
The package contains several C# scripts that work together to enable easy management of demo screens.
BaseScreenManager.cs
: This abstract class is the foundation for managing in-scene screen objects. It provides essential functionality and basic structure for derived classes, allowing you to create custom screen managers tailored to your project’s needs.MediaDisplayManager.cs
: This script offers utility methods for setting up and managing a screen object in Unity. It handles initializing and updating screen textures, transformations, and webcam streams.TextureChangeEvent.cs
: This script triggers when a material’s main texture changes, allowing for efficient texture updates and seamless transitions between different media sources.AddCustomDefineSymbol.cs
: An Editor script that automatically adds a custom scripting define symbol to the project after the package installs.ShaderIncludePostprocessor.cs
: This build postprocessor script includes the shader used by the screen material in the build process and only runs in the Unity Editor.
Code Explanation
In this section, we will delve deeper into the Unity Media Display package by examining the purpose and functionality of each C# script. We’ll discuss how these scripts work together to enable seamless management of demo screens for displaying test images, videos, and webcam streams.
BaseScreenManager.cs
This script provides an abstract class for managing in-scene screen objects, allowing you to create custom screen managers tailored to your project’s needs. The complete code is available on GitHub at the link below.
The BaseScreenManager
class includes several serialized fields for configuring scene components and settings, webcam settings, and toggle-key settings.
// Scene components and settings
[Header("Scene")]
[Tooltip("Screen object in the scene")]
[SerializeField] protected GameObject screenObject;
[Tooltip("Camera object in the scene")]
[SerializeField] protected GameObject cameraObject;
[Tooltip("A test texture to display on the screen")]
[SerializeField] protected Texture testTexture;
[Tooltip("A framerate cap to reduce lag")]
[SerializeField] protected int maxFrameRate = 500;
// Webcam settings
[Header("Webcam")]
[Tooltip("Option to use webcam as input")]
[SerializeField] protected bool useWebcam = false;
[Tooltip("Requested webcam dimensions")]
[SerializeField] protected Vector2Int webcamDims = new Vector2Int(1280, 720);
[Tooltip("Requested webcam framerate")]
[SerializeField, Range(0, 60)] protected int webcamFrameRate = 60;
// Toggle key settings
[Header("Toggle Key")]
[Tooltip("Key to toggle between image and webcam feed")]
[SerializeField] protected KeyCode toggleKey = KeyCode.Space;
It also contains protected variables for managing the current texture, webcam devices, and the webcam texture.
protected Texture currentTexture;
protected WebCamDevice[] webcamDevices;
protected WebCamTexture webcamTexture;
protected string currentWebcam;
Lastly, the script has some constant values for working with webcam devices.
// The value to multiply the webcam frame rate by to determine the application's target framerate.
protected const int WebcamFrameRateMultiplier = 4;
// The value for the width of an uninitialized webcam device
protected const int UninitializedWebcamWidth = 16;
The script includes several methods:
OnEnable
Subscribes to the TextureChangeEvent
for handling changes in the mainTexture
of a Material
.
// Subscribe to the texture change event
protected virtual void OnEnable()
{
.OnMainTextureChanged += HandleMainTextureChanged;
TextureChangeEvent}
Initialize
Sets the application’s target frame rate, configures the webcam devices, and initializes the current webcam if the useWebcam
option is enabled.
// Initializes the application's target frame rate and configures the webcam devices.
protected virtual void Initialize()
{
// Limit the application's target frame rate to reduce lag.
.targetFrameRate = maxFrameRate;
Application// Get the list of available webcam devices.
= WebCamTexture.devices;
webcamDevices // If no webcam devices are available, disable the useWebcam option.
= webcamDevices.Length > 0 ? useWebcam : false;
useWebcam // Set the current webcam device to the first available device, if any.
= useWebcam ? webcamDevices[0].name : "";
currentWebcam }
UpdateDisplay
Updates the display with the current texture, sets up the webcam if necessary, and starts a coroutine to update the screen texture asynchronously.
// Updates the display with the current texture (either a test texture or the webcam feed).
protected virtual void UpdateDisplay()
{
// Set up the webcam if necessary.
SetupWebcam();
// Update the current texture based on the useWebcam option.
UpdateCurrentTexture();
// Start a coroutine to asynchronously update the screen texture.
StartCoroutine(UpdateScreenTextureAsync());
}
SetupWebcam
If the useWebcam
option is enabled and there are available webcam devices, this method initializes the webcam device and stops it if the option is disabled.
// Sets up the webcam if the useWebcam option is enabled.
protected void SetupWebcam()
{
// If there are no webcam devices, return immediately.
if (webcamDevices.Length == 0) return;
// If the useWebcam option is enabled, initialize the webcam.
if (useWebcam)
{
// Initialize the webcam and check if it started playing.
bool webcamPlaying = MediaDisplayManager.InitializeWebcam(ref webcamTexture, currentWebcam, webcamDims, webcamFrameRate);
// If the webcam failed to start playing, disable the useWebcam option.
= webcamPlaying ? useWebcam : false;
useWebcam }
// If the useWebcam option is disabled and the webcam texture is playing, stop the webcam.
else if (webcamTexture != null && webcamTexture.isPlaying)
{
.Stop();
webcamTexture}
}
UpdateCurrentTexture
This method updates the current texture and target frame rate based on the useWebcam
option.
// Updates the current texture and target frame rate based on the useWebcam option.
protected void UpdateCurrentTexture()
{
// Set the current texture to the webcam texture if useWebcam is enabled, otherwise use the test texture.
= useWebcam ? webcamTexture : testTexture;
currentTexture // Set the target frame rate based on whether the webcam is being used or not.
.targetFrameRate = useWebcam ? webcamFrameRate * WebcamFrameRateMultiplier : maxFrameRate;
Application}
UpdateScreenTextureAsync
A coroutine that waits until the webcamTexture is ready (if useWebcam
is enabled) and updates the screen texture with the current texture (either the test texture or the webcam feed).
// Coroutine to update the screen texture asynchronously.
protected IEnumerator UpdateScreenTextureAsync()
{
// Wait until the webcamTexture is ready if useWebcam is enabled.
while (useWebcam && webcamTexture.isPlaying && webcamTexture.width <= UninitializedWebcamWidth)
{
yield return null;
}
// Update the screen texture with the current texture (image or webcam feed).
.UpdateScreenTexture(screenObject, currentTexture, cameraObject, useWebcam);
MediaDisplayManager}
HandleMainTextureChanged
This method handles the TextureChangeEvent
, updating the current texture and stopping the webcam if the new main texture differs from the webcam texture.
// Handle the texture change event.
protected virtual void HandleMainTextureChanged(Material material)
{
// Update the current texture.
= material.mainTexture;
currentTexture // If the new main texture is different from the webcam texture and the webcam is playing, stop the webcam.
if (webcamTexture && material.mainTexture != webcamTexture && webcamTexture.isPlaying)
{
= false;
useWebcam .Stop();
webcamTexture}
}
OnDisable
Lastly, the OnDisable()
method unsubscribes from the TextureChangeEvent
when the script is disabled.
// Unsubscribe from the texture change event when the script is disabled.
protected virtual void OnDisable()
{
.OnMainTextureChanged -= HandleMainTextureChanged;
TextureChangeEvent}
MediaDisplayManager.cs
The MediaDisplayManager
script contains a static utility class that provides various methods to set up and manage media displays in Unity. This script sets screen textures, updates screen transforms, and initializes camera and webcam configurations. The complete code is available on GitHub at the link below.
SetScreenTexture
This method sets the texture for the screen object and raises a texture change event.
/// <summary>
/// Sets the texture for the screen object and raises a texture change event.
/// </summary>
/// <param name="screenObject">The GameObject to be used as the screen.</param>
/// <param name="displayTexture">The Texture to display on the screen.</param>
public static void SetScreenTexture(GameObject screenObject, Texture displayTexture)
{
// Attempt to get the MeshRenderer component from the screen object
if (screenObject.TryGetComponent<MeshRenderer>(out MeshRenderer meshRenderer))
{
// Create a new material with the Unlit/Texture shader and set its main texture
= new Material(Shader.Find("Unlit/Texture"))
Material screenMaterial {
= displayTexture
mainTexture };
// Assign the material to the mesh renderer
.material = screenMaterial;
meshRenderer
// Raise a texture change event to inform other scripts
.RaiseMainTextureChangedEvent(meshRenderer.material);
TextureChangeEvent}
}
UpdateScreenTransform
This method updates the screen object’s rotation, scale, and position based on the display texture and mirror settings.
/// <summary>
/// Updates the screen object's rotation, scale, and position based on the display texture and mirror settings.
/// </summary>
/// <param name="screenObject">The GameObject to be used as the screen.</param>
/// <param name="displayTexture">The Texture displayed on the screen.</param>
/// <param name="mirrorScreen">Optional parameter to mirror the screen horizontally. Default is false.</param>
public static void UpdateScreenTransform(GameObject screenObject, Texture displayTexture, bool mirrorScreen = false)
{
// Get the width and height of the display texture
float width = displayTexture.width;
float height = displayTexture.height;
// Set the rotation, scale, and position of the screen object
.transform.rotation = Quaternion.Euler(0, mirrorScreen ? 180f : 0f, 0);
screenObject.transform.localScale = new Vector3(width, height, mirrorScreen ? -1f : 1f);
screenObject.transform.position = new Vector3(width / 2, height / 2, 1);
screenObject}
InitializeCamera
This method initializes the camera used for displaying the screen.
/// <summary>
/// Initializes the camera used for displaying the screen.
/// </summary>
/// <param name="cameraObject">The GameObject with a Camera component to be used for displaying the screen.</param>
/// <param name="screenDimensions">The dimensions of the screen.</param>
public static void InitializeCamera(GameObject cameraObject, Vector2Int screenDimensions)
{
// Attempt to get the Camera component from the camera object
if (cameraObject.TryGetComponent<Camera>(out Camera camera))
{
// Set the position of the camera object
= new Vector3(screenDimensions.x / 2, screenDimensions.y / 2, -10f);
Vector3 position .transform.position = position;
cameraObject
// Configure the camera for orthographic mode
.orthographic = true;
camera
// Calculate the aspect ratios of the screen object and the camera's viewport
float screenAspectRatio = (float)screenDimensions.x / screenDimensions.y;
float cameraAspectRatio = (float)camera.pixelWidth / camera.pixelHeight;
// Set the camera's orthographic size based on the aspect ratios
if (screenAspectRatio > cameraAspectRatio)
{
// Wider screen object
.orthographicSize = screenDimensions.x / 2 / camera.aspect;
camera}
else
{
// Taller screen object
.orthographicSize = screenDimensions.y / 2;
camera}
}
}
InitializeWebcam
This method initializes and plays a webcam stream with the specified settings.
/// <summary>
/// Initializes and plays a webcam stream with the specified settings.
/// </summary>
/// <param name="webcamTexture">A reference to the WebCamTexture instance to be initialized and played.</param>
/// <param name="deviceName">The name of the webcam device to be used for streaming.</param>
/// <param name="webcamDimensions">The desired resolution of the webcam stream.</param>
/// <param name="webcamFrameRate">The desired frame rate of the webcam stream. Default is 60.</param>
/// <returns>Returns true if the webcam stream has started playing, false otherwise.</returns>
public static bool InitializeWebcam(ref WebCamTexture webcamTexture, string deviceName, Vector2Int webcamDimensions, int webcamFrameRate = 60)
{
// If the webcam texture is not null and it is playing, stop it
if (webcamTexture != null && webcamTexture.isPlaying)
{
.Stop();
webcamTexture}
// Create a new WebCamTexture instance with the specified settings
= new WebCamTexture(deviceName, webcamDimensions.x, webcamDimensions.y, webcamFrameRate);
webcamTexture
// Start playing the webcam stream
.Play();
webcamTexture
// Return true if the webcam stream has started playing, false otherwise
return webcamTexture.isPlaying;
}
UpdateScreenTexture
This method updates the texture, transform, and camera object.
/// <summary>
/// Updates the texture, transform, and camera of the screen object.
/// </summary>
/// <param name="screenObject">The GameObject to be used as the screen.</param>
/// <param name="displayTexture">The Texture displayed on the screen.</param>
/// <param name="cameraObject">The GameObject with a Camera component to be used for displaying the screen.</param>
/// <param name="mirrorScreen">Optional parameter to mirror the screen horizontally. Default is false.</param>
public static void UpdateScreenTexture(GameObject screenObject, Texture displayTexture, GameObject cameraObject, bool mirrorScreen = false)
{
// Update the texture of the screen object
SetScreenTexture(screenObject, displayTexture);
// Update the transform of the screen object based on the new texture dimensions and mirror settings
UpdateScreenTransform(screenObject, displayTexture, mirrorScreen);
// Get the screen dimensions from the updated screen object
= new Vector2Int(displayTexture.width, displayTexture.height);
Vector2Int screenDimensions
// Initialize the camera for displaying the screen
InitializeCamera(cameraObject, screenDimensions);
}
TextureChangeEvent.cs
The TextureChangeEvent
class defines and manages a custom event that triggers when the main texture of a Material object changes. This event allows other scripts to be notified and respond accordingly when the displayed texture updates. The complete code is available on GitHub at the link below.
OnMainTextureChangedDelegate
: This delegate defines the signature for the event handlers that get called when the main texture changes.OnMainTextureChanged
: A static event of theOnMainTextureChangedDelegate
type. It allows other scripts to subscribe and respond to texture change events.RaiseMainTextureChangedEvent
: This static method gets called when the main texture of a Material object changes. The method invokes theOnMainTextureChanged
event, calling any subscribed event handlers and passing the updated material as an argument.
public class TextureChangeEvent : MonoBehaviour
{
// Define a delegate with the desired signature
public delegate void OnMainTextureChangedDelegate(Material material);
// Create a static event with the delegate type
public static event OnMainTextureChangedDelegate OnMainTextureChanged;
// Method to call when the mainTexture has been changed
public static void RaiseMainTextureChangedEvent(Material material)
{
?.Invoke(material);
OnMainTextureChanged}
}
AddCustomDefineSymbol.cs
This Editor script contains a class that adds a custom define symbol to the project. We can use this custom symbol to prevent code that relies on this package from executing unless the Unity Media Display package is present. The complete code is available on GitHub at the link below.
public class DependencyDefineSymbolAdder
{
// Constant string representing the custom define symbol.
private const string CustomDefineSymbol = "CJM_UNITY_MEDIA_DISPLAY";
// This method is called on Unity editor load to ensure the custom define symbol is added.
[InitializeOnLoadMethod]
public static void AddCustomDefineSymbol()
{
// Get the currently selected build target group.
var buildTargetGroup = EditorUserBuildSettings.selectedBuildTargetGroup;
// Retrieve the current scripting define symbols for the selected build target group.
var defines = PlayerSettings.GetScriptingDefineSymbolsForGroup(buildTargetGroup);
// Check if the custom define symbol is already in the list of define symbols.
if (!defines.Contains(CustomDefineSymbol))
{
// Append the custom define symbol to the list of define symbols.
+= $";{CustomDefineSymbol}";
defines
// Set the updated list of define symbols for the selected build target group.
.SetScriptingDefineSymbolsForGroup(buildTargetGroup, defines);
PlayerSettings
// Log the successful addition of the custom define symbol.
.Log($"Added custom define symbol '{CustomDefineSymbol}' to the project.");
Debug}
}
}
ShaderIncludePostprocessor.cs
The ShaderIncludePostprocessor.cs
script includes the shader used by the screen material in the build by adding it to the list of Always Included Shaders
in the Graphics Settings. This script implements the IPostprocessBuildWithReport
interface and runs only in the Unity Editor. The complete code is available on GitHub at the link below.
The ShaderIncludePostprocessor
class includes a property to specify the order in which the build postprocessor gets called.
/// <summary>
/// The order in which this postprocessor is called.
/// </summary>
public int callbackOrder { get { return 0; } }
OnPostprocessBuild
This method gets called after the build completes. It calls the AddShaderToAlwaysIncludedShaders
method with the "Unlit/Texture"
shader name, ensuring the built project includes it.
/// <summary>
/// Called after the build has been completed.
/// </summary>
/// <param name="report">The BuildReport object containing information about the build.</param>
public void OnPostprocessBuild(BuildReport report)
{
AddShaderToAlwaysIncludedShaders("Unlit/Texture");
}
AddShaderToAlwaysIncludedShaders
This method adds the specified shader to the Always Included Shaders
list in the Graphics Settings.
/// <summary>
/// Adds the specified shader to the list of always included shaders in GraphicsSettings.
/// </summary>
/// <param name="shaderName">The name of the shader to add to the always included shaders list.</param>
private static void AddShaderToAlwaysIncludedShaders(string shaderName)
{
= Shader.Find(shaderName);
Shader shader if (shader == null)
{
.LogWarning($"Shader '{shaderName}' not found.");
Debugreturn;
}
// Load the GraphicsSettings asset
var graphicsSettings = AssetDatabase.LoadAssetAtPath<GraphicsSettings>("ProjectSettings/GraphicsSettings.asset");
if (graphicsSettings == null)
{
.LogWarning("GraphicsSettings.asset not found.");
Debugreturn;
}
// Create a serialized object from the GraphicsSettings asset
= new SerializedObject(graphicsSettings);
SerializedObject serializedGraphicsSettings // Find the "m_AlwaysIncludedShaders" property in the serialized object
= serializedGraphicsSettings.FindProperty("m_AlwaysIncludedShaders");
SerializedProperty alwaysIncludedShadersProp
// Iterate through the shaders in the Always Included Shaders list
for (int i = 0; i < alwaysIncludedShadersProp.arraySize; i++)
{
= alwaysIncludedShadersProp.GetArrayElementAtIndex(i);
SerializedProperty shaderProp if (shaderProp.objectReferenceValue == shader)
{
.Log($"Shader '{shaderName}' is already in the Always Included Shaders list.");
Debugreturn;
}
}
// Add the specified shader to the Always Included Shaders list
.InsertArrayElementAtIndex(alwaysIncludedShadersProp.arraySize);
alwaysIncludedShadersProp= alwaysIncludedShadersProp.GetArrayElementAtIndex(alwaysIncludedShadersProp.arraySize - 1);
SerializedProperty newShaderProp .objectReferenceValue = shader;
newShaderProp.ApplyModifiedProperties();
serializedGraphicsSettings
.Log($"Shader '{shaderName}' has been added to the Always Included Shaders list.");
Debug}
Conclusion
This post provided an in-depth walkthrough of the code for the Unity Media Display package. The package helps simplify creating and managing demo screens for displaying test images, video, and webcam streams in Unity projects.
You can continue to explore the package by going to its GitHub repository linked below, where you will also find instructions for installing it using the Unity Package Manager.
- GitHub Repository: unity-media-display
You can find the code for the demo project shown in the video at the beginning of this post linked below, along with links for other demo projects that use the Unity Media Display package.
- Unity Media Display Demo: A simple demo project demonstrating how to use
the unity-media-display
andunity-cv-image-gallery
packages in Unity. - Barracuda Image Classification Demo: A simple Unity project demonstrating how to perform image classification with the
barracuda-inference-image-classification
package. - Barracuda Inference PoseNet Demo: A simple Unity project demonstrating how to perform 2D human pose estimation with the
barracuda-inference-posenet
package. - Barracuda Inference YOLOX Demo: A simple Unity project demonstrating how to perform object detection with the
barracuda-inference-yolox
package.
- I’m Christian Mills, a deep learning consultant specializing in computer vision and practical AI implementations.
- I help clients leverage cutting-edge AI technologies to solve real-world problems.
- Learn more about me or reach out via email at [email protected] to discuss your project.