Person Tracking

MixCast can now use the person tracking capabilities of ZED depth cameras for several features:

• Estimating the depth of the subject optically rather than using a tracked device.
• Masking the input feed to only draw areas around the person as part of background removal.
• Sending the person’s pose to an experience to render a shadow approximating their body or simulate physics interactions against them.

Subject Relighting

MixCast’s ‘subject relighting’ functionality has been restored, allowing the physical subject to be brightened/tinted by the virtual lights found in a MixCast SDK-enabled experience. This functionality can be found under the Effects section of the Virtual Cameras tab for VR cameras.

Chromakey Shadow Isolation

MixCast’s chromakeying algorithm can now be configured to detect which pixels represent shadows cast from the subject onto the greenscreen surface (generally the floor) and composite them into the final output, making it appear like the user is casting a shadow onto the virtual floor (provided that the virtual ground is relatively flat).

Mirror Output Horizontally

The MixCast output window can now be configured to mirror the output horizontally if desired. The toggle can be found in the top right corner of the window while holding SHIFT.

Improvements & Fixes

• A warning is shown in the Status Window if the running experience uses a newer version of the MixCast SDK than the Client supports.
• If Present as Webcam is active and the user switches to a new camera on the desktop, that camera takes over as the camera with Present as Webcam active.
• Resolved a bug where configuration data became corrupted on computers set to use a language other than English, which would result in several visible issues including black output from experiences.
• Resolved a bug with Azure Kinect initialization that would sometimes cause the compositing pipeline to stall.

Improvements & Fixes

• The SDK updates the Client upon becoming active to avoid a race condition that was causing the Client to believe that the experience wasn’t running in a supported mode (ex: Tilt Five mode).
• The SDK, when running in the Unity Editor, combines compositing layers using a less-optimized but more stable manner to avoid sporadically encountered alignment issues. Standalone builds continue to use the more optimized approach.
• The SDK no longer generates deprecation warnings when imported into a Unity 2023 project.

Known Issues

• Running projects using Unity 2020 and the OpenXR plugin in the Editor will produce exceptions. To eliminate this Editor-only issue, please disable MixCast from running in the Editor through the MixCast Project Settings or upgrade your version of Unity to 2021+.

Improvements & Fixes

• Unreal Engine versions 5.0 – 5.3 are now supported.
• MixCast rendering of transparent objects (to the foreground compositing layer) has had GPU usage optimized further.
• The render settings that MixCast uses for Global Illumination and Reflections (Lumen, Raytracing, etc) can now be configured in the Unreal Project Settings.
• Resolved warnings about the SDK not declaring certain plugins as dependencies.

Known Issues

• UE 4.27 and later: Projects using OpenXR as the active XR platform only function correctly with MixCast as long as the user hasn’t ‘re-centered’ their play space in Oculus or SteamVR during their session.
• UE 4.27 – 5.0: Projects using OpenXR as the active XR platform are only supported when using the “Stage” origin type.
• UE 4.26 – 5.0: Blueprint-only projects aren’t supported due to restrictions with the TextureShare plugin. This guide can walk you through converting a Blueprint-only project to a C++ project.