Person Tracking

MixCast can now use the person tracking capabilities of ZED depth cameras for several features:

• Estimating the depth of the subject optically rather than using a tracked device.
• Masking the input feed to only draw areas around the person as part of background removal.
• Sending the person’s pose to an experience to render a shadow approximating their body or simulate physics interactions against them.

Subject Relighting

MixCast’s ‘subject relighting’ functionality has been restored, allowing the physical subject to be brightened/tinted by the virtual lights found in a MixCast SDK-enabled experience. This functionality can be found under the Effects section of the Virtual Cameras tab for VR cameras.

Chromakey Shadow Isolation

MixCast’s chromakeying algorithm can now be configured to detect which pixels represent shadows cast from the subject onto the greenscreen surface (generally the floor) and composite them into the final output, making it appear like the user is casting a shadow onto the virtual floor (provided that the virtual ground is relatively flat).

Mirror Output Horizontally

The MixCast output window can now be configured to mirror the output horizontally if desired. The toggle can be found in the top right corner of the window while holding SHIFT.

Improvements & Fixes

• A warning is shown in the Status Window if the running experience uses a newer version of the MixCast SDK than the Client supports.
• If Present as Webcam is active and the user switches to a new camera on the desktop, that camera takes over as the camera with Present as Webcam active.
• Resolved a bug where configuration data became corrupted on computers set to use a language other than English, which would result in several visible issues including black output from experiences.
• Resolved a bug with Azure Kinect initialization that would sometimes cause the compositing pipeline to stall.

Improvements & Fixes

• The SDK updates the Client upon becoming active to avoid a race condition that was causing the Client to believe that the experience wasn’t running in a supported mode (ex: Tilt Five mode).
• The SDK, when running in the Unity Editor, combines compositing layers using a less-optimized but more stable manner to avoid sporadically encountered alignment issues. Standalone builds continue to use the more optimized approach.
• The SDK no longer generates deprecation warnings when imported into a Unity 2023 project.

Known Issues

• Running projects using Unity 2020 and the OpenXR plugin in the Editor will produce exceptions. To eliminate this Editor-only issue, please disable MixCast from running in the Editor through the MixCast Project Settings or upgrade your version of Unity to 2021+.