Present Output as Webcam

MixCast now allows you to present your MixCast camera’s output to webcam-based software such Skype, Zoom, Google Hangouts, and Discord! While in VR, you can now communicate with others on video calls while providing them with the full digital context of your environment. We’re very excited about these capabilities and plan to continue work on this area as we see great value in bridging the powers of both 2D and VR/AR tools to the benefit of both. Note that this is a significant improvement over sharing an MR window on your screen with your video call, as your camera feed is generally transmitted at a higher quality than your desktop with these programs.

Viewfinder Expansion

In addition to the changes allowing Viewfinders to be edited from the Preferences window mentioned above, Viewfinders have the following new options:

• Select Desktop as Image Source and let MixCast mirror your desktop in VR. This is a powerful addition to MixCast’s roster as it now allows you to have a clear view of all your PC’s operations while using any MixCast-ready VR experience. When combined with the Webcam functionality above, you can now bring your full VR toolset to video conferences with effective two-way communication!
• Configure Viewfinders to follow any tracked object including the HMD.
• Configure Viewfinder display scale (a welcome addition to all!)

[Film License] Enhanced BlackMagic DeckLink Support

Improvements continue on MixCast’s virtual-production capabilities. In this release, specific support for BlackMagic DeckLink capture/broadcast cards is being added, with the following initial functionality:

• Decklink Devices being used as Video Inputs will have their input parameters automatically detected rather than needing to be specified exactly by the user.
• MixCast Camera video can be piped out of a Decklink Device via HDMI or SDI cable for further processing. High end setups can use this to great advantage in capturing footage in various stages of the rendering/compositing pipeline.

Quick Align Improvements

• Quick Align has added support for the following controllers:
  • HTC Cosmos
  • Valve Knuckles/Index
  • Rift S/Quest Link
• Presentation of instructions and camera feed is clearer and more flexible.
• The Quick Align flow can be cancelled out of during the process.

This is the first of several passes to the Quick Align process to make it more powerful and accessible, so stay tuned for more improvements to come!

Expanded Backdrop Options

MixCast now allows for mixed reality rendering in multiple styles: Physical Backdrop and Virtual Backdrop. Virtual Backdrop represents the existing behaviour, where the background of a video input, as determined by WildKey, is hidden, and the active experience renders the background that appears behind the user. Physical Backdrop mode is our new offering, which allows the experience to render some objects behind the user, but draws the physical background of the video input as the ‘Backdrop’ behind those virtual objects. This is generally the desired approach for AR experiences though it can have its place in VR experiences too!

For experiences that intend to offer both styles of Backdrop rendering, objects can be flagged as representing the background of the virtual scene (eg: meshes representing the sky, ground, etc) so that they don’t conflict visually with the physical environment that the user is shown in.

Preferences Window Expansion

MixCast now exposes more of its existing configuration options through the Preferences window rather than needing to open the MixCast Configuration app, allowing for more options to be configured/adjusted on the fly! This process of migrating options out of the configuration app is an ongoing effort (notable omissions in this release are WildKey parameters and Quick Alignment) but the changes are already providing benefits with regards to ease of use. The new sections that can be found in the Preferences window are “Physical Devices”, “Virtual Cameras”, and “Viewfinders”.

Improvements & Fixes

• The Desktop Display window can now be ‘detached’ from the experience window while running. This is helpful if access to the underlying experience UI is desired while still displaying the MixCast output. You can trigger this functionality by selecting the MixCast Desktop Display or Experience windows and either:
  • Holding down Shift and clicking the button that appears in the top right of the MixCast Desktop Window, or
  • Holding down Ctrl and pressing the Tab key
• Azure Kinect devices have their Field of View and Lens Distortion data read in automatically rather than needing to be input manually. Kinect Lens Distortion correction is applied automatically.
• Exported configuration files can replace a Video Input configuration’s Hardware ID field with an asterisk (*) to flag that this configuration should be assigned to a matching device that is connected on the machine that loads the configuration file.
• Performance improvements in several areas of MixCast’s general operations.
• Resolving unintentional interactions between the MixCast Client and the Unreal Editor.
• Fixed issue where Windows Startup entry wasn’t supplied the correct path.
• Enabled full compression on MixCast Client Installer for better file sizes.

Depth Filtering Improvements

In the past, RealSense cameras could have their depth maps processed by the RealSense SDK to refine the quality of data received. This was exposed through MixCast as optional Temporal and Spatial Filters. Since MixCast now supports the Kinect Azure, we took it upon ourselves to introduce our own filters that could be applied to any supported depth camera. These are exposed as the following two features in the Input Device section for this release:

• Temporal Filter: Keeps track of multiple frames of depth data so that areas where depth can’t be determined in this frame can be filled with recent determined values.
• Fill Gaps in Depth: Areas where depth can’t be determined are filled with the nearest found values.

The Visualize Depth functionality in Studio also now displays the depth map using a rainbow gradient instead of in grayscale for readability.

Improvements & Fixes

• Additional Azure Kinect sensor parameters exposed for user control: Brightness, Contrast, Saturation, Sharpness.
• Automatic Branding Image files can now be hot-swapped without needing to restart MixCast.
• Intel RealSense cameras disable their IR emitters during Quick Setup to avoid SteamVR 2.0 tracking interference when in close proximity to the device.
• Fixing compatibility issue with WiGig adapter OpenVR driver.
• Added check to ensure MixCast windows don’t launch in an invalid position.

Archived Download: MixCast Client for Windows (2.3.3)

[Arcade Edition] Media Delivery Improvements

• A new Local Network Media Transfer feature allows you to transfer your captured media to a second PC on your Local Network. Useful only under certain situations so ping us if you have questions!
• Media that hasn’t completed uploading to the Cloud or transferring on the LAN before the MixCast Client exits will be queued for uploading/transferring again on next launch.

Recordings Showcase Window (Experimental)

Users can open a Recordings Showcase window to review the most recent media they’ve captured – both images and videos.

Improvements & Fixes

• Implemented a hardware acceleration optimization in the video recording/streaming pipeline that has resulted in a meaningful bump in performance.
• Fixed Preferences Window not reloading values when using Reset Data or Import Settings.
• Fixed issue where some SteamVR paths weren’t been read correctly.
• Continuing to improve reliability of connection establishing between MixCast processes.
• Fixed issue where the first frame of a video recording was black.
• (Arcade) Fixed displayed QR code not updating immediately when Server URL is modified by the user.

Archived Download: MixCast Client for Windows (2.3.2)

Improvements

• Multiple Azure Kinect cameras can now be connected and used in MixCast at the same time.
• [Arcades] Changing Cloud Server URL in Preferences applies immediately instead of after restart.

Fixes

• Fixed Session Code persisting through Reset Data
• Fixed Session Code overlay displaying before MixCast was configured
• Fixed MixCast Configuration app launching to a white screen from a race condition
• Fixed Hotkey to Toggle the Desktop UI Controls (Take Screenshot, etc) not applying during an experience

Known Issues

• Due to API limitations, Azure Kinect hotplugging is limited: Kinects can be plugged in while MixCast is running but shouldn’t be unplugged.

Archived Download: MixCast Client for Windows (2.3.1)

2.3.0

Azure Kinect Support (Experimental)

The Azure Kinect DK is now a supported input device in MixCast! This is another strong option for camera input and produces great results in a number of scenarios, especially in regard to depth-based (no-greenscreen) background removal. Here are the specs:

• Image resolution up to 4K at 30fps
• Narrow and Wide Field of View depth modes supported
• CPU performance comparable to Intel RealSense
• Single Kinect currently, with multi-Kinect support on the near-term roadmap

Also on the roadmap is updates to improve both quality of background removal with this camera and its performance. In the meantime if you have an Azure Kinect and want to give it a try with MixCast, please share your experience on our Discord!

Compositing Engine Improvements

A number of under-the-hood tweaks to how MixCast combines digital and physical layers for a variety of benefits. Updates to the MixCast Client will be capable of providing more new functionality even in older SDK titles without requiring SDK updates going forward. Frame Timing/Syncing can also receive improvements under this new compositing scheme which it couldn’t before.

Improvements

• Optimizing save/load process in configuration to avoid texture copies.
• Enforcing HTTPS URL format with Cloud configurations
• Improved automatic error reporting during operation
• Correcting for non-standard image mapping with a couple specific cameras

Fixes

• Fixed some preferences not applying their intended behaviour when changed.
• Fixed Decklink capture cards not initializing correctly.
• Fixed webcam device settings (exposure, white balance, etc) resetting on each load.
• Fixed localization failing to display non-Latin characters.

Known Issues

• Streaming Settings require restarting the Configuration app to take effect
• Subject Lighting isn’t available through the MixCast SDK at this time while compositing undergoing upgrades. This will return later in the 2.3.x cycle.

Archived Download: MixCast Client for Windows (2.3.0)

2.2.4

Fixes

• Fixed an issue with establishing the SDK to Client connection.
• Fixed an issue with tracking the active experience’s window (affected hotkeys and display window positioning).
• Fixed an issue with determining whether a SteamVR experience window is full-screen.

Archived Download: MixCast Client for Windows (2.2.4)

2.2.3

Arcade Edition Improvements

• Cloud/Kiosk parameters are now accessible through the Preferences window.

Unreal SDK Updates

Under the hood updates to support upcoming improvements to the MixCast SDK for Unreal.

Fixes

• Expanded automation of MixCast OpenVR Driver installation to handle some less-standard machine configurations.
• Fixed image flickering when using a tracked camera in certain configurations.
• Fixed some Experiences triggering the wrong mode of MixCast behaviour.
• Fixing case where Configuration wouldn’t save correctly on closing.

Archived Downloads: MixCast Client for Windows (2.2.3)

2.2.2

Cleaner Compositing w/ Non-SDK Titles

Eliminated the black seam visible between the foreground and the background at the user’s depth when running MixCast with a non-SDK SteamVR title.

Arcade Edition Improvements

• MixCast Cloud has its own status icon displaying session state.
• SpringboardVR and SynthesisVR clients will manage session state automatically while running in tandem with MixCast.

Fixes/Optimizations

• Video Recording is less taxing on the CPU (about 20-40% from testing) and handles dropped frames more gracefully.
• Resolved Video Recording continuing to use CPU after completion.
• Fixed regression bug present when video inputs with fractional frame rates are accessed.
• Fixed OpenVR Driver not accepting communications from the MixCast Service if the Service is closed and reopened while SteamVR is running.
• Fixed the presence of certain background removal artifacts when using Fixed Depth with a RealSense camera.

Known Issues

• Oculus SDK usage currently requires SteamVR process to be running in parallel with MixCast and the MixCast-enabled experience. This will be resolved shortly.

Archived Downloads: MixCast Client for Windows (2.2.2)

2.2.1

Fixes

• New Video Input library improving compatibility/stability and a small boost to performance.
• Cycling between cameras doesn’t restart SteamVR titles unexpectedly.

Known Issues

• Previous MixCast SDK titles can’t be imparted with the new video input library capabilities meaning newly-functional devices won’t function in them.

Archived Downloads: MixCast Client for Windows (2.2.1)

2.2.0

Multi-Camera Support in Non-SDK Titles

• When running recent SteamVR titles with MixCast, you can now switch between multiple cameras’ perspectives on the fly.

Camera Inputs

• The Intel RealSense D435i Depth Camera is now correctly recognized by MixCast
• Input Configuration Controls rebuilt for more stability
• External Compositing no longer requires an input device to be configured

Additional SteamVR Support Improvements

• Titles are restarted more aggressively to ensure load of configuration values
• MixCast now detects SteamVR being open more reliably
• SteamVR Safe Mode is no longer triggered by the MixCast OpenVR Driver

Archived Downloads: MixCast Client for Windows (2.2.0)

2.1.0

Major Improvements

• New MixCast Virtual Driver takes all the pain out of managing a third controller or virtual driver. The MixCast Virtual Driver works in the background making mixed reality output from more titles even easier!
• Expanded title support for experiences with non-standard alpha channels like Gorn, VRChat, Belko VR, and many more!
• Greatly improved performance and compatibility with Direct Window Capture mode, allowing mixed reality output from Tilt Brush, Blocks, and many other titles!
• New Picture in Picture mode automatically puts your video feed in the corner of the screen for VR experiences that don’t support MixCast mixed reality output, ensuring that no matter what you’re playing, you’re captured!
• Added support for Blackmagic DeckLink capture cards.
• Oculus 3rd Controller Support (Experimental) – MixCast can now drive the position of the mixed reality camera in SteamVR non-SDK titles from the position of your 3rd Oculus controller (despite not being recognized by SteamVR!), once it’s been set up as a “Tracked Object” in the Oculus software
• New UI to provide fast access to a variety of MixCast subsystems as well as information about the status of MixCast

Interface

• New System Window provides detailed information about your copy of MixCast
• Ability to send a Diagnostic Report directly to Blueprint Reality to help resolve issues
• Ability to import and export your entire MixCast configuration into a self-contained file for backup and sharing
• World rotation controls are now accessible in the MixCast Service interface
• The in-VR display can be toggled in the Service interface and applies to all MixCast experiences
• First-person display can be toggled in the Service interface and applies to all MixCast experiences

WildKey

• Manual Edge Cropping allows setting a manual crop from the edge of the screen
• Support for sampling depth or RGB separately for more precise control over subject isolation

Major Improvements

• New MixCast Virtual Driver takes all the pain out of managing a third controller or virtual driver. The MixCast Virtual Driver works in the background making mixed reality output from more titles even easier!
• Expanded title support for experiences with non-standard alpha channels like Gorn, VRChat, Belko VR, and many more!
• Greatly improved performance and compatibility with Direct Window Capture mode, allowing mixed reality output from Tilt Brush, Blocks, and many other titles!
• New Picture in Picture mode automatically puts your video feed in the corner of the screen for VR experiences that don’t support MixCast mixed reality output, ensuring that no matter what you’re playing, you’re captured!
• Added support for Blackmagic DeckLink capture cards.
• Oculus 3rd Controller Support (Experimental) – MixCast can now drive the position of the mixed reality camera in SteamVR non-SDK titles from the position of your 3rd Oculus controller (despite not being recognized by SteamVR!), once it’s been set up as a “Tracked Object” in the Oculus software
• New UI to provide fast access to a variety of MixCast subsystems as well as information about the status of MixCast

Interface

• New System Window provides detailed information about your copy of MixCast
• Ability to send a Diagnostic Report directly to Blueprint Reality to help resolve issues
• Ability to import and export your entire MixCast configuration into a self-contained file for backup and sharing
• World rotation controls are now accessible in the MixCast Service interface
• The in-VR display can be toggled in the Service interface and applies to all MixCast experiences
• First-person display can be toggled in the Service interface and applies to all MixCast experiences

WildKey

• Manual Edge Cropping allows setting a manual crop from the edge of the screen
• Support for sampling depth or RGB separately for more precise control over subject isolation

2.0.3

General

• Improved RealSense Connection Logic
• Fixed Tracked Camera Subject Jitter
• Fixed Input Devices Selecting Incompatible Config by Default

Archived Downloads: MixCast Client for Windows (2.0.3)

2.0.2

• Added full support for the Elgato Cam Link
• Improved color space conversions in MixCast textures
• Added buffer delay to input feed projection
• Added support for SteamVR SDK 2.0 preview
• Added support for Unreal 4.20
• Builds including the MixCast SDK on OSX will no longer report errors
• Updated to latest Intel RealSense SDK integration
• Improved usability of chroma key colour selection
• Fixed an occasional recording crash
• The in-VR preview window now appears in all titles when using an Oculus Rift
• The RealSense infrared shader now draws to the screen

Archived Downloads: MixCast Client for Windows (2.0.2)

2.0.1

• Added option to invert camera feed on the Y axis
• Fixed media recording in Unreal titles
• Fixed subject lighting in Studio
• Fixed crash caused by USB mics
• Improved Razer Stargazer handling
• Made camera geo and viewfinder invisible to other MixCast cameras
• Added editor window to Unity SDK to debug alpha handling
• Fixed issue with auto-FOV toggle in default virtual cameras
• Improved error handling and log messaging in SDK
• Increased list of audio input sources, including VR headset mics
• Removed unnecessary memory allocation in SDK
• Optimised Oculus alignment across various SDKs
• Fixed possible memory leak in sensor pose checker

Archived Downloads: MixCast Client for Windows(2.0.1)

2.0.0

• WildKey™ Background Removal: High-quality background removal with or without a green screen, including support for depth-based subject isolation with the new Intel RealSense line of depth cameras. MixCast from anywhere!
• Improved Camera Input Support: Rewritten camera library offers wider hardware support and more flexibility in configuration.
• Snapshot Tool: Capture snapshots of the MixCast output from the desktop in one click, or automatically take snapshots at set intervals.
• Video Recording: Record video and audio of the MixCast output to disk from the desktop in one click, at any resolution.
• Video Streaming: Stream live video and audio of the MixCast output to leading platforms such as Twitch, Facebook Live, Mixer, and YouTube Live in one click, at any resolution.
• Multiple Camera Support: Configure more than one camera input and toggle between feeds at will. Multiple cameras can produce output in parallel or sequence, and record video or stream, hardware permitting.
• Splitscreen View: Display up to four cameras on the desktop simultaneously for control and monitoring.
• Supersampled Resolution: Resolution of MixCast output can now be configured to up to 4K (3840 x 2160 px) regardless of monitor size.
• MixCast now runs in the background to detect MixCast-compatible VR experiences and automatically launch mixed-reality
• MixCast now runs as an overlay on support experiences, transparently adding mixed-reality support to your favourite VR titles
• Support for new languages: Fully localised text has been added for French, Italian, German, Spanish, Portuguese, and Arabic.
• Output Options: Customize the folder where outputs such as Snapshots and Video are stored. Configure streaming options globally or per-camera.
• Multiple Camera Management: New UI enables easy navigation between cameras, customization of camera names, and the ability to quickly add or remove cameras.
• Improved Background Removal Controls: Improved UI makes background removal quicker and easier.
• Advanced Camera Settings: One-click access to advanced camera settings such as exposure, gain and focus. Available settings may vary between devices.
• Windows Mixed Reality Headset Support: MixCast can now be configured and used with a Windows Mixed Reality headset.
• Centralized Project Settings: Access and manage global MixCast project settings from within a central asset.
• Improved Define Symbol Control: Automatic symbol management can now be disabled to allow more control over project and build pipelines.
• Anti-aliasing Support: Anti-aliasing setting for MixCast cameras can be customized.
• Shader Wizard: Automates the process of updating shaders and materials to support full MR transparency in minutes.
• Preview Subject in Scene: At runtime, the subject is optionally displayed in the Scene View.
• Memory Allocation Optimizations: Eliminated all per-frame memory allocations
• Unreal VR Project Support: Unreal VR titles on 4.17+ can now integrate MixCast!
• Support for MixCast Features: All features available in MixCast 1.5 and many from 2.0 are available to Unreal projects.

Archived Downloads: MixCast Client for Windows (2.0.0)

1.5

• Added Chromakey Desaturation feature to reduce the visibility of the keying color on the subject after background removal.
• Introducing automatic MixCast Snapshot capture allowing for periodic screenshots to be taken without user input.
• Added optional user-controlled branding feature to add a graphic to the MixCast output automatically.
• Added user option to activate MixCast as soon as a MixCast-ready application opens.
• Fixed issue where certain Camera Clear Flags in Unity 5.6+ break Buffered Mode.

1.4

• Added optional Player lighting which allows Unity lights to apply to the player
• Added optional Player-relative camera feed cropping
• Added lightweight localization system
• Added user framerate control
• Greatly improved feed synchronization in low framerate situations
• Improved OpenVR tracking device serialization to match by Serial ID
• Fixed bug involving deserializing the static subtraction textures in a linear color space project
• Fixed Oculus space mismatch bug
• Fixed camera HDR warning in Unity 5.6 and above

1.3

• Merged input feed shaders and materials using a multi_compile shader for easier custom effects.
• Added Posterize and HSV Modify (Desaturate, etc) variations on the input feed shader for custom player effects. Apply one of the shaders to the supplied Camera Feed material to activate it.
• Added tracked motion smoothing to reduce tracking and/or hand jitter.
• Improved WebcamFeed/MixCastCamera relationship for separation of concerns.
• Studio has new quick setup for FoV and Alignment.
• Fixed HDR texture allocation.

1.2

• Added support for Oculus SDK based projects. Created Oculus and SteamVR specific code branches and an Editor process to activate the appropriate one(s) using define flags MIXCAST_STEAMVR and MIXCAST_OCULUS.
• Added automatic update checking.
• Allowing “None” option for input device for a purely virtual camera that can still be tracked by a controller.
• Created additional isolation modes: “None” and “Static Subtraction”.
  • None simply disables background removal while still allowing for foreground-based Mixed Reality.
  • Static subtraction provides a rudimentary background removal system for fixed cameras which doesn’t require a greenscreen. The scene setup can influence the resulting quality greatly.
• Added ability for user to separate the In-Scene Display from the Camera location. As a result the visual representations are now in separate sub-groups of the MixCast Camera prefab.
• Expanded tracked camera capabilities to be configurable to any tracked controller.
• Created SetRenderingControllerForMixCast to be attached to the default SteamVR_RenderModel object for the controllers if you don’t want them to appear in the Mixed Reality output
• Arrow buttons can be used while camera is being tracked by a device.

1.1

• Restructured UI for clarity and expandability.
• Added buffered output mode with configurable game delay for camera latency compensation.
• Added icon on in-scene display to communicate if the camera is tracked.
• Added device resolution selection.
• Added quadrant output mode for recording.

1.0

MixCast released!