Migration of Hololens2 projects to newest MRTK V2.7.2 and Open XR SDK

Hey there!  With the advent of MRTK 2.7, it was then the obvious choice to use OpenXR as the latter is supported with 2.7 release. Furthermore, OpenXR provides a royalty-free API standard from Khronos to develop code which is applicable on a multitude of Mixed Reality devices- including Hololens 2, Oculus Quest 2 and other Windows Mixed Reality Immersive VR or AR devices. OpenXR has made a bunch of features like eye tracking, hand tracking, spatial mapping, spatial anchors etc and extensions are supposed to follow later this year. Also, starting Unity 2021.2.x OpenXR would be the only supported Unity backend for targeting Hololens 2 and Windows Mixed Reality headsets. Therefore, I took up the task of migrating existing projects with older MRTK and XR versions to the new OpenXR and MRTK 2.7. This article will outline a checklist to follow when you upgrade your older Unity projects to the new OpenXR and MRTK 2.7.

I would view it in 3 broad categories:

1.  Upgrading your older Unity project itself to a newer version supported by OpenXR i.e Unity 2020 LTS or later (although, I encountered an error with Unity 2020 LTS where the il2cpp builds were failing due to crash in Tundra and the solution was to upgrade to Unity 2021.1.10f1- see more here: https://issuetracker.unity3d.com/issues/2020-dot-3-il2cpp-build-fails-due-to-a-crash-in-tundra-introduced-by-0-change-player-build). In this project, I use Unity 2021.1.10f1 (short intro to using the new Mixed Reality Feature tool)
2.  Upgrading from Legacy XR to Mixed Reality OpenXR: In this project, I upgraded it to Mixed Reality Open XR plugin 1.0.0-preview.2 and getting the Unity Open XR plugin (if you are running your project on any conformant PC based Open XR runtimes- note that this is different from the MixedReality OpenXR plugin)
3.  Upgrading older MRTK version to newer MRTK version. This project shows you a migration from MRTK V2.4 to V2.7.2

I will also list what you need to do to address several quite commonly used features in your project, when you upgrade to the above:

•  Configuring newer MRTK profiles for OpenXR
•  Using eye tracking successfully
•  How to use the Migration Window to migrate all obsolete scripts and prefabs in older version to newest MRTK V2.7 version
•  Implementing QRcode tracking successfully with new OpenXR

So, then let’s get started!

I used an older project as an example which needed to be migrated to the newest version.

For the 1^st part: Upgrading your older Unity project itself to a newer version supported by OpenXR i.e Unity 2020 LTS or later

Here I used a project which was in 2019.4.24f1. So before you open your older Unity project with a higher Unity version you want, go to your project folder and delete the following folders and their .meta files.

MRTK/Core

MRTK/Examples

MRTK/Extensions

MRTK/Providers

MRTK/SDK

MRTK/Services

MRTK/StandardAssets (if you had made any changes to the MRTK Shaders, create a local backup of these before deleting the StandardAssets folder)

MRTK/Tools

Do not delete the MixedRealityToolkit.Generated folder or its .meta file

Delete the library and logs folder.

Now, open the project with the new version of Unity 2021.1.10f1 (or whichever you want to, but ensure its Unity 2020 LTS or later). If there is a popup to rebuild APIs then press ok, if there is a popup to enter Safe Mode, press Ignore.

Now go to Releases and import the MRTK V2.7.2 Foundation followed by MRTK V2.7.2 Tools. Please note, you will need the Tools package if you are upgrading from older MRTk to newer. Recommended is MRTK V2.7.2 or later

(A side note on Mixed Reality Feature tool – I believe it is still buggy and not fully stable to install all packages I want at once. For ex: if I want Examples, Extensions, Foundation and Tools and I try to check all of these in the tool and install them at once, it does not work. Therefore, check again in the project if these are all fully installed. Else go to the Release page and import packages one by one).

Also, check in your Assets/MixedRealityToolkit.Generated/link.xml if the following line is in there:

<assembly fullname = “Microsoft.MixedReality.Toolkit.Providers.OpenXR” preserve=”all”/>

For the 2^nd part: Upgrading from Legacy XR to Mixed Reality OpenXR: In this project, I upgraded it to Mixed Reality Open XR plugin 1.0.0-preview.2

In the Mixed Reality Feature Tool, Select under Platform Support, Mixed Reality OpenXR plugin and the 1.0.0-preview-2 release. Recommended is 1.0.0 or later

If you don’t see preview releases, go to Settings on the Mixed Reality feature tool (bottom left gear icon when you start the tool) and under Feature-Show preview releases.

Here is the link to an empty project with all the right OpenXR settings and latest MRTKv2.7 for you to get started with.

Now reopen your project and let the APIs and plugins reload.

Next for the Unity OpenXR plugin

which you need to install via the Package Manager. I took version 1.2.3. Recommended is Version 1.2 or later. If the project configurator automatically loads a window asking you to download the Unity OpenXR plugin, then get that.

And finally check in Package Manager if you have Unity OpenXR and Mixed Reality OpenXR plugins.

Now close the project and restart it. This will ensure that the library is correctly built with all the necessary APIs.

Now go to Project Settings-XR Plugin Management and change to these settings for the use of OpenXR on Hololens 2:

If there is a popup here to enable XR plugin management with OpenXR settings, then also use that:

It will anyway take you to the XR Plugin management settings:

if you have PC XR runtimes, then also choose these settings under Desktop platforms

One important note:

Click on this yellow icon next to Open XR in the XR plugin management settings, this is a Validation window which comes up. If you don’t do this, you may be missing some important things which may not work, for ex: EyeTracking etc.

Now for some more settings under XR Plugin management-OpenXR, make sure you select the two profiles if they have not been already enabled in the previous step. We need both Eye Gaze Interaction Profile and Microsoft Hand Interaction Profile for the MRTK gaze and gestures to work correctly. Also under Microsoft Hololens, choose Hand Tracking and Motion Controller Model and Mixed Reality Features (selected by default)

Also restart Unity after this. And make sure when you check under XR plugin management again, there is no more yellow icon next to Open XR

For the 3^rd part: Upgrading older MRTK version to newer MRTK version. This project shows you a migration from MRTK V2.4 to V2.7

First, in every scene of the project, delete the MixedRealityToolkit and MixedRealityPlayspace. Next, add this manually to every scene via the menu item MixedReality-Toolkit-Add to Scene and Configure

Now for the part of using OpenXR configuration profiles for the MRTK:

On your scene, click on MixedRealityToolkit, which was added in the previous step and select ObsoleteOpenXRConfigurationProfile. Clone this and call it something sensible. I called it OpenXRConfigurationProfile.

Within this I cloned the following profiles, Experience, Camera, Input and Spatial Awareness (it depends on what you need for your project, though). I will be showing Camera, Input and Spatial Awareness parameters for Hololens2 in detail, as these are most important:

Camera profile changes:

First, add a new data provider and add the OpenXRCameraSettings in Type.

I also changed the Clear Flags to Color instead of Skybox

Under Input – Input Data Providers, make sure you have all of these: I did not need to change any settings in these default profiles. These would be added automatically when you previously used the MRTK configurator to set up your XR settings:

For pointers, we need a DefaultGazeCursor if we need Eyetracking to be set up correctly, so clone the Pointer Profile and add a DefaultGazeCursor Prefab in the Gaze Cursor Prefab field. And make sure you enable Is Eye Tracking Enabled.

Next, clone the Spatial Awareness profile and remove the XR SDK Spatial Mesh Observer,

Add Spatial Observer and in Type choose OpenXRSpatialAwarenessMeshObserver

Within that, I cloned the Spatial mesh Observer profile to change the Display Option of the Wireframe to None – as I did not want this mesh in my app. But this is your preference.

So for Eye tracking to work here is the checklist:

1. Make sure you have the Eye Gaze Interaction Profile selected in XR Plugin Management – OpenXR settings, under UWP

     2.Make sure to have the OpenXR Eye Gaze Provider added under Input Data providers in MRTK

     3. Make sure to have the DefaultGazeCursor set up in MRTK-Input-Pointers with Is Eye Tracking Enabled checked

     4. Under MRTK-Input Data Providers-Input Simulation Service-clone the profile and set Eye Gaze Simulation to Camera Forward Axis. This ensures Eye Tracking also works in Editor

Next part for using the Migration Window to migrate all obsolete scripts and prefabs to newest MRTK V2.7 version :

Reference: https://docs.microsoft.com/en-us/windows/mixed-reality/mrtk-unity/features/tools/migration-window?view=mrtkunity-2021-05

You’ll have some migration of obsolete MRTK components and this you can do via the Migration Window that the Tools package in MRTK provides. Open this window using the menu MixedReality-Toolkit-Utilities-Migration Window. And for each of the options in Migration Handler selection, run the Migration tool for the Full project. You can also select individual items if you like.

Then click on Migrate for those obsolete scripts to migrate to new ones

And now for the last part of updating your QR code tracking to the latest Open XR settings:

Reference: https://github.com/yl-msft/QRTracking/tree/main/SampleQRCodes/Assets/Scripts

Make sure you follow the GitHub link above to get all the scripts and prefabs etc necessary for QRCode Tracking with Hololens 2. The main changes although were in SpatialGraphNodeTracker instead of the SpatialGraphCoordinateSystem.cs

The link to the empty git project with OpenXRSettings and MRTK V2.7 is here: https://github.com/NSudharsan/HoloLensExamples/tree/master/OpenXRSettingsProject I have just included a nearmenu prefab just to check if the scene was working fine.

Here are some sample projects from Microsoft on Open XR Mixed Reality : https://github.com/microsoft/OpenXR-Unity-MixedReality-Samples