You must have come across the problem when you use Holograms or objects in your HoloLens apps, you’d need to do either of three things at some point:
1. Move it around
2. Make the object bigger or smaller
3. Rotate the object
That is exactly what I’m going to cover in this article.
Microsoft makes it very easy for us to do this by providing what they call the App bar and the Bounding Box.
The above link should quickly explain to you about the concepts
So, let’s get started…
1. Create a new project in Unity.
It’s better if you get Unity 2017.2 version as you’d need to make it compatible with the next must have – Mixed Reality Toolkit- Unity
The Mixed Reality Toolkit- Unity is using the base code from the normal MR Toolkit, but it has scripts and prefabs specially designed to be used with ease within Unity.
2. Import the Mixed Reality Toolkit-Unity from https://github.com/Microsoft/MixedRealityToolkit-Unity Latest Master branch Release (as of today 6th May, 2018 is https://github.com/Microsoft/MixedRealityToolkit-Unity/releases/tag/2017.2.1.4 )
3. Do the usual tweaks:
a. Delete the main Camera. Add the MixedRealityCameraParent.prefab in the Hierarchy. This will serve as the Main Camera
b. Change the Camera Settings to Hololens preferred:
Skybox to Solid Color
Background to 0,0,0,0
c. Save the scene
Now let’s get some objects within the scene which we can manipulate.
I downloaded and imported the Furniture Pack 3D Asset pack from the Asset Store for free: https://assetstore.unity.com/packages/3d/props/furniture/pack-gesta-furniture-1-28237
If you want, you can even start with a simple 3D object like a cube in Unity. The furniture assets just make it a bit more fun 🙂 It will be like setting up your furniture in your living room in 3D
Once you import it, it will show up as Furniture_ges1 under your Assets
Now let’s get some furniture into the scene. You can use whatever prefab came with the pack (or your own). But I used the following:
Bed, TV, Printer, Cupboard, A big table, A little table. Simply drag the drop these prefabs from the furniture_ges1 asset pack into the scene. I also grouped them under MovableObjects, which is an empty GameObject- just to keep things organized
Now let us check if we are able to see the furniture in the scene.
Build and deploy. (It is best if you use a HoloLens for this experience, but the Emulator is also ok)
So if everything has been followed step by step, then you should be able to see the furniture in the space. Never mind for now that it is hanging in the air or is too big.
Once you’re sure that everything is visible, then adjust the scale to whatever you want. I reduced things to 0.5 and for the smaller things like printer and TV to 0.2
I just made quick camera adjustments to face the furniture and set up things in a presentable way so that it doesn’t look weird when you see through the HoloLens the first time (this is completely optional because you’ll pull and arrange objects or make bigger or smaller anyway)
Now for the part where we can move the objects around using the pinch and drag gesture
1. First add the Default Cursor.prefab from the HoloToolkit. Search for it in the Assets and drag and drop it into the Hierarchy
2. Search for the InputManager.prefab in the Assets and drag and drop it into the Hierachy as well
3. On the Inspector settings of the InputManager, under SimpleSingerPointerSelector component, drag and drop the Default Cursor into the Cursor field (if you haven’t seen the article Gaze and Tap on objects, then this is a good time to refer to it)
4. For every object that you want to pinch and move, you need to ensure that there is a corresponding Collider of some sort. I added Mesh Colliders to the individual objects which are movable
5. Now we add the script to each movable object which makes the objects draggable by pinch and move. The MR Toolkit-Unity has already provided us with this script. It is as easy as attaching this script to the object. On the object’s Add Component, search for HandDraggable.cs and click to add it
Quickly build and deploy and test it. The objects should be movable now. Great going! Now let’s add the BoundingBox part
This is the logic which will create a box around the object which you can click on to make bigger or smaller. You can also rotate the objects with the Boundingbox
1. Again, the MR Toolkit-Unity provides us with the script to do all this with ease. In the Assets folder, search for BoundingBoxRig.cs script and add it to each movable object. You can drag the drop the script to the object directly
2. In the BoundingBoxRig script, there is a BoundingBoxBasicPrefab field. Search for the BoundingBoxBasicPrefab in the Assets folder and drag and drop this into the field.
In case you are using a 2D object, make sure you select the Flattened Axis field to Flatten Auto. But for 3D objects, the default Do Not Flatten should do
Also if you are using the settings as is, you’ll see that the AppBar, which appears in front of each object is way too tiny. So let’s make that bigger.
Search for the AppBar.prefab in the Assets. This is already referenced in the BoundingBoxRig script on every movable object. So we need to change it only once in the prefab.
Change the scale of the AppBar to 1,1,1. When you test it, you can also increase it to suit your preference.
The handles on the BoundingBox are too small for me. Tapping on them is stressful with the sizes already existing and so I headed over to the BoundingBoxRig.cs Script and increased the scale
I changed the scaleHandleSize and the rotateHandleSize to 0.07f,0.07f,0.07f and 0.06f,0.06f,0.06f respectively. This way they are bigger and easier to tap and grab
I also increased the Scale Rate on the BoundingBoxRig script to 4. 1 is too low in my opinion, but you can try whatever suits you
Okay, now you need to just build and deploy and you can see the App bar in front of the movable objects.
On the AppBar’s Adjust button, Tap to see the Bounding Box.
Tap and Grab the square handles and pull up to make bigger and push down to make smaller
Tap and Grab the round handles to rotate the object
The objects can be pinched and moved around in BoundingBox mode or not
Have fun setting up the furniture!!
Attached is a video of how the output should be like.
(P.S: it is extremely tough to test out the controls on the Emulator, but if you have no other go you can. Try with smaller objects so that you don’t have to move the mouse much to adjust your gaze)
Questions, Comments? Let me know below!
Errors and Solutions:
Error: Object not moving
Solution: Check if Cursor, InputManager is in the scene
Reference Cursor in the SimpleSinglePointerSelector component of the InputManager
Make sure the object has a Mesh Collider
Make sure the HandDraggable.cs is attached to the object