Move, resize and rotate objects in your HoloLens apps

Move, resize and rotate objects

You must have come across the problem when you use Holograms or objects in your HoloLens apps, you’d need to do either of three things at some point:

1. Move it around 

2. Make the object bigger or smaller

3. Rotate the object

That is exactly what I’m going to cover in this article.

Microsoft makes it very easy for us to do this by providing what they call the App bar and the Bounding Box. 

The above link should quickly explain to you about the concepts

So, let’s get started…

1. Create a new project in Unity.

It’s better if you get Unity 2017.2 version as you’d need to make it compatible with the next must have – Mixed Reality Toolkit- Unity

The Mixed Reality Toolkit- Unity is using the base code from the normal MR Toolkit, but it has scripts and prefabs specially designed to be used with ease within Unity.

2. Import the Mixed Reality Toolkit-Unity from https://github.com/Microsoft/MixedRealityToolkit-Unity  Latest Master branch Release (as of today 6th May, 2018 is https://github.com/Microsoft/MixedRealityToolkit-Unity/releases/tag/2017.2.1.4 )

3. Do the usual tweaks:

a. Delete the main Camera.  Add the MixedRealityCameraParent.prefab in the Hierarchy. This will serve as the Main Camera

b. Change the Camera Settings to Hololens preferred:

Skybox to Solid Color

Background to 0,0,0,0

c. Save the scene

Now let’s get some objects within the scene which we can manipulate.

I downloaded and imported the Furniture Pack 3D Asset pack from the Asset Store for free: https://assetstore.unity.com/packages/3d/props/furniture/pack-gesta-furniture-1-28237

If you want, you can even start with a simple 3D object like a cube in Unity. The furniture assets just make it a bit more fun 🙂 It will be like setting up your furniture in your living room in 3D

Once you import it, it will show up as Furniture_ges1 under your Assets

Now let’s get some furniture into the scene. You can use whatever prefab came with the pack (or your own). But I used the following:

Bed, TV, Printer, Cupboard, A big table, A little table. Simply drag the drop these prefabs from the furniture_ges1 asset pack into the scene. I also grouped them under MovableObjects, which is an empty GameObject- just to keep things organized

Now let us check if we are able to see the furniture in the scene.

Build and deploy. (It is best if you use a HoloLens for this experience, but the Emulator is also ok)

So if everything has been followed step by step, then you should be able to see the furniture in the space. Never mind for now that it is hanging in the air or is too big.

Once you’re sure that everything is visible, then adjust the scale to whatever you want. I reduced things to 0.5 and for the smaller things like printer and TV to 0.2

I just made quick camera adjustments to face the furniture and set up things in a presentable way so that it doesn’t look weird when you see through the HoloLens the first time (this is completely optional because you’ll pull and arrange objects or make bigger or smaller anyway)

Now for the part where we can move the objects around using the pinch and drag gesture

1. First add the Default Cursor.prefab from the HoloToolkit. Search for it in the Assets and drag and drop it into the Hierarchy

2. Search for the InputManager.prefab in the Assets and drag and drop it into the Hierachy as well

3. On the Inspector settings of the InputManager, under SimpleSingerPointerSelector component, drag and drop the Default Cursor into the Cursor field (if you haven’t seen the article Gaze and Tap on objects, then this is a good time to refer to it)

4. For every object that you want to pinch and move, you need to ensure that there is a corresponding Collider of some sort. I added Mesh Colliders to the individual objects which are movable

5. Now we add the script to each movable object which makes the objects draggable by pinch and move. The MR Toolkit-Unity has already provided us with this script. It is as easy as attaching this script to the object. On the object’s Add Component, search for HandDraggable.cs and click to add it

Quickly build and deploy and test it. The objects should be movable now. Great going! Now let’s add the BoundingBox part

This is the logic which will create a box around the object which you can click on to make bigger or smaller. You can also rotate the objects with the Boundingbox

1. Again, the MR Toolkit-Unity provides us with the script to do all this with ease. In the Assets folder, search for BoundingBoxRig.cs script and add it to each movable object. You can drag the drop the script to the object directly

2. In the BoundingBoxRig script, there is a BoundingBoxBasicPrefab field. Search for the BoundingBoxBasicPrefab in the Assets folder and drag and drop this into the field.

In case you are using a 2D object, make sure you select the Flattened Axis field to Flatten Auto. But for 3D objects, the default Do Not Flatten should do

Also if you are using the settings as is, you’ll see that the AppBar, which appears in front of each object is way too tiny. So let’s make that bigger.

Search for the AppBar.prefab in the Assets. This is already referenced in the BoundingBoxRig script on every movable object. So we need to change it only once in the prefab.

Change the scale of the AppBar to 1,1,1. When you test it, you can also increase it to suit your preference.

The handles on the BoundingBox are too small for me. Tapping on them is stressful with the sizes already existing and so I headed over to the BoundingBoxRig.cs Script and increased the scale

I changed the scaleHandleSize and the rotateHandleSize to 0.07f,0.07f,0.07f and 0.06f,0.06f,0.06f respectively. This way they are bigger and easier to tap and grab

I also increased the Scale Rate on the BoundingBoxRig script to 4. 1 is too low in my opinion, but you can try whatever suits you

Okay, now you need to just build and deploy and you can see the App bar in front of the movable objects.

On the AppBar’s Adjust button, Tap to see the Bounding Box.

Tap and Grab the square handles and pull up to make bigger and push down to make smaller

Tap and Grab the round handles to rotate the object

The objects can be pinched and moved around in BoundingBox mode or not

Have fun setting up the furniture!!

Attached is a video of how the output should be like.

 

(P.S: it is extremely tough to test out the controls on the Emulator, but if you have no other go you can. Try with smaller objects so that you don’t have to move the mouse much to adjust your gaze)

Questions, Comments? Let me know below!

Errors and Solutions:

Error: Object not moving

Solution: Check if Cursor, InputManager is in the scene

Reference Cursor in the SimpleSinglePointerSelector component of the InputManager

Make sure the object has a Mesh Collider

Make sure the HandDraggable.cs is attached to the object

39 response to "Move, resize and rotate objects in your HoloLens apps"

  1. By: Nischita Posted: June 7, 2018

    I’m glad it helped! Thanks for the feedback Marco!

  2. By: ArielF Posted: June 20, 2018

    Hi we just tried you manual (10x by the way its awesome!)
    but we stopped at the point when you need to drag BoundingBoxRig.prefab in to the bounding box rig script component under the bounding box prefab filed ().

    For some reason it doesn’t let us to choose it, any ideas?

    Thanks:)

  3. By: Nischita Posted: June 21, 2018

    Hey ArielF,
    Thanks for the feedback! 🙂 You need to drag in BoundingBoxBasic.prefab into the Bounding Box Rig script and not the BoundinfBoxRig.prefab.

  4. By: Erick Posted: July 2, 2018

    Hi Nischita, in your first demo it appears that you are rotating the object in the Y axis, can you explain how did you manage to do that? Thanks

  5. By: Erick Posted: July 2, 2018

    Hi Nischita, in your first demon with the wooden furniture it appears you rotate the object in the Y axis, can you explain how you did that?
    In my test the object rotates from the right side only.

  6. By: Nischita Posted: July 3, 2018

    Hi Erick, If you are using the BoundingBox, there should also be the rotate (round) handles on the top for Y Axis rotation. Do you see them? If not, maybe your object is sort of flattened which may hide the rotate handles on the top. Maybe send a screenshot and I can help you more?

  7. By: Joshua Posted: July 17, 2018

    Dear Nischita, In the latest MRTK, I cannot seem to find the BoundingBoxRig.cs script and the BoundingBoxBasic prefab. Is this the case or am I missing something? Thanks

  8. By: Joshua Posted: July 22, 2018

    Another question if I may: I am using the recommended Unity 2017 LTS with the latest MRTK 2017.4.0.0 but I have noticed that when you use the native menu option to configure your scene, firstly the camera’s Clear Flags do not change from Skybox to Solid Color (so we have to do it manually) and also the little hands are missing for air-tapping. These were always sorted when you configured the project via that menu option. Have you noticed these?

  9. By: Nischita Posted: July 23, 2018

    If you’re talking about the MRTK – Configure option and Apply MRTK settings and scene settings, then yes -> I have also noticed that Clear Flags needs to be manually changed to Solid Color. Even in Unity’s 2018 version this happens. About the hands, I’ve really never needed them so not observed this. It’s so complicated to keep track of these tiny changes since each upgrade something new pops up or something old gets discarded 😉

  10. By: Rupe Posted: September 5, 2018

    The bounding box isn’t working for me. When I click on Adjust, nothing happens and the console displays this error:

    NullReferenceException: Object reference not set to an instance of an object
    HoloToolkit.Unity.InputModule.InputManager.RaiseBoundingBoxRigActivated (UnityEngine.GameObject boundingBoxRiggedObject) (at Assets/HoloToolkit/Input/Scripts/Utilities/Managers/InputManager.cs:852)

    Any help?

    • By: Nischita Posted: September 11, 2018

      Hi Rupe, can you send me a screenshot of your project’s inspector? Especially the object’s inspector on which the BoundingBox is added. That’ll help me see what else could be wrong. thanks!

  11. By: Wawan Satria Posted: October 22, 2018

    hey.. after I followed the tutorial that you provided, everything went well when playing on unity, but why can’t build to hololens emulator?

  12. By: Nischita Posted: October 23, 2018

    Hi Wawan, Did you try the build procedure explained in the article https://codeholo.com/2017/11/23/get-started-with-hololens-development-with-mr-toolkit/. In this article, look for the section where it specifically says…”To deploy to the Emulator, Choose Release, x86, Hololens Emulator on the top menu. If you don’t see the Hololens emulator, then download it from here (https://go.microsoft.com/fwlink/?linkid=852626) and restart Visual Studio.”

  13. By: Wawan Satria Posted: October 24, 2018

    yes, i understand about that, but why the object can’t move after i deploy in hololens ?

  14. By: wawan satria Posted: October 25, 2018

    how to order the movement can faster when i pinch ?

  15. By: YS Posted: November 1, 2018

    My application works almost perfectly.
    But the AppBar does not face the front. How do i fix it?

    • By: Nischita Posted: November 2, 2018

      Hi, The AppBar should by default appear in front of the user. If it doesn’t just play around with it’s settings in the Inspector. Maybe you are moving and the AppBar moves with your gaze, so it then appears as not in front of the object. If you can, also send me a screenshot of the object and the settings in the Inspector and I can also check what the problem could be.

  16. By: Mathilde Posted: December 13, 2018

    Dear Nischita,
    I had a problem with your tutorial. Indeed, it works as you when I apply what you did on a simple object. However, I have a problem when I create a game object which is composed by an object and a button (which is on it). When I want to apply your tutorial to my game object, it does’nt work and I don’t know why… I can’t seperate my button and my object so could you help me ? Thank you very much.

  17. By: Nischita Posted: December 17, 2018

    Hi Mathilde, Please send me a screenshot of both your object in Unity and where you are trying to manipulate- also the inspector settings on the object would be good to see before I can help debug. Thanks!

  18. By: Rampravesh Bhardwaj Posted: December 18, 2018

    Hi,
    I need some help from you. When we load a 3D object in a space/room it doesn’t retain its initial position if we move any other new room/space.
    How can i fix these issue. Please help

  19. By: James Posted: January 18, 2019

    Hello Nischita,

    I am trying to use the bounding box to reshape an object and then remove the bounding box app bar but can not figure out how to remove the app bar without removing the object as well. I would appreciate any help to this problem.

  20. By: Nischita Posted: January 20, 2019

    Hi James, If I understand you right, then all you have to do is go to the Bounding Box rig script on the Inspector. Under the field ‘App bar Prefab’ just select and choose None. I hope I have understood your problem, if not a screenshot would help me better understand.

  21. By: Gary Gorby Posted: February 1, 2019

    This works great! However, I want the object to stop moving or rotating if it collides with a room mesh created with spatial mapping or spatial understanding. Colliders don’t seem to accomplish what I would expect intuitively. Any tips or tutorials out there to add this functionality? Thanks for your help and the great tutorial!

  22. By: Nischita Posted: February 2, 2019

    Hi Gary, Another thing I stumbled upon which could help you: https://docs.microsoft.com/en-us/windows/mixed-reality/mr-azure-310 – check the spatial mapping and the GazeCursor part. That would show you how to set the SpatialMapping colliders corerctly and then you can disable moving when the collider of your GameObject interacts with the SpatialMapping collider

  23. By: Gary Gorby Posted: February 5, 2019

    Thanks, Nischita. I successfully completed the Mixed Reality 230 tutorial, but the setup doesn’t do exactly what I want. By adding a spatial mapping collider to the scanned room mesh and adding a rigid body to each object, I could get physical interaction object to object and object to floor or wall. When I move a cube object against the wall it will bounce off or if I hold onto it, it sort of squirms around. If I constrain the rigid body in space in the x, y, and z direction it will stay put when I let go, but it again twitches and squirms rotationally. Plus, I can drag it right through the wall if I want to. What I’d like it to do is orient itself flat against a surface when I move it there, but I am prohibited from dragging it through the wall. Basically, I want it to behave like all the standard app windows in HoloLens, where one can place them anywhere in midair or against a surface. Disabling movement once the colliders interact seems to be the problem….

  24. By: Enrique Avina Posted: February 10, 2019

    Thank you so much Nischita, this was exactly what I was looking for!

  25. By: Nischita Posted: February 11, 2019

    Glad to know Enrique 🙂 Welcome!

  26. By: James Scowcroft Posted: February 27, 2019

    I do not get the bounding box with handles it just blinks.

  27. By: James Scowcroft Posted: February 27, 2019

    Sorry should have been a little more clear. I have the objects and can drag them around the room I just cannot scale them. I am just trying to scale the dresser for now. The AppBar appears and seems to work. when clicked that is when the dresser just blinks.

  28. By: Godfrey Dsouza Posted: February 28, 2019

    Hey Nischita!
    Thanks for the awesome tutorial. I did this once before and it worked perfectly. However, I’m building another app and the bounding box doesn’t show up which i think is because the app bar doesn’t respond to my inputs at all. Could you please let me know how I can go about fixing this? Thanks in advance!

  29. By: Nischita Posted: March 7, 2019

    Hey Godfrey, Thanks for the feedback. Hmm, have you included the BoundingBoxBasicPrefab in the Bounding Box Prefab Inspector field of the BoundingBoxRig script? Also you can drop that in the AppBar inspector value in the AppBar component. If not, send me a screenshot at codeholo@gmail.com and I can help you better.

Leave a Reply

Your email address will not be published. Required fields are marked *

%d bloggers like this: