Designing An Ideal Adjustable Wrist Menu
This is an example of a real design challenge that I have worked through.
The brief was: an ergonomic wrist menu which can provide useful functions and also have customizations to maximize user comfort and suit their preferences.
It is important to start with research and ground your work in the best practices of others. Although toolkits are getting better, such as Unity’s XR Interaction Toolkit (XRI) and Meta’s Presence Platform for example, it is still ideal for an Immersive/Spatial designer to have the perspective and skills to bring forth XR user interfaces from first principles.
Here I will go through a design process I use often consisting of three main phases.
Research and discovery
Concept Exploration
Mechanics Validation
These three phases can be used cyclically, or you can use just the last two if you are satisfied with the research and initial discovery you did at the onset.
Research
Since I want to make this wrist menu’s exact position user-customizable, I need to have a framework for the limits of that customization. I need to understand what ergonomics are best for wrist mounted UI in general so that I can have a solid ground to build from.
Turning to academic work can often help give a concrete basis for certain decisions you want to make and generally grounds you in experimentation which will pay off later. Many ideas you might try have already been tested in some form out there. Make use of Google Scholar to help narrow down to just academic work.
For this wrist menu design I found a work exploring exactly how to optimize for placement of touch targets on the opposite arm titled “Armstrong: An Empirical Examination of Pointing at Non-Dominant-Arm-Anchored UIs in VR”.
Armstrong: An Empirical Examination of Pointing at Non-Dominant Arm-Anchored UIs in Virtual Reality
This research gave me a perfect zoning of where the ideal areas for wrist mounted UI is in an empirical way. This was essential for me, since I wanted to make the interface easy to use while also providing a range of customization. All positions a user can choose should be optimized so that the user does not create an uncomfortable customization. So I can utilize this research’s regions in my design and have grounding that what I’m offering will still be ergonomic.
Discovery and Inspiration
Gathering inspiration from other talented designers’ open work is a great way to break ground on what is possible. And of course exploring games that already implement some of the mechanics you are after is ideal. This is a small subset of comparables that particularly inspired me:
https://twitter.com/DavidSHolz/status/1265409000868216832
This twitter thread from Leap Motion designer David Holz contains tons of great reference on the early (still path leading) experiments done with that first hand interface. Check out the whole thread, which shows their iterations to this final wonderful interface.
Demeo uses a fanned out card approach. I like the use of hand open gesture, there’s clear visibility to the elements.
https://twitter.com/Protodeep/status/979499009055211520
And this amazing concept by Barrett Fox is something I still return to often. The use of a single hand for the interface sets this apart and helps set a North Star for what is possible with hand menus outside of the ‘normal’ two handed approach.
Concept Design
Explorations
Moving from research to concept work, it is important to first understand what aspects you need to consider in your prototypes. I often use Miro or Figjam alongside ShapesXR in early exploration. I’ll gather up a moodboard that features some ideas, areas I want to tackle, and some logic where applicable and then screenshot that to bring into ShapesXR for some explorations.
I often try to experiment with two or three different or complementary approaches to solving a given problem, so I can have lots of variance to see what’s working and not. The trick here is how to accommodate the customization. I start by establishing a frame of reference for how the menu component will be positioned, and how the user will change that position.
Above, the overview stage I usually make where I show all my work at once for reference. Then I go into specific stages for different concepts. Let’s walk through some of them.
First I tried a zone-based approach trying to encompass the whole hand, inspired by Demeo. I actually brought in a 3D fibonacci spiral to help inform this design. The concept is to have different predetermined zones where the menu can occupy that will accommodate different ways of holding ones’ hands: some people like to hold their hand fingers facing up, while others prefer thumb facing up, and angles in between.
Here I made use of the library of assets available in ShapesXR. Making sure to work at 100% scale in ShapesXR to have 1:1, I could use the stage system and manual puppeteering to feel out the hand interaction.
Then I tried a more ‘traditional’ straight line area on the inside of the user’s wrist that allows repositioning within the fixed area.
These experiments let me see two different extremes of customization. My takeaways were:
Radial placement of hand menu has odd edge cases and is more complex to implement than it rewards in ergonomics.
Straight along is simple yet excludes users from working with a thumb up palm position if they desired, limiting flexibility.
So next I synthesized the ideas together. A concept to set the position along a curve constraint. This could enable flexible positioning along a wider range of possible optimal zones, while removing edge cases of inefficient / unergonomic placements
Menu Aspects
Ambidextrous in two main areas: ulnar and above fingertips
Curve enables ‘analog’ placement along any desired place along side to top
Scaling constrained to reasonable small and large dimensions
Menu can be rotated tall or wide
Animation in Blender for details
ShapesXR works wonderfully for blocking out keyframes and major app states for your designs, but sometimes you need to add more in between detail and explore more complex animations or materials and shading.
You can easily export from ShapesXR just a certain selection, which will save as a GLB file in the web portal. From there you can import the GLB into Blender (Or any other 3D software) and add animations, explore materials and rendering.
Test in Unity using MRTK
ShapesXR and Blender can give insight into which direction to take, yet when considering committing to an entire mechanic, it is ideal to test your hypothesized design with a technical proof of concept. At this step, you can use any available parts you have access to. You are ‘kitbashing’ or hacking together a way to validate some core assumptions of the design.
Remember, for hand menus it is important to not parent them directly to the hand, but rather create a rig with smoothed tracking to avoid unnecessary fatigue from micro hand movements messing up your UX and UI targets. Thus I need to check the validity of my ideas using frameworks.
Core Focus:
Testing feasibility of using a slider to position, scale and rotate a hand menu
Framework:
Unity + MRTK with Quest Passthrough
Results:
Possible! Many more details and behaviors to define
Limitations:
Slider prefab limited to line profile, MRTK introduces other menu following behaviors and constraints of scaling options
Next steps would be:
Refining the desired behaviors to forward from shapes concept + demo constraints
Creating a curved slider element to test ergonomics of continuing to hold slider along a non-linear curve. Risk, difficult to constrain? Odd edge cases?
Conclusion
It is possible to establish a customizable hand menu system with constraints. Directly implementing in MRTK provided a grounding on what the minimum viable customization might look like, while also showing the limits of a straight line slider. Providing validation to keep exploring this idea this curve ‘menu on rails’ customization approach could be explored more broadly.
I hope you can see ways you can use this broad design process of exploration with rapid iteration in ShapesXR alongside grounding validation using development toolkits to craft some really delightful and novel XR interactions. Take advantage of the freeform nature of ShapesXR, that was my biggest personal takeaway from this project. I could really immerse myself in different permutations of how the possible menu could work, and become one with the problem and use feeling to help guide what was working and not. Because I could freeform draw alongside the prefab items I found myself being more ambitious about what shapes I would want to have ergonomically.
Final thought, as someone working daily on XR software for scientists where effectiveness is part of the delight, I want to encourage folks that the ‘industry’ does not have it all figured out. There are still innovations to make, and standards to set for years to come. We currently stand on the shoulders of giants with nice frameworks for smooth-tracked menu prefabs and such. However, there is still much untapped potential of the medium so make sure you explore freeform first and design to your requirements and goals, don’t just design around what someone else has given you. Even if you end up using a totally off the shelf paradigm or solution, you did your due diligence as a designer to explore the possibilities and understand why you made your decision. Try things out and see what you learn, we all will benefit as this new medium is still being created.