Director, Research Science, Meta

From 2012 to 2014, Robert co-founded and was CEO of Nimble VR (née 3Gear Systems), which created technology for tracking and understanding the hand, or hand-tracking, to increase presence and embodiment in augmented and virtual reality (AR/VR). Nimble VR was awarded a NSF Small Business Innovation Research grant in 2012 as well as Augmented World Expo’s “Best Startup” in 2013. In 2014, Nimble VR was acquired by Facebook after raising $2M in venture capital. Nimble VR was also identified as one of the “Top 10 inventions of 2015” by Popular Science.

At Facebook, Robert is a Director of Research Science, supporting a research and engineering team to further advance hand-tracking technology for AR/VR as well as other natural input technologies, including body-tracking, object-tracking and rapid text input. In 2019, the team shipped hand-tracking on the popular Oculus Quest VR platform, enabling users to directly interact with VR content with their hands, without picking up a controller. Hand-tracking makes VR more approachable, particularly for people not used to game controllers. It also enables people to be more expressive, capturing natural hand gestures like a thumbs up and a peace sign. Hand-tracking is now embedded into the core VR experience and has been adopted by popular applications such as Vacation Simulator. To deliver this technology, the team had to significantly advance the state-of-the-art of computer vision, enabling hands to be tracked smoothly (without jitter) from a single monochrome camera with a fraction of the compute of a mobile processor already running games and applications.

In 2020, the team also proposed a method that enables touch typing on any flat surface, i.e., without a keyboard, at speeds comparable to typing on a physical keyboard, i.e., over 70 words per minute. Faster text entry is critical to enabling better communication and productivity in AR/VR. Rather than relying on capacitive touch or other peripherals, the team used hand-tracking, which can run from just an AR/VR headset, to unlock always-accessible rapid text entry.

Robert received a Ph.D. in EECS from MIT in 2011. Robert’s thesis work on using an inexpensive color-glove and webcam for hand-tracking received media coverage from Wired, CNN, and New Scientist.