Update 25 June 2021
Oculus appears ready to significantly update it’s AR/Mixed reality experiences when using the Oculus Quest 2 headset. On their developer blog, Oculus has shown off a developer toolkit called Passthrough API Experimental, designed to make it easy for app builders to seamlessly merge VR/AR models with the real world video footage generated by Oculus Quest 2’s two front facing cameras.
Oculus states that the software will allow you to collaborate with remote teams or friends through virtual monitors, while accessing physical keyboards and desks, create games that blend the virtual world with the real world (think aliens in your living room or Zombies on your bed), and allows what they call Co-located Social presence, giving users the ability to interact with people and pets in the same room at the same time.
These features are a big step up from the addition of virtual monitors and keyboards in recent updates, but it still doesn’t address the main problem with mixed reality on the Quest 2: black and white cameras. It doesn’t matter what virtual elements are overlaid on the real world if the raw footage of the outside world is distorted, grainy, and in black and white. High definition colour, without distortion, is a must if this is ever going to take off.
Perhaps we’ll have to wait for the Oculus Quest 3 or the Oculus Quest Pro that is rumored to be released in 2022.
Read all about these AR/Mixed Reality updates on the Oculus blog here.
Previous article content from 2019
Last year at its annual conference, Oculus showed a mixed reality demonstration that combined virtual reality and augmented reality views using their upcoming Oculus Quest mobile headset. Virtual app icons, calendar entries, notifications, video screens, a word processing application, and more, were overlaid on a rough black and white outline of the real world in real time. The outline of a user’s world was generated by the headset’s external sensors with coloured 3D elements fixed in various locations in space. There was even a virtual room you could enter and meet others, jumping between an enhanced real-world view and a completely virtual space. A user typed on a real-world keyboard and his words were shown inside their augmented reality word processor.
Watch the mixed reality presentation from the Oculus 5 Connect conference in the video above. Starts at 20:37.
Passthrough plus leaks
Recent leaks of the Oculus Quest’s user set up, as well as reports from people who have attended demos at GDC 2019, suggest Oculus has already solved the problem of accurately capturing the real-world using headset sensors. Instead of taking off your headset to draw boundaries around your play area (as you do in the current Oculus Rift), when setting up the Oculus Quest, users see a representation of the real world in black and white (not merely a vague outline) and then draw boundaries over real world objects. Oculus calls this setup experience and outside world view “Pass-through plus”. Users don’t need to take off their headsets as their surroundings are visible.
See the leaked set up video below.
Problem solved
If the Oculus Quest already allows you to draw over real world objects (all be it in black and white) then the barriers to achieving their mixed reality environment shown in last year’s video have been removed. Add colour and a software interface to support this augmented reality view, and they are up and running.
A bold mixed reality future with Oculus Quest
It’s likely Oculus will not only be releasing their first mobile headset with six degrees of tracking, but also their first augmented reality headset too. Seamlessly jumping between augmented reality and virtual reality arguably offers greater opportunities than a simple augmented reality view. The scope of the experience would be infinite. Imagine being able to work from home without any draw backs. You can visit your workplace by simply walking through a virtual door, attending virtual meetings with other staff who work in different places around the globe.
If the software is not in place when the Oculus Quest is first released, no doubt it will arrive shortly afterwards.
Present Augmented reality limitations solved
Presently there are five main draw backs to augmented reality hardware: the cost of the devices; the limited field of view; the resolution of the 3D images; and the comfort and size of the headsets. HoloLens, Magic Leap, Meta 2 and other augmented reality devices all suffer from these problems.
The Oculus Quest immediately solves three of these four glaring issues. The Oculus Quest has a higher field of view and and a higher resolution than any augmented reality device on the market, and is significantly cheaper (for example, US$399 compared to US$3,500 for the Hololens 2 or US$2,200 for the Magic Leap One). This would leave headset size and comfort as the only obstacles to widespread adoption.
A pre-order announcement for the Oculus Quest is expected at Facebook’s F8 conference that runs from 30 May to 1 April. Hopefully we’ll hear more about their mixed reality aspirations and software then.
Leave a Comment