Oculus CTO John Carmack on the VR challenges of the Quest and Rift S

Give Doom co-creator and Oculus CTO John Carmack 90 minutes to talk, and you can expect be inundated with tons of geeky technical details. That basically describes his keynote for the second day of the Oculus Connect 6 conference, where the legendary developer spoke frankly about the challenges the company faced as it launched the Quest ($399) and Rift S ($399) headsets this year. But of course, he also couldn't help but give us a taste of where things are headed.

It turns out, Carmack and his team had many of the same concerns reviewers did with this year's headsets. He worried that the Quest was too heavy, expensive and underpowered to succeed. And similarly, he was aware that the Rift S could be seen as a "step sideways," since it had a lower refresh rate than the original Rift. But he says the two devices allowed the company to do a sort of A/B test to see what style of device people preferred. Resoundingly, they opted for the Quest, which he describes as the "most retentive" Oculus platform. Basically, it's the one people keep coming back to the most, which is a key factor in making VR more mainstream.

Carmack says the Rift S also made him realize he was undervaluing the gaming VR content coming to PCs. With the $200 Go, the company saw widespread adoption and decent retention gains, and he initially believe that would be the direction Oculus would lean into. But the Rift S proved there's still a healthy market for high-end PC VR. And while the Quest can't fully reach the level of desktop gaming on its own, it's still capable of running far more immersive experiences than the Go. Both treat VR as a more active medium, whereas the Go is best suited for being a fairly passive device for diving into 360-degree videos.

The first huge bombshell from the conference was Oculus Link, a new feature that'll let the Quest run high-quality VR experiences by connecting to a PC over a USB-C cable. It turns the Quest into a PC headset -- effectively making the Rift S somewhat obsolete -- and based on my short demo, Link works surprisingly well. It relies on remote rendering, whereby the host PC sends the VR content to the Quest over a compressed video feed.

Carmack says he pushed for remote rendering on the Quest from the start. It's something developers have been toying with since the Gear VR, but it was never fast enough to be viable. Colleagues at Oculus also thought it might require an additional accessory to work properly on the Quest, like a 60GHz WiFi adapter or a Displayport over USB-C cable. But Carmack was confident they could it with the headset's onboard hardware and a single cable. It turns out he was right.

"Internally, we have a lot of perfectionists at Oculus and I have a lot of arguments about value, where you could look at something and say this is terrible for X,Y and Z reasons," Carmack said. "But if people find value in it, I tend to be of the position that you let people choose to do things, even if other people think it's not a sufficiently high quality level. We have a lot of debates about minimum bars of quality and poisoning the well [of interest for VR with bad solutions]."

Due to the speed and latency variabilities of WiFi, it made more sense to launch Link over a direct USB-C connection, where those issues don't exist. The video encoder Oculus is using is also limited to 150Mbps, which is easily handled by the USB 3.0 standard (and technically should be fine with USB 2.0 ports too). Carmack says the company would still like to optimize Link to work well over WiFi eventually.

And while Oculus might never officially support it, he said you could conceivably even connect to a remote computer over the cloud and stream a VR experience. There are ways developers could optimize their games for cloud rendering too, for example by handling some processes locally, but he's found it's hard to convince developers to adopt risky new architectures. (He's still bitter devs aren't widely adopting his "precious Timewarp layers" to make VR scenes render more clearly.)

Because Oculus controls both the hardware and desktop side of things, it's also able to implement remote rendering more efficiently than earlier attempts. Carmack says its implementation peels away a few layers of complexity, allowing the remote renderer to talk directly the Quest's Android OpenMAX driver. The big takeaway? There's far less latency than you'd expect while playing a VR video feed on the Quest.

Down the line, Carmack hopes to work more closely with Qualcomm to get low-level access to its processors. That would let him craft custom microcode to essentially treat the Quest like a remote monitor. He'd be able to do things like compress the VR video feed one scanline at a time, instead of dumping the entire frame buffer all at once. Surprisingly enough, Carmack says that could even let the Link-equipped Quest offer lower latency than the Rift S. That's mostly due to their display differences: the Rift S's LCD screen has a global shutter, so it has to scan the entire image frame at once. The Quest's OLED screen has a rolling shutter, which could allow it to offer up to a frame less latency.

As it stands, Carmack says you can expect the current implementation of Link on the Quest to feel like a mobile app when you're moving your head around. But you'll probably feel a bit of latency when you're moving side to side, or using the controllers heavily. I didn't notice too much of that during my demos, but then again, I didn't really encounter anything truly fast-paced.

The rest of Carmack's talk covered a wide variety of topics, here are some highlights:

• He held a sort of eulogy for the Gear VR, a device that sold well, but didn't keep users interested for long. He blamed the friction of using a smartphone as the main reason, since it involved taking your phone out of a case, plugging it into the Gear, and then losing access to your most important computing device. The Quest almost completely reduces the friction of entering VR, which is likely a big reason why it's so successful.

• Carmack is very happy about adding an Oculus Go compatibility layer to the Quest. He wants future VR users to be able to experience a "retro VR" scene, where they can easily play all of the first generation games. He wants to avoid something like Apple's shift towards 64-bit iOS apps, which killed access to any older 32-bit apps.

• Eventually, Carmack wants to have access to every 3D film ever made on the Quest. Its two screens can mimic the stereoscopic effect of 3D glasses, a feature that could be incredibly useful as we move away from 3D TVs (most 4K TVs no longer support that).

• Now that Fandango is on the Oculus platform, you'll also be able to watch any of your MoviesAnywhere purchases in VR. Carmack foresees VR headsets being among the best screens in your house for viewing films.

• Carmack admitted that the Quest's OLED screen can actually run up to 90Hz, but the company stuck with 72Hz since that was a more achievable framerate for mobile games. He was considering unlocking the 90Hz refresh rate for Oculus Link content, but he was warned that would voice the Quest's FCC certification. (I'm sure an intrepid hacker will figure out a way to make that happen.)

Follow all the latest news from Oculus Connect 6 here!