valve-steamvr-icons — Based on images courtesy Valve

Valve Revamps Steam Survey to More Accurately Count VR Headsets

By

-

Apr 1, 2020

Valve’s long running Steam Survey has been one of the most transparent data sources for gauging which VR headsets are the most used. Now the company says it’s revamping the way it collects data to more accurately count VR headsets in use on Steam.

Although Steam represents just one of several major VR platforms, it’s the only one of the bunch that transparently presents data about which VR headsets users are using via the Steam Survey which collects data about hardware used by the Steam population. The data helps developers and the rest of the VR industry understand important trends which can inform development, marketing, support, and more. Our last major analysis of the Survey data is here.

Valve tells Road to VR that it’s revamping the way it collects data about VR headsets used on Steam, which should increase the accuracy of the figures in the Survey. The company says that both wireless and less common headsets will now be represented in the data.

Previously, the company explained, VR headsets were counted by the Survey only if they were plugged in via USB at the time the data was collected. The new method will allow SteamVR to report to the Survey any headsets that have been used by the system in the last month, regardless of whether they are plugged in at the time of data collection.

SteamVR Gets a New Dashboard, Finally Ditching Steam Big Picture

Valve expects the change will result in a bump in the number of headsets counted in the survey. While the change in data collection hasn’t happened yet, Valve has already tweaked the Survey in a way that shows headsets being used on Steam which never previously appeared in the data, including Pimax headsets and PlayStation VR (which can be used with Steam via some third-party workarounds).

We’ll see the biggest changes when the Survey rolls over to the next month; at last count, it shows that 1.01% of Steam users used a VR headset in February.

Martin355

Yeah, I got the survey once, but I only plug in my Index when I’m using it. I think this change will result in a sizeable bump.
- Immersive Computing
  
  In 4 years of PCVR from Vive and Rift CV1 through Lenovo Explorer and now Index I’ve never received the survey?
  - James Cobalt
    
    It’s just a little popup window in Steam that asks to send your hardware information to them. I’ve seen it only a handful of times over the years. Perhaps once you grant permission it doesn’t need to get it again for a while?
  - Andrew Jakobs
    
    You’re not the only one, I also never had such a survey EVER..
    - Immersive Computing
      
      Yes it’s strange, and how many PCVR owners are not counted?
  - ArSh
    
    It only pops up on starting up Steam, not SteamVR. And only if you’re randomly selected. I’ve had it a handful of times.
kuhpunkt

Finally.
Scott Arthur

I noticed this when I got the pop-up survey this morning, sure enough it picked up my Quest (which was not Linked at the time) and both Touch controllers. Finally!
- Matteo Valles
  
  How does it pick up the Quest? Did you plug it into the computer at some point?
  - Scott Arthur
    
    Yes, I use the link cable to play SteamVR and Oculus games on my gaming PC through the Quest Link feature. Works great!
impurekind

Cool.
JACrazy

Within the last month is an improvement, but maybe they should have a separate tracker with an even larger time span and present both numbers. A lot of users out there, including me, that havent touched their headset in months. Would also be a good way to track the retention of keeping players engaged with using VR.
- Blaexe
  
  The surve – in general – includes monthly active user, so one month makes perfect sense.
gamechanger

This is the only way that makes sens if you want to count Quest Link users.
NooYawker

Now we’ll see that WMR have outsold all other headsets :p
Sven Viking

According to UploadVR the March 2020 survey initially showed VR ownership at 1.07% of Steam users, but if you check it right now it is much higher at 1.29%:

https://store.steampowered.com/hwsurvey/
Schadows

My Pimax 5K+ is always plugged in, but not powered up unless I want to use it, hence last time I had the survey popup, I check the results and no VR headset was reported. I tried powering it up and go back, but the data was already collected and the survey answer never changed.
DickDastardly

“Previously, the company explained, VR headsets were counted by the Survey only if they were plugged in via USB at the time the data was collected. The new method will allow SteamVR to report to the Survey any headsets that have been used by the system in the last month, regardless of whether they are plugged in at the time of data collection.”

If that’s accurate then the inevitable result of the new methodology will be to inflate the proportions for headsets whose primary store is Steam (whilst reducing the proportion of Oculus headsets reported).

Consider, for example, a user who only has their headset connected to their PC on average one day a month. With the old methodology they only had a 1 in 30 chance of having their headset connected and detected when the survey ran.

But compare that to their chances with the new system: If they exclusively use Steam VR (let’s say because they’re an Index owner) then they’re now guaranteed to be counted in the survey because they’ll show up in the Steam VR logs for the month, whether or not they happen to have the headset plugged in at the time of the survey. In other words the chance of their headset being counted has gone up by a factor of 30!

If, however, they’re a Rift owner who has Steam installed for pancake games but only uses the Oculus store for VR content, then the chances of their headset being counted hasn’t changed – it’s still only 1 in 30. In fact this may even also apply if they play VR content on Steam which uses the Oculus API rather than Steam VR (as presumably they wouldn’t show up in the Steam VR logs then either and so would still only be counted if their headset happened to be plugged in at the time of the survey).

Of course, this is just an example – I suspect most VR users have their headsets plugged in for far more than one day a month, but it explains why there was such a large drop in CV1 users at exactly the time the methodology changed whilst OG Vive users actually went up.

Presumably the fact that the proportion of Index users reported also increased significantly as a result (despite largely being unavailable to buy during the period in question) was just an accidental by-product (an “unforeseen consequence” if you will) and not the entire reason Valve made the change ;).