Client overview

Our Client is a leading social media organization. The AI/ML algorithms required for their innovative new VR headset needed ground truth data on an unprecedented scale to make sure the model recognizes everyone in every scenario.

Big ground truth data requires enormous pool of participants

To ensure the VR headset would be flexible enough to work with all kinds of individuals, under varying environments, clothing and accessories. With a target of 1,500 participants, the Client needed to find a way to recruit a large and varied pool of people for whole body capture activities.  They also needed a very specific range of demographic targets to capture to eliminate bias from the ground truth data. This needed to reflect a diversity of skin tones, BMI, height, gender, age etc. Only with this quantitative and qualitative data set would the AI perform inclusively and capture diversity in humans, body actions, gestures and environments.

Further complexity was added to the training of the AI/ML algorithms for the VR headset in the requirement for comprehensive data representing various dynamics and control factors including groups, clothing, accessories, room environments like staged living rooms, bedrooms, home offices, dining rooms, kitchens etc.

Quality Engineered comprehensive ground truth data delivered

To deliver the vast tome of data, Qualitest sourced 1,500 participants to visit Qualitest’s pioneering ground truth data collection laboratory in Kirkland, Washington, USA.

We first established a comprehensive plan to capture specific demographic targets and to capture diversity of skin tones, BMI, height, gender and age. Whole body data captures were completed including groups and varied clothing, accessories. There was huge diversity in captured room environments too with staged living rooms, bedrooms, home offices and dining rooms.

Key benefits

By selecting Qualitest as a partner for this pioneering project, the Client was able to achieve:

  • 98% QA success rate – this means that, if we collected 100 recordings, 98 or more of those passed the bar for quality. In each recording, we collect specific protocols. If these protocols are not performed correctly by the moderator, the client can reject the entire recording.
  • 20 participants per week full body motion captured for 75 weeks (100% of target).
  • 97% demographic match vs minimum Client requirement of 90% – outperforming target by 7%.
  • 30% monthly cost savings for Client through onsite staging vs other vendor’s traditional participant engagement methods.

New call-to-action