Rise in extended reality technology patents suggests market revival

Augmented reality (AR) and virtual reality (VR) are on the cusp of revolutionising our lives – from the way that we shop to the way that we consult doctors and interact with our communities.  

Extended reality (XR) is a fairly recent addition to the tech dictionary and the world of AR, VR and mixed reality (MR). It refers to all real and virtual combined environments and human-machine interactions generated by computer technology and wearables. As such, ‘XR’ is an umbrella term that includes and unites AR, VR and MR.

There are many practical applications of XR, which can be broadly classified into the following industries:

• Retail – XR enables customers to try a product before they buy it. Rolex, one of the world’s most coveted watchmakers, has an AR app that allows users to try on virtual watches on their real-life wrist, while Swedish furniture company IKEA enables customers to place furniture items into their home via their smartphone. 
• Training – especially in circumstances involving extreme danger, XR can provide training tools that are hyper-realistic and can help soldiers, healthcare professionals, pilots, astronauts and chemists figure out solutions to problems or learn how to respond to dangerous situations without putting their lives or anyone else's at risk. 
• Remote work – workers can connect to their office or with professionals located anywhere in the world in a way that makes both sides feel like they are in the same room. With online meeting tools set to greatly affect the future of work, as demonstrated during the ongoing coronavirus pandemic, XR will play a critical role in guiding the direction of innovative products that seek to make remote meetings more realistic. Video conferencing using XR is examined in more detail below.
• Marketing – XR creates a host of new possibilities to engage with prospective customers and consumers.  
• Real estate – finding buyers or tenants might be easier if individuals can walk through available spaces while they are in another location. 
• Entertainment – as an early adopter, the entertainment industry will continue to find new ways of utilising immersive technologies in cinema, gaming and other offerings.  

The mass adoption of XR applications across many industries is explained not only by their ability to ensure higher quality services, but also because AR and VR devices are becoming less expensive and more accessible. But with such incredible possibilities, it is not surprising that there is a high degree of sensitivity surrounding the topic. There is a growing need to transform XR from a concept into a technology that can be quickly applied in our daily lives. This article examines the IP landscape of XR technology.    

IP leaders in XR technology

It comes as no surprise that LG is heavily invested in XR technologies, considering that the company is a market leader in electronics and home appliances. Figure 1 reveals that its portfolio is almost 10 times the size of other players in the market (eg, Sony or Qualcomm). What is more, LG looks set to consolidate its strong presence by introducing various XR-centred products in its innovation.

Figure 1 also shows dedicated XR rivals Oserhout Design Group (ODG) and Magic Leap among the leading patent owners. ODG seemed to have been bolstered by a $58 million investment by 21st Century Fox in 2016, but its market presence has dissipated in subsequent years. Nevertheless, the 20-year old company has already seen its early work in foundational AR patents pay off. In 2014 Microsoft paid around $150 million to acquire a trove of ODG patents after deciding not to buy the company outright. Moreover, ODG claims that a number of AR patents in its collection have been infringed by existing products from companies such as Magic Leap, Google and Facebook, specifically pointing to diagrams of systems like Magic Leap 1 and Oculus Quest, which it claims conflict with its prior art. With a patent sale, ODG’s leadership is looking to recoup enough to pay back the company’s debts.

Meanwhile, Magic Leap glasses have also plummeted in sales, and the company’s position is of grim concern, especially in light of the ongoing coronavirus pandemic. Yet over the years, Magic Leap has managed to hire some top engineers who have developed some great technology that will likely be included in future AR headsets that the company has patented. Therefore, even if Magic Leap's products expire and the organisation collapses, it will still have intellectual property that other contenders will likely have to license in order to bring their own products to market.

The presence of Japanese giants Canon and Panasonic among the top patent owners highlights the surge in development of XR technology in imaging and optical products such as cameras, TV sets, film equipment and smart glasses. Indeed, it will be exciting to see how diverse applications of XR take shape in the coming years.

Evolution of XR patents over time

Figure 2 depicts the evolution of XR technology over the years. As XR includes AR and VR technology, it makes sense to view the data from the earliest years of AR and VR inventions. Beside the significant drop in filings in 2014, which reflects the overwhelming disillusionment surrounding the technology following the failure of Google Glass, the number of patents being registered in relation to XR technology has been steadily increasing. In the last two years, in particular, the hype around XR as one of the most exciting new technologies has been regenerated.

XR patents filed by geography

Although global patent officers have not been as busy with XR technologies, there has been an increase in filings over the years. If we examine the global spread of XR technology from 2011 to 2020, it is safe to say that the world leaders in innovation see the promise of XR. Figure 3 reveals that the USPTO and Japan Patent Office top the chart with 841 and 687 patents, respectively. Australia, India and Canada are also emerging jurisdictions for filing XR patents.

XR use cases

Let us drill down further into core use cases of XR.

XR conferencing

One of the most significant advantages of XR is the possibility for cross-functional adaptation. In a world that is becoming smarter with communication methods, XR in conferencing technologies is an interesting use case to explore. XR certainly looks to play a critical role in the development of technical standards. Figure 4 examines the patents of declared SEP holders for XR conferencing, revealing 190 SEPs across 57 families. It is no surprise that market leaders in this space are organisations such as Microsoft, Dolby, Qualcomm and Nokia.

XR-powered solutions are also penetrating aftersales services. Among the main use cases are rescued assist apps, virtual technical handbooks and augmented damage analysis.

One notable example is the BMW Group, where production has been focusing on XR for some time. By implementing XR, BMW has enabled assembly planners to assess the entire production process in a virtual environment, as well as test new processes in 3D.

Other prominent use cases include the following:

• Offline sharing of 3D objects – this looks to use 3D image messaging, which has already been seen in multiple social media and streaming apps. Apps such as SnapChat and Facebook have cashed in to create further functionalities with respect to image sharing. 3D image messaging is a highly patented topic with 41,338 patents currently filed across 17,169 International and Cooperative Patent Classification families. Market leaders Samsung, Microsoft, Intel, Apple and Google complete the top five as they look to bring the technology into core/application layers. As XR patents form a part of this set, they will be instrumental in shaping the future of 3D image messaging.
• XR multimedia streaming – the future of television is predicated on the rise in multimedia streaming governed by XR. It is of no surprise that LG leads this space as it has consistently incorporated innovative technologies to improve its TV and streaming products.
• XR mission critical – this focuses on using XR in police missions to improve safety and planning.
• XR gaming – more immersive gaming and game spectator modes are consistent areas of innovation in an industry worth $18.4 billion. With more than 1,183 XR patents related to gaming, XR is revolutionising untethered online gaming.
• Spatial multiparty call – videocalling and front-facing camera multi-party calls will see further innovation in XR, helping to make the world smaller and more connected.
   

XR in automotive

In the fast-paced automotive world, companies experience huge pressure from constantly emerging innovations. In order to stay ahead and be able to create the cars of tomorrow, vehicle manufacturers must not only keep track of new market trends but also anticipate and foresee them. In the wake of widespread digitalisation, the automotive market is becoming more consumer-focused. As such, automotive companies must create more personalised marketing models to satisfy their audience. By applying XR solutions, car producers can now offer their customers augmented and virtual car configurators, as well as showcasing exterior accessory parts, virtual test driving and virtual showrooms.

Data and metrics

To understand a bit more about the market itself and what these IP portfolios represent, Table 1 highlights various parameters such as market cap, coverage and technical relevance. IPlytics makes use of statistical patent valuation indicators to analyse strengths, weaknesses and the positioning of patent portfolios.

The market coverage (MC) indicator estimates the patent family size by counting each country of a patent family divided by the country’s gross domestic product. A high MC value reflects a global strategy and wide legal market protection. Further, a high MC value reflects a high perceived patent value for the applicant. Patent application and patent maintenance fees create considerable costs. The more countries a patent has been filed in, the higher the perceived international market potential for the patented invention.

The technical relevance (TR) indicator is calculated by counting the number of prior art citations that a patent receives (forward citations). A high TR value reflects the high technical relevance of a patent, as subsequent patents must cite the patent as prior art. This helps to identify leading technologies in a particular market segment.

Table 1. Leading XR patent market data

XR and standards contribution

XR is starting to influence a lot of 3GPP SA standards development, as demonstrated in Figure 5. It seems that industries such as gaming and online communication will require grounding and normalisation in order for future technologies to be built. With 5G technology set to greatly affect many aspects of our lives, advanced interactive services will use XR and 5G in tandem to develop products and services. Organisations such as Qualcomm, Samsung, Nokia and KPN lead the counts, with Qualcomm having contributed to 133 standards documents.   

Notable standard document IDs that focus on XR technologies include:

• TR 26.928, which focuses on network controlled interactive services, which form the basis of cloud gaming with a total of 46 contributions (contribution counts are led by Qualcomm with 35 counts and KPN with 3 counts); and
• TR 23.758, which focuses on edge computing, a type of network architecture that uses XR for enabling edge applications with a total of 436 contributions (Samsung leads this area with 190 contributions followed by Huawei and Intel with 72 and 45 contributions, respectively).

Challenges

Organisations developing XR technologies are battling with various challenges when it comes to mainstream adoption. First, XR technologies collect and process huge amounts of highly detailed and personal user data, which needs to be regulated and protected. Second, the cost of implementing the technology needs to be more development-friendly in order for the technology to become accessible.

It is also essential that the wearable devices that allow a full XR experience are fashionable and comfortable, as well as always connected, intelligent and immersive. There are significant technical and hardware issues to solve that include but are not limited to the display, power and thermal issues, motion tracking, connectivity and common illumination, where virtual objects in a real world are indistinguishable from real objects, especially as the lighting shifts. 

However, as each day passes, we come one step closer to solving these issues, and we will likely see many more mainstream applications of all XR technologies over the coming years.