A look into facebook’s metaverse Patents
Recent headlines about Facebook’s transition to Meta and Mark Zuckerberg’s momentum towards building “the metaverse” indicate that we are within reach of Zuckerberg’s idea of an immersive virtual reality.
As the virtual space begins to take form, Founders Legal analyzed a variety of the patent trends and technologies Facebook could potentially utilize to power the virtual world.
What is a “Metaverse?”
Metaverse, a term coined to describe an iteration of the internet involving immersive virtual worlds and experiences, stems back to a reference from Neal Stephenson’s 1992 science fiction novel Snow Crash. However with the rapid rise and development of virtual reality technologies, the concept of diving into the metaverse is no longer a distant science fiction fantasy.
What is Facebook’s Involvement in the Metaverse?
Fully immersive VR worlds are still at the beginning stages. In a recent video stream with Gary Vaynerchuk, Zuckerberg noted that since the founding of Facebook, the social internet format has progressed from text to photos to videos, making online experiences “more natural and immersive.” Zuckerberg believes that VR and AR are the next major progression toD enhance togetherness and collaboration online.
In Facebook’s Metaverse unveiling, Zuckerberg shares his visions of a fully immersive homespace that allows users to create their own virtual environments by blending real-world comforts with virtual landscapes and spaces that inspire them. Based on the introduction, it’s evident that Zuckerberg’s metaverse intends to incorporate rich user-to-user interaction with facial expressions, sensory aspects, communication platforms, interactive experiences, and every aspect imaginable to ensure that the user truly feels like they are in an alternate reality.
Metaverse and innovation – Patent Trends
Patents can be an important indicator to understand the innovation in a technological field. Additionally, patents grant a 20 year monopoly to exclude others from using the disclosed technology. Therefore, patents can provide insight to both the progression of a technology and how the technology might provide a competitive advantage over others in the same market.
We analyzed Facebook’s patents and patent applications that may potentially relate to the company’s metaverse platform. At the time of our analysis, our findings indicated that overall Facebook holds roughly 6,730 patents and 5,607 patent applications. Of these patents, roughly 238 refer to Augmented Reality or Virtual Reality within the patent claims or abstract. Additionally, the company also has 147 public patent applications that refer to Augmented Reality or Virtual Reality within the patent claims or abstract. Essentially, AR and VR-focused patents only account for 3.5% of Facebook’s patent portfolio, and 2.6% of Facebook’s public patent application filings.
Facebook’s Launch into AR/VR
Facebook’s first granted AR/VR U.S. patent for a “Multidirectional communication system” was issued on January 10, 2017, nearly a year and a half after it’s initial filing on August 10, 2015. The patent describes wavelength-shifting (WLS) materials to facilitate high data rate communication, a head-mounted display, and a detector.
The patent background outlines the need for high data rate wireless communication systems, further stating, “there exists a need for devices capable of receiving high data rate communications without imposing onerous restrictions on the system or user.” This is an excellent example of initial development within a new technological space. Facebook had to first establish a sufficient data connection that could balance natural interaction with the virtual world and a transmitting enough data to the system to enable the virtual environment to respond to the users inputs.
Interesting Facebook AR/VR Patents
Over time, Facebook’s AR/VR patent portfolio has expanded to encompass a variety of unique technologies. From real-time customizable AR makeup to generating virtual landscape paths based on the actual space within your own physical room.
Below we highlight a selection of these patents, including:
- Comfortable navigation in large-scale virtual worlds when constrained by small-scale physical environment
- Use of neuromuscular signals to provide enhanced interactions with physical objects in an augmented reality environment
- Virtual spaces, mixed reality spaces, and combined mixed reality spaces for improved interaction and collaboration
- Systems and methods for generating and displaying artificial environments based on real-world environments
- Shared three-dimensional user interface with personal space
- Integrated augmented reality head-mounted display for pupil steering
- Systems and methods for rendering avatars with deep appearance models
- Virtual reality garment capable of jamming user movement
- Apparatuses, systems, and methods for adjusting fluid lenses
- Generating realistic makeup in a digital video stream
- Visual indicators of user attention in AR/VR environment
- Techniques for using perception profiles with augmented reality systems
- Virtual reality collision interpretation
- Suggestion of content within augmented-reality environments
- Systems and methods for providing content items associated with objects
Mixed Reality World patents
Comfortable navigation in large-scale virtual worlds when constrained by small-scale physical environment
U.S. Patent No. 11,170,575 | Filed 1/28/2020 | Issued 11/9/2021
INVENTION OVERVIEW:
The background states, “when the user’s play space includes not just a single room but also additional rooms and corridors, it would be undesirable to have the alerting feature of the entire play space boundary be activated. Alerting the user to potential hazards in other portions or segments of the play space would unnecessarily disrupt the user’s experience, since those hazards are irrelevant to safety due to their proximity to the user.”
This patent segments a layout of a physical space surrounding a user into physical segments and uses that segment to generate virtual paths for a virtual environment.
Use of neuromuscular signals to provide enhanced interactions with physical objects in an augmented reality environment
U.S. Patent No. 10,970,936 | Filed 10/4/2019 | Issued 4/6/2021
INVENTION OVERVIEW:
This patent describes computerized systems, methods, kits, and computer-readable media storing code that utilize a plurality of neuromuscular sensors for interacting with a physical object in an augmented reality (AR) environment.
Virtual spaces, mixed reality spaces, and combined mixed reality spaces for improved interaction and collaboration
U.S. Patent No. 10,921,878 | Filed 12/27/2018 | Issued 2/16/2021
INVENTION OVERVIEW:
This patent describes a method for users to create and use joint virtual spaces. The method involves checking spatial parameters of the virtual room before admitting additional users. When an empty space is identified in the virtual room, the additional users are then added to the room without interfering with the users already present in the virtual space.
Systems and methods for generating and displaying artificial environments based on real-world environments
U.S. Patent No. 10,726,633 | Filed 12/29/2017 | Issued 7/28/2020
INVENTION OVERVIEW:
This patent describes systems and methods for using sensors to identify objects of interest located within a real-world environment, which map and generate a virtual environment with the object.
The background describes, “traditional options for displaying real-world environments to remote users are limited. For example, displaying real-world environments to remote users as video images, such as via traditional video conferencing applications, may rob people located within such real-world environments of desired privacy. Additionally, such traditional displays of real-world environments may raise security concerns, as real-world environments may contain objects (e.g., people, documents, computer screens, areas, etc.) that may be subject to various security policies. Traditional video conferencing applications may display images of such objects to remote users, thereby potentially circumventing such security policies.”
Shared three-dimensional user interface with personal space
U.S. Patent No. 10,503,349 | Filed 10/4/2016 | Issued 12/10/2019
INVENTION OVERVIEW:
The patent describes a personal space associated with a user that displays content items visible only to the current user.The content item can be moved from the personal space to the shared three-dimensional computing environment for plurality of users.
Head-mounted and Garment Patents
Integrated augmented reality head-mounted display for pupil steering
U.S. Patent No. 11,112,613 | Filed 12/17/2018 | Issued 9/7/2021
INVENTION OVERVIEW:
This patent describes a head-mounted AR display device that includes an eye tracker, a light projector, a beam steerer and a combiner. The eye tracker is configured to determine a position of a pupil of an eye of the wearer. Virtual reality requires precise data collection from users to make the experience fully immersive. Eye tracking technology is crucial for both immersion and interaction with virtual environments. If sensory inputs are inaccurate it can become disorienting and potentially nauseating for the user.
Systems and methods for rendering avatars with deep appearance models
U.S. Patent No. 11,087,521 | Filed 1/29/2020 | Issued 8/10/2021
INVENTION OVERVIEW:
This patent describes a computer system capable of receiving geometry information and images of a subject, which can be associated with a respective viewpoint and may include a view-dependent texture map of the subject. A rendering module can be configured to render a reconstructed image of the subject for the predicted viewpoint using the inferred geometry and the inferred view-dependent texture.
According to the patent’s background, “With the advent of modern virtual reality (VR) and augmented reality (AR) headsets, there is a need for improved computer graphics models to enhance immersion. Human faces may be particularly challenging, but critical, for immersive social experiences. For example, in order to provide a virtual experience with the richness and depth of a face-to-face interaction, the nuances of the social signals between participants should be captured and displayed with accuracy. However, achieving real-time rendering of human faces has shown to be an arduous task. For example, the human face may contain fine geometric structures, such as eyelashes, pores, and vellus hair, that may be difficult to model. These and other factors may make it difficult to provide photorealistic rendering of dynamic and socially interactive scenes.”
Virtual reality garment capable of jamming user movement
U.S. Patent No. 10,976,826 | Filed 11/25/2019 | Issued 4/13/2021
INVENTION OVERVIEW:
This patent describes a system for a VR garment capable of jamming movement of a portion of the user’s body by increasing rigidity. “For example, to simulate the sensation of holding a coffee mug, the jamming mechanism prevents the user’s fingers from curling after the user’s fingers have reached a position equivalent to making physical contact with the coffee mug.” Haptic feedback is not a new concept, however connecting haptic feedback in the real world to interactions in the metaverse is no easy feat.
Apparatuses, systems, and methods for adjusting fluid lenses
U.S. Patent No. 11,048,075 | Filed 6/26/2018 | Issued 6/29/2021
INVENTION OVERVIEW:
Optical lens assemblies with a deformable medium that can alter an optical property of the optical lens assembly. Lenses like these could be used to change the focus of a display to reduce user eye-strain when dealing with three-dimensional virtual environments.
Avatar/User Patents
Generating realistic makeup in a digital video stream
U.S. Patent No. 11,069,094 | Filed 5/13/2019 | Issued 7/20/2021
INVENTION OVERVIEW:
This patent describes a method and system for rendering makeup on a user’s face within an augmented reality environment in real-time.
Visual indicators of user attention in AR/VR environment
U.S. Patent No. 11,022,794 | Filed 7/16/2019 | Issued 6/1/2021
INVENTION OVERVIEW:
This patent describes a method of notifying a user about attention from another user in an augmented reality/virtual reality (AR/VR) system.
Techniques for using perception profiles with augmented reality systems
U.S. Patent No. 10,755,487 | Filed 5/22/2018 | Issued 8/25/2020
INVENTION OVERVIEW:
This patent describes techniques to help social networking users manage their augmented reality identity. Facebook has stated its commitment to privacy, with the metaverse constantly developing it will be technologies like these that can assist in ensuring users have control of how much of their identity is shared with the metaverse. This includes user one or multiple profiles and the specification of which profile elements to display over a view of the user. “In this way, users gain more control over what image to project to different categories of people and is thereby empowered to express their identities in a context-appropriate way
Virtual reality collision interpretation
U.S. Patent No. 10,780,349 | Filed 11/19/2018 | Issued 9/22/2019
INVENTION OVERVIEW:
“A rhythm-based video game (“game”) is disclosed. In the game, a player slashes blocks representing musical beats using a pair of energy blades resembling a lightsaber. A gaming console renders multiple digital objects, e.g., digital blocks, digital mines and digital obstacles, that are approaching a player in a virtual space. The gaming console also renders a digital representation of an instrument, e.g., a lightsaber (“digital saber”), using which the player slashes, cuts or otherwise interacts with the digital blocks to cause a digital collision between the digital saber and the digital blocks. The player can score by slashing the digital blocks, not hitting the digital mines and avoiding the digital obstacles. The game presents the player with a stream of approaching digital objects in synchronization with music, e.g., a song’s beats, being played in the game. The pace at which the digital blocks approach the player increases with the beats.”
Digital Content integration patents
Suggestion of content within augmented-reality environmentsU.S. Patent No. 10,719,989 | Filed | Issued 7/21/2020
INVENTION OVERVIEW:
A method for rendering an augmented-reality environment on a display associated with a user and determining a suggested augmented-reality content item based on social-networking information.
Systems and methods for providing content items associated with objects
U.S. Patent No. 10,354,694 | Filed 12/30/2016 | Issued 7/16/2019
INVENTION OVERVIEW:
This patent describes a system and method to identify an object included in video that can be associated with another content item that can be presented in 2D or 3D.
