Apple/Lighthouse Patent Acquisition

Aaron M. Trippe

IP Litigation

Client Bulletin

April 18, 2019

Background

In late 2018, Apple acquired three issued patents and five patent applications from the defunct home security camera startup Lighthouse AI (the "Lighthouse portfolio").

Lighthouse AI developed and sold a smart security camera and supporting software system. Competing with companies like Google's Nest Labs and Amazon's Ring, Lighthouse differentiated its product with features powered by artificial intelligence and advanced computer vision, such as facial recognition, 3D sensing, and discriminating human from non-human motion. Accordingly, the patents in the Lighthouse portfolio describe various computer vision techniques such as visual authentication, foreground/background separation, incident detection, and low-light sensing with infrared illuminators.

Several parties reporting on the acquisition noted the similarities between the technologies in the Lighthouse portfolio and Apple's Face ID feature. Certain patents (such as the US20180367962A1) also disclose inventions that seem similar to FaceTime and video chat/conferencing. More generally, the Lighthouse portfolio parallels Apple's interests in cameras, optics, and video technology.

Portfolio Summary

Three of the eight properties in the Lighthouse portfolio are issued patents. The three issued patents are projected to expire in 2035, 2036, and 2037. Should any of the remaining five applications issue, their projected expiration would be sometime after 2037.

There are six patent families represented among the eight properties. The 20170039455 application is a continuation of the 20170032192 application, which itself is a continuation of the issued 9,396,400 patent.

Publication	Title	General Summary
9,396,400 20170032192 20170039455	Computer-vision based security system using a depth camera	Probabilistically designate pixels as foreground or background using depth data.
9,965,612	Method and system for visual authentication	Techniques for visual authentication of a person at a mobile device.
10,009,554	Method and system for using light emission by a depth-sensing camera to capture video images under low-light conditions	Use infrared illuminator to capture depth information in low light conditions.
20180246964	Speech interface for vision-based monitoring system	Natural language querying of detected objects.
20180374325	Method and system for incident sharing in a monitoring system	Incident detection.
20180367962	Two-way communication interface for vision-based monitoring system	Upon recognizing an individual (e.g. via face recognition), enabling a communication channel to a remote individual.

Review of Issued Patents

9,396,400

The 9,396,400 describes probabilistically designating pixels as foreground or background using depth data. It could be used in a wide variety of contexts related to computer vision and object recognition.

According to three metrics of claim strength (forward citation count, Aon Intellectual Property Solutions' proprietary claim breadth score, and Innography's proprietary PatentStrength score), the 9,396,400 is the strongest of the patents in the Lighthouse portfolio. Since it has two pending continuations, there is the opportunity for Apple to obtain further coverage of this invention with broader or more flexible claim language than is currently issued.

Key terms in the 9,396,400 include "persistent depth dropout" and "defining a foreground object based on the plurality foreground pixels;" the specifics of these terms could become fodder for claim construction debates. Furthermore, the independent claims require data to be sent to a "remote processing service," a term which could be relevant to parties designing around the patent either during litigation or in an attempt to avoid litigation.

9,965,612

The 9,965,612 describes a technique for visual authentication. The invention bears a similarity to Apple's existing Face ID technology. The independent claims contains a large number of limitations (including both first and second "validation result[s]" with two different outcomes, based on comparisons of different visual authentication elements) as well as a number of terms that could require further definition, which could make this patent more difficult to assert than the 9,396,400. Certain claim terms that could become important in claim construction include "pre-provision," "reference visual authentication element" and "visual authentication element," "matching score," and "performing matching."

10,009,554

The 10,009,554 describes a technique for activating an infrared illuminator to capture depth information in low light conditions. One relevant part of the invention is determining whether an infrared illuminator needs to be activated based on lighting conditions, and if so, obtaining a frame with an exposure time based on an "infrared illuminator parameterization" and "desired frame exposure time."

The 10,009,554 is related to the 9,396,400 in that it requires an "object [to be] determined based on data from [a] depth-sensing camera." Because of this requirement, along with the need for specialized hardware, the 10,009,554 is likely the narrowest of the issued patents in the Lighthouse portfolio.

Related Litigation

Since 2014, Apple has been sued in several cases pertaining to biometric authentication technologies. Following the release of the Face ID feature in November 2017, these cases begin to mention Apple's Face ID technology, and it is likely that companies will continue to assert patents against these Apple technologies. Therefore, acquiring the Lighthouse AI patents could have been a defensive measure against lawsuits implicating Face ID specifically or visual authentication and recognition technology more generally, though this would be limited to practicing entities that could be targeted in a countersuit (as opposed to non-practicing entities which have no products and thus could not infringe the patents). The acquisition also prevents the Lighthouse portfolio from being acquired by other parties (such as non-practicing entities) that might have used it to sue Apple.

Conclusions

The projected expiration dates of the patents (mid-2030s) suggest that Apple's acquisition of the Lighthouse portfolio is part of a long-term strategy to protect technologies in the computer vision space. The patents could be effective in protecting further advancements of Face ID or FaceTime, such as facial recognition in combination with a verbal passphrase or incorporating facial recognition into video chat. However, since most of the patents pertain to technologies beyond just visual authentication, they could be pertinent to more speculative technologies, like improved image processing technology, natural language querying of video data, or augmented reality.

Apple could venture further into the smart home space as well, as Google and Amazon have done with Nest Labs and Ring, respectively. The business strategy of Lighthouse AI aligns well with that of Apple, in that both companies aimed to develop products with more advanced features than their competitors for a higher price point. In this light, Apple could intend to improve upon the features that originally differentiated Lighthouse AI's product, namely those related to object recognition.