Automatic Content Recognition (ACR)

Automatic content recognition (ACR) refers to the ability of a client application (typically a smartphone or media tablet app) to identify a content element within its proximity _ audio, video or digital image _ based on sampling a portion of the audio or video (or image), processing the sample and comparing it with a source service that identifies content by its unique characteristics such as audio or video fingerprints or watermarks.[1]

In 2011, ACR technology was applied to TV content by the Shazam service, which captured the attention of the television industry. Shazam was previously a music recognition service which recognized music from sound recordings. By utilizing its own fingerprint technology to identify live channels and videos, Shazam extended their business to television programming. Also in 2011, Samba TV, at the time known as Flingo, introduced its patented video ACR technology, which uses video fingerprinting to identify on-screen content and power cross-screen interactive TV apps on Smart TVs. In 2012, satellite communications provider DIRECTV partnered with TV loyalty vendor Viggle to provide an interactive viewing experience on the second screen. In 2013, LG partnered with Cognitive Networks (later purchased by Vizio and renamed Inscape), an ACR vendor, to provide ACR driven interaction. In 2015, ACR technology spread to even more applications and smart TVs. Social applications and TV manufacturers like Facebook, Twitter, Google, WeChat, Weibo, LG, Samsung, and Vizio TV have used ACR technology either developed by themselves or integrated from third party ACR providers. In 2016, additional applications and mobile OS embedded with automatic content recognition services were available including Peach, Omusic and Mi O.[2]

How Automatic Content Recognition Works[3]
ACR works across a wide variety of media platforms, such as CTV systems, linear cable television, and even video games. The technology works by comparing the small in-the-moment snippet to a vast library of properly cataloged data to find exactly what the snippet is. In greater detail:

  • Media companies create a reference library of content. This is the archive of everything on television that future snippets will be compared to. Data centers catalog each moment, adding in all the metadata about when the media plays (on linear television), what video game, movie, or television show the snippet comes from, and any other useful information. Managing companies expend a lot of time and effort cleaning the data to create a refined, reliable library of listening posts.
  • ACR pulls a snippet from a viewer's current media and compares it to the library.
  • The content is matched with the right file full of metadata. These images or sound elements can happen once every 10 seconds or multiple times a second; as more snippets are compared in the library, the library continues to grow.

"To create the reference library, you need 'listening posts,'" explains Jane Clarke, CEO and managing director of CIMM. "Computers in data centers 'watch' TV and catalog what they see. Then, that reference library is matched against a schedule of what ran, so the computer can match the image and audio to a 9 p.m. prime-time episode, for example."

How ACR Works
Source: OTTVerse

Uses of Automatic Content Recognition[4]
There are several uses of ACR technology. Some of the more prominent ones are –

  • Detection of copyright infringement: Copyrighted material such as video and audio are often used indiscriminately without attributing or paying royalties to the original content creators. If a database of copyrighted content exists, then large UGC platforms such as YouTube, TikTok, Vimeo, etc. could check to see if user-uploaded content contains copyrighted material or not.
  • Ad-tracking: ACR has found a lot of use in the advertising industry and for good reason. Here’s why –
    • Unless you have the ability to determine if an ad was played and watched by the end-user (instead of being buried at the end of a long landing page), then your metrics don’t make a lot of sense and it could lead to inflated data with respect to ad impressions, plays, and completion rates. This requires SDKs and changes to the players that can consume a lot of effort and development cycles.
    • However, ACR has the ability to recognize the content that is being played by sampling certain pixels of video, or by recognizing the audio. This enables ACR to provide a better picture to the advertisers and publishers on the ad delivery and engagement.
  • Collating information from different sources: This is a very interesting use-case of ACR. In most homes, there is one big TV in the living room where people gather to watch movies. However, the content streaming to the TV could come from an STB, Chromecast, Roku, FireStick, or an Xbox. Instead of embedding code inside all these devices, SmartTVs with ACR can recognize the content being played (from the “glass”) and report on it. This allows for content attribution and normalization across a variety of sources.
  • Understanding Audiences and their preferences: Similar to other methods of gathering usage analytics, ACR allows broadcasters and content providers to know how their audience is responding to their content, marketing, strategies, etc. By having fine-grained information about their audience and their usage patterns, broadcasters can better invest their dollars and get a much higher ROI.
  • Ad Retargeting by OEMs: Samsung includes ACR technology in their SmartTVs and sells ad inventory and provides ad-retargeting services. According to their website, “Samsung Ads offers TV Ad Retargeting that empowers brands to identify audiences who saw or missed their TV spots and reconnect with them via mobile, tablet, desktop or OTT.” And, “Samsung Smart TVs have built-in Automated Content Recognition (ACR) technology that can understand viewing behavior and usage including programs, movies, ads, gaming content and OTT apps in real-time”.

Applications Automatic Content Recognition[5]
Content identification ACR technology helps audiences easily retrieve information about the content they watched. For smart TVs and applications with ACR technology embedded the audience can check the name of the song which is played or descriptions of the movie they watched.[17] In addition to that, the identified video and music content can be linked to internet content providers for on-demand viewing, third parties for additional background information, or complementary media.

  • Content enhancement: Because devices can be "aware" of content being watched or listened to, second screen devices can feed users complementary content beyond what is presented on the primary viewing screen. ACR technology can not only identify the content, but also it can identify the precise location within the content. Thus, additional information can be presented to the user. ACR can enable a variety of interactive features such as polls, coupons, lottery or purchase of goods based on timestamp.
  • Audience measurement: Real-time audience measurement metrics are now achievable by applying ACR technology into smart TVs, set top boxes and mobile devices such as smart phones and tablets. This measurement data is highly essential to quantify audience consumption to set advertising pricing policies.
  • Broadcast monitoring: For advertisers and content owners, it is vital to know when and where their content has been played. Traditionally agencies or advertisers have to manually audit the presentation. At scale it only can be checked through a statistical sampling method. ACR technology enables automatic monitoring of the content played in TV. Information like the time of play, duration, frequency can be achieved without any manual effort. Many people have expressed some concern however on the information that these smart TVs are sending out to the companies collecting this data. However there is an option in almost every set to disable this feature.

The alternative approaches are video based automated content recognition technologies. These are a suite of technologies that revolve around the convergence of video and TV Everywhere which will render the audio and digital watermarking methods incapable of handling the millions of unique streams going out and billions of hours of footage to be reviewed with metadata extracted or enriched in relation to the content in real-time. Where acoustic fingerprint fails in its reliance on a database of reference fingerprints. Digital watermarking relies on intrusive frame by frame production stage imprinting on every piece of content. The effectiveness of these techniques have been challenged based on their presumed inability to effectively scale to the amount of video being generated. In practice for monetization and other user based ACR applications the reference database or presence of watermarks only has to cover those videos that are targets of monetization. For example, a video that is hosted on YouTube and viewed only once does not need to be present in a world wide ACR database or be impressed with a watermark.

How Advertisers Can Benefit from ACR[6]
ACR’s most significant benefit is its ability to enable advertisers to align their broadcast TV and digital media strategies with a cohesive and consistent message across all device types. Because ACR data is collected via smart TVs, advertisers are able to gain insight into ad views on linear TV and can use that data to inform granular targeting decisions on digital. Let’s look at a few examples of how advertisers can use ACR to enhance their holistic media strategy.

1. Retarget linear TV audiences across screens
ACR vendors can verify that a user has seen a particular ad on linear TV and use that information to send the user a follow-up message.

ACR Example 1 Linear TV

2. Target non-exposed TV audiences on other devices
ACR vendors can verify that a user has not seen a particular ad on linear TV. Advertisers can extend their reach by targeting those users through additional channels.

ACR Example 2 Additional Devices

3. Conquest audience categories exposed to ads
Advertisers can use ACR to target audiences based on linear ads they’ve been exposed to.

ACR Example 3 Exposed Ads

4. Target show level syndicated audiences
Advertisers can use ACR technology to target viewers of a particular show in a digital environment.

ACR Example 4 Syndicated Audiences

Data providers create segments beginning with the data they collect from ACR and then model out that data to reach an appropriate scale. Because advertisers run campaigns of varying sizes, advertisers should consider widening their audience pool with additional targeting components when building out campaigns to ensure reach.

The Benefits and Downside of ACR[7]

Brands utilize ACR TV for multiple reasons. The most obvious are frequency optimization, unique reach abilities, and improved targeting. Frequency optimization is important for ad efficiency since marketers can’t always tell if their ads are getting optimal display density to reach the needed effect. Reach control helps to ignore households or users that were previously targeted by your ads to augment unique customer reach (or vice versa). Improved targeting refers to the ability to programmatically target specific audiences that consume specific types of media.

The technology is also valuable thanks to its data collection capabilities. Behavioral information tied to IP addresses and/or device MAC (Media Access Control) addresses greatly enhance household targeting. Along with this, ACR data allows us to learn actionable specifics about audiences to better understand how to reach them with ACR marketing campaigns. Through ACR, marketers are able to determine which households were exposed to specific types of content, and therefore advertising messages, on both linear and connected TV.

For brands that are considering automatic content recognition for their digital marketing campaigns, we suggest following one of two strategies: Linear-first or CTV-first. The Linear-first approach is best suited for companies striving to improve incremental unique reach and optimize frequency in the first place. The CTV-oriented approach focuses more on maximizing the spend on a target audience and driving a full-funnel sales strategy.

Naturally, ACR brings tangible benefits to those marketers who utilize it properly and carries real advantages compared to classic data sources. However, this technological approach is not without its flaws. Since smart TVs are currently not as widespread as, say, smartphones, automated content recognition analytics cover only a comparably small share of total US households.

Another limitation comes from the lack of significant personal data to enable precise enough targeting for niche advertisers. It follows that ACR data attribution is more suited for mass products and big brands, rather than startups and market newbies.

Since ACR-related analytics builds on pre-stored databases, this creates an inconvenience for marketing researchers. Say, if we want to dig a few years back to see how a particular campaign performed, we might encounter a dead end. Since data storage isn’t free, the costs associated with that storage rise significantly when it comes to huge video databases. That means that the data needed by the researcher might have been deleted, lowering the accuracy of the analytics.

See Also


  1. Definition - What Does Automatic Content Recognition (ACR) Mean? Gartner
  2. History of Automatic Content Recognition (ACR) [Wikipedia]
  3. How Automatic Content Recognition Works Strategus
  4. What are the different uses of ACR? OTTVerse
  5. What are the Automatic Content Recognition? Wikipedia
  6. How can advertisers benefit from ACR? SpotX
  7. What Benefits does ACR Bring and what are the downsides? The View Point