At its heart, the idea behind “big data” is pretty simple: computers take billions of pieces of information, process them, and churn out insights and connections that were previously unimaginable. It’s like the classic room full of monkeys sitting at typewriters — except, instead of typing Shakespeare, they’re breaking down the chain of events that connect a butterfly flapping its wings in Japan to a monsoon hitting the California coast. And, as the computing power needed to manipulate big data becomes cheaper, its insights are available to more and more business leaders.
But how can you use that data? How could it affect your hiring decisions, or the advertising you buy, or the products you develop? To answer those questions, it helps to look at how big data has transformed the media industry, and how it’s affecting several top companies.
The Deep End of Data
Up until the mid-2000s, media consumption measurement tools were rare and fairly inaccurate. A newspaper writer interested in the reach of his or her story basically had one clear metric: how many copies the newspaper sold. But just because a newspaper sold didn’t mean that it was read, or that a particular story resonated with readers. Maybe the writer might see his or her work quoted or overhear someone discussing it, but he or she had no clear, scientific way to measure audience engagement.
TV analysis was a little better: the Nielsen television ratings system, developed in the 1950’s, electronically monitors the viewing habits of a selected group of households — 40,000 as of 2016 — and extrapolates the data to make conclusions about viewership. Using it, TV writers and producers could immediately track the popularity and impact of their work, enabling them to more accurately tailor it to their audience. This breakthrough led to the golden days of “event TV” or “Must See TV,” when tens of millions of families switched on their TVs at the same time to watch the last episode of MASH or find out who shot JR on Dallas.
With the rise of VCRs and — later — Tivo, streaming, and other technologies, it wasn’t necessary to tune in at a specific time or on a specific device—a factor that made viewership a lot harder to track. “Back in the day, you’d run home to watch MASH, but today, consumers are coming in through a variety of devices and mechanisms,” says Jane Crisan, President of R2C, a data-driven creative and media agency. “If your goal was perfection in the measurement of all those, you literally could not do it.”
Today, writers and producers are drowning in data. In addition to traditional sources, like Nielson, they can draw meaningful conclusions from social media postings on Twitter and Facebook, as well as mentions on other sites. And then, of course, there is direct data collection, which makes it possible to track how many people visit your site, how long they spend there, where they are coming from, and where they are going next. For streaming video sites, it’s possible to track when viewers turn on a program, when they pause it, when they turn it off, what device they’re watching it on, and hundreds of other metrics.
Using Data to Decide Features
In this sea of information, the problem isn’t finding a metric, it’s finding the right metric, as well as the best question to ask. Are you trying to convince your customers to buy a product? Call a number? Go to a site? The answer will drive all your other decisions.
In the case of online streaming services, the bottom line is time: the more hours that subscribers spend tuned in to Hulu or Starz or Netflix, the lower the chance that they will cancel. To put this in actual numbers, Hulu’s 12 million subscribers watch, on average, 2.9 hours of programming per day, compared with 2.2 hours on Netflix, 2.1 hours on YouTube, and 2 hours on Amazon video. That basic metric — number of hours watched — has steadily risen over the past few years, in large part due to big data.
One way that data has changed the streaming video experience is autoplay. Analytics showed that customers often leave after a show is over, so many streaming sites began to automatically start a new episode of a show as soon as the last one ended. For binge watchers, autoplay eased the transition from episode to episode. And, once you’ve finished watching a show, it tees up another program that’s been selected to appeal to your tastes. If all goes well, the binge process repeats itself, you stay glued to the screen, and the number of hours that you watch rises.
The impact of data on metrics and goals extends far beyond streaming services. For example, instead of following traditional goals like “capturing eyeballs,” or “buying an audience,” data can help companies focus on more specific aims, like making sales. “There isn’t always a correlation between capturing an audience and capturing sales,” Crisan says. “We now know what is really driving sales, because we’re really getting down to that granular data about who is buying and which shows are driving the growth and sales.”
Using Data for Creative Decisions
Big data can also drive creative decisions. Unlike the heyday of newspapers, it’s now easy to learn how many people read a particular story, and which factors drove them to it.
On a simple level, news sites like AOL, HuffingtonPost, and Business Insider can use data to drive aesthetic choices, like webpage design and headlines. On a deeper level, though, data can also drive which stories a website promotes from partner sites, and even which stories it commissions. After all, for editors interested in driving traffic, knowing what readers want to see can make the difference between a Kardashian expose that is read by millions and a story on gerrymandering that only draws a few thousand eyeballs.
The value of big data-driven content is even more obvious at studios and streaming sites, where buying rights to a movie or commissioning a series can be a matter of millions of dollars. Historically, TV has used a winnowing process, in which a flood of new TV shows at the beginning of every season would slow to a trickle as roughly 70 % of them failed to find an audience. Even today, with the improved metrics available to TV producers, 50 % of new shows on network TV don’t make it past their first season. By comparison, Netflix claims that 93 % of its original shows are renewed, and other streaming sites, like Hulu, have canceled only a handful of their original programs.
The difference is data. On an individual level, this influences which movie Hulu suggests for you, or which Amazon movie shows up in your feed. On a larger level, viewer data can be rolled up into aggregate analyses that reveal how an audience will respond to a show — or even to a particular ad. This recently played out in The Defenders, Netflix’s series featuring four b-level superheroes, each of whom had already had their own show. Analytics revealed several overlapping fan groups: for example, fans of Grace and Frankie also gravitate toward Iron Fist, while fans of Stranger Things gravitate toward Luke Cage. After The Defenders aired, the company was able to use viewing data to determine how to make those overlaps translate into actual sales. Or, to put it another way, they explored how they could convince a Grace and Frankie fan to tune into The Defenders to watch Iron Fist.
In some ways, the show functioned both as a product and as a litmus test to determine which future products will be greenlit, and the tools that will be used to market them.
Crisan notes that this use of aggregate data is transforming traditional ways of buying and selling advertising. “Historically, media was purchased in upfront packages,” she says. “You’re given a big plan and you take inventory, not knowing if it works or not.”
Now, however, companies can directly view the specific impact of every ad on every show, both in how it drives web traffic and even in how it drives sales. “You can pull in a customer’s web activity and their TV airings and marry the two on top of each other,” Crisan says. “We can see a spike in response when we aired a TV commercial during a program and we literally can see a spike in an advertiser’s web activity.”
Using this information, companies can choose the specific ad buys that will yield the biggest reward. “We have the ability to go into those packages and say I don’t want those five shows, they’re not driving sales,” Crisan says. This can lead to some unexpected conclusions. For example, she notes, commercials that air during children’s programs might have a lot of sway over 35-50 year old female consumers. By purchasing commercials during somewhat unlikely programs, companies can tap unexpected pools of customers — and maximize the impact of their ad budget.
Hire the Right People
Regardless of what a company does, big data conclusions — specifically, a deeper understanding of information around audience, advertising, and products — are an increasingly vital part of the business landscape.
As such, a deep understanding of data analytics is becoming more important for all employees across a company. “It’s getting deeper and more data-centric, and some of the people that can’t play in that space are finding themselves at a loss,” says Crisan.
For many companies, this means bringing data analytics in-house, where the people who know how to read and use data can work more directly with the people who design products and ads. “Gone are the days when you necessarily just let your agency have all the intellectual property around that,” says Crisan. “We’re seeing a movement towards bringing expertise on the media or the analytics side in-house”
In the case of big companies, this translates into hundreds of job postings for data analysts, engineers and scientists. But small and medium businesses hardly have to go to those lengths for the expertise they need — at least in the beginning. There are numerous analytics programs that provide real-time information about how customers are engaging with a product, as well as broader conclusions about where they’re are coming from, why they’re leaving, and what needs to be done to engage and retain them. With the right analytics and a little creativity, even the smallest company can start getting that roomful of monkeys working at their typewriters.
Bruce Watson is a New York-based writer and editor. His work has appeared in the Guardian, Esquire, the New York Times, and several other publications.