Not long ago, you could board a flight and the plane would have so many open seats, you could pick a whole row for yourself. Then, even if it were a short journey and you'd just had lunch, you still might enjoy a full meal, compliments of the airline. There was also a decent chance that you bought your flight cheaply. Not coincidentally, the industry wasn't doing so well then. Today, everything from flight capacity to food demand to fuel futures to advance sale behavior is meticulously analyzed, not just because the data is there, but because the data analysis is highly profitable. The airlines now know what flights will fill up fast, instead of whether they might be full.
And it's not just the airlines.
The way you consume and the things you're interested in are recorded every time you swipe your grocery store card, browse products online or make a purchase from Amazon. Data collection is nothing new. There have always been focus groups and questionnaires. What's new is that it's now ubiquitous, a have-to-have for companies both large and small, not a nice-to-have. What's changed is that data has become the automated gathering of real-time behaviors, as opposed to a search for the right random sample. Data is now collected in the cloud and analyzed by everyone from teens to CTOs.
"There is tremendous value in having a dialogue with consumers," says Kimberley Wasiljew (‘08), who works as a strategy and operations manager at Google, "but there are time constraints and money constraints, and you can only talk to so many people at once."
According to a survey conducted by consulting firm Enterprise Management Associates and released in January of 2015, 53.4% of companies polled across the globe had big data projects in operation in 2013, as did 52.1% in 2014. The same survey reported that in 2013, 35.4% of companies were in the process of seriously planning for big data projects, as were 36.5% in 2014.
Data is generally gathered, says Michael Bale ('15), who recently worked as a manager at Netflix, by programs that record a piece of software or a machine's activities and then save that information for perusal in the future. "It's kind of like recording history," Bale says. "So you're just recording what the world looked like at different points in time, or across different dimensions, so you can go back and look at it again and analyze it."
The increased speed of processing and using data have changed the face of business. Quick decisions are no longer made based on instinct alone. Companies can evaluate partnerships with more accuracy. And smaller companies now harness data and utilize it to usurp their larger competitors.
Bale witnessed this phenomenon at Netflix. The entertainment and media company began using data analytics in 2008 to track the content customers were watching and has since provided recommendations for movies and TV shows based on individual consumers' viewing history. The result? Netflix expanded exponentially while rivals folded.
"[Data] allowed Netflix, [which] is a lot smaller than its competitors, to make really informed decisions on the business," Bale says. "They can be really focused and efficient with how they spend their money and make decisions. It can be a differentiator if your competitors aren't really using it."
"It's kind of like recording history," says Bale. "So you're just recording what the world looked like at different points in time so you can go back and look at it again and analyze it."
Data analytics also allowed Netflix to determine how much to invest in original programming — on shows like Orange Is the New Black, for instance, or House of Cards — based on how successful the data suggested they will be.
"You don't spend that much money [on shows] and have no data to back up your decisions," he says. "Those were calculated bets based on what the company saw."
Bale notes that the use of data analytics has also allowed Netflix to make big money decisions as strategically as possible: for instance, knowing how to forecast inventory. He says, "You can triangulate how many copies you need of each title. You have to preorder them, so you have to make multi-million-dollar decisions weeks in advance — and there's a big cost to being inaccurate."
Netflix isn't alone when it comes to translating data analysis to lower costs. John Bixby ('08), a data analyst at Intuit until his recent death in a car accident said, "Intuit has been collecting data going back probably into the '90s and earlier. There's actually a former Intuit analyst who did a lot of work to pioneer analytics at this company, and that was 2000-ish." He further explained that Intuit implemented data analytics in order to test products in the marketplace without having to mass produce them.
"Companies can build a viable product [using] the least amount of stuff, then put their product in front of customers and test and get feedback, before they build something really robust and find out it didn't meet their customers' needs," Bixby says. "That's what we're doing with our products: incremental changes and putting different versions of a product in front of different users simultaneously to find out which performs best."
Because of the way that data is gathered and stored, Bixby says, information about how well products and the company are doing is also available to stakeholders in ways that it never was before.
"Data speaks to what's working and what's not, and what the right course of action might be in certain cases. You can't lie with the data."
"[We use] a data visualization and platform into which you can plug all kinds of data, creating summary reports and trend charts," he says, "and, that way, you can get basic reporting and data into the hands of stakeholders. We can give them nicely groomed data."
While companies such as Intuit and Netflix added data to their operations, for other organizations, big data is at the very heart of their original business plan. Sarah Sanger ('09) is the director of data assets at Elevate Credit, and her company found a way to utilize data to offer loans to individuals who may not qualify for credit otherwise.
"We're lending to the type of people who don't have access to traditional credit," she says. "Because of our use of data, we are able to better underwrite these consumers. Most companies will use traditional credit checks, but that data doesn't often tell the full picture of their identity or willingness to repay. We use a huge variety of different variables to weed out fraud, which enables us to lower default rates and make sure we are making appropriate offers to these consumers."
As companies continue to utilize data analytics to maximize their profits, what many are finding is that, at its core, data's most valuable benefit is eliminating what used to be educated guesswork in business.
"It's made conversations less about who's got the loudest voice in the room," Bixby says. "Data speaks to what's working and what's not, and what the right course of action might be in certain cases. You can't lie with the data. If I have to deliver bad news, people can't get upset. It is what it is, and we have to deal with that and make informed decisions."
"We are able to use a huge variety of different variables and make sure we are making appropriate offers to these consumers."
That's an important distinction to keep in mind. While computers are now being used to make adjustments to business strategy based on certain programmed parameters, the art of human decision-making is still a necessary partner to the science of big data.
"Data has always played a role and will continue to play a role in business," says Kimberley Wasiljew ('08), who works as a strategy and operations manager at Google. "The extent to which different companies use data will vary, but I imagine that data-driven decision-making will only increase as companies gain access to bigger data and more efficient ways of analyzing and visualizing it."
That also means that small businesses that have moved quickly to implement data analytics may only have a leg up on their larger, slower-to-adapt competitors for a short while longer.
"Most successful businesses will have data analytics in the future," Bale says. "Soon, I don't think you will be able to differentiate between companies just on strong data analytics, because everyone is going to have this capability."