Big data is playing an increasing role in people’s lives.
Loosely defined as data that is too massive to be contained or processed by any one machine or person, it includes information on individuals’ Facebook likes, supermarket loyalty cards, and other seemingly innocuous personal information that was used by both the Trump and Brexit campaigns to reach voters who wouldn’t normally disclose their political opinions.
Finding out more about people
Apps created by researchers in Britain and the United States can guess how old a person is, their IQ, and to whom they are sexually attracted.
“There’s lots of different sources for sentiment data,” said John Kreisa, a London-based executive at Hortonworks, a California software company. “One probably very obvious one is things like Twitter. Social media in general is a way that people express themselves and express a like or dislike, sentiments obviously positive or negative,” Kreisa said.
By themselves, the trillions of bits of information would amount to a pile of worthless junk. Add the power of the human mind, and it is a different story.
When Julian Dailly graduated from the University of Sussex with a degree in English and philosophy and faced a world dominated by machines and technology, he wondered how he was going to make a living. Now, he is part of a mega industry that, considering it includes Google and Facebook, is so large no one seems able to estimate its worth.
Predicting what people will do
Dailly’s research company, Morar Consulting, started three years ago with five employees. With annual revenues up by 25 percent each year, Dailly’s firm now has a staff of 90.
“What we ultimately do here is we try to discover what’s meaningful for people, and we correlate that to their economic behavior.” Dailly said.
Diversity, he adds, is important.
“There’s always the risk I think if you have people from too similar backgrounds that you end up with a bit of group think. You don’t consider all the alternatives, and you maybe take too many bets on the same thing,” Dailly said. “We have people from traditional research backgrounds, in addition to some in social sciences, economics, people from tech backgrounds, and sales people.”
Companies like Dailly’s also draw recent college graduates who bring perspectives of youth to the industry.
New forms of data also make it possible for analysts to predict the future. They are thus more valuable to companies and campaigns than the traditional forms of recordkeeping, recording or reporting data.
“We have access to the core information inside people’s heads,” Dailly said. “They tell you what people are going to do as opposed to what they’ve done. That helps people take preemptive action. This makes it much more useful for strategy.”
Brexit, Trump and Clinton used this new data
Both the Clinton and Trump campaigns used big data, with Trump hiring the London-based Cambridge Analytica. The company employs a trademark method known as “psychographs” that uses psychological profiling to influence consumers. The company also provided its services to the Brexit campaign.
Both the Trump and the Brexit campaigns faced charges of xenophobia, a topic many people preferred not to discuss.
For both Trump and Brexit, polls had predicted Election Day losses. But their victories hinged on voters who had remained quiet during the campaign, often declining to publicly state their opinions out of concern of being labeled as racist.
Big Data beat the polls
Polls did not tell the truth, but big data did.
“You just rely on data that you collect at the polls, you’re not going to be able to gather the same amount of data, but also people might lie or might not reveal whom they will vote for or what way they are going to vote,” said Tamara Chehayeb Makarem, a user experience designer at Scott Logic, a British software development consultancy.
“Using big data, you would be able to generate certain patterns, and what people like,” she said. “Let’s say what you like something on Facebook, if you publish certain articles, follow certain people. That could give them (campaign strategists) an indicator about your views, and based on that they could get a better indication about how likely you are to vote for someone or something.”
The prospect of having companies read minds is spooky, and there has been pushback. Facebook has blocked the use of much of its content and the European Union has enacted some of the world’s toughest privacy protections.
Many uses for Big Data
Proponents of the industry are eager to show that big data is a force for good.
Laurie Miles, director of analytics at SAS UK, an analytics company whose clients include HSBC. He says the ability to capture and process data in real time is crucial for protecting credit card users.
“You go into a shop, you swipe your card,” she said. “In real time, it’s determined whether that transaction is likely to be fraudulent or not.”
Dailly, of Morar Consulting, discounts concerns that computers have finally taken over.
“There is a moment of suspension of disbelief when we allow ourselves to believe that humans will be allowed to be replaced by machines. I think it’s a fantasy,” he said.
Artificial intelligence will evolve and so will their autonomy to make decisions, he said.
“But fundamentally, they will always be plugged into the wall. They can be turned off. With that in mind, humans will still remain in control.”
