Big Data and The Benefits From The Bubble

A recent article in Bloomberg Businessweek worries that a new bubble is forming in Silicon Valley, and that this one will be different and worse than the ones that came before. The argument presented by author Ashlee Vance is that while previous tech industry bubbles, like the one that burst in the spring of 2000 and turned dot-com into dot-bomb, left behind important innovations, this one won’t.

This one – let’s call it Bubble 2.0 – consists mainly of companies using the Net to market, sell, and advertise, and the only innovations they’re coming up with are better ways to get us to click and/or buy. As former Facebook employee Jeff Hammerbacher says, “The best minds of my generation are thinking about how to make people click ads. That sucks.”

If that were really all they were ever going to do, I’d agree about the suckiness. But it’s not. When I look at the valuations and sale prices of many tech companies these days, it does seem like Bubble 2.0 is a reality. If so, it’ll burst, and leave some investors poorer and some people without jobs. But it will also leave behind technical and intellectual advances that are powerful, beneficial, and broadly applicable.

As Vance nicely summarizes, the core of Bubble 2.0 is a set of companies, technologies, and specialists that “poke around in data, hunt for trends, and figure out formulas that will put the right ad in front of the right person.” Of course, the first two of these activities have been going on as long as there have been geeks (which is to say, as long as there have been people). So what’s new now? The amount of data available, and the power of the digital tools — databases, memory, processors — available (at ever-lower price points) to poke around in it and hunt for trends.

Today’s environment is one of ‘big data.’ And it’s an absolute playground for statisticians, algorithmicists, and other applied math whizzes. These folk get to flex their mental muscles, identify patterns, make predictions, test hypotheses, and so on in ways that were just not possible when digital data were more scarce and computers had less horsepower. As a result, their skills are now highly valued, and they’re commanding lots of attention, good jobs, and high salaries.

Applied math empiricists are taking over the prestige from their pure math theoretical cousins, which is great news for those of us who value evidence over ideology. I have posted on my real-world wall Vaclav Havel’s advice to “Seek the company of those who search for truth; run from those who have found it.” In the era of big data, that advice is more valuable than ever, and also easier to follow.

So maybe it’s true that too many applied math whizzes are currently working on ever-better ways to get us to click ads. But so what? After Bubble 2.0 bursts, they and their tools will be redeployed to many other purposes. To see how beneficial this will be, go back to Vance’s sentence above and replace “figure out formulas that will put the right ad in front of the right person” with phrases like

“determine how proteins fold into their final knotty forms”

“learn which genetic alleles are most closely associated with different diseases”

“map how and when humans spread out around the world from our African birthplace”

“estimate how quickly irregular English verbs become regular over time”

“specify how much more censorship and propaganda exists in totalitarian regimes compared to democratic ones”

“learn which employees are most likely to be stealing, and how they can be persuaded to stop”

Wouldn’t it be cool if the people and machines of big data could be applied to do these things? Well, they already are; I cherry-picked the list above to include only areas of current study. The list is only going to get longer over time, and will get longer quicker once Bubble 2.0 bursts.

Big data is a big part of Bubble 2.0, but the former won’t end when the latter does. So I’m not that bothered that right now a lot of really smart people and powerful computers are working to do nothing more exalted than getting me to click somewhere. They’ll be doing other things soon enough — things that will greatly help our economy, society, and understanding of the world.

Do you agree? Or do you take a dimmer view of big data and/or Bubble 2.0? Leave a comment, please, and let us know.