Amazon HQ2: a great data collection initiative

I probably should be cautious writing about Amazon, especially as I’m talking to the good folks at Amazon Web Services (AWS) about a job right now. But what strikes me about the whole soon-to-be-done “Headquarters 2” process is the sheer brilliance of the data collection exercise that Amazon has just completed.

In the end, they’re not doing anything revolutionary. They’re going to expand their existing #2 and #3 locations significantly, but they probably would have done that anyway. Being as connected as they are to the media and publishing business will demand more presence in New York, and the AWS business is built — like much of the internet — around a series of hubs in northern Virginia.

The audacious “contest” will, most likely, enable them to extract tax concessions that they wouldn’t have gotten otherwise, though it remains to be seen how this changes when the locations find out that they’re only getting half of what they expected, much of which they might have received anyway. But as this commentator notes, they’re going with the familiar and nobody should have really expected much else from what is now a large and well-run organization.

For myself, there’s no big deal here. If I get through the Amazon interview process and end up working for them, a move east would not bother me. From a family perspective it might be quite nice. In any case, at this point I have no stake in this, though I certainly hope that I might soon.

But it was brilliant!

As a data engineer/scientist I’m always thinking about how to collect data, especially data that people aren’t all that comfortable sharing. Figuring out how to get people to share this is one of the more challenging aspects of data science, behavioral sciences, consumer marketing and many other fields. I ran into this years ago doing my MBA field study project for a Vegas casino. Compared to what could be done with the tools we have today, our efforts seem amateurish. But being literally on the front lines of talking to customers taught me a lot about how hard it can be to get people to open up about such things as how they spend their vacation money, how much they gamble, etc. The mere presence of a spouse or group of friends always seemed to strongly influence the answers, though it was hard to quantify how much, or in what precise way. (As a rule,  men seemed to minimize their tendency to gamble when a spouse was around, and to exaggerate their high-stakes prowess when surrounded by a group of male friends. I wish we had been able to quantify that in some way.)

Amazon received proposals from almost 240 locations. And they received more detailed proposals from 20 more. All those proposals are covered by an NDA. Only the people submitting them and Amazon know how much those 220 + 20 locations are willing to offer in exchange for development commitments.

Which means that Amazon is now better equipped than any company in existence, to optimize the cost of their physical infrastructure around North America. How serious they were about the competition in the first place is impossible to assess , but the result is a trove of data about what concessions they can expect from various locations, and with it the ability to be the best negotiators, the best at trading off one place against another, and the best at pitting rival bids against each other. In a world where information rules, they just collected a whole boat-load of critical information that nobody else has.

They found the perfect way to get virtually every major metropolitan region in the US and Canada to happily give up the information that nobody ever wants to give up.

That’s a brilliant result, even if the public “outcome” is a bit of a yawner.