This is a continuation of the article " Secrets of Googlonomics ".
The self-focus on engineering moments, mathematical formulas and data-mining has made Google a whole new kind of company. But to fully understand why, it’s worth going back a bit and looking under the roof of AdWords.
Most people think that the Google ad auction is a very "straightforward" tool. But in fact, there is another key point that not only a minority of users of the system itself know about, but not even all experienced ad people have any idea that it exists. Bidding itself is only part of the system that determines the winner of an auction. Perhaps the most important factor determining the winner of an auction is the quality score. This value is used to determine whether the ads shown to the users exactly match the descriptions and requirements for them, i.e. to roughly define the "truth" of the advertising message. For if not, the whole system suffers and the company ends up getting less money.
The quality score is calculated based on a number of other indicators, including the relevance of the ad to the keywords, the quality of the page to which the link leads, and above all the percentage of unique clicks on the ad unit when it appears on the final page (obviously, there are other factors, but Google will never discuss them for obvious reasons). In addition, there is also a mechanism for "punishing" advertisers if the quality of their ads is below average – in such cases, the company automatically assigns the advertiser a minimum rate. Google explains that such tactics, brought to the court of many companies and individuals who are automatically "underbid", to protect users from the irrelevant or annoying advertising, which defame the name of the contextual advertising or paid links in general. There have even already been several court cases from individuals who claim to have been victims of arbitrary quasi-monopoly.
And if we can argue about the fairness of the algorithm, then it is possible to call it "arbitrary" with a great stretch. In order to set the "quality indicator", the company sometimes has to calculate how many users will click on the ad ahead of time. This is a very subtle point, especially when we’re talking about billions of mini-auctions. But since the entire model depends on accurate click prediction, Google has to collect and process every "curl" of information as perfectly as possible. Susan Wojcicki, who oversees all of the company’s advertising business, calls it "click physics."
When Hal Varian spent his second summer at Mountain View, he still only came in once or twice a week. That’s when another turning point occurred – Mr. Varian asked a newly hired Stanford computer scientist, Diane Tang, to create a CPI (Consumer Price Index – Consumer Price Index ), which they called the Keyword Price Index (KPI). "Instead of a basket of commodities like beer and donuts, we have keywords in it, " says Tang, who is known internally as the "Click Queen."
KPI is a kind of reality and compliance check. This metric alerts Google if there are abnormal "price bubbles" in the system, a sure sign that the auction is not working as it should. Categories are determined by the cost per click an advertiser usually has to pay, and are ranked by their distribution, and then divided into three branches : top, middle and bottom. "The top branch is very popular words like ‘hotel’ or ‘flowers, ‘" Tang explains, "In reality, in the middle branch we have words whose popularity can be seasonal, so for example ‘snowboarding’ is unlikely to be popular at the end of the season. Well, and the lower branch is rarely used keywords and phrases, such as "Chelyabinsk Metallurgical Plant.
The index invented by Diane Tang is not a bad example of how, with a lot of information, you can create effective mechanisms to improve and regulate the system itself. In addition, it is obvious that such tools must exist and function in order for AdWords to work and bring in money.
"The people working with me are mostly a mix between economists and those interested in statistics, " says Varian, who finally moved to Google permanently in 2007 and now leads two groups, one of which focuses closely on analysis.
"Google needs mathematical formulas and algorithms to build tools to find signals in noise, " are the words of one statistician, Daryl Pregibon, who joined the company in 2003 after 23 jobs as lead scientist at Bell Labs and ATT Labs. "Our rule is simple — one statistician per 100 computer scientists."
Keywords and click ratings are the bread and butter of the system. "We try to understand our own mechanisms not in terms of numbers and indicators, but in general and general terms, like a living organism, " notes Qing Wu, one of Varian’s handmaidens. Wu specializes in forecasting, so he has to deal with predicting how business will change depending on the season, climate, international holidays, even the time of day. "We have information on temperature, weather and requests, so we can do statistical modeling and adjust that result during different changes. All these results are delivered in a digestible form to the other end of the system, the advertiser, which allows us to manage our own advertising more effectively.
In order to track and test forecasts, Wu and his colleagues use many different spreadsheets that receive real-time information. A kind of Bloomberg terminal analogue for the Googleosphere. Wu constantly checks the results to compare the accuracy of the prediction with reality : "It’s a handy tool to see not only the underside of the queries, but also how much money and advertisers we have, what keywords are popular at the current time and what the ‘return’ rate is for each of the advertisers."
Wu calls the company "the barometer of the whole world." Indeed, studying clicks is in some ways similar to looking out an imaginary window from which one can see the entire planet. We can trace the changing seasons – the clicks react to the change toward skiing and warm clothing during the cold season, while also showing how popular bikinis are in the summer. This is also how we see, for example, who the most popular pop culture people are. Many of us remember some key events from newspapers and television, while Google displays all this information in the form of peaks in various charts. "One of the biggest takeaways happened a few years ago during the SARS epidemic. " says Tang. But King Wu didn’t even have to read the newspapers to learn about the financial crisis – he saw an extraordinary jump in search queries for the word "gold. From all this you can see how important forecasts and analysis of information are, every drop of which we can see makes a difference.
After Google hired Varian, many other companies, like Yahoo, also decided they needed a head of economics to oversee a team doing thorough work related to auctions, advertising and economic models in order to lick their businesses to a shine. In 2007, another Harvard economist, Susan Athey, was quite surprised to receive an invitation for a face-to-face meeting with Steve Ballmer in Redmond. "It’s not just a call, it’s a challenge you take, " recalls Susan, who had worked in Microsoft’s Cambridge and Massachusetts offices in previous years.
So, it turns out the rest of the world is left somewhere behind? Eric Schmidt thinks that if it’s going to happen, it’s not going to happen as fast as many people think. And he also thinks that auctions like the one Google invented can be used for any type of transaction. What’s the solution to the oversaturation of the car market? Put the entire spectrum of unsold cars on auction, and it will be cleared out in record time. The situation is the same with the real estate market: "Now people go to auction when they are dissatisfied, when there are almost no other options to sell something. But imagine a situation where this approach would be normal and quite natural for most people, "- said Schmidt.
Hal Varian believes that a new era is just around the corner, which he himself calls "informational assertion"-its essence being the mastery of supply and demand. "What’s convenient and cheap?" – Varian asks and immediately gives the answer: "Information." So what is missing to make it all emerge now? The analytical ability to analyze this very information en masse. Hal Varian believes that there will come a time when someone with a technical background or an aptitude for engineering will work for hedge funds on Wall Street and make daring decisions based on analyzing information and statistics, with unique results.
Perhaps this approach is indeed satisfactory for Hal Varian, who makes no secret of the fact that he chose a career in economics inspired by science fiction : "In Isaac Asimov’s first book of The Founding trilogy there was a character who was building mathematical models of society and I thought it was really entertaining. When I went to college, I looked everywhere for something that would allow me to do the same thing. That turned out to be economics." Hal Varian tells this story sitting at his ranch, where he sometimes stays to avoid the 40-mile trek from company headquarters to his home. The ranch is now owned by Google, which is where Brin and Page started the company.
The wild contrast between how Spartan the house is furnished and what has happened as a result of its existence-the emergence of dozens of geek millionaires on the planet, as well as the new rules of doing business in the information world-is impressive. And perhaps this contrast is far more surprising than what Asimov wrote some 60 years ago. What could be more surprising than a major company operating in a capitalist world and, at the same time, giving its best services for free to anyone who wants them? Varian, of course, does not suffer from a God complex and is well aware that he owes his success not to inspirational madness, but to an early identification of the Internet’s potential and a focus on speed, volume, analysis, statistics and customer satisfaction. A pinch of auction theory doesn’t get in the way of general taste. And today there is a word to describe this whole dish : Googlonomics. You have to learn it, or pay the price and still weave in at the end.
Source of the English text : Wired Magazine
Secrets of Googlonomics, Part 2
This is a continuation of the article " Secrets of Googlonomics ".