"I was thinking about you all day today and what a great
person you are."
"I wanted to be #1...After 2 months I reached the top position for my most popular keywords."
Above The Fold!
Matt Cutts Interview
9 October 2005
How and when did you get into search?
I got into search in 1999. I was working on my Ph.D. at the University of North Carolina at Chapel Hill (go Tarheels!). The computer science department there makes you take two outside classes, so I took classes from the Information and Library Science department about search engines. It turned out that search engines were really exciting stuff: term frequency times inverse document frequency and all that stuff. During the whole class, I found myself scribbling fun ideas to try in the margins of my notes. I was hooked.
Is all SEO spam?
Personally, I have a theory that some spammers want everyone practicing solid, common-sense SEO to believe that they're spamming, or that everyone is a black hat. To me, that fails at first blush; was I a spammer for changing the word "howto" on a page to "how to," because more people type it the second way? Of course not. I have a similar theory about doing spam reports--why would someone who claims that they will use any trick in the book whatsoever then turn around and try to convince people not to do spam reports? Those two beliefs ("I'll do anything to get ahead in rankings" and "Don't report a competing site that is using spam tactics") aren't especially consistent. There's a lot more nuance that I'd like to go into sometime, but I'll throw those points out there for people to discuss. :)
What do you think of tomatoes? Have you ever been spammed by a waiter offering them?
Your readers will need some background on this one. :) The last time Aaron and I ate together, our table received unsolicited tomatoes. It was down at the WMW Pubcon in New Orleans, and it turns out that a couple webmasters sent the tomato dish over as an appetizer. It was very thoughtful, but a little surreal. One quick thing I'd like to mention: if you're sending a reinclusion request to Google, please don't send us cookies, pens, chocolate, etc. It doesn't happen often, and we'd rather not get them. We don't want any appearance of doing a favor for a gift, plus the "do you think these cookies are safe to eat?" jokes get old after a while. :)
Some SEO firms cold call saying they can rank people in first place. Can they guarantee this?
Not on Google. No one can guarantee this, not even Google, since our ranking algorithms are often updated. I've seen scams where the "#1 placement" is really buying ads. I've seen scams where the "keywords" that they sell are really for people who have scumware hidden in their browser. I've seen stuff where the guaranteed keywords are 5-6 word phrases that only have nine results, and no one would ever really type that really long, specific phrase. I've seen situations where the guarantee is that they'll try to get a #1 spot, and if they don't, then they'll try again. Cold calling and cold emailing is a bad sign in the first place, frankly. So I'd be skeptical of that in the extreme, and read the fine print carefully.
As a search engineer who knows search algorithms inside out, what would you personally look for when hiring an SEO?
References, probably. Proven success with clients who are willing to show their results. An emphasis on crawlability/site architecture is a plus. Experience with both SEO and PPC. Good industry knowledge and contacts. Probably a track record with at least one medium to large company, to show that they can work with marketing/PR/corpcomm as well. Creativity in terms of finding a hook that propelled one of their other clients in terms of buzz or blogosphere attention. If they've been around for a while, that's a plus. They should be willing to explain what they do, and make the explanation clear enough that you can follow it. There are a lot of great SEOs out there, but if I were hiring one for my most important site, I'd be pretty conservative.
Do you think there will ever become a time when Google will recommend certain SEO services or products outside of things related to AdWords?
It wouldn't surprise me if it happened some day, but probably not really soon. One thing about being a certified Google Advertising Professional (GAP) is that we ask you to follow Google's quality guidelines in regular SEO practices. If you look at the GAP program, I think that's worked out pretty well so far. But because of the difficulty of knowing that a specific person/company is abiding by those guidelines, I wouldn't expect it in the short-term.
When people aim to extract many alternate meanings out of your words, and everyone and their dog has a blog about search or Google how do you interact with the public so much without it backfiring often?
It's true that I think more these days about how someone could interpret what I'm saying. I've learned that it's usually a bad idea to talk/post when I'm angry, and that making promises about the future (e.g. when something might or might not launch) is dumb. Sometimes I still do it, but at least I know when I'm doing something stupid. I like having a blog so that if the Red Herring picks a quote of mine that sounds weird, I can give more background.
What is your favorite spider?
Googlebot is a fine little fellow, but my sentimental favorite was probably Grub (now a part of LookSmart). It was a distributed crawler that let anyone run a client and crawl pages. I thought that was a neat idea. Very spammable in some way unless you're careful, but a neat idea nonetheless. But if you pay attention to bandwidth pricing, a distributed crawl causes a few extra headaches that probably aren't worth it. If you're talking non-computer spiders, I'd have to go with Anansi (with a shout out to Neil Gaiman for stopping by Google :)
What do you think of Big.com?
I think it's a cool idea that doesn't take a lot of work to implement. Kudos to them. In terms of viral buzz and creativity, I have to give props to http://www.milliondollarhomepage.com/ though. The laptop I'm typing this on has 1.47M pixels, but it never occurred to me to sell the pixels for $1 apiece in blocks of 100. Of course, I hope that the people buying pixels don't assume that those pixels will flow PageRank. :)
As a person who probably gets hours of email every day what tips do you have for keeping a tidy inbox and fast email processing?
Ach. I could still use some help keeping my email under control. I still use mutt--I haven't made the webmail leap yet. A few things that help me:
- aggressively procmail/filter out mailing lists and stuff you don't need to reply to
- try to spend blocks of time away from email or you'll spend the whole day responding to it
- make sure your mail client can show you threads on the same topic
- I color-code my emails:
- white: to me, and from a Googler
- green: cc'ed to me, and from a Googler
- blue: I'm nowhere in the to/cc lines, and from a Googler
- yellow: from outside Google
- That lets me prioritize a little better.
Mike Grehan wrote an article about the rich get richer effect of sites ranking at the top of the search results. Does Google counteract this? If yes, how so?
We try to. A non-trivial fraction of our time is spent thinking about how to return the right sites for queries, and small or mom-and-pop sites are a large part of that. It's a real challenge when some of these sites have one or two (or zero!) links to them. At this point, when I go on a trip or visit a street arts fair, I end up writing down 3-4 urls each time, so I can go back to Google and see whether the sites will come up in Google. I carry a hipster PDA (a little notebook from Office Max that costs like $1.09) so that I can write down stuff when I'm away from a computer.
If search engines give people whatever they want, does the effect of acting as an oracle of information hurt society as much as it helps it, since search services passively (and perhaps even actively as search personalization improves) reinforce our worldviews?
As far as self-reinforcement, I think that Google News does a good job of showing opinions from a diversity of sources. And I do think diversity can be important in search results as well. But I think the larger question is not just worldview-reinforcement, but what people do with knowledge. Do a Google search for [computer scientist as toolsmith]. I was lucky enough to spend some time with Fred Brooks at the University of North Carolina at Chapel Hill, and my take on this was strongly influenced by his keynote at SIGGRAPH several years ago. If you enjoyed his book The Mythical Man Month, you should read the article [PDF]. One notable quote is "a toolmaker succeeds as, and only as, the users of his tool succeed with his aid." I think you have to have a fundamental belief that people will benefit from the ability to amplify their intelligence with better access to information.
When you guys change your business models or market focus it has a profound effect on what publishing business models are functional. What are the biggest changes and surprises to the structure of the web you have seen due to changes in your business model or algorithms?
I think someone who has a niche hobby has a better chance of being able to write or blog about that hobby, which is a big change from a few years ago. That's not as much of a surprise--the hope was that AdSense would allow good content to be created and monetized. What recently surprised me was the "13 startups in 90 minutes" panel at Web 2.0. I was struck by how many tiny companies were able to put AdSense on a site to defray their costs. Let me see if I get the list right. Zvents.com had AdSense. RealTravel had AdSense. I believe Wink had AdSense. Probably 1-2 more. It was a huge surprise to me, and I think it's fantastic. If AdSense means that more startups can come to market, that's great for everybody.
When you guys roll out new algorithms, filters, and patches some good sites end up getting filtered out with the bad. Do you pre-test most of the algorithms prior to launching them? How do you know how strongly to apply filters? By default do you usually lean on one side or the other and then tweak your way back?
We always put algorithmic changes into our test harnesses to poke and prod in lots of different ways. But you also have to be adaptive. If someone in the outside world notices an issue after a launch that you didn't notice, it's important to take that feedback and act on it, and also to try to improve the testing procedure to cover that in the future. We usually have a pretty strong sense of whether something will be a large-impact launch or not. But you can't completely avoid having a large impact with a launch. An example might be if you're replacing a large subsystem in the crawl-index-serve pipeline. We continually go back and improve or replace sections of our system. Sometimes the results can't be bit-for-bit compatible in output, so you have to do the best you can. Update Fritz in 2003 is the canonical example of that; you can't go from a batch-based search engine to an incrementally-updated search engine without some visible impact. To answer your last question, I personally lean toward softer launches; webmasters never need any extra stress. But sometimes launches can't be made completely soft or invisible, as I mentioned.
As a webmaster how do I minimize the chances that my site may be the baby thrown out with the bathwater in the updates?
Make a great site, and try to make sure that site is recognized and thus earns organic links. To be safe, pick a stranger and ask them whether the site is great--sometimes you'll be surprised. If Google doesn't return a reputable, great site for a query, then we're going to be working to figure out why and fix it.
Do you like volley ball?
I do! When I have the free time, I usually play roller hockey though. A friend talked me into wearing a pedometer, and I want to dunk that thing into the sink sometimes. 10K steps a day is hard for a computer geek without much free time.
Graph theory vs social networking vs buzz marketing: which of them is most important for a new webmaster to study? What resources would you recommend for learning about each of them?
I'd study buzz marketing. If you capture the fancy of the web, you won't need to worry about graph theory--you'll get links on your own. Plus, once you know what a clique is in graph theory, you can never go back. Instead of asking "does this link make sense for my users?" you'll be wondering "Am I too close to a clique?" and that's just not healthy. :) Other people could provide better resources than me, but The Tipping Point and Freakonomics are good reads.
If a person wanted to learn IR what books and papers would you consider vital?
I would read:
- Modern Information Retrieval
- Managing Gigabytes
- Readings in Information Retrieval
- plus the Google papers on PageRank, The Anatomy of a Large-Scale Hypertextual Web Search Engine [PDF], Mapreduce, Digital Image Processing and GFS.
but mainly: just freaking write one. Get a CD with SIGGRAPH papers or IEEE papers or pull down the ODP or whatever, convert the PDFs to text, process the documents to compute term frequency for each word, then try to write a scorer. Google provided a CD with a ton of .edu pages a while back for one of our programming contests. Find that CD and write a search engine. Start small if you need to, but write some kind of search and try it and use it and tweak it when it doesn't work, and in a month or two you'll be a an on-page zen scorer and know how to write relevant documents. Then take another month or two and write a crawler. It's hard, but it's not magic and you'll learn a ton.
In "The Anatomy of a Large-Scale Hypertextual Web Search Engine" it was written, "We expect that advertising funded search engines will be inherently biased towards the advertisers and away from the needs of the consumers." What have you done to ensure you minimize advertiser bias?
We make sure that our ads are clearly marked and separate from our search results. We don't allow any pay-for-inclusion in our organic results. And if an advertiser is spamming our index, we still take action.
I originally started a service-selling site and then later sort of stopped selling services and started blogging. An interview with you will likely go on that static site and is likely to be a well linked page, but my blog is by far the more popular of the two sites. Am I effectively hurting my end rankings by splitting up the content?
I don't think that's ultimately hurting your end rankings much. There's always going to be people who do some attribution by linking to your main seobook.com page, and some people that link directly to a post or to your blog--and some people that do both, because they're different urls. So having a primary service that's off the main page can sometimes even help a little bit. I think you'll get the same total number of links (or even a little more if they link to the root and the blog or specific post). After that, it's up to you how to handle that with internal linkage.
What would be the best ways to integrate the link popularity?
I think having a main site with a large feature like a blog somewhere near the main page is actually a pretty good structure. If you run a blog, it's good to spend some effort to have one main url for each post so that there's a single well-known permalink. I haven't been as nitpicky about that on my own site, but if you do SEO for a living I'd pay a little more attention to that.
When does it make sense to use one site or multiple domains?
If you're a whitehat, I'd almost always go for one site. Promoting multiple sites and keeping them distinct is a lot more of a challenge.
In the past I teased you about creating a Google hedge fund, using search demand related AI to automate stock, bond, & commodity orders. What prevents you guys from doing something like that?
If search becomes a nearly winner-take-all game, and Google out executes its competitors, how can you prevent that market position from eventually causing bad things to happen years and years after you, Mr Page, Mr Brin, & Mr Schmidt are no longer around?
I think you have to get Google's core values deep into our corporate DNA. You have to make sure each new Googler understands the importance of our users' trust. And ideally the company can make public promises (e.g. Google does not do pop-up ads) which act as guides for future behavior. And you need to listen to your users. I hope we get even better at that last one; handling the scale of communications is hard but important.
From your mission page: "Google's mission is to organize the world's information and make it universally accessible and useful." You guys have already helped make storage free and fairly limitless in size (Gmail & Blogger). Do you think Google will eventually be able to provide free access to information (ie: like a free ISP)?
I have no clue. I just try to make search quality better. You'd have set up an interview with some kind of product manager-ish person, not an engineer. :)
If I bought a domain that was already listed in DMOZ and looked like it had a clean history but found that it was banned from Google what should I do?
File a reinclusion request. DMOZ does a pretty fair job of pruning stale domains in lots of instances.
If I got a site banned what is the procedure to get it re indexed?
This is boilerplate that we're sending out to some site owners as a pilot program if we detect spam, but it's the most current info:
"If you wish to be reincluded, please correct or remove all pages that are outside our quality guidelines. When you are ready, please submit a reinclusion request at http://www.google.com/support/bin/request.py
You can select "I'm a webmaster inquiring about my website" and then "Why my site disappeared from the search results or dropped in ranking," click Continue, and then make sure to type "Reinclusion Request" in the Subject: line of the resulting form."
If that procedure changes, I'll blog it.
Do most sites that ask for a reinclusion get reincluded? If not, what are the common problems?
We don't discuss the reinclusion ratio, but if you're a mom/pop site with a single domain compared to an SEO site that had industrial-strength spam, I would request the reinclusion. Check your own site for spam before you request reinclusion! Look for hidden text on the home page. Do a site: query and check a few random pages. If you were doing a weird link scheme, stop it. That's the biggest problem we run into--the site isn't really cleaned up.
Eventually there will come a time when SEO will be too hard to attempt to teach and traditional SEO will likely no longer be a viable and profitable business model for many SEOs. Knowing what you know about SEOs, how would you suggest they could leverage their experience into other fields? What other fields do you think would be good for SEOs to go into?
If you're creative, I'd look at the marketing/buzz aspect of things. A person who is savvy about marketing will often have a good leg up on interactions with people. If you are a talented backend person, there's a ton of neat start-ups right now. 2-3 people in different places can collaborate on some nice stuff. If you're a button-pusher, I'd try to diversify that skillset. ;)
What is your favorite Stones song?
Probably "Wild Horses." But the interesting thing is that I'm not really that much of a Stones fan. When they start playing The Who songs at the intro to CSI and CSI: Miami--that's some good stuff.
What does it feel like being the Mick Jagger of Search?
Honestly, there's a ton of people at Google who work to improve our quality and tackle spam, so it's a little arbitrary to call out one person when there's so many people working hard on search. That got put into perspective for me when a friend told his Mom the "Mick Jagger of Search" thing, and the Mom said "Who is Mick Jagger?" :)
What are the biggest SEO-related changes you can forecast for the upcoming year? ;)
If you're doing whitehat SEO, the same principles will apply going forward: make a great site (e.g. that provides a useful service or relevant information) and think of creative ways for people to find out about it. But spamming attempts in six months or a year from now will not be the same, just as spam attempts today are quite different from a year or 15 months ago. I think the lifetime of off-topic spam in search engines will continue to drop.
What is the most evil thing Google has ever done?
Oy, saving the stinger for the last question, eh? Sneaky, sneaky. :) I don't agree with every single decision ever made at Google, but I think overall we've made good choices for our users and that the world is a better place for Google's influence. I think the flip side of the "don't be evil" core value is that people tend to judge Google against ideal behavior instead of against our competition. That can be frustrating sometimes when it feels like we get dinged for something that is not a Google-only issue, but in general it's good because we have to set a high bar for ourselves. I'm proud of what we've done, but I think there's still room for us to do more as an advocate for our users. Does "needing to do even more for our users" count close enough to evil?
If you would like to read the latest on Gadgets, Google, & SEO check out Matt's blog.
Thanks again Matt.