Key Relevance Review of Google Automatic Match – Measuring the Cost of Skipping Keyword Research

by Jim Gilbert and Mike Churchill of Key Relevance

Automatic Match is Google’s new feature that allows AdWords managers to receive clicks in their PPC campaigns without the need to select specific keywords. According to Google:

 

Automatic matching

Automatic matching is an optional feature that helps your ads reach targeted traffic missed by your keyword lists. It works by analyzing the content of the landing pages, ads, and keywords in your ad group and shows your ads on search queries relevant to this information.

The automatic matching system continually monitors your ad performance and aims to show your ads only on queries that yield a comparable or better cost-per-click (CPC) than that of your current traffic. Automatic matching will only use your unspent budget and will never deliver more traffic than your budget allows for.

 

In two previous posts, Jim took a somewhat negative shot at Google’s “Automatic Match” feature, with little to go on but past experience with “new features”.

 

Well, we’re back and have the results of our real-life testing of this new feature.

After a fairly lengthy process of testing “Automatic Match”, we can now report these facts — Facts at the time of this writing, but Google can always change at any moment without warning or notice.

Facts:

  1. The following statement is still true, so be on the lookout for when Google rolls “Automatic Match” out to your account!
    Quote from an Official Google email dated 23May2008: “The feature will be enabled by default..” – see Automatic Match to be Default
  2. Automatic Match does not start spending immediately… once activated, it takes up to a couple weeks for it to learn what it thinks it should do. So, don’t get complacent if it does don’t start spending on Day One — KEEP TABS — it could take off like a rocket at any time.
  3. Google’s “Automatic Match” IS greedier than expanded broad match! See our original post at: More Greedy than Expanded Broad Match
  4. Still true (if you notice when it shows up): But there is GOOD news — you can OPT out!

    Look for this in your Campaign Settings:


  5. Automatic Match is especially invasive in adgroups that have relatively few keywords.
  6. You can “negate” bad impressions and clicks with negative keywords — if you keep very, very close tabs on them. We use the PPCProbe keyword tool to allow us real-time tracking of the search phrases.

NOW, some actual results:

The Scenario:
One adgroup with 1 PHRASE match keyword, residing in a campaign enabled for automatic match.

The Results:

  • Adgroup spend increased 600%! That’s right… 600%!

While the increase in spend in and of itself is not a bad thing (assuming that’s why we were using Automatic Match in the first place), there is a problem with ther results of our test: spending more money is a good thing only if it is bringing targeted traffic to the site. One of the shortcomings of using Automatic Match is that you don’t get to see the search terms that the searchers are using in the Campaign/AdGroups management screen. We used PPCProbe to allow us to gain the insight into the actual search terms that Automatic Match was matching to in real time.

  • 88% of all clicks were from “Automatic Match”. Only 12% were from the actual phrase match keyword. The CPC of the Automatic Match keywords was a little cheaper than the CPC of the actual phrase in the account, but…
  • The majority (4 out of 5) of “Automatic Match” clicks came from keywords I consider to be not relevant. As we shall see, this makes the Effective CPC much worse in our case.

We ran our test with a single phrase match term in the AdGroup: “wedding table decorations”. Of the clicks collected during our test, clicks for the phrases in the AdGroup broke down as follows:

 

Category Example Percentage of Clicks
Actual Phrase from AdGroup
(non-Auto Match – Very Relevant Hits)
“wedding table decorations” 11.9%
Automatic Match (Relevant Hits) “weddiing table decor”
“decorations for wedding tables”
“wedding cake table decorations”
“wedding table ideas”
7.5%
Automatic Match (Non-Relevant/Close Hits) “table settings”
“party table numbers”
“table numbers for weddings”
9.0%
Automatic Match (Non-Relevant Hits) Chocolate “Hersheys”
“chocolate wedding favors”
“chocolate lollipops”
10.4%
Automatic Match (Non-Relevant Hits) Flowers “wedding flowers”
“wedding florists”
“wedding lily flowers”
47.8%
Automatic Match (Non-Relevant Hits) Wedding Gowns “discount wedding gowns”
“discount wedding dresses”
3.0%
Other (Non-Relevant Hits)   10.4%

 

As result, if we consider the Actual Phrases, the Automatch Hits, and the Near Misses (to give the benefit of the doubt – it is just a computer making these KW decisions and we are being lazy by using Automatic Match in the first place), you can see that only 28.4% of the ad spend generated relevant traffic to the site. This effectively made the CPC of the KW buys in this AdGroup 3.5 times more expensive with Automatic Match turned on compared to manually selecting keywords because of the ad spend wasted on the mis-targeted Keyword clicks.

So, where are these mismatched keywords coming from? Are chocolates, flowers and wedding gowns featured on the landing page for the AdGroup? The short answer is “No”. The word “flower” is mentioned once in the plain text of the page, the words “hershey” and “chocolate” appear in the sidebar navigation that points to other pages of the site, and the words “gown” and “dress” are not on the page at all. None of these off-target keywords appeared in the AdWords Ad Copy or anywhere else in the AdGroup. A new AdGroup was created for this test, so no deleted words were previously in the AdGroup. From this we conclude that Automatic Match seems to be using a variation of the Expanded Broad Match algorithm.

Recommendations

  • Turn Automatic Matching off until you understand the ramifications of what it will do to your ad spend, traffic, and conversions.
  • If you should decide to use it, watch it closely, and track the actual phrases that are being used to drive traffic to the site.
  • Be prepared for nothing to happen on the Automatic Match lines of your AdGroups immediately – Automatic Match takes some time to “kick in”.
  • Compute an “effective CPC” [total $$ / (total clicks – off-target Automatch clicks)] for the AdGroup. Once you discount the off-target traffic you will be better able to determine the real cost of using Automatic Matching in your ad campaign.

Pay Per Click

Share and Enjoy:
  • Print
  • Digg
  • StumbleUpon
  • del.icio.us
  • Facebook
  • Yahoo! Buzz
  • Twitter
  • Google Bookmarks

24 thoughts on “Key Relevance Review of Google Automatic Match – Measuring the Cost of Skipping Keyword Research

  1. Nice work, I’ll definitely keep an eye on this. At this rate I’ll be turning the feature off immediately, especially for some of our more frugal clients.

  2. It would be fascinating to see Automatic match in action and the results produced by it. As rightly pointed out by Jim and Mike, it does give us a reason for not doing our homeworks because things can go terribly wrong. It sure can be used to complement your efforts not replace it.

  3. Excellent, original post – thanks for sharing that. I did an interview with Andrew Goodman and Amy Konefal for Youmoz a while back explaining to folks how expanded broad match works. Perhaps we might do one on automatic match?

  4. Wow, nice work on actually providing some solid illustrations of the output from Automatic Match. It’s interesting to see it picked up some great matches, but mostly really poor matches.

    I wonder if it slowly fixes these mistakes if you have conversion tracking enabled?

    Even so, it would probably bankrupt you before it has done that..!

  5. Looks like a great research tool to me. Personally I would make a page for “wedding flowers” “wedding florists” “wedding lily flowers”. The Automatic Match has found more targeted keywords and tested them for you. Those keywords are almost 5 times more efficient than the ones you were using.

  6. Why is it that your blog post is literally the *only* quantitative review of Automatic Match that I can find anywhere on the web?

    Thanks for putting this together – I appreciate it!

  7. Wow. I love how in depth you guys went with that!

    Seeing that Automatic Match only provided you with 15 vaguely relevant keywords, it’s a no-brainer to not use Automatic Match anymore. Anyone can come up with 15 keyword phrases much more relevant to the actual topic. Thanks again!

  8. Charles Said:
    > I wonder if it slowly fixes these mistakes
    > if you have conversion tracking enabled?

    Unfortunately, the metric Google is using is CPC, not conversions.

  9. I’ve tryied to use the automatic matching tool and gave it a shot for 2 weeks and nit only i didn’t get more conversions but my ctr and amount of sales got dropped – very disappointing.

  10. Seeing that Automatic Match only provided you with 15 vaguely relevant keywords, it’s a no-brainer to not use Automatic Match anymore.

  11. Although i tought it would be fascinating to see Automatic match in action, but after i started running it, I was very disappointed from its results. This is the first time i got disappointed from one of Google’s new tools.

  12. My suggestion (after trying it) is to turn to auto-match on for a week or two and after that turn it back off, by doing that you will get the words from the “google system” and you will be the one who decide if you want to use it or not (the words can be seen in the report section in your google account)

  13. Don’t fall for this one, this new feature (surprisingly) tries to take as much money as it can from your budget but at the end of the day brings no new conversions

  14. My philosophy is that anything that sounds too easy, is more than likely very complex. Taking shortcuts on something as important as keyword research will come back to haunt you. Its always better to take the time upfront to do the research correctly…or you’ll end up paying a lot more than you need. In today’s economy, being lean and smart is survival.

    Great article.

    Christine

  15. I think the most insidious aspect to this is increasing bid prices. If my ad with keywords “emergency locksmith” shows on a search for “emergency dentist” – and it did – that means the price that emergency dentists must pay for position is going up.
    So Google makes more money even if irrelevant automated matches aren’t clicked on.

  16. I see no reason to ever use this feature of Google Adwords. I`m better off researching keywords myself with a little brainstorming, than to rely on Google. I think leaving this on can seriously drain your Adwords funds, just like the advertising on google AdSense can. Too broad for my taste.

Leave a Reply

Your email address will not be published. Required fields are marked *

*
To prove you're a person (not a spam script), type the security word shown in the picture. Click on the picture to hear an audio file of the word.
Anti-spam image