Feeds




















SES Toronto - Combine Professional Development with Family Fun

2:14 pm   -   May 8th, 2008

by Christine Churchill

Toronto CN TowerIf you’re looking for inspiration for a summer trip that combines professional development, culture, and natural beauty, here’s a suggestion: combine professional training with a family vacation. Treat yourself to the high powered Search Engine Strategies Toronto Conference and instead of flying straight home, bring the family and explore Toronto and Canada.

SES Toronto is scheduled for June 16-18.  The first day is a full day of training consisting of  four-hour blocks of personalized instruction given by some of the top experts in the country.  Toronto’s training sessions include Link Building taught by the Link Goddess herself, Debra Mastaler, SEO Tools taught by the creative and fun loving Todd Malicoat, Search Engine Marketing Metrics and Myths taught by ex-Googler Adam Goldberg, and Social Media Marketing taught by KeyRelevance’s own social media dynamo, Liana Evans.

June 17 and 18 are the actual conference dates. I was pleased to see my dear friends Fredrick Marckini and Bryan Eisenberg giving the keynotes. When you walk out of their keynotes, your mind will be going a hundred miles an hour, filled with a fresh perspective and stimulating ideas.

Alan K’necht, President of K’nechtology and I are teaming up on the second day of the conference to do the Keyword Research session. Alan and I have spoken together on panels in the past and he is one of my favorite people. Alan doesn’t hold back on his advice. He is one of the most open and forthright people in the industry. I’m excited about our session and our goal is that after our presentations you will have an arsenal of keyword research techniques at your disposal.

Plan a few extra days to explore Toronto and Canada. Toronto is an exceptional Canadian city, and you would be well served to take advantage of the opportunity to get to know it better. The CN Tower, the Royal Ontario Museum, and the Harbourfront on Lake Ontario offer something for every member of the family. Hope to see you there.

Online Marketing Strategy, Where It All Begins

1:26 pm   -   April 28th, 2008

By Li Evans

Whether you are doing a PPC campaign, an SEO initiative or even social media outreach you have have to have a plan. Without an online marketing strategy in place, how will you every know where to go, how to get there and what to do once your get there? If you don’t have a strategy, it’s kind of like throwing spaghetti against the wall to see if its sticks (and is done). That’s not good for the cook - you waste spaghetti, time and have to clean the wall, and it’s certainly not good in online marketing - you waste resources, time and are left scratching your head in bewilderment.

When you are looking at starting an online marketing strategy, there’s some basic things you should take into account. Making sure all your bases are covered, will save you a lot of hassle and a lot of headaches in the long run. Sure, it may take a few more hours in planning, but it can gain you so much more in the end.

Research First
Research is probably the single most, foundational thing any marketer can do. Know your industry inside and out. Know your industry and how it relates in both an online and an offline marketing environment. Who’s your competition? Nine times out of ten, your competition is different offline than it is online.

How are people searching for you, the products or services you provide and is it different than the jargon you use? Do you want to focus on brand building or focus on the services / products? Can you do something totally different online than you do offline? By doing your research first, you can be prepared to make the right decisions and most likely get a leg up on the competition.

Decide What Online Marketing Facets To Utilize
Do I need to do PPC first? How about starting an online forum? What low hanging fruit can I pick from in the SEO world? Do I need to re-energerize my email marketing campaign? Maybe I need to do some videos & and images? Maybe, you need a piece of everything?

Knowing what approach you are going to utilize, SEO, PPC, Social Media, Email, Multimedia, Online PR or any other segments of online marketing is key to making a strategy work. Doing the research first will help you to determine what segments you really need to hone in on. If it’s a brand new site, you’ll likely need to boost that PPC campaign first, then bring in the SEO. If its an established site, maybe a little bit of social media is needed. Make sure your plan spells out what exactly you will need to use so that all the players on your team know where to put their time & resources.

Coordinate Offline & Online Marketing Initiatives
One of the biggest blunders large brands make is not coordinating their offline marketing with their online efforts. What happens when a commerical hits really big on TV? Most people head to YouTube looking for it, or they head to the company’s site. If offline and online don’t mesh, someone else can take advantage of that company’s “miss”.

It’s not enough to just put up a video or a photo, or even a blog post these days. Coordinating offline, with all facets of online marketing is needed. More often these days, people do not use just one source to find what they are looking for. They may start at a search engine, but then the go off to social sites looking for information, too. Planning and strategizing for this is essential to your marketing success.

Decide What To Measure
targeting what to measure in online marketingOne of the most important things to do from the very beginning of any online marketing effort is to decide what is going to be measured. Is it incoming traffic, is it time on site, is it number of pages in a visit or maybe its conversions? Not only is this important, but it’s important to segment that measurement.

By segmenting the measurements, you are going to know where and how these items are succeeding or failing. You can know whether its a landing page or the home page that’s driving the bounce rate up. Is that particular keyword really focused on what it needs to be? Is there a problem with your shopping cart? All of these can be seen if you decide what to measure and what segments to look at before you implement that online marketing initiative.

What Are Your Success Factors?
So you’ve got an idea of what you want to do online marketing wise, right? Well how are you going to determine whether or not that initiative is successful? Traffic & hits alone can’t give you a whole picture. Just because your traffic went up, doesn’t mean that your plan was a success.

Did you get new subscribers? Did anyone buy the product or service you were promoting? Did you get new links pointing into the marketing piece you launched? Did that email get opened more and did the receivers click through? Without setting goals and deciding on what is deemed a success, you are never going to know whether or not that marketing strategy was truly a success or a colossal blunder that shouldn’t be repeated again.

What Happens Next?
It’s not enough to just plan for “right now”, you need to also plan for “what’s next”. Is your marketing strategy agile enough to be able to capitalize on a successful viral marketing piece? Is your online marketing plan taking into account that PPC effort is tanking and costing too much? What are the contingencies you’ve set in place?

If you don’t plan for what’s next, you could miss out on some great opportunities that come your way from both successes and failures. Also planning for what comes next gives your team something to strive for an attain - the next milestone in the online marketing plan. Knowing what you’d like to do next, also helps you to keep an eye out both budget and resource wise before you actually implement. If your team also knows what is coming next, you have multiple eyes looking out to make the next action in your online marketing strategy a success.

Travel Industry: Online Marketing Going Beyond 10 Blue Links

1:45 pm   -   April 8th, 2008

By Li Evans

Last week I had the delightful opportunity to present on the Travel & Search panel at Search Marketing World 2008 in Dublin, Ireland. It was a really great event that featured not only search marketing in Ireland, which is growing by leaps and bounds, but search marketing across the UK and Europe. Not only did Chris Sherman of Search Engine Land give a great keynote speech, but Mike Grehan of Acronym Media presented a panel on linking, Bill Hunt of Global Strategies and, Jon Myers of Mediavest spoke on Integrating Search Into Your Media Plan, and Dave Davis of Ireland’s own RedFly Marketing presented on Blogging & Search.

Travel Industry Expanding Beyond 10 Blue LinksThe Travel & Search panel brought up some interesting topics. From why the travel industry should care about search marketing, to why it should care about mobile and finally my topic of social media and travel, the panel covered a lot of information that businesses in the hospitality industry should be looking at to be competitive in this online market. Both of my industry colleagues, Nick Walsh and Teddie Cowell brought some great information to the audience.

Businesses who find themselves right in the center of the hospitality industry like hotels, resorts, restaurants and tour companies, find themselves in an ever evolving market space when it comes to online marketing. 3rd party booking engines are a godsend and a problem, social media is this big area that is rapidly growing and a place that can make or break a business, and mobile search is coming on so fast, no one really knows what to prepare for. Understanding all of this and being able to handle it is a big task for a lot of businesses in the hospitality industry.

The big question that really came out of the panel is “Why should the Travel Industry care?” … about online marketing, which encompasses SEO, PPC, Mobile, and Social Media. There are a lot of reasons to care, and they all boil down to the company’s bottom line of making a profit. Although some of it is hard to directly relate to it, every segment of online marketing can have a direct affect on the bottom line.

Looking at hotels, making sure your site is optimized for the search engines can help ensure their site is the rankings for their listings. Not only that, but making sure that their own sites have a capable booking engine and prices are in line with both the competition and the 3rd party booking sites like Expedia, Hotels.com and Priceline, can help these hotels keep the relationship personal with the customers booking on their sites. In the end, it saves the hotels commissions, which helps their bottom line, and creates a valuable relationship with the customer that usually brings them back to their hotel again and again.

Social media has increasingly had a growing affect on businesses in the hospitality industry. From forums to review sites and then tack on blogs, businesses have a lot on their hands when it comes to not only promoting their services, but also when monitoring their brands and services. One bad post about a guest not having their sheets changed for 17 days is enough to drive enough potential customers away that it makes a difference in the bottom line. Knowing how to handle and embrace the good as well as the bad, is essential to this industry.

Finally mobile is about to take this industry by storm. Ensuring websites are mobile friendly is not just a “nice” thing to have; soon enough it will be essential if you want to keep up with the competition. From maps to find the business, to making reservations with smart phones, mobile will soon be a force that can no longer be ignored.

So where does this leave the travel industry? To be honest, it leaves them in a place with a lot of great marketing possibilities. It’s a great time for the hospitality industry to embrace a lot of new ways to market their businesses and improve their bottom lines, to reach out and engage customers and create rewarding relationships. Are you a hotel, a restaurant or even a tour company, are you thinking about embracing any of these great ways to market your business? Perhaps now, is the perfect time to take a second look at online marketing!

Stop over at Search Marketing Gurus for additional coverage of Search Marketing World.

Yahoo on Web Mining and Improving the Quality of Web Sites

8:59 am   -   March 18th, 2008

by Bill Slawski

A successful web site is one that fulfills the objectives of its owners and meets the expectations of the visitors that it was created to serve.

This is true of ecommerce web sites, news and informational sites, personal web pages, and even search engines. And, it’s a topic that even the search engines are exploring more deeply. A recent patent application from Yahoo tells us that:

The Web has been characterized by its rapid growth, massive usage, and its ability to facilitate business transactions. This has created an increasing interest for improving and optimizing websites to fit better the needs of their visitors. It is more important than ever for a website to be found easily on the Web and for visitors to reach effortlessly the content for which they are searching. Failing to meet these goals can mean the difference between success and failure on the Internet.

User Query Data Mining and Related Techniques, (US Patent Application 20080065631), by Ricardo Alberto Baeza-Yates and Barbara Poblete.

The patent filing discusses how information about queries that people use, collected from search boxes on a site (if one is used) and from search engines bringing people to a site, can provide useful and helpful information about how people use that site.

The collection of this kind of information is often referred to as Web Mining, and looking closely at the words people use to find information on a site can tell us something about the actual information needs of those visitors.

Search engines have studied searchers’ queries mostly to try to make search engines work better, but looking at the words people use to find a site, and to search within it once they have found it, could help to make the web sites themselves better.

The abstract of Yahoo’s patent filing notes:

Methods and apparatus are described for mining user queries found within the access logs of a website and for relating this information to the website’s overall usage, structure, and content. Such techniques may be used to discover valuable information to improve the quality of the website, allowing the website to become more intuitive and adequate for the needs of its users.

One tool that many site owners use on their pages are analytics programs, though often those are looked at to see how much traffic is coming to a site, and possibly to determine which words people are using to find a site. Analytics programs can provide a stronger role in helping people with web sites improve the experience of people visiting their pages, and the success of their sites.

Web Mining

The Yahoo patent is interesting in that it focuses less on how a search engine works, and more on how the owners of web sites can use the process of Web mining to discover patterns and relations in Web data. Web mining can be broken down into three main areas:

  • Content mining,
  • Structure mining, and;
  • Usage mining.

These relate to three kinds of data that can be found on a web site:

  • Content — the information that a web site provides to visitors such as the text and images and possibly video and audio, that people see when they come to a site.
  • Structure data — this is information about how content is organized on a site, such as the links between pages, the organization of information on pages, the organization of the pages of the site itself, and the links to pages outside of the site.
  • Usage data — this information describes how people actually use the site, and may be reflected in the access log files of the server that the site is on, as well as data collected from specific applications on the site, such as people signing up for newsletters or registering with a site and using it in different ways.

Knowing which pages people visit and which pages people don’t can be helpful in figuring out if there are problems with a site. They can uncover a need to rewrite pages, or to reorganize links, or make other changes.

Mining User Queries

Understanding query terms used to find a site and to search on the site can help improve the overall quality of a site. Yahoo’s approach would be to create a model to use to understand how people are accessing a site, and navigating through it:

According to specific embodiments of the invention, a model is provided for mining user queries found within the access logs of a website, and for relating this information to the website’s overall usage, structure, and content. The aim of this model is to discover valuable information which may be used to improve the quality of the website, thereby allowing the website to become more intuitive and adequate for the needs of its users.

This model presents a methodology of analysis and classification of different types of queries registered in the usage logs of a website, including both queries submitted by users to the website’s internal search engine and queries from global search engines that lead to documents on the website. As will be shown, these queries provide useful information about topics that interest users visiting the website. In addition, the navigation patterns associated with these queries indicate whether or not the documents in the site satisfied the user’s needs.

Queries uncovered might be related to categories drawn from such things as navigational information found on a site.

Traffic through the site could tell someone using this invention how effective the site was at meeting the information needs of the people using certain queries. It could also provide suggestions for:

  1. The addition of new content
  2. Changes or additions in words found in anchor text in links
  3. New links between related documents
  4. Revisions to links between unrelated documents

Information Scent

Visitors to a site will follow links that use words within the links that provide some level of confidence that the information being looked for will be upon the other side of those links (The Right Trigger Words as User Interface Engineering’s Jared Spool calls them). Likewise, when someone searches at a search engine, and sees a page title and a snippet of text for a site in search results, the words used in the title and snippet may persuade someone to visit the page. This is true both for search results from a search engine, and search results from an internal search for a specific site.

Understanding what kind of information is being searched for regarding a specific query, and how the words used in search results, on web pages, and in links to other pages may provide some insight into making those search results, those pages, and that anchor text better.

The patent application describes how pages and the queries used to reach them can be classified based upon how they are typically used by a visitor - from external searches through a search engine, from internal searches through a web site search, or through navigation on the site itself.

It also classifies queries as successful or unsuccessful, based upon things such as whether someone visited a page in response to the display of a search result showing the page, or if they followed other links on pages visited to explore a site in more depth.

Seeing how pages are typically reached on a site in response to certain queries, and seeing which queries are successful and unsuccessful in bringing people to information that they want to find can help a site owner make positive changes to a site.

Example

The patent application provides an example using a portal targeted at university students and future applicants.

It focuses upon exploring how effective the site is when searchers use the queries “university admission test” and “new student application” in searches for the site both on search engines and on a site search for the site. Two initial reports evaluated how effective the site was without making any changes. Twenty of the top suggestions generated from reviewing the model described in this patent application were incorporated into the site’s content and structure:

The suggested improvements were made mainly to the top pages of the site and included adding Information Scent to link descriptions, adding new relevant links, and suggestions extracted from frequent query patterns, and class A and B queries.

Other improvements included broadening the content on certain topics using class C queries, and adding new content to the site using class D queries. For example the site was improved to include more admission test examples, admission test scores, and more detailed information on scholarships, because these were issues consistently showing up in class C and D queries.

The “class C” queries mentioned are ones where there was very little information available on the pages of the web site. The “class D” queries were ones for which there was no information available on the site.

One significant result of these changes showed an increase in traffic from external search engines of more than 20%, due to improvements in content, and in link text.

Conclusion

It’s interesting that a search engine would apply for a patent that explores how to use data mining to improve the quality, content, and navigation of a web site. It’s difficult to tell what Yahoo might do with the method describe in this patent application - whether they will only use it internally, or will offer it to others for a fee, or for free.

Many of the concepts described in this patent application are ones that site owners can presently use to improve how well their site meets their objectives, and the objectives of people visiting their pages.

Understanding the terms that people will try to use to find your pages, and the words and concepts that they expect to see on the pages of your site can make a difference in how successful your site may be.

Using analytics tools to understand how visitors who use certain queries will explore your pages and navigate from one page to another can provide even more value to both searcher and site owner, by pointing out changes that can be made to improve the experience of those visitors.

And those changes may just lead to more visits from search engines.

Google Says Users Won’t be able to Tell Paid Ads from Natural

5:26 pm   -   March 10th, 2008

by Jim Gilbert

By Scott Morrison, Of DOW JONES NEWSWIRES reports a top Google executive (Tim Armstrong, Google’s North American president for advertising and commerce.) of saying:

“Speaking at the Bear Stearns Media Conference in Palm Beach, Fla., Armstrong said Google’s advertising platform will evolve over time so that it won’t distinguish between search and display ads.”

Anyone care to comment on what the heck that means?

Google’s Automatic Match - More Greedy than Expanded Broad Match

6:38 pm   -   March 8th, 2008

by Jim Gilbert

Google Automatic Match Beta

So far this “Automatic Match” option is only a beta and accessible by invitation only. BUT, If this monster goes live and removes our ability to “opt out” (like in Expanded Broad Match), something very, very ugly may happen:

  •   No matter how large your budgets, they WILL be spent — every penny (and dollar)!

I was going to let you read the Beta help file, but it disappeared… IT’S BACK, but you will have to be logged into an AdWords account to get to it: Automatic Match

Summarizing:

Just build your campaigns and they will come. Heck, you no longer even have to offer any keywords — Google will look at your ad and your site and make sure your ads show for any search query that even “smells” relevant.

 

Did Google’s revenue drop in January scare them that badly?

Hanes ‘Wedgie Free’ Campaign Misses Out on Online Marketing

8:13 am   -   March 7th, 2008

By Li Evans

Madison Avenue advertising agencies may be good at TV commercials, and highly paid PR Firms may know how to write a press release, but when it comes to translating that across to an online medium (i.e. the internet), the majority of them have a lot to learn. I came across a post on AdFreak about Hanes’ new ad campaign for their new product “Wedgie Free” underwear, which features actress Sarah Chalke of Scrubs fame. The commercials really hit the mark by capturing Sarah’s comedic timing and her all around good looks. It can appeal to women by them thinking “wow, ‘She Gets Wedgies Too?’”, yes I know kind of corny, but all of us have been in that situation at least once in our lives.

While the commercials are catchy, and even premiered on American Idol (trying to capture that ‘young adult female’ demographic), I stopped and wondered how this was translating online. To any online marketer, it’s probably not a surprise that it hasn’t translated yet. If you’re a major online brand, maybe even Hanes, you are probably wondering “what is she talking about?” Well lets take a look at this a little closer.

Hanes PR people sent out a press release. It’s nice, contains images of Sarah Chalke from the commercials and also includes the ability to play the videos on PR Newswire. Great! Hanes’ PR company has at least managed to figure out how to get the videos and images into the press release, but that’s where it seems to have stopped. The PR Release isn’t optimized for search - at least the way normal people search - especially if the aim is “Wedgie Free”, “Wedgies”. I’m sorry, but not many women refer to their underwear crawling up their backsides as “no ride up”, its a “wedgie” plain and simple. It make work in a commercial, but that’s not how people search.

When they launched this campaign, they probably didn’t even stop to think about an online strategy. I’m pretty certain it was more of an after thought. Why? Well because if you look at the search results, you’ll see they (meaning Hanes’ website) doesn’t rank for the main phrase “Wedgie Free”, nor “Wedgie Free” Hanes. They could own this term but they don’t and they are missing out - especially with their PR people contacting blogs like AdFreak.

Google Search “Wedgie Free”

Wedgie Free Search Results in Google

Google Search “Wedgie Free Hanes”

Search for Wedgie Free Hanes in Google

Google Blog Search “Wedgie Free Hanes”

Search for Wedgie Free in Google's Blog Search

You can see the results (in both regular search and blog search) brought back are minimal, and probably until this point, not a lot of search were conducted on “wedgie free”. However, if you launch a campaign on American Idol touting “Wedgie Free” underwear, what do you think will happen? Hello - the audience of American Idol is the demographic that uses the internet the most, they are going to go on and search for videos, images and information on “Wedgie Free”. With as little competition as there is for the key phrases around this campaign, they could have really hit the mark online with this campaign without a lot of effort. Instead their Press Release on PR Newswire gets the search results as does AdFreak, who points to the PR NewsWire and Wall Street Journal pieces, not even to the Hanes website.

Multi-Media wise Hanes is sorrily missing out too. They could really capitalize on this campaign if they only took the time to contact an online marketing agency to help them “get more bang for the buck” when it comes to their online efforts. In taking a closer look, I’ll show you some examples of where they are really missing out. First we’ll look at images and then go to video.

A search in Google Image Search shows the screen capture below for “Hanes Wedgie Free”. I also did a search on “Sarach Chalke”. Granted the search for Sarah might be a bit more competitive, but had Hanes optimized their images on their site and in their press release for theses terms, they could be capturing another segment of search, and it’s quite possible they could invoke that these images produce “blended” search results in the search engines (where the images will appear in the search results).

Google Image Search “Hanes Wedge Free”

Results for Hanes Wedgie Free in Google Image Search

Google Image Search “Sarah Chalke”

Search for Sarah Chalke in Google Image Search

Now lets go to video. Here’s another chance that Hanes could quite possibly get “blended” search results to start appearing for these phrases that undoubtedly people are looking for after the appearance of the commercials on American Idol, however, again they are missing out. Google now incorporates relevant YouTube videos into their search results, Yahoo incorporates Yahoo, YouTube, Metacafe and a few others - Hanes is really missing out here!

Hanes doesn’t have a YouTube channel (as of this writing!), and they don’t have any of their videos/commercials out there. Instead other users on YouTube do. This actually does say a lot for their commercials - they are clever and witty, and Cuba Gooding, Jr is just hilarious in those commercials with Michael Jordan. People really LIKE them. It’s too bad Hanes isn’t taking advantage of this, people would subscribe to the channel and it could be another channel to disseminate their message in a quick and easy manner. Instead, with these new commercials, only one video is out there and it’s put up by a division of a PR Company. Plus the video isn’t even optimized for what it should be, it just has that “PR Spin” in the description.

YouTube Search for “Hanes”

YouTube Search Results for Hanes

YouTube Search for “Hanes Wedgie Free”

YouTube Search Results for Hanes Wedgie Free

YouTube Search for “Hanes Wedgie”

YouTube Search Results for Hanes Wedgie

YouTube Video of Wedgie Free Commercial Uploaded By Another Company

Hane's Wedgie Free Sarah Chalke Commercial / Video Uploaded in Another Users Account

The point here is that this ad campaign is clever, it hits its mark in speaking to its target audience, and it’s got a likable spokesperson, but wow, is it missing out on taking this to the next level. The video of how the commercials were made that’s included in the press release on PR Newswire is great but only included there, why they haven’t put together an online marketing strategy to take advantage of this is really befuddling!

Hanes, if you listening, at least get your own YouTube channel! (That’s a little free advice!) :)

Advances in Crawling the Web

2:33 pm   -   February 29th, 2008

By Bill Slawski

There are 3 major parts to what a search engine does.

The first is crawling the Web to revisit old pages and find new pages. The second is taking those crawled pages, and extracting data from them to index. The third part is presenting pages to searchers in response to search queries.

There’s been some interesting research published recently on the first of those parts.

Crawling Challenges

Crawling the Web to discover new pages, and identify changes in pages that a search engine already knows about can be a challenge for a search engine. The major issues that search engines face in crawling sites involve:

  • How many pages they can crawl without becoming bogged down,
  • How quickly they can crawl pages without overwhelming the sites that they visit, and;
  • How much resources do they have to use to crawl and then revisit pages.

A search engine needs to be careful on how it spends its time crawling web pages, and choosing which pages to crawl, to keep these issues under control.

A recently published academic paper describes this important aspect of how a search engine works, the Web crawl, in more detail than most papers that have been published on the subject before.

Enter IRLBot at Texas A&M

The Department of Computer Science at Texas A&M University has been running a long term research project know as IRLBot which “investigates algorithms for mapping the topology of the Internet and discovering the various parts of the web.”

In April, researchers from the school will be presenting some of their recent reseach in Beijing, China, at the 17th International World Wide Web Conference (WWW2008).

The title of their presentation is IRLbot: Scaling to 6 Billion Pages and Beyond (pdf), and the focus of the paper is this primary function that a search engine performs - crawling the Web and finding new Web pages.

Their research describes some interesting approaches to finding new pages on the Web, handling web sites with millions of pages, while also avoiding spam pages and infinite loops that could pose problems to web crawlers.

In a recent experiment that they performed that lasted 41 days, their crawler “IRLbot” ran on a single server and “successfully crawled 6.3 billion web pages at an average download time of approximately 1,789 pages per second.” This is a pretty impressive feat, and it’s even more impressive because of some of the obstacles faced while finding those pages.

Problems Facing Crawling Programs

Politeness

One challenge that faces Web crawling programs is that those programs shouldn’t ask for too many pages from the same site at a time, or they could use up too many resources of the site and make the site inoperable. Keeping that from happening is known as politeness, and search crawlers that aren’t polite often find themselves blocked by site owners, or complained about to the internet service provider hosting the crawler.

URL Management

As a crawling program indexes a site, it needs to pay attention to a file on the site known as a robots.txt file, which provides directions on pages and directories that the crawling program shouldn’t visit, so that it doesn’t crawl pages that it isn’t supposed to see. The program also needs to track which pages it has seen, so that it doesn’t try to crawl the same pages over and over again.

Avoiding Spam While Crawling

The crawling process described in the paper also tried to limit the crawling program from accessing pages that might more likely be spam pages. If a crawling program spends a lot of its time on spam pages and link farms, it has less time to spend on sites that may be more valuable to people searching for good results to their queries at search engines.

One key to the method used by this research team in determining how much attention a site should get from their web crawler was in looking at the number of legitimate links into the site there are from other sites, which is what they refer to as domain reputation.

Why This Paper is Important

The authors of the paper tell us that there are:

…only a limited number of papers describing detailed web-crawler algorithms and offering their experimental performance.

The paper provides a lot of details on the crawling process and the steps that the Texas A&M researchers took that enabled them to index multi-million page web sites, avoid spam pages, and remain “polite” while doing so. It explores the experiments that they conducted to test out ideas on how to handle very large sites, and crawls of very large amounts of pages.

They conducted their experiment using only a single computer. The major commercial search engines have considerably more resources to spend on crawling the web, but the issues involving managing which pages they choose to index, being polite to sites that they visit, and avoiding spam pages are problems that commercial search engines face too.

Learning about how search engines may crawl pages can help us understand how a search engine might treat individual sites during that process. If you are interested in learning about the web crawling process in depth, this paper is a good one to spend some time reading.

Retailers & Blended / Universal Search

11:32 pm   -   February 28th, 2008

By Li Evans

This past week I presented at SMX West in Santa Clara, California on the Retail and Blended Search panel. It was quite interesting to be on a panel that also included representatives from both MSN and Yahoo Shopping divisions. There was a lot of information given, from making sure your images had feeds to looking beyond feeds for promotion of online retail products.

As we progress further and further with technology and the availability of broad band to shoppers, searchers are looking for more than just a blue link on a search results. Searchers are becoming more savvy as technology progresses, and as more and more options are being provided to them, they actually WANT more than just a blue link. So where does that leave the retailers on the web who have invested so much in feeds?

Retailers need to start thinking outside of the box, because if they merely rely on a feed to get the traffic to your page, eventually as the search results become more engaging. Retailers who just rely on the feed links will loose out on all those people clicking on video links, picture links, social media profiles, and reviews. So what’s a retailer to do?

Images:

  • Make sure your products have images.
  • Make your put captions underneath image.
  • Make sure your images folder is accessible to the search engines.
  • Name your images properly.
  • Make sure your images are of good quality.

Google actually shows different images in blended search than it does in regular image search. The thing to remember with images is that shoppers are very visual and if you have the opportunity to take advantage of image search why not put your best foot forward?

Videos:

Here’s a perfect opportunity to engage consumers via a social media medium. It’s visual, its interesting and it engages a customer into finding out more information. Utilize a few of the social video sites, by uploading some short videos of product demonstrations, humor takes, or even “how to” videos. If you make them fun and interesting, there’s even a chance for them to become viral and while not directly on your site, if the description is optimized with a link to your site or the page the product is on, this can be another traffic driver, beyond the search engine.

Rating & Reviews:
If you can start reviews of your products on your site, this could be a powerful resource to help raise the quality of the page. Amazon uses this very wisely and to their advantage. Rating & review sits such as Epinions and Yelp, also hold a lot of value and can help with with yet another way to “indirectly” hold another position in the results.

Social Media Profiles:
Having profiles on various - relevant - social media sites are another way to help bring awareness to your brand, as well as your products or services. People link to social media profiles, so just like with rating and review sites, it is possible to own another spot on the SERPs in an indirect manner. Make sure that your profile on all of the social media site you belong to is properly filled out with the right URL, emails, contact information, etc.

These are just a few ways retailers can broaden their reach, beyond the regular product feed. Starting to think beyond the feed and planning a full online marketing strategy will open a lot more opportunities for retailers in the new realm of blended / universal search.

A Chat with Analytics Guru Jim Sterne

11:19 am   -   February 15th, 2008

By Christine Churchill

SES London 2008 is nearly upon us. Looking over the agenda and speaker list I was happy to see Analytics expert and all around good guy Jim Sterne. I’ve known Jim for a couple of years and I continue to be in awe of the man.

Jim Sterne PictureJim is a prolific author of books and articles, a famed speaker, and the producer of the eMetrics Summit Conferences. Energetic and engaged in life, Jim runs Target Marketing and is the Founding President and current Chairman of the Web Analytics Association, a wonderful organization of which KeyRelevance is a proud Premiere Corporate Member.

Jim and I will both be in London next week speaking at SES London. Jim is legendary on stage and I’ve heard other speakers playfully call him the PowerPoint King. He’s one of those rare people you occassionally meet in life who exude positive energy and great ideas. I caught up with Jim the other day and asked him a few questions.

Christine: Most of us running online businesses are going in many directions and have to prioritize where we spend our time. Here’s a question for all those harried business owners who are trying to make every minute count. If I only had 15 minutes a day to spend on analytics, where should I spend my time?

Jim: That would depend on my goals. If my obsession for the day were in conversions, then I would spend my time looking over the persuasion path to see where I could improve the visitor interaction. If my goal was to sell more advertising, I would be measuring what makes people look at more pages so I can display more ads. If my goal is to bring in more qualified leads, I would watch how well my advertising money is being spent - where are people coming from and are they the sort of people I’m after?

Christine: Now here’s a related question. When evaluating a new site, what’s the first measurement you would look at?

Jim: It completely depends on the goals of the website. The first thing would be to get the basic numbers, just as a benchmark. How many visitors? How many events (what we used to call pageviews) per visit? How often do they come back? That way, we have something we can compare with tomorrow’s numbers.

Christine: Since both of us will be in London next week, let me as a question related to analytics and geography. Key Performance Indicators (KPIs) are important measurements that companies track. Do you see any regional or national differences in KPIs?

Jim: None at all. KPI’s are particular to industries and website types. Even the ability to track those KPI’s is the same. There are very smart companies in every corner of the globe and some of the most advanced places harbor the least capable companies.

Christine: That is excellent news for businesses with global aspirations and your answer makes perfect sense. Jim, you are well known for your clear thinking and forward looking approach to life. Where do you see Analytics heading?

Jim: Given enough traffic for statistical significance, I think we can use the activity on a website to measure the impact of all our marketing. The web is so much a part of daily life now that an ad in the newspaper, an ad on the radio and a direct mail postcard will all have an effect on the behavior on the website. Capture that activity, sift through it and the impact of your marketing spend will be revealed. We’re not there yet, but it’s right around the corner.

Christine: In the good old days of the web, the “hit” was the unit of visitor interaction, at least until we figured out that it was a lousy metric. Then, the “impression”or “page view” became the standard. Now along comes Web 2.0 sites with their richer, more interactive mode of interfacing with a visitor. Google Maps, for example, may occupy a visitor for several minutes, without the URL on the Address line changing. When a visit can no longer be accurately be measured in “impressions” how do we properly quantify web site traffic?

Jim: While those better and brighter than me are working on measuring “engagement”, I am happy to break down a visit to a website into “actions”. Searching for an address is an action. Scrolling the map is an action. Zooming in on a location is an action. Commenting on a blog is an action. These actions add up and spell out the flow of individual activity and quite readily replace the pageview as a means of understanding behavior.

Christine: The search engines are offering analytics tools as a part of their offerings. I’m frequently asked this question by students and clients who invest heavily into online paid advertising. From a web advertiser’s perspective, is there any danger in letting the search engines have such a detailed view of a company’s conversions, revenue, and other business metrics?

Jim: One first has to assume that your website is interesting enough and the data about your website is valuable enough to put a $166.2 billion enterprise at risk.

Christine: Jet lag or bad food? What’s the worst part about travel?

Jim: The worst part about travel is yet to come: when they allow mobile phone on airplanes.

Christine (laughing): Thanks much Jim. I’ll see you in London!

Jim: Looking forward to it!

Next Page »