StanleyWong.org

A recent research report from Microsoft (NY Times article) claimed that approximately 11% of search results and as much as 30% of some competitive search results contain Internet Content SPAM. Spammers create these pages for the sole purpose of profiting from advertisements by fooling the search engines (i.e. Google, Yahoo!,etc.) to rank their pages well. Typically these SPAM sites are computer generated to create huge numbers of pages each targeted to rank well with a specific keyword.

There are many strategies, but typically these SPAM sites are computer generated to produce a huge numbers of pages, each targeted to rank well with a specific keyword. I’ve found several SPAM content generator scripts (YACG, RSSGM, WP AutoBlog, & MyGen) on the web that take content snippets from a variety of articles cribbed from across the web. These scripts auto generate content by jumbling content snippets into search engine readable pages using statistical algorithms such as Markov Chains. Add a custom template to make the site unique and create an internal link network between pages within the site.

Once the site is built, Spammers create huge numbers of bogus blogs (Blogger) and webpages (from free hosting sites such as Yahoo!’s GeoCities) to build links directed towards the Spam site. This confirmed by the Microsoft study where most free web hosting and free blog sites have over 70% Internet Content SPAM.

Internet Content SPAM is a huge problem for Search Engines because it degrades the quality their search results. As Albert Einstein famously said “We can’t solve problems by using the same kind of thinking we used when we created them.”, it is tough for Search Engines to identify Internet Content SPAM. Typical approaches entail usage of filters and/or blacklists when a SPAM signature or domain is discovered. This makes Internet Content SPAM one of the most vexing issues for this generation of Search Engines.

Filed by SWong at 6:25 am under Interactive Advertising, SEM, SEO, Search
No Comments

              

Last month, I found out the patent for the Yahoo! Buzz Index (US Patent 7,146,416) was granted. I had a great time working with Janice Yoo, Elliot Yasnovsky, and KT Lim on building this project. From Search Engine Land:

Filed by SWong at 7:51 pm under Marketing, Search, Yahoo!
No Comments

              

I just read Rick Skrenta’s great blog post,
“Winner-Take-All: Google and the Third Age of Computing”

Rick is right on the money with a lot of his observations, especially the fact that Google has built their huge lead on the backs of the Search and Advertising dominance.

One thing I’d like to add is zero switching costs also has a downside for Google if a new entrant is **significantly better** than the incumbent. So far, the efforts of Yahoo!, Ask, and Microsoft in the search space are building on essentially the same technology platform as Google. The best they can do is incrementally better than Google and therefore have a huge challenge to overcome the brand gap and technology refinement from Google’s band of top notched engineers.

Significantly better requires one to make a dramatic leap (technology or business model) beyond what Google is doing today. When Google came into the market they introduced PageRank (link based algorithm) which was significantly better than the prevailing keyword based ranking technologies employed by the incumbents (AltaVista, Excite, Infoseek, Inktomi, etc). PageRank introduced, for that time, a relatively SPAM free search environment that plagued the keyword based search engines. So for users comparing searches performed on Google vs. the incumbents the results were startling.
The biggest challenge for search continues to be search SPAM. SEO practitioners (white hat and black hat) have continued to arm themselves with tools to SPAM the current generation of search engines. These tools include keyword based semantic page generators (using Markov chain models) while linking them together into artificially generated networks to improve ranking.

Having worked in the Internet space for over 10 years, I personally think that Google is not invincible. I’ve seen too many dominant players come and go during my career to be bold enough to make that claim. However for the potential competitors, it requires one to have a talented team that can approach this space from a completely different vantage point and offer up a new technology paradigm to advance the state of the art towards the next generation. This requires a healthy combination of Smarts (Innovation) + Money (to Scale the business) + Flawless Execution to achieve.

Filed by SWong at 8:55 pm under Search
1 Comment