A Coasian Approach to Beating Google

15 May 2008 at 9:14 am Peter G. Klein 4 comments

| Peter Klein |

The always-interesting and provocative Mark Cuban (yes, him) suggests a “Coasian” strategy, based on side payments, for beating Google in the search-engine wars. If I understand correctly, the strategy works like this. Websites have the option (through robots.txt protocols) to prevent their inclusion in search engine databases. At some price, popular websites would presumably be willing to block Google’s spiders. Once word gets around that Google’s database no longer includes these sites, consumers would start to use alternative search engines. Doing this wouldn’t be cheap, but Cuban points out that that upstart search provider wouldn’t need the whole internet to play along, just the top sites, the ones at the left side of Chris Anderson’s long tail. It might cost, say, $1 billion, but the upstart would recover that $1 billion pretty quickly.

One problem Cuban doesn’t consider is that this kind of fragmentation in search-engine space would presumably reduce the value of search, cutting the revenue stream that Google is presently gobbling up. But the upstart might be better off with a larger slice of a smaller pie.

A more fundamental problem is that Google would presumably bid to get the top sites back. Do the search engines bid away all the rents, such that the returns from search accrue entirely to the top sites (as in the old Demsetz-Posner franchise-bidding model)? Or is the process more like a common-values auction with uncertainty and hence subject to the winner’s curse?

Entry filed under: - Klein -, Strategic Management.

Greetings from Leuven Something to Abuse Graduate Students

4 Comments Add your own

1. Ann | 15 May 2008 at 10:57 am

I’m confused. Couldn’t search engines just refuse to honor the robots.txt protocol if it threatens their existence? It seems like would be the first thing to give in this scenario.
2. Peter Klein | 15 May 2008 at 3:44 pm

I don’t know if there are other ways to force removal of one’s information from a search-engine database. If not, then commercial sites could refuse to buy ads on Google unless it honors their removal requests (again, in exchange for appropriate compensation from the rival search service).
3. GringoSalado | 16 May 2008 at 8:14 pm

I’m w/ Ann, that is not any kind of silver bullet. Some bots already ignore robots.txt

I thought Cuban was smarter than that.
4. Jongwook Kim | 18 May 2008 at 11:19 pm

Actually, what the blog is showing is not really a way to beat Google but doesn’t this provide a glimpse into how much website owners value continuing to be in the Google index (and putting a $$ next to it)? Or, conversely, at what point (how much would they need to be compensated) would website owners no longer want to be in the Google index?

This little mental exercise can provide an estimation of the maximum that Google can charge for being included in the index. It seems to be a nice example of the Coasean bargaining process as a way to discover preferences. Or is it an entrepreneurial discovery process as discussed by Kirzner, since Google or website owners (or both) can find ways to generate economic value out of something that had been in the “public domain”?