Social Networks/Zittrain: To be sure, we can Google ourselves, but this does not capture those databases open only to “friends of friends”—a category that may not include us but may include thousands of others. At the same time, we may have minimal recourse when the information we thought we were circulating within social networking sites merely for fun and, say, only among fellow college students, ends up leaking to the world at large. (1)
Early on, some wanted to be able to publish material to the Web without it appearing in search engines. In the way a conversation at a pub is a private matter unfolding in a public (but not publicly owned) space, these people wanted their sites to be private but not secret. The law offers one approach to vindicate this desire for privacy but not secrecy. It could establish a framework delineating the scope and nature of a right in one’s Web site being indexed, and providing for penalties for those who infringe that right.
Today, nearly all Web programmers know robots.txt is the way in which sites can signal their intentions to robots, and these intentions are respected by every major search engine across differing cultures and legal jurisdictions. (2)
1. It does not just happen on social networking sites; constitutional law scholar Laurence Tribe was distressed when a statement he posted on a family Web site became the subject of public attention. ROSEN, THE NAKED CROWD: RECLAIMING SECURITY AND FREEDOM IN AN ANXIOUS AGE, (2004); Jeffrey Rosen, The Naked Crowd: Balancing Privacy and Security in an Age of Terror, 46 ARIZ. L. REV. 607, 610 (2004) at 164—65.
2. See, e.g., Yahoo!, Search Help, How Do I Prevent You from Indexing Certain Pages?, http://help.yahoo.com/lus/yahoo/search/webcrawler/slurp-04.html (last visited Dec. 1, 2007); Microsoft Live Search, Site Owner Help: Control Which Pages of Your Website Are Indexed, http://search.msn.com.sg/docs/siteowner.aspx?t=SEARCH_WEBMASTER_REF_RestrictAccessToSite.htm (last visited Dec. 1, 2007); Baidu, http://www.baidu.com/search/robots.html (last visited June 1, 2007); Google, Webmaster Help Center: How Do I Request that Google Not Crawl Parts or All of My Site?, http://www.google.com/support/webmasters/bin/answer.py?answer=33570&topic=8846 (last visited June 1, 2007). See generally Posting of Dan Crow to the Official Google Blog, Controlling How Search Engines Access and Index Your Website, http://googleblog.blogspot.com/2007/01/controlling-how-search-engines-access.html (Jan. 26, 2007, 11:36)._____________Explanation of symbols: Roman numerals indicate the source, arabic numerals indicate the page number. The corresponding books are indicated on the right hand side. ((s)…): Comment by the sender of the contribution. The note [Author1]Vs[Author2] or [Author]Vs[term] is an addition from the Dictionary of Arguments. If a German edition is specified, the page numbers refer to this edition.
The Future of the Internet--And How to Stop It New Haven 2009