On my way home I turned on the Tablet PC at Oakland International Airport. WiFi here costs $7 on Wayport. I see that the RSS copyright story is still raging. Here's a few I just saw come through my aggregator....

But what kind of argument can I expect from someone who emails me to say:

Erik J. Barzeski said on January 17, 2005:

Go fuck yourself, you tedious DaveWinerWannabe cunt.

Mmmmm… I'll pass.

If I wrote a spider that downloads every page on this site, ROT13s it, then pipes it to /dev/null, you'd (rightfully) be pissed off if I ignored robots.txt. I'd be wasting your bandwidth. That's what robots.txt is there to protect.

Robots.txt is not designed for the case where a user-agent (whether fat client on a PC, or a service on a remote server) requests one file on a scheduled basis.

In this situation, the content has been repackaged for profit.

If BlogLines starts selling ads, someone buy an ad on BlogLines, selling their newsfeed reader, educating those reading BlogLines they can be their own editor. Or buy an ad touting Creative Commons.

Engadget pumps ads into their feeds, making the content LESS usable, for keyword searching for example.

Erik is right, Bloglines is wrong..

I don't even want to use a service like Blogger to host a possible blog I might fancy, because it seems so tied in to a possibly for profit service (Google? Yahoo?) that I am afraid it will be caged off somehow.

1. CONSUMPTION

2. RE-PUBLICATION

They're not the same.

PulpFiction doesn't respect robots.txt because PulpFiction doesn't re-publish. It merely consumes. Bloglines re-publishes (takes credit for, makes money from, etc.) and does so without the creator's permission.

Roger insists that I "think again" that Google doesn't re-publish my feed content. I'd like Roger to find the spot on Google where my feed content is re-published. Not my Web content - my feed content.

I'd also like Roger to note that if I wanted to block Google, I could. The power still rests in my hands to control how my content is handled. I still have the ability and power as creator of copyrighted material to say what Google can and cannot do. No such power exists with Bloglines - they automatically assume they're free to rape creative content for their own benefit, regardless of license or copyright.

Consumption. Re-publication. Big difference, guys.

Bloglines has no need to, according to robotstxt.org who define a robot as A robot is a program that automatically traverses the Web's hypertext structure by retrieving a document, and recursively retrieving all documents that are referenced.. There is no automatic traversal involved.

Rod.

Microsoft takes in your HTML, reconfigures it, strips parts, modifies the display, and sends it on to be viewed by hundreds of thousands of paying MSNTV users. Just as Earthlink's proxy compresses the JPG of your company logo, modifying its appearance while allowing dial-up users to download it faster. Corporate firewalls examine your content, strip anything they don't like, and cache the results, doling them out to hundreds (or in some cases, thousands) of users.

The Web's entire infrastructure violates any self-servingly rigid interpretation of copyright, every second of every day. The caches, proxies, and firewalls of the world take in content, archive it, alter it, and make it available to others. That's the way this whole thing was designed, that's why it works. Anyone who doesn't like it is free to opt out of the Web. (Or begin panting breathlessly about suing Cisco, Microsoft, most of the Fortune 500, and so on.)

Ditto for syndication and RSS. The technology is designed to enable services like Bloglines to do precisely what services like Bloglines do. If you don't like that, RSS is not for you.

"Google doesn't re-publish my feed content."

Ummm... think again. Do a Google search for your site and click the "cached" link... Google does quite a bit more than simply republish your some text. It republishes your layout, too.

"Also, Google respects my robots.txt file - something Bloglines (et al) do not."

A marginally useful point. I say "marginally" because robots.txt was not designed to restrict single page accesses by applications under human control... its purpose was to limit the automated link crawling of spiders. Bloglines isn't obligated to act as if they're Google, or even YahooFeedSeeker.

But it probably wouldn't hurt if aggregators (including desktop apps) supported robots.txt. It would give site owners another tool in controlling the RSS Bandwidth Monster... they could just disallow the most popular clients during peak traffic periods.

"In no way should a search that turns up my content provide a bloglines URL."

Another arguably valid point, and one easily addressed by "noindex,follow" meta tags. The content isn't indexed by Google, but the author gets the full benefit of the aggregator's Google juice. Unfortunately, Bloglines doesn't *appear* to be including such meta tags at this time... their bad.