Yahoo! Pipes for open and extreme feed manipulation

A growing body of content is now available as a structured feed. While feed readers make use of this structure to organise various feeds in groups etc, Yahoo! Pipes allows the content of feeds to be manipulated programmatically. Yahoo! Pipes is not just a visual programming platform, the pipes created by the users are open source and can be incorporated or even cloned and modified. Pipes can be thought of as feeds of feeds (meta feeds) and can get to be extremely complex and powerful.

Pipes Blog

From http://blog.pipes.yahoo.net/about-pipes/

About Pipes

Pipes is a powerful composition tool to aggregate, manipulate, and mashup content from around the web.

Like Unix pipes, simple commands can be combined together to create output that meets your needs. Here are a few popular ways the service can be used:

  1. create your ultimate custom feed by combining many feeds into one, then sorting, filtering and translating them.
  2. geocode your favorite feeds and browse the items on an interactive map.
  3. remix your favorite data sources and use the Pipe to power a new application.
  4. build custom vertical search pages that are impossible with ordinary search engines.
  5. power widgets/badges on your web site.
  6. consume the output of any Pipe in RSS, JSON, KML, and other formats.

Pipes – Frequently Asked Questions

From http://pipes.yahoo.com/pipes/docs?doc=overview

What is Pipes?

Pipes is a free online service that lets you remix popular feed types and create data mashups using a visual editor. You can use Pipes to run your own web projects, or publish and share your own web services without ever having to write a line of code.

Building a Pipe

Sharing a Pipe

Are Pipes I create private?

What does “Publishing” a Pipe do?

Some popular Pipes:

From http://pipes.yahoo.com/pipes/pipes.popular

Google and Yahoo News

http://pipes.yahoo.com/pipes/pipe.info?_id=1nWYbWm82xGjQylL00qv4w

Searches Google and Yahoo news, then merges them together into one feed.

Webpage-to-RSS

http://pipes.yahoo.com/pipes/pipe.info?_id=NlNQKdO62xGAq1ZgZoQMOQ

Webpage-to-RSS Pipe generates RSS from any webpage which doesn’t have native syndication. It utilizes a data mining service called Feedity ( http://www.feedity.com ) to dynamically build the RSS, and then only return unique items. This pipe accepts the URL of the source webpage for which the feed is to be rendered, and it returns a valid RSS for reuse in other pipes. This pipe is designed to be used as a modular sub-pipe in your own pipeline. Clone and reuse!

Aggregated News Alerts

http://pipes.yahoo.com/danielraffel/fELaGmGz2xGtBTC3qe5lkA

Setup a persistent search at Bloglines, Findory, Google Blog Search, Google News, IceRocket, MSFT Live News, Technorati, and Yahoo! News.

eBay Price Watch

http://pipes.yahoo.com/pipes/pipe.info?_id=avkEShi32xG_EF6KZVUMqA

This pipe is designed to use eBay’s RSS API to find items within a certain price range.

New York Times thru Flickr

http://pipes.yahoo.com/danielraffel/vvW1cD212xGMiR9aqu5lkA

This Pipe takes the New York Times homepage, passes it thru Content Analysis and uses the keywords to find Photos at Flickr.

Apartment Near Something

http://pipes.yahoo.com/pipes/pipe.info?_id=1mrlkB232xGjJDdwXqIxGw

Ever wanted to find an apartment close to a park, a school or Whole Foods? Now you can…

GeoAnotated Reuters News

http://pipes.yahoo.com/pipes/pipe.info?_id=gGThvN_62xG2JH50ZoQMOQ

Uses the geonames.org RSS-to-geoRSS webservice to add location information to Reuters newsfeed. The result is displayed using the Yahoo!Maps AJAX API (thanks to Pipes Location Extractor). It can be viewed there : http://www.80n.dreamhosters.com/

Formats, Tags, Sources and Modules

To get an idea of where the action is, here is some data on the popular formats, tags, sources and modules used. Click here to download the csv file.

pipes_7494_bytes_0980832132cdbc8663d188eec4a14db5_md5.csv

Size: 7,494 bytes

MD5: 0980832132cdbc8663d188eec4a14db5

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: