Google's War on Hierarchy, and the Death of Hierarchical Folders

by John Hiler

Hierarchical Folders have dominated info organization since they first appeared over 40 years ago. But in industry after industry, a strange thing is happening: hierarchy is under severe attack, and even dying out.

Just take a look at some of the most successful info-based businesses of all time:

* Yahoo's Web Directory
* Microsoft Outlook's email folders
* Microsoft Window's file system

All three businesses have achieved enormous success using hierarchical nested folders. And yet, one of these businesses has already abandoned Hierarchies, and the other two Hierarchies are under attack.

The company behind all three of these attacks? Google, the web's leading search engine.

TagCloud is an automated Folksonomy tool. Essentially, TagCloud searches any number of RSS feeds you specify, extracts keywords from the content and lists them according to prevalence within the RSS feeds. Clicking on the tag’s link will display a list of all the article abstracts associated with that keyword.

The technology behind TagCloud.com was created just for fun by IonZoft
developer John Herren, and word quickly spread through the blogosphere.
After numerous requests for his source code, we decided to produce this
service based on John's original idea.

"...one sure tip-off to the fact that you're being assaulted by an Architecture Astronaut: the incredible amount of bombast; the heroic, utopian grandiloquence; the boastfulness; the complete lack of reality. And people buy it!
...Remember that the architecture people are solving problems that they think they can solve, not problems which are useful to solve... "

"There has been a lot of interest recently in e-Portfolios, and in particular the relationship with feeds and blogs. Having been involved in a lot of work recently in the area of standards for e-portfolios, I think we're quite close to achieving some very simple de-facto solutions. In this post I'll describe a very basic approach using Atom and FOAF..."

"...In most community's metadata practice, you'd make an application profile to determine which elements to use and define community specific vocabularies. All the grubby business of constructing the records in interoperable ways could be taken care of by using all the documents from the IMS specification. That gave the datamodel, binding instructions and a schema to validate records against. The latter was crucial, because that's what most developers used to build tools with and check them. Now, the IMS Meta-Data spec continues to provide best practice, and the two IEEE LOM standards, the datamodel and the binding instructions. But the IEEE doesn't really provide a single schema to validate records against. That is effectively the job of the community now."

Google has introduced a new beta service that I hope we'll be able to leverage:

"By placing a Sitemap-formatted file on your webserver, you enable our crawlers to find out what pages are present and which have recently changed, and to crawl your site accordingly. Basically, the two steps to participating in Google Sitemaps are:

1. Generate a Sitemap in the correct format using Sitemap Generator.
2. Update your Sitemap when you make changes to your site."

Including a fairly fully developed xml format and protocol:
https://www.google.com/webmasters/sitemaps/docs/en/protocol.html

and a Python-based map generator:
https://www.google.com/webmasters/sitemaps/docs/en/sitemap-generator.html

Security, as in our OAI_admin authorization plan (which is in turn based on Alexa's site information update capability), is based on the ability of the site admin to generate/upload and maintain a file on the site being indexed. No other security or authorization is necessary.

Diane and I have discussed in the past how useful somethink like this would be for crawling, and I think we should take a look at how we might integrate this into our own crawling for metadata generation. Especially now that Google has 'invented' this particular wheel, although Dave Winer proposed something very similar in 1997 (I actually remembered that post when I saw it again).

This really would solve a lot of problems for us and I think it's pretty exciting.

"We librarians flatter ourselves that we know a thing or two about organizing information. It's time we stepped up and contributed to Wikipedia: not just to its content but to its structures and technologies. This project page is intended to provide a rallying point for these activities."

"Get your Mac, a webcam, and Delicious Library and rediscover your home library. Just point any FireWire digital video camera, like an Apple iSight ™, at the barcode on the back of any book, movie, music, or video game. Delicious Library does the rest. The barcode is scanned and within seconds the item's cover appears on your digital shelves filled with tons of in-depth information downloaded from one of six different web sources from around the world.

Once your whole library is cataloged, you can find and use your items like never before. Browse, sort, and search through your digital shelves. Sync your cataloged library onto your iPod or print a color catalog and take it with you. Find and purchase new items using Delicious Library's personalized recommendations. Keep track of the items your friends are borrowing using Delicious Library's loan management system, which integrates with Apple's Address Book and iCal."

Makes me wish I had a Mac.

Cory Doctorow: A Ben and Jerry's customer requested that the ice-cream come "in stainless steel, bulletproof containers with a little padlock." The company didn't go that far, but they did create this lockable pint-lid that fits over your ice-cream …

Sajax is an open source tool to make programming websites using the Ajax framework — also known as XMLHTTPRequest or remote scripting — as easy as possible. Sajax makes it easy to call PHP, Perl or Python functions from your webpages via JavaScript without performing a browser refresh. The toolkit does 99% of the work for you so you have no excuse to not use it.

OK, so have people create custom screen scrapers for extracting site semantics, add a shared central repository, shake and you have…

Piggy Bank is an extension to the Firefox web browser that extracts information from existing web pages and stores it in RDF. If a web page already links to RDF information, extraction simply means retrieving that information. Otherwise, Piggy Bank employs custom software code that untangles the “pure” information from the web page’ formatting.

Having extracted the “pure” information and stored it on your computer, Piggy Bank can now apply its own user interface to let you browse through that information independent of the original web sites. For example, Piggy Bank can call upon Google Maps to display geographical information even if the original web sites do not offer cartographic views of their data.

Furthermore, by storing “pure” information from different web sites in the same data model, Piggy Bank can offer a unified view on the “pure” information regardless of its many origins.

The piece of software code that Piggy Bank uses to “purify” information within a web page is called a screen scraper. Different screen scrapers are made for different web pages. Piggy Bank supports an easy way to install screen scrapers, so that getting better use of a web page’s information is just a few clicks away.

Now add a large scoop of iVia and you have the idea I presented last year to extract the semantically rich information from a site as metadata via screen-scraping. Personally I prefer ‘semantically rich’ to ‘pure’

Interesting discussion of e-portfolios and how they might be assembled by aggregating parts from different institutional repositories. Scott has talked about this before, but this presentation brings a lot of his ideas together. Remember this?

2005-05-10: The Semantic Web Best Practices and Deployment Working Group released three First Public Working Drafts: SKOS Core Guide, SKOS Core Vocabulary Specification, and a Quick Guide to Publishing a Thesaurus on the Semantic Web. The drafts explain how to express classification schemes, thesauruses, subject heading lists, taxonomies, terminologies, glossaries and other types of controlled vocabulary in RDF. Previous SKOS work was supported by the European project SWAD-Europe.

There’s also a ‘thumbs-down’ icon for a page that has been NOT recommended. When there are no recommendations (reports) about a page it looks like this:

Clicking on the icon opens a sidebar. Here’s the ‘No reports were found’ entry:

Clicking on ‘Enter Report’ posts the recommendation to the Outfoxed server and results in a report being displayed (from me in this case):

Clicking on one of the hyperlinked tags displays (incredibly slowly) a list of my recommendations containing that tag. Hope they speed this up:

Navigating to a page with the sidebar open displays a list of reports about that page (if any) and the site (if any):

Note the ‘Agree’ button. Clicking on that opens up a new report for me to edit that contains the contents of the report I’m agreeing with:

Note the ‘2 hops’ hyperlink in the reports sidebar:

This displays how far down my personal ‘chain of trust’ this recommendation is. 2 hops indicates that someone that I’ve said I trusted — an ‘informer’ in the Outfoxed ontology — has indicated that they trusted the person making this recommendation. The more hops, the less reliable. Clicking on that link will display the chain, in this case indicating that user damianpenney is trusted by Stan:

Clicking on the recommender’s id will display their personal page and show the pages/sites they’ve reported as well as the users that they’ve trusted. A button allows me to instantly add them as an informer, which will raise their reports in my sidebar:

I think there’s more there under the hood, but this hits most of the high points and should give you an idea about why I’m so excited. One of the things I didn’t show was the integration into Google search results — you can see an example at the Outfoxed screenshots page — but that doesn’t seem to be working for me (not surprising considering how junked-up my browser is).

Lots of potential for integrating special purpose controlled vocabularies into the ‘Enter Report’ interface. It’s open source and we can setup our own server as well.

Now all Outfoxed needs is an RSS/Atom feed from my personal page or an API that allows me to include my Reports in my weblog and insert them including tags into de.lici.us and Furl and I’m ready to abandon all other forms of blogging.

Well, and some things need to happen much faster. Performance is always a challenge, ain’t it?

Oh man, this is GOOD.  Way, way more powerful than anything we’re currently doing. Must restrain enthusiasm until I’ve actually used it for awhile…

Outfoxed is the implementation side of my master's thesis at the University of Osnabrück, Germany. The thesis title is Trusted Metadata Distribution Using Social Networks. In a nutshell, I'm exploring ways for you to use your network of trusted friends to determine what's good, bad, and dangerous on the internet. Outfoxed does this by adding functionality to the Firefox web browser. Coding began on Dec 27th, 2004.

…The essential idea of Outfoxed is that people make decisions based primarily on a few people whom they trust. The average person has a set of experts whom they consult in designated areas: the computer expert, the car expert, the fashion expert, the financial expert. If the opinions of these experts can be collected, they are incredibly useful: it is this metadata (data about other data) that gives the most intelligent filtering and sorting of the information on the internet...

via Steven Downes:

“This is a very important development, probably the closest thing to the semantic social network I've seen, and absolutely the way forward. Study this item carefully. Outfoxed "uses your network of trusted friends and experts to help you find the good stuff and avoid the bad." In a nutshell, it captures evaluations and recommendations from your network of friends (which are stored in XM:L and harvested by your harvester). These recommendations are then used to annotate such things as Google searches, application lists and more (for good measure the author tosses in Phishing & spyware protection). There is a Master's Thesis attached to the (free, open source) demonstration software. By Stan James, May, 2005”