The Dilemmas of Storage

Discussion of information inevitably leads to the question of storage and, more importantly, information management. Again, there are wider issues here than those usually considered.

Storage is certainly now cheap, very cheap. But, storage cost alone isn’t the issue – managing, finding and retrieving the information in the storage is the real cost. Throwing more storage at the problem will not, alone, solve the information management issues.

Speed and reliability are, of course, fundamental requirements. If storage is not fast and demonstrably reliable, users will replicate needlessly ‘just in case’, so exacerbating the problem and creating an escalating demand for storage.

There are also hidden issues to do with information availability. Having reliable mass storage is of no use if the information cannot be found when needed or isn’t in the store in the first place. Data stored on individual PCs is at risk; people are too busy or too lazy to make backups. Locally installed applications may be faster but also are a single point of failure – even if the data is replicated it is often inaccessible if the application itself is not available because the PC has a fault.

The Web 2.0 approach to provision of office type services, like that for Web-based email, is to have all the information stored on the provider’s servers. A well-managed data centre can achieve very high reliability and availability standards (the goal being the “five nines”, i.e. 99.999% uptime), much higher than those of a desktop PC. Try to remember the last time Yahoo or Google was not working or lost your information and reflect on how many users they serve and how much data they manage, especially in their picture and video databases.

However, this does leave the problem of what to do when the information servers are inaccessible, due to network faults or Internet traffic overload, or when offline working is required (e.g. while travelling). However, very interesting new Web 2.0 techniques are emerging from some companies. Zimbra is trialling a two way sync of mail, calendar, contacts, and documents between offline stores and online Web database. If these ideas prove workable (there are others, like the Moxie rich text editor which uses Dojo Storage, and there will, certainly, be many more), they will go a long way towards making the Web 2.0 approach a more comprehensive solution.

(Written November 2006)