Much has been written about the need to manage and control the ever-expanding mountains of data. Arguably the area within IT that has made the most progress in this effort is email archiving.

But vendor GlassHouse Technologies recently conducted a survey on email archiving and found that in a surprising number of organisations, this is not the case. It found that, in 65% of the 100-plus companies surveyed, IT was the primary decision maker for email retention policy, where regulatory and business requirements should be the primary drivers for data management policies.

However, in contrast to other application data categories, email is relatively straightforward. Dominated by a couple of applications (Exchange and Domino), with well established and understood data formats, it has been possible for a variety of third parties to develop tools to archive email in an automated, policy-driven manner.

In fact, an email archive can be viewed as a special-purpose data warehouse, and one of the primary duties of an email archiving application is to perform the extract, transform, load (ETL) function. Once in the repository it is easy to apply appropriate retention policies and perform searches for e-discovery as well as other kinds of analysis (e.g. questionable use).

But email archiving applications, unfortunately, can't set policies. It can implement and enforce them, but ultimately the tool is only as good as the policies applied to it. This is where the GlassHouse survey shows many organisations fall short.

The survey found that significant number of organisations either have no defined policy or are applying a "one size fits all" approach to email, which can be both risky and inefficient. The majority of respondents (35%) said they archive everything for a fixed period, while by contrast 26% said they had no retention policies in place at all. A further 22% said they employed multiple, tailored retention policies, and 17% said they attempt to keep all email files forever.

Factors affecting email policy include compliance mandates to maintain email records for specified periods of time and its increasing use as a knowledge base.