ExaGrid's Eye on Deduplication

As we wind through this intense political season, discussions of taxes are everywhere.   Taxes are a fact of life.  But before this post is misunderstood as a political position, let me explain the type of tax I am discussing today:  the dedupe tax.

The fact is that deduplication is an extremely important storage technology, especially in disk-based backup products.  It is deduplication that allows organizations to store large amounts of data in a small amount of disk and to transfer backups over wide area networks to disaster recovery sites by moving a very small amount of data.

However, as with most compelling technologies, there are trade-offs to be made.  Each disk-based backup vendor is deciding which trade-offs are right for your data center and your backups. Given the importance of your backup window and restore times, the most critical trade-offs to be considered are the performance trade-offs -- or what I call the "dedupe tax".  The dedupe tax is a performance hit that could show up as a longer backup window or a dramatically slowed restore.

The assessment of the tax varies with the deduplication method employed by the vendor:

• Post-process de-duplication (implemented by ExaGrid Systems) - backups are written directly to disk in their entirety and maintained for rapid restore of your most recent backup. Since 90% or more of all restores are done from your most recent backup, this method avoids the de-dupe tax for 90 to 95% of restores and 100% of backups.
• In-line deduplication - is performed on backup data on its way to disk. This method charges you the de-duplication tax for every backup into the system and every restore out of the system. The promise is the use of less disk (but not lower cost) and simplicity. With in-line, you are in the 100% tax bracket!

So, what is the cost of the dedupe tax? It can be substantial.  It can slow your backups down by as much as 2x to 5x versus raw disk speeds.  Similarly, it can dramatically slow down restores and force your organization to wait much longer to recover data when you can least afford it - during a critical recovery scenario.

Look for implementations that allow you to determine when and how much dedupe tax should be paid.  With ExaGrid's post-processing GRID architecture, all backups and most restores are tax free.  Only restores from older, deduplicated data incur the de-dupe tax and these are generally smaller restores with much less urgency associated with them.

Taxes must be paid. But, do not pay a gigabyte per second more in dedupe tax than needed!

Marc Crespi is the Vice President of Product Management for ExaGrid Systems, Inc.