Data deduplication
Data deduplication or dedupe is an approach to information storage and transmission that leverages natural data redundancy to improve performance and conserve resources. Repeated data is identified by analysis, and if the data needs to be stored or transmitted multiple times, a brief reference to the data can be used.
Data deduplication is similar to, but not the same as, data compression. Whereas data compression creates efficient encodings of redundant data, deduplication permits a single instance of data to be shared by multiple objects in a file system or data stream. Deduplication analysis can be performed after the data is completely written ("out-of-band" deduplication), or while a stream of data is being transmitted ("in-band" deduplication).
Deduplication systems
The following are examples of data systems that offer deduplication features.
- Btrfs
- Dropbox
- FreeNAS
- Microsoft Azure StorSimple 8000.
- OpenDedup
- RHEL (Red Hat Enterprise Linux) VDO
- SoftNAS Cloud Enterprise.
- Veritas CloudCatalyst NetBackup.
- Windows Server
- ZFS