dedup

data deduplication program
git clone git://git.2f30.org/dedup.git
Log | Files | Refs | README | LICENSE

README (1101B)


      1 What is it?
      2 ===========
      3 
      4 dedup is a simple data deduplication program.
      5 
      6 Getting started
      7 ===============
      8 
      9 To use dedup you have to first initialize the repository.
     10 
     11     dedup -r ~/repo -i
     12 
     13 dedup only handles a single file at a time, so using tar is advised.
     14 For example, to dedup a directory tree you can invoke dedup as
     15 follows:
     16 
     17     tar -c ~/dir | dedup -r ~/repo -m "$(date)"
     18 
     19 This will create .{snapshots,store} files in the ~/repo directory.
     20 The store file contains all the unique blocks.  The snapshots file
     21 contains all the revisions of files that have been deduplicated.  The
     22 -m flag is used to attach an arbitrary message to the snapshot.
     23 
     24 To list all known revisions run:
     25 
     26     dedup -r ~/repo -l
     27 
     28 You will get a list of hashes.  Each hash corresponds to a single file
     29 (in this case, a tar archive).
     30 
     31 To extract a file from the deduplicated store run:
     32 
     33     dedup -r ~/repo -e <hash> > dir.tar
     34 
     35 Portability
     36 ===========
     37 
     38 dedup works on Linux, BSD and possibly other UNIX-like systems.
     39 
     40 Dependencies
     41 ============
     42 
     43   - liblz4
     44 
     45 Contact
     46 =======
     47 
     48 You can reach us through irc.2f30.org/6697 at #2f30.