Re: Finding duplicate data

[Adam Nave <nave () MACALESTER EDU>]

Lifehacker.com regularly covers the topic of duplicate files.

http://lifehacker.com/tag/file-diff/

A novice or uninformed user can easily mess up their system by
de-duplicating system files and such. You might be better off providing a
search tool.

Revision control systems (CVS or SVN, typically used for organizing
programming projects) can be adapted easily for document management. The
learning curve is a bit steep for the average user, unfortunately. There are
also other more friendly products that have versioning built in. SharePoint
might be one (I'm not familiar with it).

--Adam

On Wed, Feb 25, 2009 at 12:01 PM, Dexter Caldwell <
Dexter.Caldwell () furman edu> wrote:

> I use Duplicate File Detective:
>
> http://www.duplicate-file-detective.com/
>
> D/C
>
>
>
> The EDUCAUSE Security Constituent Group Listserv
> <SECURITY () LISTSERV EDUCAUSE EDU> writes:
> > This is an availability issue (it is also an information management
> > issue).  Sometimes people have so much stuff on their hard drives that
> > important stuff gets misplaced, or they operate from old versions etc,
> > with wrong conclusions reached (or previous decisions rehashed).  And,
> > yes, I have experienced some of that myself.
> >
> > I have found backup vendors working in this space, but they seem to just
> > try to get as much onto a tape as is possible.
> >
> > What I am looking for is an easy to use gui that we could put in
> > people's hands that does a couple of things well.
> >
> > 1) Detects pure duplicates, (probably by hash and size) and then allows
> > for checks for deleting in various places.  May even place a "moved to"
> > message where deleted.
> >
> > 2) Checks for "versions" near matches, then displays sizes and
> > modification times, and then has similar types of deletion/elimination.
> > This is probably harder.
> >
> > Anyone know of anything like this?  As we try to improve personal data
> > management, something like this could be a great tool.
> >
> > Jim
> >
> >
> > - - - -
> > Jim Moore, CISSP, IAM
> > Information Security Officer
> > Rochester Institute of Technology
> > 151 Lomb Memorial Drive
> > Rochester, NY 14623-5603
> > (585) 475-5406 (office)
> > (585) 255-0809 (Cell - Incident Reporting & Emergencies)
> > (585) 475-7920 (fax)
> >
> >
> > If you consciously try to thwart opponents, you are already late.
> > Miyamoto Musashi, Japanese philosopher/samurai, 1645
> >
> >
> > "If we do not, on a national scale, attack organized criminals with
> > weapons and techniques as effective as their own, they will destroy us."
> > Robert F. Kennedy, 1960
> >
> > Confidentiality Notice:  Do the right thing.  If this has the words
> > "Confidential" or "Private" in the subject line, or similar language in
> > the email body, or as a label on any attachment, then think.  Do you
> > know me?  Did you expect to receive this?  Do you recognize and work
> > with the other addressees?  If not, then you probably received this in
> > error.  Please, be respectful and courteous, and delete it immediately.
> > Please, don't forward it to anyone.
> >
> > Now, wasn't that simple.  Just, if you had made an error in a sensitive
> > email, and I received it, what would you want me to do with it?



--
Adam Nave, CISSP
Academic Technologist
Macalester College