Today I had need to find out if some files in a directory structure were the same or not and came across a couple of pages that may be of use if anyone else needs this.
My use case is text files, however, these should work for arbitrary file types.
rmdupe compares files of the same size using a byte-by-byte compare. Usefully has a simulation mode too.
http://embraceubuntu.com has links to lots of useful programs. It’s an old blog entry, but still very useful.