SPUG:Re: Appropriateness of MD5

[David Dyck]

On Thu, 20 Mar 2003 at 21:02 -0800, Jeremy Mates <jmates at sial.org> wrote:

> perl -MDigest::MD5 -nl \
> -e '$h = sprintf "%08x", unpack "N", Digest::MD5::md5($_);' \
> -e 'print "$s{$h} $_ $h" if exists $s{$h}; $s{$h} = $_' /usr/share/dict/words
>
> My words file has 234937 lines, and reports the following duplicates:
>
> backvelder draftily dcb75a20
> burlily hobbly 32e38c3a
> footlock monosporous 68c7041c
> Harrovian skag 68db67e7
> Palaeothentidae trabacolo 8a52b103
> preceptorially untaughtness 2a759a38

but you are only looking at the first 4 bytes of the 16 byte sum

 change
     sprintf "%08x", unpack "N",
 to
     unpack "H*",
and you'll get no duplicates

perl -MDigest::MD5 -nl -e '$h = unpack "H*", Digest::MD5::md5($_);' -e 'print "$h $_" ' words | sort
2a759a38153db6bb9246c4e93f2ced65 preceptorially
2a759a38905794ea80656c41b64d7977 untaughtness
32e38c3a36344b13aa24a054c2815783 hobbly
32e38c3a510460fdc436e2afc364b6e9 burlily
68c7041c485a9cdea92d3a400ef8b32d monosporous
68c7041c4c7e90a98b2af43c0f9f20c7 footlock
68db67e78ff999044702487ce55e4322 Harrovian
68db67e790b6edb560bdee7ea6770a85 skag
8a52b103fa1ee0f20937425c6d7aaadb trabacolo
8a52b103fb136b3801414fd241a59826 Palaeothentidae
dcb75a206bebc34159d9ca7d473e484a backvelder
dcb75a20a7a9ff1d5843b166528117e3 draftily