uniq command gives wrong and different results

[Fong Tin Joen, Raimund]

Dear all,

I hope that someone can help me out with the next issue on a system running 
RedHat Fedora Core release 3 (Heidelberg).

When using the uniq command on the attached file Foo  <<Foo>> the result is not 
as expected.
It also gives different results depending on using it several times after 
eachother.

I can supply some extra information about the kernel and the rpm version ( 
which includes the uniq command ) used.
# type uniq
uniq is /usr/bin/uniq
# rpm -qf /usr/bin/uniq
coreutils-5.2.1-31
# uname -a
Linux tux06 2.6.9-1.667 #1 Tue Nov 2 14:41:25 EST 2004 i686 i686 i386 GNU/Linux 
# rpm -qa | grep coreutil
coreutils-5.2.1-31

My Environment settings are as follows.
# env
_=/bin/env
SSH_CONNECTION=::ffff:144.44.217.242 1218 ::ffff:144.44.110.66 22 
PATH=/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/X11R6/bin:/root/bin
SHELL=/bin/bash
HOSTNAME=tux06
USER=root
MAIL=/var/spool/mail/root
HOME=/root
OLDPWD=/root/scripts/bingo
HISTSIZE=1000
PWD=/tmp
SSH_CLIENT=::ffff:144.44.217.242 1218 22 LESSOPEN=|/usr/bin/lesspipe.sh %s 
TERM=xterm INPUTRC=/etc/inputrc KDEDIR=/usr
G_BROKEN_FILENAMES=1
SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass
LANG=en_US.UTF-8
QTDIR=/usr/lib/qt-3.3
SHLVL=1
SSH_TTY=/dev/pts/1
LOGNAME=root
LS_COLORS=no=00:fi=00:di=00;34:ln=00;36:pi=40;33:so=00;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:ex=00;32:*.cmd=00;32:*.exe=00;32:*.com=00;32:*.btm=00;32:*.bat=00;32:*.sh=00;32:*.csh=00;32:*.tar=00;31:*.tgz=00;31:*.arj=00;31:*.taz=00;31:*.lzh=00;31:*.zip=00;31:*.z=00;31:*.Z=00;31:*.gz=00;31:*.bz2=00;31:*.bz=00;31:*.tz=00;31:*.rpm=00;31:*.cpio=00;31:*.jpg=00;35:*.gif=00;35:*.bmp=00;35:*.xbm=00;35:*.xpm=00;35:*.png=00;35:*.tif=00;35:

Should it be stripped of the mail the content of the file Foo is :
# cat Foo
./usr/share/apps/kturtle/examples/fr_FR/carré.logo
./usr/share/apps/kturtle/examples/fr_FR/carré.logo
./usr/share/apps/kturtle/examples/fr_FR/enroulé.logo
./usr/share/doc/man-pages-da-0.1.1/læsmig
./usr/share/doc/man-pages-da-0.1.1/læsmig
./usr/share/apps/kturtle/examples/fr_FR/enroulé.logo

When using "sort Foo | uniq -u" I get an unexpected result whick looks like
- Beginning -
# sort Foo | uniq -u
./usr/share/apps/kturtle/examples/fr_FR/carré.logo
./usr/share/apps/kturtle/examples/fr_FR/carré.logo
./usr/share/apps/kturtle/examples/fr_FR/enroulé.logo
./usr/share/doc/man-pages-da-0.1.1/læsmig
./usr/share/doc/man-pages-da-0.1.1/læsmig
./usr/share/apps/kturtle/examples/fr_FR/enroulé.logo

The result looks random when trying the same command several times after 
eachother. 
- Again -
# sort Foo | uniq -u
./usr/share/apps/kturtle/examples/fr_FR/enroulé.logo
./usr/share/doc/man-pages-da-0.1.1/læsmig
./usr/share/doc/man-pages-da-0.1.1/læsmig
./usr/share/apps/kturtle/examples/fr_FR/enroulé.logo

The usage of sort -u does the job fine
#  sort -u Foo
./usr/share/apps/kturtle/examples/fr_FR/carré.logo
./usr/share/apps/kturtle/examples/fr_FR/enroulé.logo
./usr/share/doc/man-pages-da-0.1.1/læsmig

Strange result also appears when using "sort Foo | uniq -d" 
- Beginning -
# sort Foo | uniq -d

- Again -
# sort Foo | uniq -d
./usr/share/apps/kturtle/examples/fr_FR/carré.logo

Next shows the same strange result for just the uniq command
- Beginning -
# sort Foo | uniq
./usr/share/apps/kturtle/examples/fr_FR/carré.logo
./usr/share/apps/kturtle/examples/fr_FR/carré.logo
./usr/share/apps/kturtle/examples/fr_FR/enroulé.logo
./usr/share/doc/man-pages-da-0.1.1/læsmig
./usr/share/doc/man-pages-da-0.1.1/læsmig
./usr/share/apps/kturtle/examples/fr_FR/enroulé.logo

- Again -
# sort Foo | uniq
./usr/share/apps/kturtle/examples/fr_FR/carré.logo
./usr/share/apps/kturtle/examples/fr_FR/enroulé.logo
./usr/share/doc/man-pages-da-0.1.1/læsmig
./usr/share/doc/man-pages-da-0.1.1/læsmig
./usr/share/apps/kturtle/examples/fr_FR/enroulé.logo

Perhaps it has something to do with the characters in the lines, because until 
now uniq worked fine with the exception of the above example.

A file Foo2 contains the following lines
Abc
Def
Abc
Def

Now uniq works fine even when you try it several times
# sort Foo2 | uniq -d
Abc
Def

# sort Foo2 | uniq -u


PS: I've also tried the above commands on a Solaris 9 system with the same 
commands and the results are as one should expect.

Hope to hear soon.

Foo
Description: Foo