removing replicated rows from a (large) matrix

[Joao Cardoso]

Hi,

I often need to remove repeated rows from a matrix. I use the following
script, which is the fastest I could think about (I have not checked
Knuth).

Is there a faster method?

Thanks,
Joao

function ix_del = unique(data)

# ix = unique(data)
#
# return the indexes of identical rows entries in `data',
# leaving out one repeated example.
#
# Executing `data(unique(data),:) = []', will leave `data' with unique
rows,

if (isempty(data))
        ix_del = [];
        return
endif

if (columns(data) == 1)
        data = [data data]; # the alg. fail if columns(data) == 1
endif

[t ix] = sort(data(:,1));
data = data(ix,:);                      # organize data as blocks

nr = rows(data);

ix_del = []; t = ones(nr,1);

for i=1:nr
    if (any(i==ix_del))         # already matched
        continue
    endif
    
    ixx = find(data(i,1) == data(i+1:nr,1));    # get a block
    if (isempty(ixx))
       continue
    endif
    
    ixxx = find(all((data(i+ixx,:) ==
ones(length(ixx),1)*data(i,:))'))+i;
    ix_del = [ix_del ixxx];
endfor

ix_del = ix(ix_del);

endfunction



-----------------------------------------------------------------------
Octave is freely available under the terms of the GNU GPL.

Octave's home on the web:  http://www.che.wisc.edu/octave/octave.html
How to fund new projects:  http://www.che.wisc.edu/octave/funding.html
Subscription information:  http://www.che.wisc.edu/octave/archive.html
-----------------------------------------------------------------------