[task #15803] Match program builds k-d tree and later read from it

[Mohammad Akhlaghi]

Update of task #15803 (project gnuastro):

                  Status:             In Progress => Done                   
        Percent Complete:                     90% => 100%                   

    _______________________________________________________

Follow-up Comment #10:

A first, fully working (as far as I have tested it!) version of this branch
has been completed and was just merged into the 'master' branch (from Commit
c36f753a5bc4
<https://git.savannah.gnu.org/cgit/gnuastro.git/commit/?id=c36f753a5bc4> to
Commit f5d7d1a351
<https://git.savannah.gnu.org/cgit/gnuastro.git/commit/?id=f5d7d1a351>). So I
am closing this task.

It involved a lot of work, but the final result is very impressive, and also
lead to many good improvements in Table reading (in parallel) and
finding/fixing bug #61462 while testing on various large input tables.

To see the current status of the implementation of this branch, see the new
"Matching algorithms" section of the book from the links below (which also
include the tarball, in case you would like to try it):

https://akhlaghi.org/gnuastro.pdf
https://akhlaghi.org/gnuastro-latest.tar.lz

The only thing that isn't yet implemented is the automatic feature and the
benchmarking tests (to deduce when to use sort-based and when to use k-d tree
based). But I thought that its not necessary because now by default the k-d
tree method is used. The speed improvements of the sort-based method are only
relevant when the table is small, and in those cases, even 10 times speed
improvement will not be noticeable for a user (it will be a small fraction of
milli-second anyway!).

The benchmarking Python scripts have been removed in Commit a6b838c488
<https://git.savannah.gnu.org/cgit/gnuastro.git/commit/?id=a6b838c488>, but
are present in the commits before it, in case they may be necessary later. But
generally, Python is not a future-proof language, so it is discouraged. 

In that spirit, while completing the work on this branch, I added a
'bin/match/debug-1.sh' script to the version-controlled source of Gnuastro
that will allow easy creation of debugging tables. We can later implement the
benchmarking steps in shell also (if necessary).

    _______________________________________________________

Reply to this item at:

  <https://savannah.gnu.org/task/?15803>

_______________________________________________
  Message sent via Savannah
  https://savannah.gnu.org/