[gnuastro-devel] [task #14245] Root data structure

[Mohammad Akhlaghi]

Update of task #14245 (project gnuastro):

        Percent Complete:                     10% => 20%                    

    _______________________________________________________

Follow-up Comment #1:

This is just an update. 

A large number of functions have already been defined for this data structure,
please see data.h
<https://gitlab.com/makhlaghi/gnuastro/blob/datastruct/lib/gnuastro/data.h>.
The fits.h
<https://gitlab.com/makhlaghi/gnuastro/blob/datastruct/lib/gnuastro/fits.h>
functions have also been corrected (and greatly simplified) to return this
data structure (for image arrays, header keywords, and I am working on table
columns now).

Gnuastro's Arithmetic has been fully migrated to this new data structure, and
as it currently stands, it is the only program that works as expected in this
branch. The main work-horse behind arithmetic on this data structure is now a
library, so library users can also benefit from them. Until now, Arithmetic
would convert the input data to double type, do the arithmetic operations,
then save the results in the desired type. When the data sets get large (>1Gb
for example), this conversion itself consumes a large amount of CPU and memory
resources. Thanks to this data structure, we arithmetic library functions will
now do all the work on the native type of the data, so no conversion is
necessary, and is a great step towards better performance.

So in case you want to compile/test this branch please configure Gnuastro with
`--enable-arithmetic --enable-table' (the new FITS functions are now corrected
in the other programs yet).

Just one note: to efficiently account for native arithmetic operations on all
the combinations of the different data types (especially for the binary
operators that need two operands), the only way I could come up with until now
was heavy usage of macros, for example see data-arithmetic-binary.c
<https://gitlab.com/makhlaghi/gnuastro/blob/datastruct/lib/data-arithmetic-binary.c>.


These macros greatly help in efficient running of the binary arithmetic
operators, but it will greatly lengthen the compilation time. For example, to
allow all the types to be natively operated on, on my system with default
configure options, the compilation will take about 20 minutes! So I have
defined configure time options to specify which types to allow for native
binary arithmetic (`--enable-bin-op-XXXX', where `XXXX' is the type name). If
the data type is not native, it will be converted to a native type, the
arithmetic will be done and the output will be set to the proper type. I have
tried to fully describe this under these options
<https://gitlab.com/makhlaghi/gnuastro/blob/datastruct/doc/gnuastro.texi#L3131>
in the "Configuring" subsection of the book. With the currently default native
types, it takes about 2 minutes to compile by default. But if compilation time
is very important for you, you can disable optimization and shared libraries
(like in debugging mode) and compilation time will be like before (less than a
minute).

I am now working on fully implementing Gnuastro's Table program to use this
data structure. Reading of FITS tables has been implemented, and I am now
working on the writing and also reading/writing of ASCII tables. Once table
columns can also be easily read using this structure, most of the job will be
done: since the inputs/outputs of the programs are either table columns or
images/arrays.

    _______________________________________________________

Reply to this item at:

  <http://savannah.gnu.org/task/?14245>

_______________________________________________
  Message sent via/by Savannah
  http://savannah.gnu.org/