[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[gnuastro-commits] master c2da4b4: Fits & Table: new --keyvalue (Fits) a
|
From: |
Mohammad Akhlaghi |
|
Subject: |
[gnuastro-commits] master c2da4b4: Fits & Table: new --keyvalue (Fits) and date-to-sec operator (Table) |
|
Date: |
Wed, 17 Feb 2021 23:24:48 -0500 (EST) |
branch: master
commit c2da4b478f1757a34cc86976e38f39199f3000f8
Author: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Commit: Mohammad Akhlaghi <mohammad@akhlaghi.org>
Fits & Table: new --keyvalue (Fits) and date-to-sec operator (Table)
Until now, the Fits program had no easy value to just view the values of
certain keywords. You had to get the list of all keywords, then use 'grep'
or 'awk' to extract the ones you want. This was inconvenient in two
aspects: extracting string values (which have a single-quote around their
values), or the need to select a sub-set of many FITS files.
With this commit a new '--keyvalue' option has been added to the FITS
program to address this issue and greatly simplify the inspection of many
files. This option will only read the requested keywords from any number of
input FITS files and write the results in a table format (that can easily
be pipped onto Table for example for selecting a sub-set of the FITS images
based on their keyword values.
Having done this, another useful feature was implemented: the Table
program's Column Arithmetic now has a new operator called
'date-to-sec'. With it, you can easily convert the standard FITS dates
(found in the 'DATE' or 'DATE-OBS' keywords of FITS files).
The following minor changes were also made in the process:
- The FITS program can now take any number of input FITS file
arguments. So the name of the input is now put in a list of strings
instead of a simple string. However, for now, this is only used with the
'--keyvalue' option. For the others, only the variable with the input
filename has been updated.
- In the blank library, for string types, we now also count NULL pointers
(for each string) as blank. Also the 'gal_blank_number' function can now
also work on string datasets (until now it only used the Statistics
library, which was limited to numerical data types).
- The 'gal_fits_key_date_to_seconds' function will return a blank output
when the input is a blank string.
- The 'gal_fits_key_read_from_ptr' function can now find the best value
for a given keyword value when a type isn't specified.
I had been planning to add this feature for some time, but the final
motivation came after a discussion with Alberto Madrigal.
---
NEWS | 17 +++
bin/fits/args.h | 30 +++-
bin/fits/fits.c | 28 ++--
bin/fits/keywords.c | 356 +++++++++++++++++++++++++++++++++++++++++--
bin/fits/main.h | 52 ++++---
bin/fits/ui.c | 49 +++---
bin/fits/ui.h | 39 ++---
bin/table/arithmetic.c | 84 ++++++++++
bin/table/arithmetic.h | 1 +
bin/table/table.c | 2 +-
doc/announce-acknowledge.txt | 1 +
doc/gnuastro.texi | 177 +++++++++++++++------
lib/blank.c | 31 +++-
lib/fits.c | 45 +++++-
14 files changed, 756 insertions(+), 156 deletions(-)
diff --git a/NEWS b/NEWS
index 980d9eb..738a0db 100644
--- a/NEWS
+++ b/NEWS
@@ -21,6 +21,19 @@ See the end of the file for license conditions.
when you have a large dataset and just want to see a random sub-set of
the rows. It takes an integer, selects that many rows from the input
randomly.
+ - New column arithmetic operators:
+ - 'date-to-sec' Convert FITS date format ('YYYY-MM-DDThh:mm:ss') into
+ seconds from the Unix epoch (1970-01-01,00:00:00 UTC). This can be
+ very useful in combination with the new '--keyvalue' option of the
+ Fits program to sort your FITS images based on observation time.
+
+ Fits:
+ --keyvalue: Print only the values of the FITS keywords given to this
+ option in separate columns. This option can take multiple values and
+ many FITS files. Thus generating a table of keyword values (with one
+ row per file). Its output can thus either be piped to the Table
+ program for selecting a certain sub-set of your FITS files, or sorting
+ them for example.
** Removed features
@@ -28,6 +41,10 @@ See the end of the file for license conditions.
Library:
- gal_fits_key_write_wcsstr: also takes WCS structure as argument.
+ - gal_fits_key_read_from_ptr: providing a numerical datatype for the
+ desired keyword's value is no longer mandatory. When not given, the
+ smallest numeric datatype that can keep the value will be found and
+ used.
** Bugs fixed
diff --git a/bin/fits/args.h b/bin/fits/args.h
index 1e5f464..af1d90c 100644
--- a/bin/fits/args.h
+++ b/bin/fits/args.h
@@ -164,7 +164,7 @@ struct argp_option program_options[] =
UI_KEY_ASIS,
"STR",
0,
- "Write the argument string as is into the header.",
+ "Write value as-is (may corrupt FITS file).",
UI_GROUP_KEYWORD,
&p->asis,
GAL_TYPE_STRLL,
@@ -173,6 +173,19 @@ struct argp_option program_options[] =
GAL_OPTIONS_NOT_SET
},
{
+ "keyvalue",
+ UI_KEY_KEYVALUE,
+ "STR[,STR,...]",
+ 0,
+ "Only print the value of requested keyword(s).",
+ UI_GROUP_KEYWORD,
+ &p->keyvalue,
+ GAL_TYPE_STRLL,
+ GAL_OPTIONS_RANGE_ANY,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET
+ },
+ {
"delete",
UI_KEY_DELETE,
"STR",
@@ -347,6 +360,21 @@ struct argp_option program_options[] =
GAL_OPTIONS_NOT_MANDATORY,
GAL_OPTIONS_NOT_SET
},
+ {
+ "colinfoinstdout",
+ UI_KEY_COLINFOINSTDOUT,
+ 0,
+ 0,
+ "Column info/metadata when printing to stdout.",
+ GAL_OPTIONS_GROUP_OUTPUT,
+ &p->colinfoinstdout,
+ GAL_OPTIONS_NO_ARG_TYPE,
+ GAL_OPTIONS_RANGE_0_OR_1,
+ GAL_OPTIONS_NOT_MANDATORY,
+ GAL_OPTIONS_NOT_SET
+ },
+
+
diff --git a/bin/fits/fits.c b/bin/fits/fits.c
index 766e93c..19a7a1b 100644
--- a/bin/fits/fits.c
+++ b/bin/fits/fits.c
@@ -99,7 +99,7 @@ fits_print_extension_info(struct fitsparams *p)
/* Open the FITS file and read the first extension type, upon moving to
the next extension, we will read its type, so for the first we will
need to do it explicitly. */
- fptr=gal_fits_hdu_open(p->filename, "0", READONLY);
+ fptr=gal_fits_hdu_open(p->input->v, "0", READONLY);
if (fits_get_hdu_type(fptr, &hdutype, &status) )
gal_fits_io_error(status, "reading first extension");
@@ -257,7 +257,7 @@ fits_print_extension_info(struct fitsparams *p)
if(!p->cp.quiet)
{
printf("%s\nRun on %s-----\n", PROGRAM_STRING, ctime(&p->rawtime));
- printf("HDU (extension) information: '%s'.\n", p->filename);
+ printf("HDU (extension) information: '%s'.\n", p->input->v);
printf(" Column 1: Index (counting from 0, usable with '--hdu').\n");
printf(" Column 2: Name ('EXTNAME' in FITS standard, usable with "
"'--hdu').\n");
@@ -287,7 +287,7 @@ fits_hdu_number(struct fitsparams *p)
int numhdu, status=0;
/* Read the first extension (necessary for reading the rest). */
- fptr=gal_fits_hdu_open(p->filename, "0", READONLY);
+ fptr=gal_fits_hdu_open(p->input->v, "0", READONLY);
/* Get the number of HDUs. */
if( fits_get_num_hdus(fptr, &numhdu, &status) )
@@ -307,7 +307,7 @@ fits_hdu_number(struct fitsparams *p)
static void
fits_datasum(struct fitsparams *p)
{
- printf("%ld\n", gal_fits_hdu_datasum(p->filename, p->cp.hdu));
+ printf("%ld\n", gal_fits_hdu_datasum(p->input->v, p->cp.hdu));
}
@@ -323,14 +323,14 @@ fits_pixelscale(struct fitsparams *p)
double multip, *pixelscale;
/* Read the desired WCS. */
- wcs=gal_wcs_read(p->filename, p->cp.hdu, 0, 0, &nwcs);
+ wcs=gal_wcs_read(p->input->v, p->cp.hdu, 0, 0, &nwcs);
/* If a WCS doesn't exist, let the user know and return. */
if(wcs)
ndim=wcs->naxis;
else
error(EXIT_FAILURE, 0, "%s (hdu %s): no WCS could be read by WCSLIB, "
- "hence the pixel-scale cannot be determined", p->filename,
+ "hence the pixel-scale cannot be determined", p->input->v,
p->cp.hdu);
/* Calculate the pixel-scale in each dimension. */
@@ -343,7 +343,7 @@ fits_pixelscale(struct fitsparams *p)
{
printf("Basic information for --pixelscale (remove extra info "
"with '--quiet' or '-q')\n");
- printf(" Input: %s (hdu %s) has %zu dimensions.\n", p->filename,
+ printf(" Input: %s (hdu %s) has %zu dimensions.\n", p->input->v,
p->cp.hdu, ndim);
printf(" Pixel scale in each FITS dimension:\n");
for(i=0;i<ndim;++i)
@@ -433,11 +433,11 @@ fits_skycoverage(struct fitsparams *p)
double *center, *width, *min, *max;
/* Find the coverage. */
- if( gal_wcs_coverage(p->filename, p->cp.hdu, &ndim,
+ if( gal_wcs_coverage(p->input->v, p->cp.hdu, &ndim,
¢er, &width, &min, &max)==0 )
error(EXIT_FAILURE, 0, "%s (hdu %s): is not usable for finding "
"sky coverage (either doesn't have a WCS, or isn't an image "
- "or cube HDU with 2 or 3 dimensions", p->filename, p->cp.hdu);
+ "or cube HDU with 2 or 3 dimensions", p->input->v, p->cp.hdu);
/* Inform the user. */
if(p->cp.quiet)
@@ -453,7 +453,7 @@ fits_skycoverage(struct fitsparams *p)
}
else
{
- printf("Input file: %s (hdu: %s)\n", p->filename, p->cp.hdu);
+ printf("Input file: %s (hdu: %s)\n", p->input->v, p->cp.hdu);
printf("\nSky coverage by center and (full) width:\n");
switch(ndim)
{
@@ -474,7 +474,7 @@ fits_skycoverage(struct fitsparams *p)
}
/* For the range type of coverage. */
- wcs=gal_wcs_read(p->filename, p->cp.hdu, 0, 0, &nwcs);
+ wcs=gal_wcs_read(p->input->v, p->cp.hdu, 0, 0, &nwcs);
printf("\nSky coverage by range along dimensions:\n");
for(i=0;i<ndim;++i)
printf(" %-8s %-15.10g%-15.10g\n", gal_wcs_dimension_name(wcs, i),
@@ -506,7 +506,7 @@ fits_hdu_remove(struct fitsparams *p, int *r)
hdu=gal_list_str_pop(&p->remove);
/* Open the FITS file at the specified HDU. */
- fptr=gal_fits_hdu_open(p->filename, hdu, READWRITE);
+ fptr=gal_fits_hdu_open(p->input->v, hdu, READWRITE);
/* Delete the extension. */
if( fits_delete_hdu(fptr, &hdutype, &status) )
@@ -568,12 +568,12 @@ fits_hdu_copy(struct fitsparams *p, int cut1_copy0, int
*r)
hdu=gal_list_str_pop(&list);
/* Open the FITS file at the specified HDU. */
- in=gal_fits_hdu_open(p->filename, hdu,
+ in=gal_fits_hdu_open(p->input->v, hdu,
cut1_copy0 ? READWRITE : READONLY);
/* If the output isn't opened yet, open it. */
if(out==NULL)
- out = ( ( gal_fits_hdu_format(p->filename, hdu)==IMAGE_HDU
+ out = ( ( gal_fits_hdu_format(p->input->v, hdu)==IMAGE_HDU
&& p->primaryimghdu )
? fits_open_to_write_no_blank(p->cp.output)
: gal_fits_open_to_write(p->cp.output) );
diff --git a/bin/fits/keywords.c b/bin/fits/keywords.c
index 41a3525..9d4a950 100644
--- a/bin/fits/keywords.c
+++ b/bin/fits/keywords.c
@@ -30,6 +30,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <gnuastro/wcs.h>
#include <gnuastro/fits.h>
+#include <gnuastro/pointer.h>
#include <gnuastro-internal/timing.h>
#include <gnuastro-internal/checkset.h>
@@ -52,7 +53,7 @@ static void
keywords_open(struct fitsparams *p, fitsfile **fptr, int iomode)
{
if(*fptr==NULL)
- *fptr=gal_fits_hdu_open(p->filename, p->cp.hdu, iomode);
+ *fptr=gal_fits_hdu_open(p->input->v, p->cp.hdu, iomode);
}
@@ -320,8 +321,8 @@ keywords_verify(struct fitsparams *p, fitsfile **fptr)
" - CHECKSUM: verifies data and keywords.\n"
"They can be added-to/updated-in an extension/HDU with:\n"
" $ astfits %s -h%s --write=checksum\n"
- "--------\n", PROGRAM_STRING, p->filename, p->cp.hdu,
- ctime(&p->rawtime), p->filename, p->cp.hdu);
+ "--------\n", PROGRAM_STRING, p->input->v, p->cp.hdu,
+ ctime(&p->rawtime), p->input->v, p->cp.hdu);
/* Print the verification result. */
printf("DATASUM: %s\n",
@@ -352,7 +353,7 @@ keywords_copykeys(struct fitsparams *p, char *inkeys,
size_t numinkeys)
if(p->copykeysrange[0]>=numinkeys)
error(EXIT_FAILURE, 0, "%s (hdu %s): first keyword number give to "
"'--copykeys' (%ld) is larger than the number of keywords in this "
- "header (%zu, including the 'END' keyword)", p->filename, p->cp.hdu,
+ "header (%zu, including the 'END' keyword)", p->input->v, p->cp.hdu,
p->copykeysrange[0], numinkeys);
/* If the user wanted to count from the end (by giving a negative value),
@@ -367,7 +368,7 @@ keywords_copykeys(struct fitsparams *p, char *inkeys,
size_t numinkeys)
if(p->copykeysrange[0]>=p->copykeysrange[1])
error(EXIT_FAILURE, 0, "%s (hdu %s): the last keyword given to "
"'--copykeys' (%ld, or %ld after counting from the bottom) "
- "is earlier than the first (%ld)", p->filename, p->cp.hdu,
+ "is earlier than the first (%ld)", p->input->v, p->cp.hdu,
initial, p->copykeysrange[1], p->copykeysrange[0]);
}
@@ -375,7 +376,7 @@ keywords_copykeys(struct fitsparams *p, char *inkeys,
size_t numinkeys)
if(p->copykeysrange[1]>=numinkeys)
error(EXIT_FAILURE, 0, "%s (hdu %s): second keyword number give to "
"'--copykeys' (%ld) is larger than the number of keywords in this "
- "header (%zu, including the 'END' keyword)", p->filename, p->cp.hdu,
+ "header (%zu, including the 'END' keyword)", p->input->v, p->cp.hdu,
p->copykeysrange[1], numinkeys);
@@ -419,7 +420,7 @@ keywords_date_to_seconds(struct fitsparams *p, fitsfile
*fptr)
/* Print the result (for the sub-seconds, print everything after the */
if( !p->cp.quiet )
{
- printf("%s (hdu %s), key '%s': %s\n", p->filename, p->cp.hdu,
+ printf("%s (hdu %s), key '%s': %s\n", p->input->v, p->cp.hdu,
p->datetosec, fitsdate);
printf("Seconds since 1970/01/01 (00:00:00): %zu%s\n\n", seconds,
subsecstr?subsecstr:"");
@@ -448,25 +449,25 @@ keywords_distortion_wcs(struct fitsparams *p)
/* If the extension has any data, read it, otherwise just make an empty
array. */
- if(gal_fits_hdu_format(p->filename, p->cp.hdu)==IMAGE_HDU)
+ if(gal_fits_hdu_format(p->input->v, p->cp.hdu)==IMAGE_HDU)
{
/* Read the size of the dataset (we don't need the actual size!). */
- insize=gal_fits_img_info_dim(p->filename, p->cp.hdu, &ndim);
+ insize=gal_fits_img_info_dim(p->input->v, p->cp.hdu, &ndim);
free(insize);
/* If the number of dimensions is two, then read the dataset,
otherwise, ignore it. */
if(ndim==2)
- data=gal_fits_img_read(p->filename, p->cp.hdu, p->cp.minmapsize,
+ data=gal_fits_img_read(p->input->v, p->cp.hdu, p->cp.minmapsize,
p->cp.quietmmap);
}
/* Read the input's WCS and make sure one exists. */
- inwcs=gal_wcs_read(p->filename, p->cp.hdu, 0, 0, &nwcs);
+ inwcs=gal_wcs_read(p->input->v, p->cp.hdu, 0, 0, &nwcs);
if(inwcs==NULL)
error(EXIT_FAILURE, 0, "%s (hdu %s): doesn't have any WCS structure "
"for converting its distortion",
- p->filename, p->cp.hdu);
+ p->input->v, p->cp.hdu);
/* In case there is no dataset and the conversion is between TPV to SIP,
we need to set a default size and use that for the conversion, but we
@@ -499,7 +500,7 @@ keywords_distortion_wcs(struct fitsparams *p)
{
if( asprintf(&suffix, "-%s.fits", p->wcsdistortion)<0 )
error(EXIT_FAILURE, 0, "%s: asprintf allocation", __func__);
- output=gal_checkset_automatic_output(&p->cp, p->filename, suffix);
+ output=gal_checkset_automatic_output(&p->cp, p->input->v, suffix);
}
gal_checkset_writable_remove(output, 0, p->cp.dontdelete);
@@ -528,6 +529,330 @@ keywords_distortion_wcs(struct fitsparams *p)
+static void
+keywords_value_in_output_copy(gal_data_t *write, gal_data_t *key,
+ size_t in_counter)
+{
+ char **strarrk, **strarrw;
+
+ /* Small sanity check. */
+ if(write->type != key->type)
+ error(EXIT_FAILURE, 0, "%s: the input datasets must have "
+ "the same data type. The 'write' and 'key' arguments "
+ "are respectively '%s' and '%s'", __func__,
+ gal_type_name(write->type, 1),
+ gal_type_name(key->type, 1));
+
+ /* Copy the value. */
+ if(key->type==GAL_TYPE_STRING)
+ {
+ strarrk=key->array;
+ strarrw=write->array;
+ strarrw[ in_counter ] = strarrk[0];
+ strarrk[0]=NULL;
+ }
+ else
+ memcpy(gal_pointer_increment(write->array, in_counter,
+ write->type),
+ key->array, gal_type_sizeof(write->type));
+}
+
+
+
+
+
+/* Write the value in the first row. The first row is unique here: if there
+ is only one input dataset, the dataset name will not be in the
+ output. But when there is more than one dataset, we include a column for
+ the name of the dataset. */
+static gal_data_t *
+keywords_value_in_output_first(struct fitsparams *p, gal_data_t *topout,
+ char *filename, gal_data_t *keysll,
+ size_t ninput)
+{
+ char **strarr;
+ gal_data_t *out=NULL;
+ gal_data_t *write, *key;
+ size_t in_counter=0; /* This function is only for the first row. */
+
+ /* If a name column is necessary. */
+ if(topout)
+ {
+ /* Small sanity check. */
+ if(topout->next)
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to "
+ "fix the problem. The 'next' pointer of 'topout' should "
+ "be NULL", __func__, PACKAGE_BUGREPORT);
+
+ /* The size of the output should be the same as 'ninput'. */
+ if(topout->size!=ninput)
+ error(EXIT_FAILURE, 0, "%s: a bug! Please contact us at %s to "
+ "fix the problem. The number of elements in 'topout' "
+ "(%zu) is different from 'ninput' (%zu)", __func__,
+ PACKAGE_BUGREPORT, out->size, ninput);
+
+ /* Write the filename. */
+ strarr=topout->array;
+ gal_checkset_allocate_copy(filename, &strarr[in_counter]);
+ }
+
+ /* Add the new columns into the raw output (only keyword values). */
+ for(key=keysll; key!=NULL; key=key->next)
+ {
+ /* If the keyword couldn't be read for any reason then 'key->status'
+ will be non-zero. In this case, return a string type and put a
+ blank string value. */
+ if( key->status )
+ {
+ key->type=GAL_TYPE_STRING;
+ if(p->cp.quiet==0)
+ error(EXIT_SUCCESS, 0, "%s (hdu %s): does not contain a "
+ "keyword '%s'", filename, p->cp.hdu, key->name);
+ }
+
+ /* Allocate the full column for this key and add it to the end of
+ the existing output list of columns: IMPORTANT NOTE: it is
+ necessary to initialize the values because we may need to
+ change the types before fully writing values within it. */
+ write=gal_data_alloc(NULL, key->type, 1, &ninput, NULL,
+ 1, p->cp.minmapsize, p->cp.quietmmap,
+ key->name, key->unit, key->comment);
+
+ /* Copy the value of this key into the output. Note that for strings,
+ the arrays are initialized to NULL. */
+ if( key->status )
+ {
+ strarr=write->array;
+ gal_checkset_allocate_copy(GAL_BLANK_STRING,
+ &strarr[in_counter]);
+ }
+ else
+ keywords_value_in_output_copy(write, key, in_counter);
+
+ /* Put the allocated column into the output list. */
+ gal_list_data_add(&out, write);
+ }
+
+ /* Reverse the list (to be the same order as the user's request). */
+ gal_list_data_reverse(&out);
+
+ /* If a first row (containing the filename) is given, then add the
+ allocated datasets to its end */
+ if(topout) { topout->next=out; out=topout; }
+
+ /* Return the output. */
+ return out;
+}
+
+
+
+
+
+static void
+keywords_value_in_output_rest_replace(gal_data_t *list, gal_data_t *old,
+ gal_data_t *new)
+{
+ gal_data_t *parse;
+ new->next=old->next;
+ for(parse=list; parse!=NULL; parse=parse->next)
+ if(parse->next==old)
+ {
+ gal_data_free(old);
+ parse->next=new;
+ break;
+ }
+}
+
+
+
+
+
+/* This function is for the case that we have more than one row. In this
+ case, we always want the input file's name to be printed. */
+static void
+keywords_value_in_output_rest(struct fitsparams *p, gal_data_t *out,
+ char *filename, gal_data_t *keysll,
+ size_t in_counter)
+{
+ int goodtype;
+ char **strarr;
+ gal_data_t *write, *key;
+ gal_data_t *goodkey, *goodwrite;
+
+ /* Write the file name in the first column. */
+ strarr=out->array;
+ gal_checkset_allocate_copy(filename, &strarr[in_counter]);
+
+ /* Go over all the keys are write them in. */
+ write=out;
+ for(key=keysll; key!=NULL; key=key->next)
+ {
+ /* Increment the write column also. */
+ write=write->next;
+
+ /* If the status is non-zero then the keyword couldn't be read. In
+ this case, put a blank value in this row. */
+ if(key->status)
+ {
+ gal_blank_write(gal_pointer_increment(write->array, in_counter,
+ write->type),
+ write->type);
+ if(p->cp.quiet==0)
+ error(EXIT_SUCCESS, 0, "%s (hdu %s): does not contain a "
+ "keyword '%s'", filename, p->cp.hdu, key->name);
+ continue;
+ }
+
+ /* This key is good and the type is string (which is the type for a
+ key that doesn't exist in the previous file(s)). In this case,
+ check if all the previous rows are blank. If they are all blank
+ then this keyword didn't exist in any of the previous files and
+ this is the first one that has the keyword. So change the type of
+ the column to the final type. */
+ else
+ {
+ if( write->type==GAL_TYPE_STRING
+ && write->type!=key->type
+ && gal_blank_number(write, 1)==write->size )
+ {
+ goodwrite=gal_data_alloc(NULL, key->type, 1, out->dsize,
+ NULL, 0, p->cp.minmapsize,
+ p->cp.quietmmap, key->name,
+ key->unit, key->comment);
+ gal_blank_initialize(goodwrite);
+ keywords_value_in_output_rest_replace(out, write,
+ goodwrite);
+ write=goodwrite;
+ }
+ }
+
+ /* If the previous files didn't have metadata for this keyword but
+ this file does, use the metadata here. */
+ if(write->unit==NULL && key->unit)
+ { write->unit=key->unit; key->unit=NULL; }
+ if(write->comment==NULL && key->comment)
+ { write->comment=key->comment; key->comment=NULL; }
+
+ /* If the column types are the same, then put them in. */
+ if(key->type==write->type)
+ keywords_value_in_output_copy(write, key, in_counter);
+ else
+ {
+ /* Find the most inclusive type. */
+ goodtype=gal_type_out(key->type, write->type);
+
+ /* Convert each of the two into the same type. */
+ goodkey = ( key->type==goodtype
+ ? key
+ : gal_data_copy_to_new_type(key, goodtype) );
+ goodwrite = ( write->type==goodtype
+ ? key
+ : gal_data_copy_to_new_type(write, goodtype) );
+
+ /* Copy the row into the output. */
+ keywords_value_in_output_copy(goodwrite, goodkey, in_counter);
+
+ /* If the "good" writing dataset has been changed, then
+ replace it in the output (correct its 'next' pointer, and
+ set the previous column to point to it. */
+ if(goodwrite!=write)
+ {
+ keywords_value_in_output_rest_replace(out, write,
+ goodwrite);
+ write=goodwrite;
+ }
+
+ /* If a different key has been used, clean it. */
+ if(goodkey!=key) gal_data_free(goodkey);
+ }
+ }
+}
+
+
+
+
+
+static void
+keywords_value(struct fitsparams *p)
+{
+ int status;
+ fitsfile *fptr=NULL;
+ gal_list_str_t *input, *tmp;
+ size_t i, ii=0, ninput, nkeys;
+ gal_data_t *out=NULL, *keysll=NULL;
+
+ /* Count how many inputs there are and allocate the first column with the
+ name. */
+ ninput=gal_list_str_number(p->input);
+ if(ninput>1)
+ out=gal_data_alloc(NULL, GAL_TYPE_STRING, 1, &ninput, NULL, 0,
+ p->cp.minmapsize, p->cp.quietmmap, "FILENAME",
+ "name", "Name of input file.");
+
+ /* Allocate the structure to host the desired keywords read from each
+ FITS file and their values. But first convert the list of strings (for
+ keyword names), (where each string can be a comma-separated list) into
+ a list with a single value per string. */
+ gal_options_merge_list_of_csv(&p->keyvalue);
+ nkeys=gal_list_str_number(p->keyvalue);
+
+ /* Parse each input file, read the keywords and put them in the output
+ list. */
+ for(input=p->input; input!=NULL; input=input->next)
+ {
+ /* Open the input FITS file. */
+ fptr=gal_fits_hdu_open(input->v, p->cp.hdu, READONLY);
+
+ /* Allocate the array to keep the keys. */
+ i=0;
+ keysll=gal_data_array_calloc(nkeys);
+ for(tmp=p->keyvalue; tmp!=NULL; tmp=tmp->next)
+ {
+ if(tmp->next) keysll[i].next=&keysll[i+1];
+ keysll[i].name=tmp->v;
+ ++i;
+ }
+
+ /* Read the keys. Note that we only need the comments and units if
+ '--colinfoinstdout' is called. */
+ gal_fits_key_read_from_ptr(fptr, keysll, p->colinfoinstdout,
+ p->colinfoinstdout);
+
+ /* Close the input FITS file. */
+ status=0;
+ if(fits_close_file(fptr, &status))
+ gal_fits_io_error(status, NULL);
+
+ /* Write the values of this column into the final output. */
+ if(ii==0)
+ {
+ ++ii;
+ out=keywords_value_in_output_first(p, out, input->v,
+ keysll, ninput);
+ }
+ else
+ keywords_value_in_output_rest(p, out, input->v, keysll,
+ ii++);
+
+ /* Clean up. */
+ for(i=0;i<nkeys;++i) keysll[i].name=NULL;
+ gal_data_array_free(keysll, nkeys, 1);
+ }
+
+ /* Write the values. */
+ gal_checkset_writable_remove(p->cp.output, 0, p->cp.dontdelete);
+ gal_table_write(out, NULL, NULL, p->cp.tableformat,
+ p->cp.output, "KEY-VALUES", p->colinfoinstdout);
+
+ /* Clean up. */
+ gal_list_str_free(p->keyvalue, 0);
+}
+
+
+
+
+
+
@@ -563,6 +888,11 @@ keywords(struct fitsparams *p)
gal_list_str_t *tstll;
int status=0, numinkeys;
+ /* Print the requested keywords. Note that this option isn't called with
+ the rest. It is independent of them. */
+ if(p->keyvalue)
+ keywords_value(p);
+
/* Delete the requested keywords. */
if(p->delete)
{
diff --git a/bin/fits/main.h b/bin/fits/main.h
index 40debc4..9c66221 100644
--- a/bin/fits/main.h
+++ b/bin/fits/main.h
@@ -54,31 +54,33 @@ struct fitsparams
{
/* From the environment. */
struct gal_options_common_params cp; /* Common parameters. */
- int hdu_in_commandline; /* HDU wasn't given in config. file. */
- char *filename; /* Name of input file. */
- char *outhdu; /* HDU of output (only when necessary). */
- gal_list_str_t *remove; /* Remove extensions from a file. */
- gal_list_str_t *copy; /* Copy extensions to output. */
- gal_list_str_t *cut; /* Copy ext. to output and remove. */
- uint8_t numhdus; /* Print number of HDUs in FITS file. */
- uint8_t datasum; /* Calculate and print HDU's datasum. */
- uint8_t pixelscale; /* Calculate and print HDU's pixelscale. */
- uint8_t skycoverage; /* Calculate and image coverage in WCS. */
- uint8_t primaryimghdu; /* Copy/cut HDU into primary HDU. */
- uint8_t printallkeys; /* Print all the header keywords. */
- uint8_t date; /* Set DATE to current time. */
- gal_list_str_t *asis; /* Strings to write asis. */
- gal_list_str_t *delete; /* Keywords to remove. */
- gal_list_str_t *rename; /* Rename a keyword. */
- gal_list_str_t *update; /* For keywords to update. */
- gal_list_str_t *write; /* Full arg. for keywords to add. */
- gal_list_str_t *history; /* HISTORY value. */
- gal_list_str_t *comment; /* COMMENT value. */
- uint8_t *verify; /* Verify the CHECKSUM and DATASUM keys. */
- char *copykeys; /* Range of keywords to copy in output. */
- char *datetosec; /* Convert FITS date to seconds. */
- char *wcsdistortion; /* WCS distortion to write in output. */
- uint8_t quitonerror; /* Quit if an error occurs. */
+ int hdu_in_commandline; /* HDU wasn't given in config. file. */
+ gal_list_str_t *input; /* Name of input file. */
+ char *outhdu; /* HDU of output (only when necessary). */
+ gal_list_str_t *remove; /* Remove extensions from a file. */
+ gal_list_str_t *copy; /* Copy extensions to output. */
+ gal_list_str_t *cut; /* Copy ext. to output and remove. */
+ uint8_t numhdus; /* Print number of HDUs in FITS file. */
+ uint8_t datasum; /* Calculate and print HDU's datasum. */
+ uint8_t pixelscale; /* Calculate and print HDU's pixelscale. */
+ uint8_t skycoverage; /* Calculate and image coverage in WCS. */
+ uint8_t primaryimghdu; /* Copy/cut HDU into primary HDU. */
+ uint8_t printallkeys; /* Print all the header keywords. */
+ uint8_t date; /* Set DATE to current time. */
+ gal_list_str_t *asis; /* Strings to write asis. */
+ gal_list_str_t *keyvalue; /* Strings to write asis. */
+ gal_list_str_t *delete; /* Keywords to remove. */
+ gal_list_str_t *rename; /* Rename a keyword. */
+ gal_list_str_t *update; /* For keywords to update. */
+ gal_list_str_t *write; /* Full arg. for keywords to add. */
+ gal_list_str_t *history; /* HISTORY value. */
+ gal_list_str_t *comment; /* COMMENT value. */
+ uint8_t *verify; /* Verify the CHECKSUM and DATASUM keys. */
+ char *copykeys; /* Range of keywords to copy in output. */
+ char *datetosec; /* Convert FITS date to seconds. */
+ char *wcsdistortion; /* WCS distortion to write in output. */
+ uint8_t quitonerror; /* Quit if an error occurs. */
+ uint8_t colinfoinstdout; /* Print column info in output. */
/* Internal: */
int mode; /* Operating on HDUs or keywords. */
diff --git a/bin/fits/ui.c b/bin/fits/ui.c
index deff1d8..c9cbc63 100644
--- a/bin/fits/ui.c
+++ b/bin/fits/ui.c
@@ -120,7 +120,6 @@ ui_initialize_options(struct fitsparams *p,
case GAL_OPTIONS_KEY_SEARCHIN:
case GAL_OPTIONS_KEY_IGNORECASE:
case GAL_OPTIONS_KEY_TYPE:
- case GAL_OPTIONS_KEY_TABLEFORMAT:
case GAL_OPTIONS_KEY_DONTDELETE:
case GAL_OPTIONS_KEY_LOG:
case GAL_OPTIONS_KEY_NUMTHREADS:
@@ -177,12 +176,7 @@ parse_opt(int key, char *arg, struct argp_state *state)
case ARGP_KEY_ARG:
/* Only FITS files are acceptable. */
if( gal_fits_name_is_fits(arg) )
- {
- if(p->filename)
- argp_error(state, "only one input file should be given");
- else
- p->filename=arg;
- }
+ gal_list_str_add(&p->input, arg, 1);
else
argp_error(state, "%s is not a recognized FITS file", arg);
break;
@@ -322,13 +316,15 @@ ui_check_copykeys(struct fitsparams *p)
static void
ui_read_check_only_options(struct fitsparams *p)
{
+ int checkkeys;
uint8_t stdoutcheck=0;
/* If any of the keyword manipulation options are requested, then set the
mode flag to keyword-mode. */
- if( p->date || p->comment || p->history || p->asis || p->delete
- || p->rename || p->update || p->write || p->verify || p->printallkeys
- || p->copykeys || p->datetosec || p->wcsdistortion )
+ if( p->date || p->comment || p->history || p->asis || p->keyvalue
+ || p->delete || p->rename || p->update || p->write || p->verify
+ || p->printallkeys || p->copykeys || p->datetosec
+ || p->wcsdistortion )
{
/* Check if a HDU is given. */
if(p->cp.hdu==NULL)
@@ -340,16 +336,18 @@ ui_read_check_only_options(struct fitsparams *p)
if(p->copykeys)
ui_check_copykeys(p);
- /* Currently 'datetosec' must be called alone. */
- if( (p->datetosec || p->wcsdistortion)
- && ( (p->datetosec && p->wcsdistortion)
- || p->date || p->comment || p->history || p->asis || p->delete
- || p->rename || p->update || p->write || p->verify
- || p->printallkeys || p->copykeys
- )
- )
- error(EXIT_FAILURE, 0, "'--datetosec' and '--wcsdistortion' cannot "
- "currently be called with any other option");
+ /* Keyword-related options that must be called alone. */
+ checkkeys = ( (p->keyvalue!=NULL)
+ + (p->datetosec!=NULL)
+ + (p->wcsdistortion!=NULL) );
+ if( ( checkkeys
+ && ( p->date || p->comment || p->history || p->asis
+ || p->delete || p->rename || p->update || p->write
+ || p->verify || p->printallkeys || p->copykeys ) )
+ || checkkeys>1 )
+ error(EXIT_FAILURE, 0, "'--keyvalue', '--datetosec' and "
+ "'--wcsdistortion' cannot currently be called with "
+ "any other option");
/* Identify the requested distortion. Note that this also acts as a
sanity check because it will crash with an error if the given
@@ -400,7 +398,7 @@ ui_read_check_only_options(struct fitsparams *p)
if(p->cp.output)
gal_checkset_writable_remove(p->cp.output, 1, p->cp.dontdelete);
else
- p->cp.output=gal_checkset_automatic_output(&p->cp, p->filename,
+ p->cp.output=gal_checkset_automatic_output(&p->cp, p->input->v,
"_ext.fits");
}
@@ -431,8 +429,15 @@ ui_check_options_and_arguments(struct fitsparams *p)
{
/* Make sure an input file name was given and if it was a FITS file, that
a HDU is also given. */
- if(p->filename==NULL)
+ if(p->input==NULL)
error(EXIT_FAILURE, 0, "no input file is specified");
+ gal_list_str_reverse(&p->input);
+
+ /* More than one input is currently only acceptable with the '--keyvalue'
+ option. */
+ if( gal_list_str_number(p->input) > 1 && p->keyvalue==NULL)
+ error(EXIT_FAILURE, 0, "one input file is expected but %zu input "
+ "files are given", gal_list_str_number(p->input));
}
diff --git a/bin/fits/ui.h b/bin/fits/ui.h
index 6212d49..4ee5333 100644
--- a/bin/fits/ui.h
+++ b/bin/fits/ui.h
@@ -44,29 +44,30 @@ enum program_args_groups
/* Available letters for short options:
- b e f g i j l m x y z
- A B E G J L O W X Y
+ b e f g i j m x y z
+ A B E G J L W X Y
*/
enum option_keys_enum
{
/* With short-option version. */
- UI_KEY_REMOVE = 'R',
- UI_KEY_COPY = 'C',
- UI_KEY_CUT = 'k',
- UI_KEY_NUMHDUS = 'n',
- UI_KEY_PRINTALLKEYS = 'p',
- UI_KEY_ASIS = 'a',
- UI_KEY_DELETE = 'd',
- UI_KEY_RENAME = 'r',
- UI_KEY_UPDATE = 'u',
- UI_KEY_WRITE = 'w',
- UI_KEY_COMMENT = 'c',
- UI_KEY_HISTORY = 'H',
- UI_KEY_DATE = 't',
- UI_KEY_VERIFY = 'v',
- UI_KEY_QUITONERROR = 'Q',
- UI_KEY_DATETOSEC = 's',
-
+ UI_KEY_REMOVE = 'R',
+ UI_KEY_COPY = 'C',
+ UI_KEY_CUT = 'k',
+ UI_KEY_NUMHDUS = 'n',
+ UI_KEY_PRINTALLKEYS = 'p',
+ UI_KEY_ASIS = 'a',
+ UI_KEY_KEYVALUE = 'l',
+ UI_KEY_DELETE = 'd',
+ UI_KEY_RENAME = 'r',
+ UI_KEY_UPDATE = 'u',
+ UI_KEY_WRITE = 'w',
+ UI_KEY_COMMENT = 'c',
+ UI_KEY_HISTORY = 'H',
+ UI_KEY_DATE = 't',
+ UI_KEY_VERIFY = 'v',
+ UI_KEY_QUITONERROR = 'Q',
+ UI_KEY_DATETOSEC = 's',
+ UI_KEY_COLINFOINSTDOUT = 'O',
/* Only with long version (start with a value 1000, the rest will be set
automatically). */
diff --git a/bin/table/arithmetic.c b/bin/table/arithmetic.c
index 809c318..0b8a0e2 100644
--- a/bin/table/arithmetic.c
+++ b/bin/table/arithmetic.c
@@ -29,6 +29,7 @@ along with Gnuastro. If not, see
<http://www.gnu.org/licenses/>.
#include <stdlib.h>
#include <gnuastro/wcs.h>
+#include <gnuastro/type.h>
#include <gnuastro-internal/checkset.h>
@@ -134,6 +135,7 @@ arithmetic_operator_name(int operator)
case ARITHMETIC_TABLE_OP_ATAN2: out="atan2"; break;
case ARITHMETIC_TABLE_OP_WCSTOIMG: out="wcstoimg"; break;
case ARITHMETIC_TABLE_OP_IMGTOWCS: out="imgtowcs"; break;
+ case ARITHMETIC_TABLE_OP_DATETOSEC: out="date-to-sec"; break;
case ARITHMETIC_TABLE_OP_DISTANCEFLAT: out="distance-flat"; break;
case ARITHMETIC_TABLE_OP_DISTANCEONSPHERE: out="distance-on-sphere";
break;
default:
@@ -212,6 +214,8 @@ arithmetic_set_operator(struct tableparams *p, char *string,
{ op=ARITHMETIC_TABLE_OP_WCSTOIMG; *num_operands=0; }
else if( !strcmp(string, "imgtowcs"))
{ op=ARITHMETIC_TABLE_OP_IMGTOWCS; *num_operands=0; }
+ else if( !strcmp(string, "date-to-sec"))
+ { op=ARITHMETIC_TABLE_OP_DATETOSEC; *num_operands=0; }
else if( !strcmp(string, "distance-flat"))
{ op=ARITHMETIC_TABLE_OP_DISTANCEFLAT; *num_operands=0; }
else if( !strcmp(string, "distance-on-sphere"))
@@ -728,6 +732,82 @@ arithmetic_trig_hyper(struct tableparams *p, gal_data_t
**stack,
+/* Convert the ISO date format to seconds since Unix time. */
+static void
+arithmetic_datetosec(struct tableparams *p, gal_data_t **stack,
+ int operator)
+{
+ size_t i, v;
+ int64_t *iarr;
+ gal_data_t *out;
+ char *subsecstr=NULL;
+ double *darr, subsec=NAN;
+
+ /* Input dataset. */
+ gal_data_t *in=arithmetic_stack_pop(stack, operator, NULL);
+ char **strarr=in->array;
+
+ /* Output metadata. */
+ char *unit="sec";
+ char *name="UNIXSEC";
+ char *comment="Unix seconds (from 00:00:00 UTC, 1 January 1970)";
+
+ /* Make sure the input has a 'string' type. */
+ if(in->type!=GAL_TYPE_STRING)
+ error(EXIT_FAILURE, 0, "the operand given to 'date-to-sec' "
+ "should have a string type, but it is '%s'",
+ gal_type_name(in->type, 1));
+
+ /* Allocate the output dataset. */
+ out=gal_data_alloc(NULL, GAL_TYPE_INT64, 1, &in->size, NULL, 1,
+ p->cp.minmapsize, p->cp.quietmmap, name, unit,
+ comment);
+
+ /* Convert each input string (first try as an integer, assuming no
+ sub-second, or floating point precision). */
+ iarr=out->array;
+ for(i=0; i<in->size; ++i)
+ {
+ v=gal_fits_key_date_to_seconds(strarr[i], &subsecstr,
+ &subsec);
+ iarr[i] = v==GAL_BLANK_SIZE_T ? GAL_BLANK_INT64 : v;
+ if(subsecstr) break;
+ }
+
+ /* If a sub-second string was present, then save the output as double
+ precision floating point. */
+ if(subsecstr)
+ {
+ /* Free the initially allocated output (as an integer). */
+ free(subsecstr);
+ gal_data_free(out);
+
+ /* Allocate a double-precision output. */
+ out=gal_data_alloc(NULL, GAL_TYPE_FLOAT64, 1, &in->size, NULL, 1,
+ p->cp.minmapsize, p->cp.quietmmap, name, unit,
+ comment);
+
+ /* Convert the date. */
+ darr=out->array;
+ for(i=0; i<in->size; ++i)
+ {
+ subsecstr=NULL;
+ v=gal_fits_key_date_to_seconds(strarr[i], &subsecstr,
+ &subsec);
+ darr[i] = v==GAL_BLANK_SIZE_T ? NAN : v;
+ if(subsecstr) { darr[i]+=subsec; free(subsecstr); }
+ }
+ }
+
+ /* Clean up and put the resulting calculation back on the stack. */
+ if(in) gal_data_free(in);
+ gal_list_data_add(stack, out);
+}
+
+
+
+
+
@@ -859,6 +939,10 @@ arithmetic_operator_run(struct tableparams *p, gal_data_t
**stack,
arithmetic_wcs(p, stack, operator);
break;
+ case ARITHMETIC_TABLE_OP_DATETOSEC:
+ arithmetic_datetosec(p, stack, operator);
+ break;
+
case ARITHMETIC_TABLE_OP_DISTANCEFLAT:
case ARITHMETIC_TABLE_OP_DISTANCEONSPHERE:
arithmetic_distance(p, stack, operator);
diff --git a/bin/table/arithmetic.h b/bin/table/arithmetic.h
index e1abcd1..436e07a 100644
--- a/bin/table/arithmetic.h
+++ b/bin/table/arithmetic.h
@@ -50,6 +50,7 @@ enum arithmetic_operators
ARITHMETIC_TABLE_OP_ATANH,
ARITHMETIC_TABLE_OP_WCSTOIMG,
ARITHMETIC_TABLE_OP_IMGTOWCS,
+ ARITHMETIC_TABLE_OP_DATETOSEC,
ARITHMETIC_TABLE_OP_DISTANCEFLAT,
ARITHMETIC_TABLE_OP_DISTANCEONSPHERE,
};
diff --git a/bin/table/table.c b/bin/table/table.c
index 5ce6245..86fda2b 100644
--- a/bin/table/table.c
+++ b/bin/table/table.c
@@ -1002,7 +1002,7 @@ table(struct tableparams *p)
|| p->tail!=GAL_BLANK_SIZE_T )
table_select_by_position(p);
- /* If any operations are needed, do them. */
+ /* If any arithmetic operations are needed, do them. */
if(p->outcols)
arithmetic_operate(p);
diff --git a/doc/announce-acknowledge.txt b/doc/announce-acknowledge.txt
index 2aff751..2457afd 100644
--- a/doc/announce-acknowledge.txt
+++ b/doc/announce-acknowledge.txt
@@ -2,6 +2,7 @@ Alphabetically ordered list to acknowledge in the next release.
Mark Calabretta
Raul Infante-Sainz
+Alberto Madrigal
Sylvain Mottet
Francois Ochsenbein
Zahra Sharbaf
diff --git a/doc/gnuastro.texi b/doc/gnuastro.texi
index 06a78f8..b0daca5 100644
--- a/doc/gnuastro.texi
+++ b/doc/gnuastro.texi
@@ -397,7 +397,7 @@ Fits
Invoking Fits
* HDU information and manipulation:: Learn about the HDUs and move them.
-* Keyword manipulation:: Manipulate metadata keywords in a HDU
+* Keyword inspection and manipulation:: Manipulate metadata keywords in a HDU
Sort FITS files by night
@@ -8771,19 +8771,19 @@ Therefore as described in the paragraphs above, when no
explicit call to the @op
The operating mode and input/output options to Fits are similar to the other
programs and fully described in @ref{Common options}.
The options particular to Fits can be divided into two groups:
1) those related to modifying HDUs or extensions (see @ref{HDU information and
manipulation}), and
-2) those related to viewing/modifying meta-data keywords (see @ref{Keyword
manipulation}).
+2) those related to viewing/modifying meta-data keywords (see @ref{Keyword
inspection and manipulation}).
These two classes of options cannot be called together in one run: you can
either work on the extensions or meta-data keywords in any instance of Fits.
@menu
* HDU information and manipulation:: Learn about the HDUs and move them.
-* Keyword manipulation:: Manipulate metadata keywords in a HDU
+* Keyword inspection and manipulation:: Manipulate metadata keywords in a HDU
@end menu
-@node HDU information and manipulation, Keyword manipulation, Invoking
astfits, Invoking astfits
+@node HDU information and manipulation, Keyword inspection and manipulation,
Invoking astfits, Invoking astfits
@subsubsection HDU information and manipulation
Each FITS file header data unit, or HDU (also known as an extension) is an
independent dataset (data + meta-data).
Multiple HDUs can be stored in one FITS file, see @ref{Fits}.
@@ -8800,7 +8800,7 @@ Print the number of extensions/HDUs in the given file.
Note that this option must be called alone and will only print a single number.
It is thus useful in scripts, for example when you need to do check the number
of extensions in a FITS file.
-For a complete list of basic meta-data on the extensions in a FITS file, don't
use any of the options in this section or in @ref{Keyword manipulation}.
+For a complete list of basic meta-data on the extensions in a FITS file, don't
use any of the options in this section or in @ref{Keyword inspection and
manipulation}.
For more, see @ref{Invoking astfits}.
@item --pixelscale
@@ -8859,7 +8859,7 @@ Calculate and print the given HDU's "datasum" to stdout.
The given HDU is specified with the @option{--hdu} (or @option{-h}) option.
This number is calculated by parsing all the bytes of the given HDU's data
records (excluding keywords).
This option ignores any possibly existing @code{DATASUM} keyword in the HDU.
-For more on @code{DATASUM} in the FITS standard, see @ref{Keyword
manipulation} (under the @code{checksum} component of @option{--write}).
+For more on @code{DATASUM} in the FITS standard, see @ref{Keyword inspection
and manipulation} (under the @code{checksum} component of @option{--write}).
You can use this option to confirm that the data in two different HDUs
(possibly with different keywords) is identical.
Its advantage over @option{--write=datasum} (which writes the @code{DATASUM}
keyword into the given HDU) is that it doesn't require write permissions.
@@ -8911,15 +8911,84 @@ If we hadn't used @option{--primaryimghdu}, then the
zero-th extension of @file{
@end table
-@node Keyword manipulation, , HDU information and manipulation, Invoking
astfits
-@subsubsection Keyword manipulation
+@node Keyword inspection and manipulation, , HDU information and
manipulation, Invoking astfits
+@subsubsection Keyword inspection and manipulation
The meta-data in each header data unit, or HDU (also known as extension, see
@ref{Fits}) is stored as ``keyword''s.
Each keyword consists of a name, value, unit, and comments.
The Fits program (see @ref{Fits}) options related to viewing and manipulating
keywords in a FITS HDU are described below.
+First, let's review the @option{--keyvalue} option which should be called
separately from the rest of the options described in this section.
+Also, unlike the rest of the options in this section, with
@option{--keyvalue}, you can give more than one input file.
+
+@table @option
+@item -l STR[,STR[,...]
+@itemx --keyvalue=STR[,STR[,...]
+Only print the value of the requested keyword(s): the @code{STR}s.
+@option{--keyvalue} can be called multiple times, and each call can contain
multiple comma-separated values.
+If more than one file is given, this option uses the same HDU/extension for
all of them (value to @option{--hdu}).
+For example, you can get the number of dimensions of the three FITS files in
the running directory, as well as the length along each dimension, with this
command:
+
+@example
+$ astfits *.fits --keyvalue=NAXIS,NAXIS1 --keyvalue=NAXIS2
+image-a.fits 2 774 672
+image-b.fits 2 774 672
+image-c.fits 2 387 336
+@end example
+
+If a single dataset is given, its name is not printed on the first column,
only the values of the requested keywords.
+
+@example
+$ astfits image-a.fits --keyvalue=NAXIS,NAXIS1 \
+ --keyvalue=NAXIS2
+2 774 672
+@end example
+
+The output is internally stored (and finally printed) as a table (with one
column per keyword).
+Therefore just like the Table program, you can use @option{--colinfoinstdout}
to print the metadata like the example below (also see @ref{Invoking asttable}).
+The keyword metadata (comments and units) are extracted from the comments and
units of the keyword in the input files (first file that has a comment or unit).
+Hence if the keyword doesn't have units or comments in any of the input files,
they will be empty.
+For more on Gnuastro's plain-text metadata format, see @ref{Gnuastro text
table format}.
+
+@example
+$ astfits *.fits --keyvalue=NAXIS,NAXIS1,NAXIS2 \
+ --colinfoinstdout
+# Column 1: FILENAME [name,str10,] Name of input file.
+# Column 2: NAXIS [ ,u8 ,] number of data axes
+# Column 3: NAXIS1 [ ,u16 ,] length of data axis 1
+# Column 4: NAXIS2 [ ,u16 ,] length of data axis 2
+image-a.fits 2 774 672
+image-b.fits 2 774 672
+image-c.fits 2 387 336
+@end example
+
+Another advantage of a table output is that you can directly write the the
table to a file.
+For example if you add @option{--output=fileinfo.fits}, the information above
will be printed into a FITS table.
+You can also pipe it into @ref{Table} to select files based on certain
properties, to sort them based on another property, or any other operation that
can be done with Table (including @ref{Column arithmetic}).
+For example with the command below, you can select all the files that have a
size larger than 500 pixels in both dimensions.
+
+@example
+$ astfits *.fits --keyvalue=NAXIS,NAXIS1,NAXIS2 \
+ --colinfoinstdout \
+ | asttable --range=NAXIS1,500,inf \
+ --range=NAXIS2,500,inf -cFILENAME
+image-a.fits
+image-b.fits
+@end example
+
+Note that @option{--colinfoinstdout} is necessary to use column names in the
subsequent @command{asttable} command.
+Also, with the @option{-cFILENAME} option, we are asking Table to only print
the final file names (we don't need the sizes any more).
+
+@item -O
+@itemx --colinfoinstdout
+Print column information (or metadata) above the column values when writing
keyword values to standard output with @option{--keyvalue}.
+You can read this option as column-information-in-standard-output.
+@end table
+
+Below we will discuss the options that can be used to manipulate keywords.
To see the full list of keywords in a FITS HDU, you can use the
@option{--printallkeys} option.
-If any of the keywords are to be modified, the headers of the input file will
be changed.
-If you want to keep the original FITS file or HDU, it is easiest to create a
copy first and then run Fits on that.
+If any of the keyword modification options below are requested (for example
@option{--update}), the headers of the input file will be changed first, then
printed.
+Keyword modification is done within the input file.
+Therefore, if you want to keep the original FITS file or HDU intact, it is
easiest to create a copy of the file/HDU first and then run Fits on that (for
copying a HDU to another file, see @ref{HDU information and manipulation}.
In the FITS standard, keywords are always uppercase.
So case does not matter in the input or output keyword names you specify.
@@ -10204,6 +10273,29 @@ If it is plain-text, the string column will be written
following the standards d
@cindex Declination
Convert degrees (a column with a single floating point number) to the
Declination, Dec, string (in the format of @code{_d_m_s}).
See the @option{degree-to-ra} for more on the format of the output.
+
+@item date-to-sec
+@cindex Unix epoch time
+@cindex Time, Unix epoch
+@cindex Epoch, Unix time
+The popped operand should be a string column in the FITS date format (most
generally: @code{YYYY-MM-DDThh:mm:ss.ddd...}).
+This operator will return the corresponding Unix epoch time (number of seconds
that have passed since 00:00:00 Thursday, January 1st, 1970).
+The returned operand will be named @code{UNIXSEC} (short for Unix-seconds).
+If any of the times have sub-second precision, the numeric datatype of the
column will be 64-bit floating point type.
+Otherwise it will be a 64-bit, signed integer, see @ref{Numeric data types}.
+
+For example, in the example below we are using this operator, in combination
with the @option{--keyvalue} option of the Fits program, to sort your desired
FITS files by observation date (value in the @code{DATE-OBS} keyword in example
below):
+
+@example
+$ astfits *.fits --keyvalue=DATE-OBS --colinfoinstdout \
+ | asttable -cFILENAME,'arith DATE-OBS date-to-sec' \
+ --colinfoinstdout \
+ | asttable --sort=UNIXSEC
+@end example
+
+If you don't need to see the Unix-seconds any more, you can add a
@option{-cFILENAME} (short for @option{--column=FILENAME}) at the end.
+For more on @option{--keyvalue}, see @ref{Keyword inspection and manipulation}.
+
@end table
@@ -10571,7 +10663,7 @@ If your table is large and generated by a script, you
can first do all your oper
Then, look into that file's metadata (with @command{asttable temp.fits -i}) to
see the exact column positions and possible names, then add the necessary calls
to this option to your previous call to @command{asttable}, so it writes proper
metadata in the same run (for example in a script or Makefile).
Recall that when a name is given, this option will update the metadata of the
first column that matches, so if you have multiple columns with the same name,
you can call this options multiple times with the same first argument to change
them all to different names.
-Finally, if you already have a FITS table by other means (for example by
downloading) and you merely want to update the column metadata and leave the
data intact, it is much more efficient to directly modify the respective FITS
header keywords with @code{astfits}, using the keyword manipulation features
described in @ref{Keyword manipulation}.
+Finally, if you already have a FITS table by other means (for example by
downloading) and you merely want to update the column metadata and leave the
data intact, it is much more efficient to directly modify the respective FITS
header keywords with @code{astfits}, using the keyword manipulation features
described in @ref{Keyword inspection and manipulation}.
@option{--colmetadata} is mainly intended for scenarios where you want to edit
the data so it will always load the full/partial dataset into memory, then
write out the resulting datasets with updated/corrected metadata.
@end table
@@ -22130,11 +22222,10 @@ not have any zero values (a dimension of length zero
is not defined).
@end deftypefun
@deftypefun void gal_data_free_contents (gal_data_t @code{*data})
-Free all the non-@code{NULL} pointers in @code{gal_data_t} except for
-@code{next} and @code{block}. If @code{data} is actually a tile
-(@code{data->block!=NULL}, see @ref{Tessellation library}), then
-@code{data->array} is not freed. For a complete description of
-@code{gal_data_t} and its contents, see @ref{Generic data container}.
+Free all the non-@code{NULL} pointers in @code{gal_data_t} except for
@code{next} and @code{block}.
+All freed arrays are set to @code{NULL}.
+If @code{data} is actually a tile (@code{data->block!=NULL}, see
@ref{Tessellation library}), then @code{data->array} is not freed.
+For a complete description of @code{gal_data_t} and its contents, see
@ref{Generic data container}.
@end deftypefun
@deftypefun void gal_data_free (gal_data_t @code{*data})
@@ -23755,13 +23846,13 @@ Return the number of HDUs/extensions in
@file{filename}.
@deftypefun {unsigned long} gal_fits_hdu_datasum (char @code{*filename}, char
@code{*hdu})
@cindex @code{DATASUM}: FITS keyword
Return the @code{DATASUM} of the given HDU in the given FITS file.
-For more on @code{DATASUM} in the FITS standard, see @ref{Keyword
manipulation} (under the @code{checksum} component of @option{--write}).
+For more on @code{DATASUM} in the FITS standard, see @ref{Keyword inspection
and manipulation} (under the @code{checksum} component of @option{--write}).
@end deftypefun
@deftypefun {unsigned long} gal_fits_hdu_datasum_ptr (fitsfile @code{*fptr})
@cindex @code{DATASUM}: FITS keyword
Return the @code{DATASUM} of the already opened HDU in @code{fptr}.
-For more on @code{DATASUM} in the FITS standard, see @ref{Keyword
manipulation} (under the @code{checksum} component of @option{--write}).
+For more on @code{DATASUM} in the FITS standard, see @ref{Keyword inspection
and manipulation} (under the @code{checksum} component of @option{--write}).
@end deftypefun
@deftypefun int gal_fits_hdu_format (char @code{*filename}, char @code{*hdu})
@@ -23922,15 +24013,13 @@ The advantage of working with the Unix epoch time is
that you don't have to worr
@deftypefun void gal_fits_key_read_from_ptr (fitsfile @code{*fptr}, gal_data_t
@code{*keysll}, int @code{readcomment}, int @code{readunit})
-Read the list of keyword values from a FITS pointer. The input should be a
-linked list of Gnuastro's generic data container
-(@code{gal_data_t}). Before calling this function, you just have to set the
-@code{name} and desired @code{type} values of each node in the list to the
-keyword you want it to keep the value of. The given @code{name} value will
-be directly passed to CFITSIO to read the desired keyword name. This
-function will allocate space to keep the value. If @code{readcomment} and
-@code{readunit} are non-zero, this function will also try to read the
-possible comments and units of the keyword.
+Read the list of keyword values from a FITS pointer.
+The input should be a linked list of Gnuastro's generic data container
(@code{gal_data_t}).
+Before calling this function, you just have to set the @code{name}, and
optinally, the desired @code{type} of the value of each keyword.
+The given @code{name} value will be directly passed to CFITSIO to read the
desired keyword name.
+This function will allocate space to keep the value.
+If no pre-defined type is requested for a certain keyword's value, the
smallest possible type to host the value will be found and used.
+If @code{readcomment} and @code{readunit} are non-zero, this function will
also try to read the possible comments and units of the keyword.
Here is one example of using this function:
@@ -23939,7 +24028,7 @@ Here is one example of using this function:
gal_data_t *keysll=gal_data_array_calloc(N);
/* Make the array usable as a list too (by setting `next'). */
-for(i=0;i<N-1;++i) keysll[i].next=keysll[i+1];
+for(i=0;i<N-1;++i) keysll[i].next=&keysll[i+1];
/* Fill the datasets with a `name' and a `type'. */
keysll[0].name="NAME1"; keysll[0].type=GAL_TYPE_INT32;
@@ -23959,23 +24048,15 @@ for(i=0;i<N;++i) keysll[i].name=NULL;
gal_data_array_free(keysll, N, 1);
@end example
-If the @code{array} pointer of each keyword's dataset is not @code{NULL},
-then it is assumed that the space to keep the value has already been
-allocated. If it is @code{NULL}, space will be allocated for the value by
-this function.
+If the @code{array} pointer of each keyword's dataset is not @code{NULL}, then
it is assumed that the space to keep the value has already been allocated.
+If it is @code{NULL}, space will be allocated for the value by this function.
-Strings need special consideration: the reason is that generally,
-@code{gal_data_t} needs to also allow for array of strings (as it supports
-arrays of integers for example). Hence when reading a string value, two
-allocations may be done by this function (one if
-@code{array!=NULL}).
+Strings need special consideration: the reason is that generally,
@code{gal_data_t} needs to also allow for array of strings (as it supports
arrays of integers for example).
+Hence when reading a string value, two allocations may be done by this
function (one if @code{array!=NULL}).
-Therefore, when using the values of strings after this function,
-@code{keysll[i].array} must be interpreted as @code{char **}: one
-allocation for the pointer, one for the actual characters. If you use
-something like the example, above you don't have to worry about the
-freeing, @code{gal_data_array_free} will free both allocations. So to read
-a string, one easy way would be the following:
+Therefore, when using the values of strings after this function,
@code{keysll[i].array} must be interpreted as @code{char **}: one allocation
for the pointer, one for the actual characters.
+If you use something like the example, above you don't have to worry about the
freeing, @code{gal_data_array_free} will free both allocations.
+So to read a string, one easy way would be the following:
@example
char *str, **strarray;
@@ -24045,12 +24126,10 @@ Reverse the input list of keywords.
@end deftypefun
@deftypefun void gal_fits_key_write_title_in_ptr (char @code{*title}, fitsfile
@code{*fptr})
-Add two lines of ``title'' keywords to the given CFITSIO @code{fptr}
-pointer. The first line will be blank and the second will have the string
-in @code{title} roughly in the middle of the line (a fixed distance from
-the start of the keyword line). A title in the list of keywords helps in
-classifying the keywords into groups and inspecting them by eye. If
-@code{title==NULL}, this function won't do anything.
+Add two lines of ``title'' keywords to the given CFITSIO @code{fptr} pointer.
+The first line will be blank and the second will have the string in
@code{title} roughly in the middle of the line (a fixed distance from the start
of the keyword line).
+A title in the list of keywords helps in classifying the keywords into groups
and inspecting them by eye.
+If @code{title==NULL}, this function won't do anything.
@end deftypefun
@deftypefun void gal_fits_key_write_filename (char @code{*keynamebase}, char
@code{*filename}, gal_fits_list_key_t @code{**list}, int @code{top1end0})
diff --git a/lib/blank.c b/lib/blank.c
index b126200..dda0e91 100644
--- a/lib/blank.c
+++ b/lib/blank.c
@@ -445,7 +445,9 @@ gal_blank_present(gal_data_t *input, int updateflag)
error(EXIT_FAILURE, 0, "%s: tile mode is currently not supported for "
"strings", __func__);
strf = (str=input->array) + input->size;
- do if(!strcmp(*str,GAL_BLANK_STRING)) return 1; while(++str<strf);
+ do
+ if(*str==NULL || !strcmp(*str,GAL_BLANK_STRING)) return 1;
+ while(++str<strf);
break;
/* Complex types */
@@ -486,17 +488,34 @@ gal_blank_present(gal_data_t *input, int updateflag)
size_t
gal_blank_number(gal_data_t *input, int updateflag)
{
+ size_t nblank;
+ char **strarr;
gal_data_t *number;
- size_t num_not_blank;
+ size_t i, num_not_blank;
if(input)
{
if( gal_blank_present(input, updateflag) )
{
- number=gal_statistics_number(input);
- num_not_blank=((size_t *)(number->array))[0];
- gal_data_free(number);
- return input->size - num_not_blank;
+ if(input->type==GAL_TYPE_STRING)
+ {
+ nblank=0;
+ strarr=input->array;
+ for(i=0;i<input->size;++i)
+ {
+ if( strarr[i]==NULL
+ || strcmp(strarr[i], GAL_BLANK_STRING)==0 )
+ ++nblank;
+ }
+ return nblank;
+ }
+ else
+ {
+ number=gal_statistics_number(input);
+ num_not_blank=((size_t *)(number->array))[0];
+ gal_data_free(number);
+ return input->size - num_not_blank;
+ }
}
else
return 0;
diff --git a/lib/fits.c b/lib/fits.c
index ba3b5c3..10a0bac 100644
--- a/lib/fits.c
+++ b/lib/fits.c
@@ -1105,6 +1105,14 @@ gal_fits_key_date_to_seconds(char *fitsdate, char
**subsecstr,
size_t seconds;
void *subsecptr=subsec;
+ /* If the string is blank, return a blank value. */
+ if( strcmp(fitsdate, GAL_BLANK_STRING)==0 )
+ {
+ if(subsec) *subsec=NAN;
+ if(subsecstr) *subsecstr=NULL;
+ return GAL_BLANK_SIZE_T;
+ }
+
/* Fill in the 'tp' elements with values read from the string. */
tmp=gal_fits_key_date_to_struct_tm(fitsdate, &tp);
@@ -1208,9 +1216,11 @@ void
gal_fits_key_read_from_ptr(fitsfile *fptr, gal_data_t *keysll,
int readcomment, int readunit)
{
- void *valueptr;
- char **strarray;
+ uint8_t numtype;
gal_data_t *tmp;
+ char **strarray;
+ int typewasinvalid;
+ void *numptr, *valueptr;
/* Get the desired keywords. */
for(tmp=keysll;tmp!=NULL;tmp=tmp->next)
@@ -1227,6 +1237,15 @@ gal_fits_key_read_from_ptr(fitsfile *fptr, gal_data_t
*keysll,
"tmp->dsize");
tmp->ndim=tmp->size=tmp->dsize[0]=1;
+ /* If no type has been given, temporarily set it to a string, we
+ will then deduce the type afterwards. */
+ typewasinvalid=0;
+ if(tmp->type==GAL_TYPE_INVALID)
+ {
+ typewasinvalid=1;
+ tmp->type=GAL_TYPE_STRING;
+ }
+
/* When the type is a string, 'tmp->array' is an array of pointers
to a separately allocated piece of memory. So we have to
allocate that space here. If its not a string, then the
@@ -1292,12 +1311,26 @@ gal_fits_key_read_from_ptr(fitsfile *fptr, gal_data_t
*keysll,
fits_read_key(fptr, gal_fits_type_to_datatype(tmp->type),
tmp->name, valueptr, tmp->comment, &tmp->status);
- /* If the comment was empty, free the space and set it to zero. */
+ /* Correct the type if no type was requested and the key has been
+ successfully read. */
+ if(tmp->status==0 && typewasinvalid)
+ {
+ /* If the string can be parsed as a number, then number will be
+ allocated and placed in 'numptr', otherwise, 'numptr' will
+ be NULL. */
+ numptr=gal_type_string_to_number(valueptr, &numtype);
+ if(numptr)
+ {
+ free(valueptr);
+ free(tmp->array);
+ tmp->array=numptr;
+ tmp->type=numtype;
+ }
+ }
+
+ /* If the comment was empty, free the space and set it to NULL. */
if(tmp->comment && tmp->comment[0]=='\0')
{free(tmp->comment); tmp->comment=NULL;}
-
- /* Strings need to be cleaned (CFITSIO puts ''' around them with
- some (possiblly) extra space on the two ends of the string. */
}
}
| [Prev in Thread] |
Current Thread |
[Next in Thread] |
- [gnuastro-commits] master c2da4b4: Fits & Table: new --keyvalue (Fits) and date-to-sec operator (Table),
Mohammad Akhlaghi <=