Space Telescope Science Institute
DrizzlePac 2012 Handbook
Table of Contents Previous Next Index Print

The DrizzlePac Handbook > Chapter 4: DrizzlePac Package > 4.4 The DrizzlePac Package

drizzlepac is a suite of tasks that allow users to align HST images, combine them, and perform coordinate transformations on source positions.
AstroDrizzle uses image WCS information to align the images. For the most part, this works well for images taken in the same orbit. Images in one visit over several orbits usually align well because the same guide star pair is used, but they should be carefully inspected to check for very small offsets due to guide star re-aquisition anomalies.
Images from different visits, however, cannot be aligned based on WCS alone because guide star catalog positions have uncertainties as high as 0.3 to 0.5 arcseconds. Therefore, the tweakreg task was developed to fine-tune the alignments, using point sources common to each image to determine x and y offsets, and rotations. Other tasks in drizzlepac complement astrodrizzle and tweakreg, providing functions that would be useful, such as coordinate transformations. All these tasks rely on the same distortion information required by AstroDrizzle, and also depend on other tasks from the stwcs and fitsblender packages (described later) to perform many of their operations.
Detailed information is available in the help files for these task. Help can be accessed either from the TEAL GUI or by Python commands. These interfaces are documented in Chapter 5.
A brief description of the tasks in drizzlepac:
“Sub-task” or “pset” containing parameters to find point sources used by tweakreg to build source catalogs for each tweakreg input image
Apply an updated WCS solution created by tweakreg for a drizzled image to the constituent distorted (flt.fits) images
“Sub-task” to reset specified flt.fits data quality (DQ) values to 0
Add the names of the new ACS distortion reference files NPOLFILE and D2IMFILE, then update images to include residual distortion corrections as image extensions.
Merge the keywords from all input images used to create a drizzled product into a single output header with a table extension using rules defined for each instrument. A default set of rules have been developed for pipeline use for ACS and WFC3, with offline support of STIS, NICMOS and WFPC2 data being provided by very basic rules.
Detailed information on how to run these tasks are available in their respective help files. Tasks that handle coordinate transformations–pixtopix, pixtosky, skytopix–are relatively straightforward and won’t be covered in this document (please consult the help file).
Data retrieved from the Archive before AstroDrizzle was implemented in the pipeline in June 2012 (check the DrizzlePac website for the exact date) have to be updated for compatibility with drizzlepac tasks. WFC3 and ACS images should be processed with the updatewcs task from the stwcs package to update SIP and other distortion keywords. ACS images should also be processed with the updatenpol task to insert FITS extensions containing information about residual distortions.
tweakreg (which calls the imagefindpars sub-task) is used for aligning images. Please see Section 4.4.2 for a description of the task.
tweakback is used for additional image alignment applications; when tweakreg has been used to align drizzled products (i.e., different filters, pointings, or detectors), the tweakback task can be used to propagate the updated WCS back to the original flt.fits images. astrodrizzle can then be used to process those updated flt.fits. Note that tweakback only aligns the WCS in the image headers. To ensure that the actual pixels are aligned in the final drizzled product, a reference output frame must be defined, as described in Section 7.5.
blendheaders, a task that’s also run in the pipeline for ACS and WFC3 data, collects important keyword information from AstroDrizzle input images for inclusion in the final drizzle-combined image.
Coverage of tweakreg and imagefindpars in this section is focused on describing the tasks and parameters. Examples of how to use tweakreg are available in Chapter 7and the DrizzlePac website.
Overview of the TweakReg Software
TweakReg provides an automated interface for computing residual shifts between images before they’re combined by AstroDrizzle. It is especially useful for combining images taken in different visits.
In the Dither package that included MultiDrizzle, the tweakshifts task required distortion-free images as input, making it necessary to run multidrizzle to generate single-drizzled distortion-corrected images. TweakReg, however, supports the use of flat-field calibrated distorted images (flt.fits for ACS, STIS, and WFC3, flc.fits for ACS, and c0m.fits for WFPC2) as input images.
WCS information for each image is related to the guide star pairs used during telescope guiding. For images taken in the same visit and orbit, the alignment between images are generally very accurate, to about two to five milliarcseconds. For images in the same visit but across several orbit, guide star reaquisition could potentially introduce very small offsets between five to 20 milliarcseconds. However, for images taken at different visits using different guide stars, residual offsets that remain after the images are aligned based on their WCS information are due to uncertainties in the guide star catalog positions, which can be as large as 0.3 to 0.5 arcseconds. For more information about HST pointing stability, please see Appendix B:HST Pointing Accuracy and Stability.
The source-finding algorithm provided within TweakReg has been optimized for point sources. As a result, this task may not work optimally for fields containing primarily extended sources or even images dominated by cosmic rays. In addition, its reliance on catalog matching to determine offsets requires a large enough overlap between images and a large enough sample of valid sources in the overlap to obtain a good solution, making the use of this task on sparse fields or images with small overlap more problematic
TweakReg can be used to align sets of images if there are enough point sources to make reliable matches, in the following way.
Using flat-field calibrated images (flt.fits images1; flc.fits for CTE-corrected images for ACS; c0m.fits for WFPC2) as input, TweakReg can generate source catalogs for each image using an algorithm similar to daofind. Exclusion files, in the form of ds9 regions files or simple target-radius specifications, can be used to instruct TweakReg to avoid certain parts of the image for source detection. The software also accepts user-provided catalogs for each input image.
For each image, its distortion model is used to correct source positions. (Therefore, it is unnecessary to drizzle the images before obtaining source positions, as was the case for MultiDrizzle.)
The offsets between each image and a reference frame are determined using a catalog-matching algorithm similar to xyxymatch. This reference frame could be an image in the input list (by default, the first image), a user-specified reference image, or undistorted sky coordinates (R.A., Dec.) and fluxes provided by the user. Matching sources between images can be severely exacerbated by cosmic rays, especially for long exposures. imagefindpars (called by tweakreg) parameters may be adjusted to impose flux detection ranges to exclude likely cosmic ray events. For pre-determined catalogs with magnitude values, potential matches that don’t fall in a reasonable magnitude range could also be discarded as a false match. Alternately, detections can also be performed on cosmic ray-cleaned images.
With the culling of false detections over several iterations of parameter adjustments, TweakReg should be able to converge on an acceptable RMS for the offset solution fits. Plots showing the quality of the fits can be displayed for each reference-image offset solution.
When the user is satisfied with the result, TweakReg is run one last time with those final parameters to update the WCS information in the images to a common reference frame, making it ready for AstroDrizzle processing.
tweakreg Task Syntax in PyRAF and Python
As a reminder, the examples below (following the “-->” PyRAF prompt) illustrate how to call tweakreg from PyRAF and Python. Chapter 5 has a more in-depth guide to using drizzlepac in the PyRAF and Python interfaces.
Before tweakreg is called, the drizzlepac package must be loaded. This only needs to be done once per session. The command can be executed either in PyRAF or Python:
If the package needs to be reloaded,
To run tweakreg using the TEAL GUI,
--> epar tweakreg
To run tweakreg as a command-line in PyRAF or Python (the parameter settings are for illustrative purposes):
The command above can also be run in PyRAF or Python with the parameter values written to a configuration file (suffix .cfg) that is read by tweakreg,
The help file can be accessed either from the TEAL GUI or at the command-line as such:
To write the help contents to an ASCII file,
tweakreg Parameter Details
Text in Arial are supplemental comments to the descriptions in the tweakreg configuration file. Additional information is available in the tweakreg help file.
The reference image from which reference catalog sources are derived.A name or catalog may be specified for this field. By default, the reference image is the first image on the list. .In some situation, the image with the largest area of overlap with other images in the set should be selected as the reference.
A set of files containing image regions to avoid when finding sources. This file MUST have one line for each input image, with the name of the input file in the first column. Subsequent columns would be used to specify an exclusions file for each chip, in order, in the science extension. If a chip does not require an exclusions file, the string None or INDEF can be entered for that chip. Each chip’s exclusion file should conform to the ds9-generated “region” file format; currently, only “circle()” regions are interpreted and used, and only with units for the positions of the exclusions center of “fk[4|5]” and “image”. Support for additional region file options will be added at a later date.
A user will typically run tweakreg several times till a satisfactory RMS to the fit solution is obtained. Then, tweakreg is run with those good settings again with the parameter value True to record new WCS information to the image header.
Specify if the source catalogs for each image, created by the tweakreg source detection algorithm, should be saved.
Specify whether or not to remove the temporary files created by tweakreg, including any catalog files generated for the shift determination. The default setting is False because these files could come in useful for tracking down potential problems.
Comments displaying the execution progress of tweakreg are saved to a file; this is useful to track down potential problems.
Specify whether or not to update the headers of each input image directly with the shifts that were determined. This will allow the input images to be combined by astrodrizzle without having to provide the shift file as was required for multidrizzle, since the shifts have already been applied to each input image’s WCS.
Each WCS has a unique name assigned to it. The default value used here is TWEAK but it can be any name specified by the user.
Specify whether or not to generate headerlets at the end of the tweakreg run? If True, headerlets are created for the images, regardless of the value of the updatehdr parameter.
Name of the separately-written headerlet file. If a name specified for this parameter does not have the suffix .fits, the headerlet will be named after the rootname of the input image, the string entered for this parameter, followed by the suffix _hlet.fits. (For example, if the image rootname is j99da1f2q_flt.fits, and if hdrfile has the value hdrlet1, the FITS headerlet will be named j99da1f2q_hdrlet1_hlet.fits.
If a headerlet with the same name specified by the hdrfile parameter already exists, specify whether the old file should, or should not, be overwritten.
Unique name (HDRNAME) for headerlet
Name of an ASCII file containing details regarding the history of the headerlet WCS solution. This can include information on the catalog used for the alignment and processing notes on finalizing the headerlet WCS alignment. This information will be reformatted by TweakReg as a 70-character wide FITS HISTORY keyword section.
Name for the output shift file created by TweakReg. This shift file was formatted for use as direct input to mulitdrizzle, but is not used by astrodrizzle. This file simply serves as a human-readable summary of the shifts found by TweakReg.
Filename for the OUTPUT reference WCS file created by TweakReg. This is the reference WCS from which shifts get measured, and could be used by the now depreciated MulitDrizzle to interpret those shifts. The reference WCS file is a FITS file containing only the WCS keywords in a Primary header with no image data. The values will be derived by default from the first input image or, if given, a user-specified reference image.
Name of the file that contains a list of input images and associated catalog files generated by the user. Each line of this file will contain the name of an input image in the first column. The remaining columns will provide the names of the source catalogs for each chip in order of the science extension numbers ([SCI,1], [SCI,2], etc. ....). A sample catfile, with one line per image would look like:
Column number of X position from the user-generated catalog files specified by the parameter catfile.
Column number of Y position from the user-generated catalog files specified by the parameter catfile.
Column number for flux values in the user-generated catalog files specified by the parameter catfile. These values are only used if a flux limit has been specified by the user using the fluxmax or fluxmin parameters.
This value, if specified, is the upper limit in the range of source fluxes that will be kept for each image’s catalog of sources. Those sources will then be matched with sources from the reference catalog for the final fit. For the default entry, None, there is no upper limit in the range of fluxes for valid sources.
This value, if specified, is the lower limit in the range of source fluxes that will be kept for each image’s catalog of sources. Those sources will then be matched with sources from the reference catalog for the final fit. If the value is set to None, all objects fainter than the limit specified by the parameter fluxmax will be used. If values for both fluxmin and fluxmax are set to None, all detected objects will be used.
Units of flux for parameters fluxmin and fluxmax. Choices are counts, cps, and mag.
Integer value specifying the number of brightest objects to keep in the source catalog for each input image. The default value None means that all objects are used.
Maximum flux value (a real number) for objects in the external reference catalog. This value, if specified, is the upper limit in the range of fluxes for reference sources to be kept for the final reference catalog. For the default entry, None, there is no upper limit in the range of fluxes for sources kept in the reference catalog.
Minimum flux value (a real number) for objects in the reference catalog. This value, if specified, is the lower limit in the range of fluxes for reference sources to be kept for the final reference catalog. For the default entry, None, there is no lower limit in the range of fluxes for sources kept in the reference catalog.
Flux units for the values specified in parameters rfluxmax and rfluxmin. Choices are counts, cps, and mag.
Integer value for how many brightest objects to keep in the reference catalog. If refnbright is set to None, all objects will be used. This value is used in conjunction with the refcat parameter.
One way to determine an optimal search radius would be to use ds9 to display all the images being combined and use the "Align WCS" option to align the images using their WCS information. For images from different visits, expect there to be small offsets when the images are blinked. The maximum offset in arcseconds for a single valid source from one image to another as reported by ds9 can then be used to set this parameter.
Another way to determine an optimal search radius value is to make sure residplots is set to Both, and that use2dhist and see2dhist are set to yes. After several runs with increasingly larger searchrad values, the number of matched sources will eventually "level out," indicating that an optimal fit solution may have been found. Additionally, the peak value displayed in the two-dimensional histogram plot should be sharp and unambiguous, located far enough from the edge of the plot that there's no chance that a better solution could be excluded by the limits of the search range. Furthermore, incorrect fit solutions will result in anomalous organized flow patterns in the vector plot, and odd shapes and/or tilts in the residuals plot. For the alignment of intravisit observations (which use the same guide stars for all observations), the default value of 1 arcsecond works well. However, for the alignment of intervisit observations, the search radius may to be extended to as much as 4 arcseconds.
Matching tolerance for xyxymatch (pixels)
The matching tolerance in pixels after applying an initial solution derived from the 2dhist algorithm. This parameter gets passed directly to an algorithm similar to xyxymatch for use in matching the object lists from each image with the reference image’s object list.
The minimum separation for objects in the input and reference coordinate lists. Objects closer together than separation pixels are removed from the input and reference coordinate lists prior to matching. This parameter gets passed directly to the xyxymatch algorithm for use in matching the object lists from each image with the reference image’s object list.
Initial estimate for the offset in X between the images and the reference frame. This offset will be used for all input images provided. If the parameter value is set to None, no offset will be assumed in matching sources with xyxymatch.
Initial estimate for the offset in Y between the images and the reference frame. This offset will be used for all input images provided.If the parameter value is set to None, no offset will be assumed in matching sources in xyxymatch.
The fitting geometry to be used in fitting the matched object lists. This parameter is used in fitting the offsets, rotations and/or scale changes from the matched object lists. Choices are shift and rscale. The rscale option tells tweakreg to fit for an offset, a scale (same scale in X and Y), and a rotation (same rotation for X and Y axes). The shift option only results in a fit of the overall offset between images without fitting for any rotation or scale differences.
The choices are No plot, vector, residuals, both. If both is selected, the vector and residuals plots will be displayed in separate plotting windows at the same time.
imagefindpars Parameter Details
The task imagefindpars can be optionally called by tweakreg to fine-tune object detection settings. Its algorithm is similar to that used by daofind, and has parameter names that resemble those in the IRAF version of daofind. However, the skysigma and threshold parameters are not defined identically to those in daofind. The computesig parameter attempts to automatically determine the value of skysigma for each image, but it is prone to failure for images with low background such as those containing globular cluster regions and nebula. In those situations, a user-defined skysigma can be used for all input images.
Table 4.25: imagefind Parameters  
This parameter controls whether or not to automatically compute a sigma value for use in object identification. If set to True, the value computed will override any user input for the parameter skysigma. The automatic sigma value gets computed for each exposure as
1.5 * imstatistics(image,nclip=3,fields='stddev').This computation works fairly robustly for nearly all fields.
Only select sources with a total flux (in the units of the input image) greater than this value. The flux gets computed for all pixels in the source extraction region, typically a five pixel wide aperture depending on the value of the conv_width parameter.
Format of Exclusions Catalog
The tweakreg parameter exclusions accepts a catalog containing a set of files defining regions to avoid when finding sources. The format of an exclusions catalog requires one line for every input image, regardless of whether that image has exclusions regions files. The example below shows the contents of an exclusions catalog file containing two images; no exclusions files are specified for the first image, and for the second image, an exclusions regions file (suffix .reg) is specified for [sci,1] (that regions file is in the second column).
The next example shows an exclusions file catalog containing two images. There are no exclusions files for the first image and one exclusions regions file for the second chip in the second image. For the second image, None is entered in the second column to indicate there is no [sci,1] exclusions regions files, while the third column contains the name of the [sci,2] exclusions regions file. (INDEF can also be used instead of None.)
Format of Regions Files
Exclusions files specified in an exclusions catalog have a default file format called “regions” files that are in a format generated by ds9 using the ds9/Funtools. Shown below is an example of a regions file using the circle() regions syntax.
global color=green dashlist=8 3 width=1 font="helvetica 10 normal roman" select=1 highlite=1 dash=0 fixed=0 edit=1 move=1 delete=1 include=1 source=1
TweakReg can also work with regions files with a much simpler format where each region has two values for the source center and a third value for the radius of the region. The units of all the values can be either pixels or sky coordinates (sexagesimal R.A./Dec. or decimal degrees R.A./Dec., with a radius in arcseconds). The values on each line can be separated either by spaces or commas, as shown in the example below.
The first (non-comment) line in the file specifies the units of the regions values. Supported options are:
The terms image, physical, and pixel all refer to values in pixels, while the remaining refer to values specified in sky coordinates. A default units of pixels is assumed if the unit type is not specified at the beginning of the simple format regions file.
The format of the regions can actually be different from one line to the next, as in cases where multiple regions files, written out in different formats, are merged; regions using the simple format, for instance, could be included in the first example file that uses the ds9 format.
A Note About “Difficult” Images
Examples in this handbook utilize images with enough stellar point sources to allow alignment using the daofind-like source-finding algorith in TweakReg. There are images, however, that are filled primarily with nebulosity, or with small faint galaxies and not much else, certainly not with enough stars useful for tweaking the alignment.
What can one do in such situations? In the case of nebulosity or high galactic latitude fields with few stars, direct cross-correlation can be done on prominent features in the nebulosity. But while cross-correlation is reliable and easy to use when there are only x- and y-shifts, it is far less adept at dealing with rotations or differences in scale.
For fields with lots of small galaxies, software such as SExtractor could be used to generate catalogs for input to TweakReg. This will require removing cosmic rays first, but if there are a few images that are reasonably well-aligned (such as those taken within a single visit), then cosmic ray-cleaned images could be used as a first pass in AstroDrizzle.
The CANDLES Treasury program has settled on a hybrid approach (Ferguson, H., Koekemoer, A., private communications). Catalogs are used to obtain the rotation and scale between images to do a first estimate of the shift. Rotation and scale are then fixed, and the shift is further refined using cross-correlation. At present, there are no straightforward ways for users to give the software a delta-shift and have it immediately translated into a change in header WCS. Users wanting to do this will need to update the headers themselves (by changing the CRVAL* keywords in their images, for instance). A task may be available to do this automatically in a future version of the software.
Other techniques are being investigated, such as wavelets, for use in aligning more difficult images in a more routine and self-contained manner. For now, users will have to experiment with techniques that may be best suited for their unique images. Fortunately, most images have enough stars in them to obtain fairly robust alignments with the current TweakReg software.
Drizzled images have been fully distortion-corrected, making it easy to perform a fit between drizzled images using TweakReg to calculate the overall offset, rotation, and scale differences between the drizzled images. While applying a fit to align two drizzled images can be done very simply due to the lack of distortion, the calibrated input images (flt.fits or flc.fits) which were combined to create those drizzled images, however, still contain distortion, so a fit computed using drizzled images cannot be applied to the input images in a simple manner. This typically results in the user updating the drizzled image headers with the results of a fit without any means of updating the original distorted input images.
TweakBack takes the WCS information from a drizzled image, one that has been aligned to another drizzled image, and propagates the newly updated WCS information back to all the input images used to create the drizzled images. This will allow those input images to be combined again to produce new aligned drizzled products.
When to Use TweakBack
Some situations require alignment of drizzled images, such as:
Creating combined drizzled images for each filter results in images free of cosmic rays and detector defects, making it ideal for finding and matching sources. These cosmic ray-free drizzled images can therefore maximize the number of sources detected and minimize the work needed to find matches between sources, resulting in the best possible fit between the images using the most possible sources. Alignment of these drizzled images using TweakReg will result in highly reliable and accurate updates to the drizzled image headers that can then be passed back to the original input images.
TweakBack Algorithm
Any update to the WCS information of an image should only be done after copying the original WCS keyword values as an alternate WCS (using FITS Paper I standards). This gets done automatically when using TweakReg to update the headers of images, and the task also provides a means to recover the original alignment if errors are made in the new WCS keyword values. This task relies on the presence of the original WCS and the newly-updated WCS to be recorded in the drizzled image’s header as the last two alternate WCSs. The difference between the previous WCS values and the newly updated WCS values will be used as the basis for computing the changes that need to be applied to each distorted input image.
The algorithm used by this function follows these steps:
The D00*DATA keywords from the PRIMARY header of the drizzled image will be used to build a list of input images that will need to be updated with the new WCS solution.
If no D00*DATA keywords can be found in the drizzled image PRIMARY header, then the user must provide a list of filenames for the images that need to be updated.
If the user does not specify a list of images and no D00*DATA keywords can be found, this task will report an error and quit.
The last two alternate WCS solutions (as generated when tweakreg updates an image header with a new WCS solution) represent the drizzled images starting WCS and the newly updated WCS.
Generate footprints using the STWCS .calcFootprint() method for each WCS.
The sky positions of the corners of the drizzled image get computed using the new updated WCS solution and using the original WCS solution.
The pixel positions for corners of each WCS get computed by running the .wcs_sky2pix() method for the last (updated) WCS
These pixel positions will be used to do a fit between the corner positions for the updated WCS and the drizzled image’s original corner positions.
This algorithm essentially backs out the correction that would have been applied by TweakReg if it had been run on two distorted input (flt.fits) images instead of drizzled images, then applies that correction to the distorted images just as TweakReg does itself. This means that input images updated using TweakBack will use the same conventions for keeping track of multiple WCS solutions in the input image headers just as if TweakReg was run.
This process has been tested on both ACS/WFC and WFC3/UVIS datasets and have demonstrated that this process can update images to as low as 0.001 pixels depending on the number of sources.
tweakback Usage
The syntax for running this task in a Python task is:
An image named acswfc_mos2_drz.fits was created from four images using AstroDrizzle. This drizzled image was then aligned to another image using tweakreg and the header was updated so the keyword WCSNAME was assigned the value TWEAK_DRZ. The new WCS can then be used to update each of the four images that were combined to make up this drizzled image using:
If the same WCS should be applied to a specific set of images, those images can be updated using:
A step-by-step example of how to run tweakback on real image data is available in Section 7.5
Table 4.26: tweakback Parameters
These can be provided either as a list of images in a file entered as @filename, a comma-separated list of filenames, or using wildcards. NOTE: A blank value will indicate that the task should derive the filenames from the drzfile itself, if possible. The filenames will be derived from the D*DATA keywords written out by AstroDrizzle. If they can not be found and no images are specified in this parameter, the task will quit.
WCSNAME of WCS upon input to tweakreg
Observations retrieved from the Archive before June 2012 (please check the DrizzlePac webpage for the exact date) were processed in the pipeline by MultiDrizzle. To reprocess this data with AstroDrizzle, users need to update the World Coordinate System (WCS) in their images, which includes geometric distortion information, to a format compatible with AstroDrizzle.
The task updatewcs, in the STWCS package, is used to insert and format linear and polynomial distortion information into the image header for HST images, such as WFC3 and even STIS data.
For ACS data, new non-polynomial distortion reference files need to be added to the header and folded into the image as new image extensions. The updatenpol task in the DrizzlePac package evaluates the DGEOFILE reference file specified in the header and selects the appropriate NPOLFILE and D2IMFILE reference files to replace the DGEOFILE and updates the header with the names of those new reference files. This task can then call updatewcs to add that reference file information to the input image as new extensions along with updating the header with the current SIP keywords based on the IDCTAB, although the user can always run updatewcs manually if desired. The updatenpol task really only needs to be run on ACS data that were obtained from the HST Archive prior to AstroDrizzle becoming operational in the pipeline, and it only needs to be run ONCE on any ACS image in order to record the names of the new reference files in its image header. The updatewcs task can be run any number of times afterwards as needed to reset the WCS solution to the default solution generated by the pipeline.
updatewcs Requirements
The tasks for updating geometric distortion correction information requires reference files for:
Updating linear and polynomial distortion corrections using the distortion corrections reference file named by the header keyword IDCTAB
Updating non-polynomial distortion corrections, currently applicable only for ACS data, the reference files named in header keywords D2IMFILE (for detector-based distortions) and NPOLFILE (for optical non-polynomial distortions) are required. These reference files are available at the ACS Reference Files webpage.
If reference files are kept in a specific directory, for example, /mydata/reffiles/, directory pointers need to be defined to tell the software where the reference files are located. For WFC3, by default, reference file names are preceded by iref$ where iref is a directory location for that reference file. The iref$ pointer can be defined using the following command–for UNIX-based machines, this needs to be done in the shell window before starting PyRAF.
or, for the ACS jref$ pointer,
Using updatewcs
updatewcs can be run by using the updatewcs task itself, or by setting the updatewcs parameter in astrodrizzle to True2. This task is required for populating older (pre-AstroDrizzle Archive pipeline data) WFC3, ACS, and STIS images with linear and polynomial distortion correction information in a format compatible with AstroDrizzle. To do this, the IDCTAB reference file is required.
The IDCTAB reference file can be downloaded from the instrument webpages by following a link to their reference files. To check that the images have the best-available IDCTAB reference file, use the Best Reference Files search in StarView. If a new IDCTAB file is listed, use it instead of the older one and be sure to update the IDCTAB header keyword in the images with the new reference file name; updating the header keyword can be done in PyRAF using the hedit task, as show below.
After starting PyRAF (or Python), import the stwcs package which contains the task updatewcs.
For all flt.fits files in the working directory, the WCSs are updated to be compatible with AstroDrizzle with this command:
Using updatenpol for ACS images
updatenpol requires access to the distortion correction image that contains column width corrections and residual wavelength-dependent distortion corrections, named in the header keyword DGEOFILE. The updatenpol task updates ACS images with:
New header keywords NPOLFILE and D2IMFILE that hold the names of reference files containing information about non-polynomial distortions and detector-based distortions, respectively. It identifies these new reference files using the geometric distortion image reference file that is named by the header keyword DGEOFILE.
Distortion correction information in the NPOLFILE and D2IMFILE reference files, containing non-polynomial optical distortion corrections and detector-based distortion corrections, respectively, are inserted into the flt.fits images as new FITS extensions. If it is not already done, there’s also an option to run updatewcs to ensure that all WCS information from the IDCTAB reference file has been incorporated into the ACS images.
Making WFPC2 Images Compatible with AstroDrizzle
Since WFPC2 data resides in a static Archive, after being processed by MultiDrizzle, c0m.fits and c1m.fits data need to be updated for compatibility with AstroDrizzle. To do that, the reference files identified by the header keywords IDCTAB, OFFTAB (for time-dependent chip offsets), and DGEOFILE (for detector-based distortions) are required.
The first time WFPC2 data is processed by AstroDrizzle, simply set the astrodrizzle parameter updatewcs to yes. AstroDrizzle will extract and convert detector distortion data in the DGEOFILE to the same format as in the D2IMFILE (detector distortion correction array) and insert that information into the FITS image as an extension. Information from the IDCTAB and OFFTAB are saved as SIP coefficients and other distortion header keyword values in the image header.
DrizzlePac relies on STWCS, another software package within STScI_Python, to manage all WCS-related operations. stwcs supports not only reading in the WCS information from FITS images as WCS objects in memory, but allows users to perform coordinate transformations using the full distortion model from the image header, among many other operations, with those WCS objects.
This software also provides the capabilities to package a WCS solution and save it as a file of its own for application to a copy of the original image; more specifically, it defines and supports the use of headerlets.
The STWCS package, in turn, relies on the generally available pywcs package which provides a Python interface to the C library WCSLIB which serves as the definitive implementation of all the approved FITS standards for WCS information. In short, any operation in any DrizzlePac task (such as astrodrizzle or tweakreg) dealing with the WCS gets performed by calling the stwcs package and its tasks.
STWCS consists of two primary subpackages: updatewcs and wcsutil. The task updatewcs (from the stwcs.updatewcs package) performs corrections to the basic WCS and includes other distortion information in the science files as header keywords or file extensions. The stwcs.wcsutil package implements many tasks including the most basic HSTWCS object which extends default FITS standard implementation of the WCS (as implemented by the pywcs.WCS object) as well as all the headerlet related tasks. The HSTWCS Python object provides HST instrument-specific WCS support as well as methods for coordinate transformations and serves as the primary in-memory implementation of the WCS information used by all the tasks in the DrizzlePac package. The coordinate transformation tasks in the DrizzlePac package (pixtosky, skytopix, pixtopix) all rely on the HSTWCS object implemented in the stwcs package for performing the actual computations of the requested coordinate transformations. The wcsutil package also provides functions for manipulating alternate WCS descriptions in the headers. Most of these functions do not have TEAL GUI interfaces, as they are designed for use within Python tasks or scripts or for interactive use under Python.
The package can be loaded into either a pure Python environment or under PyRAF using:
stwcs tasks that have TEAL interfaces are listed every time the STWCS package is imported or reloaded in PyRAF and Python. They are:
These tasks can be run in a pure Python environment:
or from PyRAF:
where the taskname corresponds to the name given in the list of available TEAL tasks reported when the stwcs gets (re-)loaded.
More About updatewcs
The task updatewcs from the stwcs package recomputes the entire WCS for an image to incorporate all distortion corrections in the form of SIP and other WCS-related keyword values. This task can be run directly, or it can be invoked as a processing step in astrodrizzle and updatenpol.
Since AstroDrizzle is run in the HST pipeline, calibrated (flt.fits) data from the Archive already contains all necessary information to represent astrometrically precise positions. Therefore, distortion reference files are no longer needed for post-pipeline drizzling, as was the case for MultiDrizzle. A full description of the astrometric computations and how they include the full distortion model can be found in Chapter 3.
The example below calls updatewcs as a standalone operation using Python syntax.
The parameters recognized by updatewcs are:
EXTNAME of extensions to be archived
If True, the velocity aberration correction will be applied if the image has a VAFACTOR keyword value.
If True, a time-dependent distortion correction will be applied, if appropriate. Currently, only ACS data has been calibrated to account for this effect.
If True, a look-up table distortion correction will be applied, if appropriate. Currently, only ACS data relies on an external NPOLFILE reference file.
If True, the detector to image correction will be applied, if appropriate. Only ACS and WFPC2 images have been calibrated to account for this effect. The ACS corrections have been stored as the new D2IMFILE reference files. The DGEOFILE reference file for WFPC2 data gets interpreted by this task to generate a D2IMFILE correction that gets copied directly into the updated WFPC2 image as a new extension when the WFPC2 image gets converted into a multi-extension FITS file.
If True, the format of the input files will be checked; GEIS and waiver-FITS files will be converted to Multi-extension FITS (MEF) format. The default format for WFPC2 data retrieved from the static Archive is the Multi-extension FITS (MEF) format, with suffixes c0m.fits (science images) and c1m.fits (data quality images).
All relevant distortion reference files must be available when this task is run, and image header keywords for specifying these reference files should be updated to reflect the correct file names. All instruments require the availability of the IDCTAB. For WFPC2 data, the DGEOFILE and OFFTAB reference files are also needed for the updatewcs task if run by itself or if run by turning on the updatewcs parameter in the astrodrizzle or tweakreg tasks. The DGEOFILE for WFPC2 data is converted into a D2IMFILE (that contains the 34th row corrections) which is inserted into the header as an FITS extension. Information from the IDCTAB and OFFTAB reference files are inserted into the header as keyword values. For ACS images, this task also requires the addition of the NPOLFILE and D2IMFILE header keywords and the availability (on disk) of the appropriate reference files for the exposure. The DGEOFILE reference files do not get used by updatewcs or astrodrizzle at all (for ACS data) and neither reference file needs to be accessible.
More About Running updatenpol for ACS Images
The updatenpol task in the DrizzlePac package uses the geometric distortion image named by the DGEOFILE header keyword to determine the names of new reference files for correcting residual optical distortions and detector-specific distortion keywords. These reference files are recorded in the NPOLFILE and D2IMFILE header keywords, respectively. This task will also optionally run the updatewcs task to insure that all the WCS information from those new reference files, and the IDCTAB, has been imported into the ACS file(s) as needed by AstroDrizzle, making it unnecessary to have the updatewcs parameter in astrodrizzle turned on.
This program requires access to the jref$ directory in order to evaluate the DGEOFILE specified in the input image header. This evaluation allows the program to get the information it needs to identify the correct NPOLFILE reference file.
The updatenpol task can be called from PyRAF and Python task using this syntax:
or the TEAL interface can be used from within PyRAF using:
The definitions of all the parameters for this task can be found in the following table.
Path to directory containing new reference files, either environment variable or full path. This task understands IRAF-styled definitions for directories (like jref$) assuming those variables have been defined in the users environment. Typically, these variables need to be defined at the operating system level for the Python environment (or PyRAF) to use (i.e., setenv jref /user/myreffiles/)
Specifies whether or not to interactively ask the user for the exact names of the new reference files instead of automatically searching a directory for them.
By default, this parameter will run the updatewcs task on the input files to incorporate the new reference files and distortion information into the image headers for use by astrodrizzle. If turned off, the user can always run the updatewcs task manually or simply turn on the updatewcs parameter when running either astrodrizzle or tweakreg.
A fundamental problem exists when trying to combine multiple images into a single product; namely, how to account for the header information from all the input images that went into generating the output product. The operation of drizzling multiple input images together to create a single output product of higher scientific value runs into this problem every time. The original solution to this problem implemented by MultiDrizzle was to simply use the first input image header as the basis for the final output header, modifying only a few keywords (namely, EXPSTART, EXPEND, EXPTIME, TEXPTIME and ROOTNAME) to make it appropriate for the output image. This resulted in an output header which did not accurately reflect the full set of input image information while also containing keywords which no longer apply to a drizzled image at all.
The solution implemented for AstroDrizzle, the new blendheaders task, results in a final output image header which contains a more complete record of all the keyword values from all the input images along with a table extension (called HDRTAB) that records values which change from one input image to another while eliminating keywords that no longer apply to drizzle products. This solution relies on rules specified by the user, or by the instrument teams for pipeline use, that describe what should be done with each keyword from every input image.
Some keywords from the input images can be merged into a reasonable single value for the output image using a simple operation such as mean, or first value, or last value based on the full list of input values from all input images. Other keywords, however, vary from one image to another and can not be combined into a reasonable single value and those keywords get flagged by the rules for population of a new table extension with the name HDRTAB. This table has a column for each keyword, and each row corresponds to the values derived from each chip (not file). This allows the user to get a full record of, for example, how the CRVAL values varied for all the input images.
The merging of headers into a new header and a HDRTAB table extension has been implemented as the blendheaders task in the new fitsblender package. A TEAL interface allows users to run this task on drizzled images that were created by AstroDrizzle or MultiDrizzle.
Many users may have data that had been combined using MultiDrizzle and would like to see the full set of input image header values that went into creating that product. Running this task manually allows them to do this. Alternatively, anyone running AstroDrizzle has the option of defining their own set of rules for combining the headers, and use that set of rules to create a new drizzle product header and table that would be different from the default header generated by AstroDrizzle. For most users, though, the default rules should be sufficient as they were designed to account for the majority of keywords found in most HST imaging data. Additional documentation on the task itself can be found online under the Documentation link of the STScI_Python web page at STScI at:
This software was designed primarily to support merging headers of images which all share the same basic FITS structure. For example, blendheaders could be run on a list of images that have a PRIMARY header and two SCI extensions, or on a list where all inputs are simple FITS files with only a PRIMARY header. It may be possible to combine images with differing numbers of science FITS extensions with missing information being represented with values of INDEF or NaN in the summary table.
Running blendheaders
This task can be run (after loading the drizzlepac package) under PyRAF using the standard syntax:
or from a pure Python command:
The function can also be called from within a Python task or interactively using the pure Python syntax:
The parameters used by blendheaders are described in the following table.
The drizzled file will be updated with the new header unless the output parameter has been specified. In addition, the drizzled file header will be interpreted to get a list of all the input images that went into its creation, if inputs is not specified, so that those images can be read in to get all the input keyword values.
Filenames with extensions for each chip in this list will be used to get the headers which will be blended into the final output headers. For example, [’j9cd01kqq_flt.fits[sci,1]’,’j9cd01kqq_flt.fits[sci,2]’] would create a blended header based on these chips headers.
EXTNAME of science extension from input images
EXTNAME of extensions with science data from the input FITS files. The header of this extension will be used as the basis for the SCI header of the drizzled product FITS file.
EXTNAME of error extension from input images
EXTNAME of extensions with the error array from the input FITS files. The header of this extension will be used as the basis for the WHT header of the drizzled product FITS file. If blank or INDEF, it will use the SCI header as the basis for the output header for the WHT array.
EXTNAME of extensions with the data quality array from the input FITS files. The header of this extension will be used as the basis for the CTX header of the drizzled product FITS file (when a CTX extension gets created). If blank or INDEF, it will use the SCI header as the basis for the header of any generated CTX array.
Drizzled Image Header Summary Table
The summary table in the drizzled image contains a table of useful header keyword parameters that characterize each input image. The table itself consists of a single row for each SCI array (chip) that was combined to create the final image product. Each column of the table represents a keyword from the PRIMARY and SCI extension of each chip. The names of the columns, by default, will match the names of the original keyword from each input header. However, the rules can rename those columns (more about that later) to whatever meets the needs of the software generating the final combined product. Renaming the columns, though, introduces a level of indirection when trying to map the values in the table to original header keywords and should only be done sparingly.
This table will get written out (by blendheader) as a binary table (pyfits.BinTableHDU) extension with EXTNAME of type HDRTAB. The header for this table will only contain the column definitions for the table and other required FITS keywords for the binary table.
blendheaders Rules
The operation of blendheaders relies, as implied earlier, on a set of rules that specifies what to do with all the input values for each keyword in the input file headers. The rules which define how to manage specific keywords need to be specified as a list defining which keyword arguments are to be aggregated and how.
The default set of rules for all HST instruments are included with the package using the filename convention <instrument>_header.rules. Any local file with an extension of *.rules will be read in as a user-supplied set of rules if it meets the formatting requirements described in this section.
Second element: name of the keyword that will be used to report the output value, or the name of a table column to record all the input values
Fourth and fifth elements: these specify how to deal with error conditions and what value to report when an error occurs in processing a set of input values
These rules have been refined for specification in ASCII files, as opposed to using direct Python syntax. The format for the rules file has been defined as:
The lines !VERSION and !INSTRUMENT are used to recognize this file as a valid rules file, and to associate this file with a specific version of fitsblender and apply it to headers for data from this instrument. This format, as supported by the current implementation of the code, only supports the use of the first three elements recognized by the fitsblender engine. Later versions can be updated to support use of the error elements for rules.

With AstroDrizzle installed in the pipeline, flt.fits and flc.fits images from the Archive now have updated WCS information to make it compatible with drizzlepac tasks. AstroDrizzle has been written to operate on WFPC2 images processed using MultiDrizzle from the Static Archive.

For ACS, updatewcs can also be run while executing updatenpol.

Table of Contents Previous Next Index Print