Home

User's guide - Astromatic.net

1. SWARP crashes with error messages like gt Error cannot open for reading during the co addition phase although it had no problem accessing the same files for resampling The maximum number of open files allowed by your shell might be set too low Use the shell command limit to increase the descriptors and openfiles parameters if required you might need the root privileges to change this if it is a hard limit 10 Acknowledging SWARP Please use the following reference Bertin E Mellier Y Radovich M Missonnier G Didelon P Morin B 2002 in Astronomical Data Analysis Software and Systems XI ASP Conf Series 281 228 11 Acknowledgements Many thanks go to Mark Calabretta ATNF CSIO Epping for his great astrometric library Nicolas Devillard ESO Garching for introducing me to memory mapping techniques Chiara Marmo TERAPIX IAP for adding the computation of saturation levels and doing many bugfixes Mireille Dantel Fort Laurent Domisse Fr d ric Magnard Yannick Mellier TER APIX IAP Mario Radovich OAC Naples Roeland Rengeling Sterrewacht Leiden Roy Williams CACR CalTech Jan Kohnert AIP Postdam J rg P Dietrich ESO Garching Dafydd Wyn Evans IoA Cambridge Joe Mohr Shantanu Desai University of Illinois and all the AstrOmatic forum users for testing and suggestions and Henry Joy McCracken TER APIX IAP for help with the manual and additional testing References 1 Ansco
2. If not subtracted the compositing of all these exposures will often produce an ugly patchwork created by all the different individual backgrounds A solution to this problem is to apply background subtraction prior to resampling and co adding the data Large scale gradients of instrumental origin are commonly found on astronomical images hence subtracting a constant from each frame will generally yield poor results as shown in Fig 11 It is therefore necessary to subtract a smooth background map which contains the low spatial frequency noise components of the data including any offset Subtracting a background map may alter or destroy the signal of scientific interest thus some caution is needed in choosing parameters for this procedure Background subtraction is activated by setting the SUBTRACT_BACK configuration parameter to Y which is the default Setting it to N will disable subtraction but background estimation will still take place it is needed by other SWARP tasks and the processing time will stay approximately the same Background estimation uses SExtractor s algorithm and is controllable with the same key words The following is largely copied from SExtractor documentation Bertin 1999 To construct the background map SWARP makes a first pass through the pixel data computing an estimator for the local background in each mesh of a grid that covers the whole frame The background estimator is a combination of K o clippin
3. L o 4 L o 4 0 05 F 0 05 2 L amp 0 0 E E E E 0 05 0 05 o PA 0 1 E NA AN DS LE de LEN 0 1 Aaa ES E A E A VS laa 14 16 18 20 22 24 26 16 18 20 22 24 26 original original Figure 8 Effects of the resampling on position top and flux bottom measurements Left bilinear interpolation right Lanczos3 interpolation In both cases a simulated deep sky image with 0 7 seeing containing stars and white background noise was rotated by 20 degrees and then rotated back to match the original image Fluxes were measured in a fixed 2 aperture The dispersions seen here reflect the differences between measurements on the original and resampled images These dispersions are much smaller than what one would observe by comparing the measurements on the resampled images with the theoretical noise free positions or fluxes of the simulation Note however the significant magnitude offset and flux dispersion in the bilinear case consequences of the stronger smoothing induced by bilinear interpolation 21 Figure 9 Oversampling in SWARP The input grid is shown as small grey squares whereas the output grid resampled image is represented by the large tilted ones Left Without oversam pling only one interpolation dark spot of the input image is done at the center of each output pixel Right with oversampling several interpolated samples are obtained on a regular subgrid and then binned in each output pixel Here a 3 x
4. it can be useful to set NTHREADS manually to have SWARP using a lower number of threads SWarp throughput Mpix s NTHREADS Figure 3 Pixel throughput of SWARP 2 0 resampling Lanczos3 as a function of the number of threads on an SMP machine with 4 Opteron 242 1 6GHz processors Departure from perfect linear scaling indicated by the dashed line for reference is due to I O limitations and parts of code that are not multithreaded 6 5 Astrometry The astrometric engine at the heart of SWARP is based on M Calabretta s WCSlib library to which we added the handling of polynomial distorsion parameters FITS keywords PV xx xx as proposed in the latest WCS documents We included IRAF s TNX astrometric projection too for inputs only although it is not part of the WCS standard All celestial coordinate computations are performed in the equatorial system Galactic or ecliptic coordinates are supported in input and output Available at http www cv nrao edu fits src wcs http www cv nrao edu fits documents wes wes html 12 6 5 1 Input frames De projection parameters of input images are extracted from their respective FITS headers These are the usual CTYPEx CRVALx CRPIXx CDELTx and or CD_xx_xx WCS parameters Exter nal header files can also be provided by the user for every input xxxx fits image SWARP looks for a xxxx head header file and loads it if present A head suffix is the default
5. v f dx Zt s E d NE fie EE represents the local angular sky coordinate vector We have made use of the fact that if pixel size is small compared to the rate of change of plate scale which is almost always true Q x 9 E is equal to the Jacobian of the de projection se Now if an equal area projection is selected for the output image ES is constant and we have the nice relation Ffw x F This means that swarping properly flatfielded data using an equal area output projection produces an image with a perfectly flat response to the incoming flux 2 This is why equal area projections should be prefered to other projections whenever possible Immediately following resampling the intensity of each image is scaled according to the configu ration parameters The flux scaling parameter p is the product of two factors a photometric factor and an astrometric one Currently the photometric part must be specified by the user for instance in a pipeline it is generally produced by the photometric calibration process The photometric factor can be set directly using the configuration parameter FSCALE_DEFAULT there must be one value per input image separated by a coma It can also be read from the FITS header The FSCALE_KEYWORD configuration parameter tells SWARP what FITS keyword to look for in each input image The default is FLXSCALE If the FSCALE_KEYWORD is not found in the image he
6. 3 oversampling is sufficient Figure 10 The effect of oversampling in SWARP A deep real image with a 0 2 pixel scale and 0 8 seeing FWHM is resampled at 1 resolution Left no oversampling Right with 5 x 5 oversampling Note the lower noise level and higher depth in the right image 22 6 6 4 Weight maps The processing of the weight maps see 86 9 2 follows that of the data images except that one is dealing now with variances instead of fluxes The resampled weight at position x may be written as 1 x 3 Therefore when an input weight within the range of the interpolation function is zero the interpolated weight is also zero The general consequence is that the borders of interpolated images are trimmed by half the range of the interpolation function Similarly small holes in a provided weight map are dilated by the interpolation function footprint For example once interpolated with a Lanczos4 kernel a single isolated zero weight pixel will yield a clump of about 64 pixels in the resampled image This is another illustration of the disadvantages in using large interpolation kernels 6 7 Background subtraction The flux at each pixel is a function of the sum of a background signal and light coming from the objects of interest At most wavelengths the strongest contribution to the background is of instrumental atmospheric ecliptic origin and is therefore prone to changes between exposures
7. 64 Weighi M ps ava sis rune Lu da pie puh eB aap a be ens a 23 6 7 Background subtraction 23 6 8 Sealing the flux rira sd a to A ee ee ee a A le hee BORG 25 6 9 Combining resampled images 27 6 9 1 Various types of image combination 27 6 9 2 Weighted coaddition 29 6 9 3 Image buffering and memory constraints 6 9 4 Overlap information 7 Two step co addition and resampling 8 Examples 9 1 Example Ls a ARR ae tr Be 32 Example 2r iui es gua a ah Aa Re RR Re ee La a Sl 9 Troubleshooting 10 Acknowledging SWARP 11 Acknowledgements li 32 32 32 34 35 36 36 1 What is SWARP SWARP is a program that resamples and co adds together FITS images using any arbitrary astrometric projection defined in the WCS standard The main features of SWARP are e FITS format including multi extensions in input and output e Full handling of weight maps in input and output e Ability to work with very large images up to 500 Mpixels on 32 bit machines and 10 Tpixels with 64 bits thanks to customized virtual memory mapping and buffering e Works with arrays in up to 9 dimensions including or not two spherical coordinates e Selectable high order interpolation method up to 8 tap filters in any dimension e Compatible with WCS and TNX IRAF astrometric descriptions e Support for equatorial galactic and equatorial coor
8. Configuration parameter list Here is a list of all the parameters known to SWARP Please refer to next section for a detailed description of their meaning Some advanced parameters indicated with an asterisk are also listed They must be used with caution and may be rescoped or removed without notice in future versions BACK_DEFAULT 0 0 floats n lt nima Default background value to be subtracted in BACK_TYPE MANUAL mode BACK_FILTERSIZE 3 integers n lt Nima Size in background meshes of the background filtering mask BACK_FILTTHRESH 0 0 integers n lt Nima Difference threshold in ADUs for the background filtering BACK_SIZE 128 integers n lt Nima Size in pixels of a background mesh BACK_TYPE AUTO keywords n lt Nima Tells SWARP what background is subtracted from the images AUTO The internal interpolated background map MANUAL A user supplied constant value provided in BACK_DEFAULT BLANK_BADPIXELS N boolean If true pixels with a weight of zero are set to zero in the combined image CELESTIAL_TYPE NATIVE keyword Celestial coordinate system in output NATIVE Same as first input file PIXEL No de projection faster EQUATORIAL Equatorial a 6 coordinates GALACTIC Galactic l b coordinates ECLIPTIC Ecliptic A 4 coordinates CENTER_TYPE ALL keywords n lt Ndim Tells SWARP how to center the output frame ALL Center on the region that contains all input fields MOST Center on the region with most over
9. MEDIAN The output is the median of all scaled pixel values with non zero weights F median f 14 Assuming Gaussian noise distribution we obtain the following approximation to the com posite weight see e g Kendall amp Stuart 1977 2 2 nzo AS 1 if np is even T n 2 W a Ee 15 2 25 nzo m 2 otherwise This approximation can become inaccurate if wj varies by large proportions a factor of 3 or more from frame to frame The median is convenient for combining data polluted by unidentified glitches or noise spikes It generally provides safe robust co additions even with strongly non Gaussian noise distributions However it is suboptimal for Gaussian noise the resulting variance is increased by 60 compared to what is obtained with an average As with all non linear combinations one should check that input images have approximately the same Point Spread Function if point sources are to be co added One should also avoid the MEDIAN option when combining images with very different depths e MIN The output is the minimum of all pixel values with non zero weights F min p fi 16 The variance of F is too noise distribution dependent to allow some estimation hence we set 1 if NAH A 0 1 w f 0 otherwise 17 e MAX The output is the maximum of all pixel values with non zero weights F min p fi 18 The variance of F is too noise distribution dependent to allow some estim
10. WRITE_XML Y boolean If true meta data are written in XML VOTable format XML_NAME swarp xml string File name for the XML VOTable output of SWARP Use STDOUT for piping to standard output XSL NAME string URL of an XSL style sheet for the XML output of SWARP The URL will appear in the href attribute of the style sheet tag 6 How SWARP works 6 1 Overview of the software What SWARP does is basically to read a set of input FTTS images resample and combine them and finally save the resultant FTTS image to disk The work can be decomposed in several steps 1 Input image headers are read and checked for content If configured in fully automatic mode SWARP will set the characteristics of the output frame based on this information 2 Input images and their weight maps if available are read one by one Background maps are built and subtracted from the images if required 3 Images are resampled projected into subsections of the output frame and saved as FITS files Projected weight maps are created too even if no weight maps were given in input 4 A combined output image is created using the information stored in the projected weight maps It consists of a composite of the resampled sub sections A composite output weight map is also written in the process The global layout of SWARP is presented in Fig 1 Let us now describe each of the important steps 6 2 Image mapping and memory constraints How
11. Weightmap filename if suffix not used all or for each weight map ee ee pe E A CO ad dita ON SSSR COMBINE_TYPE WEIGHTED weight maps are provided ST EP a RRA AStrometry TS ne ne sara eS CELESTIAL_TYPE NATIVE Standard stuff PROJECTION_TYPE TAN tangent projection will do CENTER_TYPE ALL We want all the data to fit in CENTER 00 00 00 0 00 00 00 0 Not used in CENTER TYPE ALL mode 34 PIXELSCALE_TYPE MEDIAN Will be overriden by coadd head PIXEL_SCALE 0 0 Not used in MEDIAN mode IMAGE_SIZE 0 Automatic sizing A oS Ss Resampling 2 2 RESAMPLING TYPE LANCZOS3 High quality resampling OVERSAMPLING 0 Auto oversampling 1 in that case INTERPOLATE N GAIN_KEYWORD GAIN GAIN_DEFAULT 0 0 Re SSeS Background subtraction SUBTRACT_BACK Y Needed for co adding dithered fields BACK_TYPE AUTO BACK_DEFAULT 0 0 BACK_SIZE 128 BACK_FILTERSIZE 3 RESPOSTA AA Virtual memory management See VMEM_DIR VMEM MAX 2047 MEM MAX 128 128 MB should be enough to avoid swapping ESSE SEER TESES 0 Miscellaneous 2 5 52555 252 gt DELETE TMPFILES Y Delete temporary resampled FITS files VERBOSE TYPE NORMAL To implement the unusual output features required one must write a coadd head ASCII file that contains a custom anisotropic scaling matrix coadd head for pixels 0 2 large that tilts
12. avoid disk thrashing 6 9 4 Overlap information SWARP uses the Bron and Kerbosh algorithm 1973 based on FITS header information to estimate the maximum number of input frames that overlap and optimize the management of its internal buffers The maximum overlap density is displayed on screen prior to image combi nation and can be found in the output XML meta data Note that this estimate is somewhat approximative as it ignores input weights which can be zero over significant fraction of the overlap and that there can only be one single overlapping frame per multi extension FITS input file The maximum overlap density is also used for computing the maximum accumulated exposure time and maximum equivalent gain in the output frame The maximum exposure time is sim ply the sum of all the exposure times from the individual overlapping frames The maximum equivalent gain is computed using 32 for COMBINE_TYPEs AVERAGE SUM and WEIGHTED For all other COMBINE_TYPEs the equivalent gain is just the average input gain in the overlap after the scaling of pixel values The saturation level of the combined data in ADU is also computed if saturation values are provided in the input image headers through the SATLEV_KEYWORD FITS keyword or directly with the SATLEV_DEFAULT configuration parameter For any COMBINE_TYPE the output saturation level is defined as the minimum of all input saturation values after astrometric and photometric rescaling have
13. been applied 31 7 Two step co addition and resampling All the operations described so far can be done in one single run of SWARP This generally sufficient for small projects which usually involve observations conducted over a short period of time However for large sky surveys that can extend over years this implies waiting for the data to accumulate after passing through the reduction pipeline In this case SWARP would only be started once all the fields that cover a given sky area are available Much of SWARP processing time is spent in resampling the data therefore for projects that extend over a long period of time it would be more efficient to resample the images as they come out of the reduction pipeline The remaining of the work co addition could be done at a later date To solve this problem versions gt 1 32 of SWARP allow the internal processing pipeline to be split in two image resampling and image combining If the COMBINE configuration parameter is set to N SWARP stops right after having background subtracted and resampled the input images The DELETE_TMPFILES option is then automatically deactivated To combine images resampled at an earlier stage RESAMPLE should be set to N in that case SWARP will skip all the background subtraction and resampling stage and jump directly to the combine process Prior to version 2 0 this feature only worked with input resampled images produced by SWARP because of some specific
14. default swarp and to reside in the current directory If no configuration file is found SWarp will use its own internal default configuration 5 1 1 Creating a configuration file SWARP can generate an ASCII dump of its internal default configuration using the d option By redirecting the standard output of SWARP to a file one creates a configuration file that can easily be modified afterward swarp d gt default swarp A more extensive dump with less commonly used parameters can be generated by using the dd option 5 1 2 Format of the configuration file The format is ASCII There must be only one parameter set per line following the form Config parameter Value s Extra spaces or linefeeds are ignored Comments must begin with a and end with a linefeed Values can be of different types strings can be enclosed between double quotes floats integers keywords or boolean Y y or N n Some parameters accept zero or several values which must then be separated by commas Values separated by commas spaces tabs or linefeeds may also be read from an ASCII file if what is given is a filename preceded with O e g values txt Integers can be written as decimals in octal form preceded by digit 0 or in hexadecimal preceded by 0x The hexadecimal format is particularly convenient for writing multiplexed bit values such as binary masks Environment variables written as HOME or HOME are expanded 5 1 3
15. degrees 19 Figure 7 Example of moir pattern on the background noise generated by bilinearly resampling an image containing white noise at a pixel scale slightly different from 1 samples per pixel and so on Oversampling can be different in each dimension OVERSAMPLING 2 3 will oversample each pixel in a 2 x 3 grid for instance An OVERSAMPLING of 0 the default lets SWARP select automatically the most appropriate oversampling factor in each dimension by comparing pixel scales at the reference point Although it works fairly well in many cases situations where the pixel scale varies a lot over the image like in all sky projections are not yet properly handled and manual setting should then be prefered Note that oversampling considerably slows down the processing OVERSAMPLING values should therefore be selected with some caution 6 6 3 Noise stability issues So far we have ignored the influence of noise variations in the resampling process In theory the interpolation schemes described above apply only if the noise is stationary in the wide sense over the extent of the interpolation kernel Artifacts aside this can be considered as true for the background noise since the weight maps are reasonably stable at the interpolation scale However the photon noise associated with the sources themselves may vary strongly over the scale of the PSF FWHM As most astronomical images are barely oversampled the hypothesis of nois
16. galactic coordinates of a series of observed fields stored in fits 2D files with WCS info for illustration purposes The files might be dummy ones supplemented with hand made headers or obtained from virtual telescopes such as SkyView or real ones In all cases an additional full sky map is useful to Mhttp skyview gsfc nasa gov 32 delimit the full sky one may for instance download a 360 degrees Aitoff projection of COBE DIRBE data from SkyView The syntax is swarp fits A possible default swarp configuration file is IMAGEOUT_NAME WEIGHTOUT_NAME WEIGHT_TYPE WEIGHT_SUFFIX WEIGHT_IMAGE CELESTIAL_TYPE PROJECTION_TYPE CENTER_TYPE CENTER PIXELSCALE_TYPE PIXEL_SCALE IMAGE_SIZE RESAMPLING_TYPE OVERSAMPLING INTERPOLATE GAIN_KEYWORD GAIN_DEFAULT SUBTRACT_BACK BACK_TYPE coadd fits coadd weight fits MAP_WEIGHT Not used here weight fits AVERAGE This coaddition is for illustration only the weight map will contain a sum of field footprints E Astrometry ss GALACTIC Coordinate system forced to galactic AIT Code for Aitoff MANUAL Imposed to alpha delta 0 0 00 00 00 0 00 00 00 0 MANUAL The full sky area will exceed the fraction that contains the fields 1800 0 in arcsec Half a degree per pixel at image center on both axes 800 400 360 0 5 720 180 0 5 360 plus a margin gt 258 Resampling 22 p22S BILINEAR For illustra
17. information needed in the FITS header Since version 2 0 any FITS image can be used However when RESAMPLE is set to N SWARP combines input images with the implicit assumption that they all share the same CRVAL and CDELT WCS parameters images are placed in the final frame according to their NAXISN s and the integer part of their CRPIX s The RESAMPLE N feature can be used for instance to extract sub images or paste input images within an arbitrary empty frame Setting both RESAMPLE and COMBINE to N will not produce any output file but can be useful to check the content of input data or to adjust output astrometric parameters By default RESAMPLE and COMBINE are both set to Y It may sometimes be useful to create only the header of what will become the combined image for instance for generating an output head file that will define the output projection system Such a head file can then be copied to several machines that will resample input images to a common projection for later co addition This can be done by setting the HEADER_ONLY configuration parameter to Y processing will stop before resampling like in the case where RESAMPLE and COMBINE are set to N but this time the header of the output image will be written to disk 8 Examples In the following examples of use of SWARP are given together with commented configuration files 8 1 Example 1 Let us assume one wants to produce a full sky Aitoff representation in
18. on non 2D images 6 8 Scaling the flux How are fluxes modified by image warping Let us assume F is the integrated flux in units of e for simplification of a source of finite extent S that would be recorded on a perfect detector 25 array In the continuous limit we define F fede 6 where f x is the pixel value at physical position x on the detector In real life pixel values are affected by a variable efficiency q yielding a measured raw flux B je g a f x dr 7 Digital images are generally divided by a flat field and even a super flat prior to SWARPing The assumption behind flat fielding is that the light received from the sky or the dome and recorded to form the flat field has uniform radiance i e constant flux per solid angle The flux measured on the flat fielded image is therefore f x 2 2 Fy UICN ae Le 8 where dQ is the local sky area sustained by a pixel area 1 in pixel units and fo the scaling factor of the flat field which we will set to 1 for the sake of simplicity As can be seen flat fielding does not make the image flat in terms of sensitivity it introduces a dependency with astrometrical distorsion With most cameras the effect is generally small lt 1 millimag The resampling operations described in 86 6 are designed to conserve surface brightness per pixel hence the flux recorded on the warped image with new physical coordinates x is now f x f2 x
19. E TYPE MANUAL mode Must be expressed in arcseconds for angular coordinates PIXELSCALE TYPE MEDIAN keywords n lt Naim Tells SWARP how to adjust the output pixel size MEDIAN Take the median of pixel scales at the center of input frames MIN Take the minimum of pixel scales at the center of input frames MAX Take the maximum of pixel scales at the center of input frames MANUAL User defined pixel scale at image center with the PIXEL_SCALE keyword FIT Sets the pixel scale such as the full output field fits the user defined IMAGE_SIZE PROJECTION_ERR 0 001 floats n lt nima Maximum position error in pixels allowed for bicubic spline interpolation of the astro metric reprojection O turns off interpolation PROJECTION TYPE TAN string Projection system used in output in standard WCS notation see Table 2 RESAMPLE Y boolean If true resampling is performed on the input images RESAMPLE DIR string Path of the directory where resampled images are written RESAMPLE_SUFFIX resamp fits string Filename extension given to resampled images produced by SWARP RESAMPLING_TYPE LANCZOS3 keywords n lt Naim Image resampling method NEAREST Take the nearest neighbour BILINEAR Bi linear interpolation LANCZOS2 Lanczos 2 4 x 4 tap filter LANCZOS3 Lanczos 3 6 x 6 tap filter LANCZOS4 Lanczos 4 8 x 8 tap filter RESCALE_WEIGHTS Y booleans n lt Nima Tells SWARP whether input weight maps and variance maps should be
20. Minimum of all input saturation levels flux scaling applies For data management purposes it is often useful to propagate other selected FITS keywords such as FILTER or TELESCOP for instance and their values from the input image headers to the resampled and coadded image headers To this aim a COPY_KEYWORDS configuration parameter is provided It accepts a list of FITS keywords that shall be copied in the headers of all the images created by SWARP by default only the OBJECT keyword and its content are copied Because the coadded image can result from the combination of many input files only the keyword found in the first image header from the input file list is propagated up to the final coadded image It is important to note that SWARP does not check the content of the list of COPY_KEYWORDS therefore one should be cautious not to propagate structural FITS keywords like NAXISL BITPIX that may interfere with the interpretation of the output data 11 6 4 Parallel processing Versions gt 1 32 of SWARP are multi threaded Multi threading allows CPU intensive tasks in SWARP to be run in parallel on multi core or Hyper Threaded HT machines Best performance is generally achieved when the number of active threads is equal to the number of processor cores in the machine Fig 3 This is automatically done with the default settings NTHREADS 0 However if other cpu intensive tasks have to run on the same machine at the same time
21. SWarp v2 21 User s guide E BERTIN Institut d Astrophysique de Paris December 17 2010 Contents 1 What is SWARP 1 2 Skeptical Sam s questions 1 3 License 2 4 Installing the software 2 A il Obtaining S WARP 25 6 a nant D nan A A A pe Ru em ni ee 2 4 2 Software and hardware requirements 2 dd Installation las aid og ate els WS eee MODE Te ER Pe ote le 3 5 Using SWARP 3 5 1 The Configuration file 3 5 1 1 Creating a configuration file 3 5 1 2 Format of the configuration file 3 5 1 3 Configuration parameter list 4 6 How SWARP works 8 6 1 Overview of the software 8 6 2 Image mapping and memory constraints 8 6 3 Propagating FITS keywords 11 6 4 Paralel processes gas 23 ehh A a tee D ed ee E SE 12 6 5 AStrometry 2 44 48 6 ele dp den te dot ME ale Sour ist 12 6 5 1 Input frames Cass nte uces he hee en MS a Oh Poe ie Bek 13 6 5 2 Output ffames a eg da pd Seek do ele a Be 13 6 5 3 Bi cubic spline interpolation 17 6 6 Resampling su iaa da ao da pa dan ohne Landes 17 6 6 1 Image data sas Lars hae an er US du RATE OMS Pos ie Me 17 6 6 2 Oversamplin sau eee Lp tg eee pda dame del ane e a 18 6 6 3 Noise stability issues o oo a a 20 6
22. ader then the FSCALE DEFAULT value is taken instead FSCALE DEFAULT is defaulted to 1 for all images The astrometric part of the flux scaling factor corrects for the difference in pixel size between the input and output images The FSCALASTRO_TYPE configuration parameter controls the behaviour of this astrometric flux scaling The default behaviour FSCALASTRO_TYPE FIXED is to apply Warning this flux correction can be extremely inaccurate 10 error or even more for sources that are undersampled on the output image 26 a constant correction factor to account for possible mismatches in pixel size Hence flux is con served only when equal area projections are used for both input and output FSCALASTRO TYPE VARIABLE applies a pixel scale correction variable throughout the field and can therefore be used on any kind of projection The impact on processing speed is negligible when bi cubic spline interpolation is used 86 5 3 Astrometric flux scaling can be deactivated by using the FSCALASTRO_TYPE NONE option 6 9 Combining resampled images This is the last part of the processing Now at each pixel position of the output image SWARP has to combine data values coming from all the resampled image each one coming with a rough estimate of its variance from the resampled weight map Many combinations are therefore possible 6 9 1 Various types of image combination The user can choose between the following options as ar
23. ation hence we set 1 if N 0 0 E 1 id O otherwise 19 Maxima and minima can be useful for identifying defects or rare events in a set of data e SUM The output is an unweighted sum of all pixel values with non zero weights PS pifi 20 i The composite weight is 1 W 21 Si aa Subtractions can be carried out instead of additions by using negative flux scaling factors e WEIGHTED The output is a weighted average of input values D Wipi fi F D wi 22 28 The output weight is just the sum of input weights W wi 23 i This combination should be the most appropriate for detecting and measuring faint sources on properly weighted homogeneous images Because it is a linear processing new data can be added later if needed 6 9 2 Weighted coaddition Weight maps provide a convenient way to store the standard flux error assigned to each pixel For each input image 1 lt i lt N entering image combination one can define the following parameters in an arbitrarily small pixel j The local uncalibrated flux fij fis A fij where fy is the flux contributed by the sky background and A f that contributed by celestial sources the local uncalibrated variance of the flux oj o Ad where 0 is the flux variance contributed by the background noise and Ao that contributed by the photon statistics of celestial sources the local normalized weight Wij the electronic gain of t
24. automatic settings when making all sky projections 6 5 3 Bi cubic spline interpolation The trigonometric calculations involved in SWARP re projections have a major impact on pro cessing speed To accelerate the resampling phase versions gt 2 06 of SWARP implement a bi cubic spline interpolation of the astrometric mapping between the input and the output frames Interpolation is used by default for large images with a maximum allowed positional error of 1073 pixel as measured in the output frame This error tolerance can be changed independently for each input image with the PROJECTION_ERR configuration parameter A PROJECTION_ERR of O turns interpolation off Interpolation is also automatically deactivated for smaller images or inappropriate mappings like all sky projections or projection with singularities 6 6 Resampling The action of projecting a grid of pixels on another grid is called resampling Ideal image resampling involves both filtering and interpolation between pixels In SWARP filtering is naturally implemented by oversampling the destination grid Because SWARP uses reverse mapping interpolation is made on the input images 6 6 1 Image data At each position x the dot product between a local kernel k x and neighbouring pixel values f yields a local interpolated value f x k x f 1 The kernel is derived locally from an interpolation function h ki x h x xi 2 The RESAMPLING_TYPE configurat
25. automatically rescaled or not SATLEV_KEYWORD SATURATE string FITS keyword that contains the saturation level in ADUs SATLEV_DEFAULT 50000 floats n lt nima Default saturation level in ADUs set for images for which no SATLEV KEYWORD FITS header keyword is found SUBTRACT_BACK Y booleans n lt Nima If true input images are background subtracted prior to resampling VMEM_DIR tmp string Path of the directory where virtual memory and other temporary files are written VMEM_MAX 2047 integer Maximum amount of megabytes allowed for virtual memory storage VERBOSE_TYPE NORMAL keyword Tells SWARP how well operations should be commented QUIET Run silently LOG Log essential information NORMAL Display information dynamically using control characters FULL Display more complete information WEIGHT_IMAGE strings n lt Nima List of input weight map filenames WEIGHTOUT_NAME coadd fits string File name of the output weight map WEIGHT_THRESH floats n lt Nima Threshold below or above which input weights are defined equivalent to zero infinite variance i e a bad pixel WEIGHT_TYPE NONE keywords n lt Nima Sets the type of input weight maps NONE No weighting MAP_WEIGHT Relative weights i e inverse variance MAP_VARIANCE Relative variance MAP_RMS Absolute standard deviation WRITE_FILEINFO N boolean If true extended information about input files is written in the header of the output FITS image
26. both input and output images It provides a major speed up to the warping engine by bypassing all the trigonometric operations normally involved in the other modes It is useful for quickly combining mosaic images whenever astrometric information is not needed In PIXEL mode degrees are interpreted as dimensionless Cartesian coordinates The output celestial projection is set by the PROJECTION_TYPE configuration parameter The list of all presently supported projections is shown in Table 2 and illustrated in Fig 4 and 5 using a gridded map of the Earth Now what projection is the best With small fields lt 10 degrees in their maximum di mension the choice is not critical as long as the projection center lies within the frame For compatibility reasons it is advised to stay with the traditional gnomonic TAN for tangential projection in such cases With larger fields the pure tangential projection is inappropriate and one is faced with the usual problems confronted by cartographers It would be outside the scope of this document to discuss the merits of each projection For detailed information about the different projection systems the user should refer to the latest WCS document Let us just mention that equal area projections those that conserve relative areas are often to be preferred for mapping large sky surveys because they naturally conserve surface brightness and or allow summing pixel values to measure fluxes The follow
27. dinate systems e Astrometric and photometric parameters are read from FITS headers or external ASCII files e Built in background subtraction e Built in noise level measurement for automatic weighting e Automatic centering and sizing functions of the output field e Multi threaded code with load balancing to take advantage of multiple processors e XML VOTable compliant output of meta data 2 Skeptical Sam s questions Skeptical Sam doesn t have time to test software extensively but is always keen on asking agressive questions to the author to find out if a program could fit his needs S Sam What s the point in releasing another image co addition software We already had the DRIZZLE Fruchter amp Hook 1997 2002 and MSCRED Valdes 1998 packages And there are now MOPEX Makovoz amp Khan 2005 and Montage Berriman et al 2008 Author Co addition is most certainly the most critical step in reducing modern CCD mosaic data Although several powerful packages are available they did not meet all the requirements we had at the TERAPIX data centre in Paris where the project was initiated S Sam SWARP doesn t perform the astrometric calibration does it Author No it doesn t You will have to use SCAMP Bertin 2006 for that S Sam I am the kind of guy who does high precision astrometry and photometry My sources are detectable on raw frames Resampling you know would just wreck the signal so I prefer to combi
28. dition the computational cost becomes prohibitive with multi dimensional data Nearest neighbour interpolation provides a good conservation of the noise spectrum at scales close to unity unfortunately it generates a terrible aliasing when zooming in and can distort a lot object shapes at places Its usage should therefore be restricted to images such as flag or weight maps Bilinear interpolation is fast and doesn t generate negative artifacts However it creates a lot of smoothing by correlating the values of neighbour pixels On images with white noise this may lead to obvious moir effects Fig 7 Nevertheless bilinear interpolation can be useful for processing undersampled data In general Lanczos3 resampling represents the best compromise As can be seen in Fig 8 it brings a substantial benefit over bilinear interpolation in preserving the signal while creating relatively modest artifacts around image discontinuities 6 6 2 Oversampling Unfortunately the procedure described above generates aliasing when zooming out sufficiently an image by resampling it at a lower resolution Moreover the intensity of the resampled background white noise stays constant instead of being proportional to the zooming factor see Fig 10 This is because the algorithm essentially decimates the data instead of binning them within the output pixel footprint a similar effect applies in the panning windows of astronomical visualization
29. does SWARP projects input images into the output frame space There are two ways of applying a geometric transformation to an image see Wolberg 1992 The most intuitive is Figure 1 Global Layout of SWARP called forward mapping It consists in scanning the input image pixel per pixel line by line Each pixel is simply thrown to the position it is supposed to occupy in the output grid Although this technique can be used for geometric resampling or drizzling Fruchter amp Hook 1997 it is totally cumbersome with high order interpolation techniques Inverse mapping is far more efficient in this case In this procedure the output frame is scanned pixel per pixel and line by line Using the inverse projection each output pixel center is associated a position in the input frame at which the image is interpolated This technique has been implemented in SWARP Fig 2 it possesses several advantages The output image is accessed sequentially and thus can be arbitrarily large Also only positions corresponding to pixels or sub pixels within the output frame have to be mapped Figure 2 Layout of the image mapping section The most potentially critical part is the pseudo random access in the input image In most cases it will be an individual imaging array like an individual CCD and will therefore fit in memory For much larger input images however we rely on the efficiency of virtual memory mapping SWarp s vir
30. e if the FSCALE KEYWORD keyword is not found in the FITS header FSCALE_KEYWORD FLXSCALE string FITS keyword that should contain the flux scale factor in input images GAIN_DEFAULT 0 0 floats n lt Nima Default gain conversion factor in e ADU to adopt for each image if the GAIN_KEYWORD keyword is not found in the FITS header 0 means infinite gain GAIN_KEYWORD GAIN string FITS keyword that should contain the gain in input images HEADER_ONLY N boolean If true SWARP does not do anything besides creating the FITS header in the combined image This header can later be duplicated as head files to provide an identical target frame on several machines HEADER_SUFFIX head string Filename extension for external ASCII headers that override internal FITS parameters IMAGEOUT_NAME coadd fits string Name of the output image file IMAGE SIZE 0 integers n lt Naim Dimensions of the output image in PIXELSCALE TYPE MANUAL or FIT mode O means automatic MEM_MAX 256 integer Maximum amount of megabytes allowed for storing input images in memory NTHREADS 0 integer Number of threads processes allowed to run simultaneously during the resampling and combination phases 0 automatically sets one thread per CPU core OVERSAMPLING 0 integers n lt Naim Amount of oversampling in each dimension 0 means automatic PIXEL_SCALE 0 0 floats n lt Naim Step between pixels in each dimension in PIXELSCAL
31. e stationarity breaks down on bright point sources for which intrinsic photon noises dominates In the most severe cases resampled noise peaks may generate distorsions in the resampled profiles Low background noise simulations were conducted in order to evaluate the amplitude of these distorsions on correctly sampled data PSF FWHM 3 pixels The effect is small although not totally negligible on sources with intermediate intensity On profile fitting measurements for instance photometry can be affected at the level of a few millimag rms The degradation of astrometric precision was found not to exceed a few millipixels rms On typical background noise limited images the effects are even smaller It is possible to stabilize the noise variance using a non linear dynamic scale transform Anscombe 1948 see also Stark et al 1998 The transformed signal is still bandpass limited but unfortunately resampling and transforming it back biases significantly the data 20 0 2 0 2 T T T F 95 8 o 2 4 F 2 o o oo 7 o o s Zz oi H Y 01 4 a L a 4 L 67 E oH A 0 L gt L a A El E i o 2 3 S 0 1 H 5 0 1 ol L o L S 4 o o 0 2 js pet ORT roi do 1 1 0 2 mn ge SI pO LE AE O Pp fp ape eg fi He 0 1 0 0 2 0 0 1 0 0 1 0 2 X bilinear X original pixels X Lanczos3 X original pixels 0 1 LE FRE LEE DE FEAT T 0 1 TT LE NE oad To Papo wet I 2 a o 4
32. ed in the background map The effect may be almost unnoticeable on individual input images where the signal to noise ratio is low and have measurable photometric consequences on the deep coadded image It is therefore advised to use large BACK_SIZEs in SWARP Of course if the mesh size is too large it will not be able to reproduce all the variations of the background a good compromise has to be found by the user Typically for reasonably sampled images a size of 128 the default to 512 pixels should work well The final background map is a natural bicubic spline interpolation between the meshes of the grid Before interpolating a median filter can be applied to suppress possible local overestima tions due to bright stars or artifacts BACK FILTERSIZE sets the size in background meshes of HObviously in some very unfavorable cases like small meshes falling on bright stars it leads to totally inaccurate results 24 su Hat D ett tat ae et te ah he E Ea Hi 4 PA E 4 Ho Clipped Mode ADU o T 1 0 5 10 15 20 25 30 Clipped Mean ADU Figure 12 Simulations of 32 x 32 pixels background meshes polluted by random Gaussian profiles The true background lies at 0 ADU While being slightly noisier the clipped Mode gives a more robust estimate than a clipped Mean in crowded regions the median filter 1 means no filtering applied to the background grid Us
33. g and mode estimation similar to the one employed in Stetson s DAOPHOT program see e g Da Costa 1992 Briefly the local background histogram is clipped iteratively until convergence at 30 around its median if o is changed by less than 20 during that process we consider that the field is not crowded and we simply take the mean of the clipped histogram as a value for the background otherwise we estimate the mode with Mode 2 5 x Median 1 5 x Mean 4 This expression is different from the usual approximation Mode 3 x Median 2 x Mean 5 In SWarp All background configuration keywords accept a list of values one value for each input frame 23 Figure 11 Example of residual gradients in a co addition after a constant has been subtracted from input images e g Kendall and Stuart 1977 but was found to be more accurate with our clipped distri butions from the simulations we made Fig 12 shows that the expression of the mode above is considerably less affected by crowding than a simple clipped mean like the one used in FOCAS Jarvis and Tyson 1981 or by Infante 1987 but is amp 30 noisier For this reason we revert to the mean in non crowded fields The choice of the mesh size in pixels BACK_SIZE is critical If it is too small the background estimation is affected by the presence of objects and random noise But more important is the fact that part of the flux of extended objects can be absorb
34. guments to the COMBINE TYPE configu ration parameter e AVERAGE The output is simply an unweighted average of all pixel values with non zero weights F Di Pili 10 N40 where p is the flux scaling factor see 86 8 and the composite weight is 2 n W 2 11 1 aa where the w are proportional to the inverse of the scaled variance no Needless to say that this combination is not optimum in terms of S N unless all input images have identical Gaussian noise e CHI2 The output is the square root of the reduced x of all pixel values with non zero weights aus f2 p ati 12 N40 By construction the composite weight the absolute one is W 1 13 The result of the combination is a so called x image Although it does not respond linearly to the input signals it can be used for detecting sources As shown by Szalay et al 1999 the x image is indeed the optimum combination to achieve panchromatic detection on a set of images taken at different wavelengths provided the data sets are background noise limited and that the noise is uncorrelated between frames This assumes further that the Point Spread Function PSF has been homogenized in all channels y images are most often used in deep panchromatic surveys requiring photometric redshift analyses The double image mode of SExtractor allows one to detect on the x image while making the photometric measurements on each of the single band images 27 e
35. he CCD g in e ADU defined at wij 1 the relative flux scaling factor p deduced from the photometric solution to calibrate the images DA fia pA fy Vi j l 24 and the relative weight scaling factor q derived from the comparison of the background noise level with the normalized weight input images will be weighted with qiwi Afi Ao wi and g are related through Now to optimally co add calibrated images one could weight them using 26 qiwi gt iWij p o However such weight maps exhibit strong variations on small scales in the presence of celestial objects 0 increases a lot on bright pixels The modulating effect of weighting combined with variations of the PSF and sampling errors would lead to significant distorsions of stellar profiles It is therefore more appropriate to weight pixels according to the intensity of the local background noise which is far smoother qliWij 55 27 Both fluxes and weights may have gone through resampling but for the sake of clarity we shall from now drop the from 86 6 29 This has also the advantage that one can use normalized flat fields as w s without prior knowledge of the CCD gain in the case where instrumental noise is negligeable For faint objects this weighting scheme is as efficient as that of 26 and is only suboptimum for the objects with very high surface brightness when the q s vary a lot from exposure to exposure But as the
36. ing are equal area projections ZEA CEA COE BON GLS PAR MOL AIT QSC AIT Aitoff is one of the most popular projections for all sky maps Note that some of the projections CYP CEA COD COE COO COP and BON require additional PV xx xx parameters These parameters can easily be included in a xxxx head header file with the same prefix as the output coadded image which is coadd fits by default see the example at the end of this document 13 Table 2 Valid PROJECTION_TYPEs in SWARP Zenithal projections AZP TAN STG SIN ARC ZPN ZEA AIR Cylindrical projections CYP CEA CAR MER Conic projections COP COE COD Cou Pseudoconic and polyconic projections BON PCO Pseudocylindrical projections GLS PAR MOL AIT Quad cube projections TSC CSC QSC Zenithal perspective Distorted tangential Stereographic Slant orthographic Zenithal equidistant Zenithal polynomial Zenithal equal area Airy Cylindrical perspective Cylindrical equal area Plate carr e Mercator Conic perspective Conic equal area Conic equidistant Conic orthomorphic Bonne s equal area Polyconic Global sinusoidal Sanson Flamsteed Parabolic Mollweide Hammer Aitoff Tangential spherical cube COBE quadrilateralized spherical cube Quadrilateralized spherical cube 14 GE VA Ly in SA x a A 4 j Y Figure 4 Graphic illust
37. ion parameter allows the user to choose among several sym metric compact interpolation functions 8This feature is currently limited to 2D images 17 e NEAREST a square box response function with width 1 pixel Applying this function produces nearest neighbour interpolation also known as sample and hold The kernel extends over a single input pixel e BILINEAR a pyramidal response function with Full Width at Half Maximum 1 pixel This results in a bilinear interpolation The kernel extends over 2 4im pixels e LANCZOS2 a ysinc rxa sinc xa response function with 2 lt xa lt 2 Lanczos2 window The kernel extends over 4 dim pixels e LANCZOS3 a sinc 7xq sinc xq response function with 3 lt xa lt 3 Lanczos3 window The kernel extends over 6 dim pixels e LANCZOS4 a sinc rxa sinc Fxa response function with 4 lt xq lt 4 Lanczos4 window The kernel extends over 8 dim pixels As demonstrated in Fig 6 the Lanczos4 interpolation function provides the best resampling for correctly sampled data In theory one could use an even larger kernel to get a closer to perfect resampling However in practice large kernels with a sharply limited bandpass carry more problems than advantages Artifacts image borders or undersampled data generate extended ripples Gibbs phenomenon These ripples are obvious on the saturation trail and the cosmic ray impact of the Lanczos interpolations in Fig 6 In ad
38. it can be changed using the HEADER_SUFFIX configuration parameter External headers may either be real FITS header cards no carriage return or ASCII files containing lines in FITS like format with the final line starting with END yj Multiple extensions in ASCII files must be separated by an ENDuuuuu line there should not be any primary header in that case External headers need not contain all the FITS keywords normally required The keywords present in external headers are only there to override their counterparts in the original image headers or add new ones With SWARP it is possible to process a single extension from an input Multi Extension FITS MEF file be it an image or a weight map For instance to process only the 3rd extension from the MEF file called mef fits one should use mef fits 2 Note that in command lines the and characters must be escaped with or the complete name put between double quotes to avoid expansion by the shell 6 5 2 Output frames The celestial pair of components of the output coordinate system is specified with the CELESTIAL_TYPE configuration parameter and can be selected among NATIVE PIXEL EQUATORIAL GALACTIC and ECLIPTIC In NATIVE mode the output celestial coordinate system is taken from that of the first file of the input list This is the default The PIXEL option forces SWARP to ignore all the ce lestial aspects projection de projection sky coordinates of
39. lap between input fields MANUAL Manual centering using the CENTER parameter CENTER 0 0 strings n lt Naim Position of the center in CENTER TYPE MANUAL mode Can be given in floating point nota tion in hh mm ss for right ascension longitude or dd mm ss for declination latitude COMBINE Y boolean If true resampled images will be combined COMBINE_BUFSIZE 256 integer Maximum amount of buffer memory in MB used for the co addition process COMBINE_TYPE MEDIAN keyword Tells SWARP how to combine resampled images MEDIAN Take the median of pixel values AVERAGE Take the average MIN Take the minimum MAX Take the maximum WEIGHTED Take the weighted average CHI2 Take the weighted quadratic sum SUM Take the sum COPY_KEYWORDS OBJECT strings n lt 1024 Coma separated list of FITS keywords that will be propagated from the input FITS head ers to the coadded and resampled image headers DELETE_TMPFILES Y boolean If true resampled temporary image files are deleted if COMBINE is set to Y FSCALASTRO_TYPE FIXED keyword Tells SWARP how to compute the astrometric part of the flux rescaling NONE Ignore the effects of re projection FIXED Apply a constant rescaling fluxes based on the ra tio of pixel scales at the field s geometrical center VARIABLE Apply a rescaling of fluxes based on the local ratio of pixel scales variable throughout the image FSCALE_DEFAULT 1 0 floats n lt Nima Default flux scale to adopt for each imag
40. ll subsections are written to disk in the RESAMPLE DIRECTORY as swarp zzz fits FITS files and read back later during the co addition phase to be stacked together Individual subsection files are automatically removed after processing or abortion It is possible to disable the deletion by setting the DELETE_TMPFILES configuration parameter to N the default is Y This can be useful for diagnostic purposes Note that although SWARP does not use much memory the amount of temporary disk space needed during processing can be quite large In addition to the output image and weight map one should provide disk space for the individual projected images and their weight maps In the case of mappings done at unit scale this involves storing more than twice the amount of input pixels as temporary data 6 3 Propagating FITS keywords During the re gridding and co addition processes involved in SWARP not all FITS keywords present in the input image headers are automatically copied to the output image headers as many of them become irrelevant Table 1 lists the non essential keywords that can be found in the output FITS header after being propagated updated by SWARP Table 1 Non essential FITS keywords managed by SWARP Name Output value EXPTIME Sum of exposure times in the part of the coadd with the most overlaps GAIN Effective gain flux and weight scaling apply see 6 9 2 MJD OBS Modified Julian day of the earliest start of exposures SATURATE
41. mbe F J 1948 Biometrika 15 246 2 Berriman G B Good J C Laity A C Kong M 2008 ASP Conference Series 394 83 3 Bertin E 1999 SExtractor 2 1 User s manual IAP 36 10 11 12 13 14 15 16 Bertin E 2006 ASP Conference Series 351 112 Bron C Kerbosch J 1973 Communications of the ACM v 16 n 9 575 Da Costa G S 1992 in Astronomical CCD Observing and Reduction Techniques ed How ell S B ASP Conf Series Fruchter A Hook R N 1997 SPIE 3164 120 Fruchter A Hook R N 2002 PASP 114 144 Infante L 1987 A amp A 183 177 Jarvis J J Tyson J A 1981 AJ 86 476 Kendall M Stuart K 1977 The Advanced Theory of Statistics Vol 1 Charles Griffin amp Co London Makovoz D Khan I 2005 ASP Conference Series 347 81 Starck J L Murtagh F Bijaoui A 1998 Image Processing and Data Analysis Cambridge University Press Szalay A S Connolly A J Szokoly G P 1999 AJ 117 68 Valdes F 1998 ASP Conference Series 145 53 Wolberg G 1992 Digital Image Warping IEEE Computer Society Press 37
42. ne measurements from individual images than to co add pixels see http www cv nrao edu fits documents wes wes html http terapix iap fr 3http astromatic net software scamp Author It is true that resampling distorts slightly both signal and noise However if done properly and if the data are not undersampled Full Width at Half Maximum gt 2 5 pixels the degradation is generally very small For instance resampling a single ground based image atmospheric seeing of FWHM 3 pixels and conversion factor 2e ADU twice with SWARP and comparing fluxes and positions measured using PSF fitting one gets for bright stars rms differences of less than 5 1074 mag and 107 pixel between the original and the twice resampled images This is already much better than what photon noise allows for Hence apart from situations of strongly non stationary noise or undersampled data the consequences of resampling are expected to be negligible 3 License SWARP is free software you can redistribute it and or modify it under the terms of the GNU General Public License as published by the Free Software Foundation either version 3 of the License or at your option any later version SWARP is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY without even the implied warranty of MER CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE See the GNU General Pub lic License for more details You should have received a copy of the GNU Gene
43. ometric conditions experience fluctuating gains But this effect is generally small taking the quite unfavourable case of qi X p constant background noise with p s varying by a factor of 4 between two exposures and weights varying from 0 5 to 1 makes G to vary by 20 at most This should not cause significant difficulties in any profile fitting routine Therefore 33 still remains a very good approximation in the general case and the coadded gains provided by this method are still more stable than what unweighted co addition offers This does not prevent regions with lower coverage in which N is smaller having lower gains To avoid large gain drops in the gaps of CCD mosaic images it is recommended for the observations to use large random dithering patterns consisting of at least 4 5 exposures per covered sky area 6 9 3 Image buffering and memory constraints In order to maximize the efficiency of the image combination process SWARP version 2 0 and above allocates a significant amount of memory to buffer input and output data This amount of memory can be set by the user with the COMBINE_BUFSIZE configuration parameter The default value for COMBINE_BUFSIZE is 256 Megabytes If your machine is a bit short of memory you should decrease this value Conversely if you need to combine a large number of overlapping images you might want to set COMBINE_BUFSIZE to a substantial fraction of the memory available on your machine to
44. ral Public License along with SWARP If not see http www gnu org licenses 4 Installing the software 4 1 Obtaining SWARP The easiest way to obtain SWARP is to download it from the official website or alternatively from the current official anonymous FTP site At this address the latest versions of the program are available as standard tar gz Unix source archives as well as documentation and RPM binary packages for various architectures It is strongly advised to install one of the RPM packages if you intend to run the software for production purposes on a Linux machine with x86 architecture and RPM support as the RPM executables have been optimized for execution speed 4 2 Software and hardware requirements SWARP has been developed on Unix machines Compaq Tru 64 and GNU Linux and should compile on any POSIX compliant system The software is run in ANSI text mode from a shell A window system is therefore unnecessary with present versions Memory requirements are fairly modest in most cases as they do not depend on the size of the output images 100MB is sufficient when co adding images even mosaics involving current CCD chips 2kx4 5k More memory may be helpful for co adding bigger maps Although the built in virtual memory feature will almost always allow one to work with any image size the performance hit caused by file swapping may be important in some cases http astromatic net software swarp Sftp ftp iap fr p
45. ration of projections available in the WCS library see text 15 LS amp Figure 5 Graphic illustration of projections available in the WCS library continued from Fig 4 Centering of the output frame is controlled by the CENTER TYPE parameter There are three centering modes e ALL the field is centered in a way that all input images fit into the output frame This is the default e MOST the field is centered on the zone of maximum overlap between input images e MANUAL manual centering with the CENTER parameter A different centering mode can be used in each dimension for instance in 2D images with a 6 coordinates CENTER TYPE ALL MOST will apply the ALL mode in a and the MOST mode in 6 If a single mode is specified it is applied to all available dimensions The CENTER parameters are active in CENTER TYPE MANUAL mode only and must be used to spec ify the actual center of the output field in world units In the case of angular coordinates both the floating point in degrees and sexagedecimal formats are accepted right ascension longitude may be written as hh mm ss ss and declination latitude as dd mm ss ss The pixel scale which is the step between pixels at the center of the output frame can be computed automatically in each dimension by SWARP There are five modes specified by the PIXELSCALE TYPE configu
46. ration parameter e MEDIAN the default the median value of all pixel scales at the center of input frames is taken as the output pixel scale e MIN the smallest of all pixel scales at the center of input frames is taken e MAX the largest of all pixel scales at the center of input frames is taken e MANUAL manual scaling with the PIXEL SCALE configuration parameter 16 e FIT Pixel scales are automatically computed to have the projected data fitting the output frame dimensions specified with the IMAGE_SIZE configuration parameter When right ascension longitude and declination latitude are both present pixel scales computed by SWARP are made equal in both dimensions to avoid anamorphosis The PIXEL_SCALE parameters are active in PIXELSCALE_TYPE MANUAL mode only and must be used to specify the actual pixel step for each dimension in world units Note that in the case of angular coordinates PIXEL_SCALE values are read in arcseconds not degrees The dimensions of the output frame in number of pixels per axis are set using the IMAGE_SIZE configuration parameter A value of O for any axis results in an automatic dimensioning of this axis but obviously this is not possible in PIXELSCALE_TYPE FIT mode Note that the current simple algorithms used for automatic centering and scaling routines can get confused rather easily close to the pole or with some very wide field projections In particular it is recommended to turn off
47. s one can write the output coadded flux as A fy ELO map VI 28 i diWij and the resulting variance as DAN Ro Did WijPi Ad 29 i Di qiwiy Using 25 an equivalent local gain G in the coadded image can be computed Af S qiwi piA fi Get LAA S aie 30 Ped Ao X qz wp Ao 2 Ra a where w is the composite weight map of the coadded image From our definition of weights w is inversely proportional to ee and must be 1 if all input weights are at 1 hence it is easy to show that Di GW w E 31 j gt di en Substituting 31 in 30 and using 24 and 25 one gets Di GW aa oe eames a 32 Di WI Dig i Unfortunately as can be seen this coadded gain will vary with position in the general case Nevertheless some approximations can be made to simplify this expression First of all in most cases gi will be almost constant from one input image to another Second if exposures are taken under photometric conditions with constant sky brightness and negligible instrumental noise one should have q x p 1 removing the dependence with input weights and therefore position in the coadded image In that very case the resulting gain is simply GRI nr 33 30 Sadly in many bands the presence of clouds does not decrease the sky brightness as much as source brightness and doesn t act at all like a decrease in global sensitivity or exposure time Coadded regions of a survey that are taken under non phot
48. se objects are easily detected on individual exposures the most accurate photometry is still possible by combining the N measurements In practice SWARP can read several types of weight maps although they are all internally converted to variance for processing The input weight map format must be specified with the WEIGHT_TYPE keyword WEIGHT_TYPE NONE is for no input weight map the default MAP_RMS for indicating that the data contain the absolute standard deviation of pixel values MAP_VARIANCE for weight map data stored as relative variances and MAP_WEIGHT for relative weights Relative variance maps and weight maps are re scaled internally using local variance measurements made directly on the input images This has the advantage of making it possible to use for instance a single flat field image as a weight map for a whole series of background noise limited images with different exposures But automatic re scaling may sometimes lead to inaccurate weightings because of source crowding or complicated weight maps for instance In cases where accurate weighting is important one should either use absolute standard deviation maps WEIGHT_TYPE MAP_RMS or turn off weight variance rescaling by setting the RESCALE_WEIGHTS configuration switch to N default is Y When producing composite fields larger than the input images the latter must be background subtracted prior to coaddition to avoid generating discontinuites in the output image Thu
49. the image by 30 degrees and applies a 16 9 anamorphosis to the data would be CD1 1 6 4150E 5 CD1 2 2 0833E 5 CD2_1 3 7037E 5 CD2 2 3 6084E 5 END 9 Troubleshooting In case you face a problem which not listed below please do not hesitate to discuss it in the SWARP section of the AstrOmatic forums My window terminal crashes during a long SWARP run IShttp astromatic net forum 39 Unexpected crashes of XTERM windows have been reported This seems to be caused by the large number of ANSI control sequences that SWARP sends to the terminal You may either set the SWARP configuration keyword VERBOSE_TYPE to QUIET or FULL and or redirect the output to a file SWARP crashes with error messages like gt Error pthread_create failed The multithreaded version of SWARP requires a fairly large stack that may exceed the maximum value allowed by your shell Use the shell command limit to increase the stacksize parameter if required you might need the root privileges to change this if it is a hard limit SWARP crashes with error messages like gt Error Not enough memory for although I have properly set the MEM MAX VMEM MAX parameters The maximum value allowed by your shell for memory use might be set too low Use the shell command limit to increase the datasize memoryuse and vmemoryuse parameters if required you might need the root privileges to change this if it is a hard limit
50. tion purposes no need for a sophisticated interpolation 4 A small oversampling only to have pretty antialiased field limits N GAIN 0 0 3S Background subtraction gt N No background subtraction AUTO 33 BACK_DEFAULT 0 0 BACK_SIZE 128 BACK_FILTERSIZE 3 nn RES Virtual memory management VMEM_DIR i VMEM_MAX 2047 MEM_MAX 256 256 MB should be enough to avoid swapping Te eS SSS SS PER N Miscellaneous gt esssssssss DELETE_TMPFILES Y Delete temporary resampled FITS files VERBOSE_TYPE NORMAL In this application coverage maps can be generated using the output weight map instead of the image itself 8 2 Example 2 In this example one has a set of CCD images taken with a standard dithering strategy input fits and the related set of weight maps input w fits However the unusual thing is that for some reason the output has to be tilted by 30 degrees with respect to the local north south axis and the pixels must have an aspect ratio of 16 9 First one starts with a fairly standard configuration file AAA TPA RR Te ARANA EA Wut put SSSR a SSTe IMAGEOUT NAME coadd fits Output filename WEIGHTOUT NAME coadd weight fits Output weight map filename E A Input Wei ghts 25 227222722232 395 WEIGHT_TYPE MAP_WEIGHT all or for each weight map WEIGHT_SUFFIX w fits Suffix to use for weight maps WEIGHT_IMAGE
51. tools for instance This problem can be approximately solved by dilating appropriately the interpolation kernel or by pre filtering the input image like textures in 3D hardware A more exact and more efficient solution is to oversample the output pixel grid Fig 10 in order to obtain a density of samples per unit area or hypervolume at least equal to that of the input image Oversampling is controlled by the OVERSAMPLING configuration parameter If OVERSAMPLING is set to 1 no oversampling is applied An OVERSAMPLING of 2 oversamples the data by 2 dim 18 te a nearest neighbour A S B E Eos 4 E S E e Es E 3 E o f 1 L i 1 4 0 2 4 x Ep 4 bilinear g 2 E ost 4 E 2 E 5 Es p 3 E o L L L L L 4 2 o 2 4 x T Er a Lanezos2 S ka E Bost J E El z 3 ES E E o f f 1 1 i 4 p o 2 4 x T 1H 4 Lanezos3 g 2 3 E Eos E E S Es 2 E 0 Ll L L L L L 4 2 0 2 4 x LE S Lanczos4 A 3 E 205 4 E El 3 e Es E 3 i o f f 1 1 f Figure 6 Comparison between resampling methods From top to bottom nearest neighbour bilinear Lanczos2 Lanczos3 Lanczos4 From left to right Interpolation function profile Mod ulation Transfer Function result from shifting an image by half a pixel in both direction and result for a 5x zoom rotation by 20
52. tual memory engine works in the following way each input image stored as a single precision 4 byte array is loaded in physical memory if the required amount of megabytes doesn t exceed MEM MAX If it does a temporary file called vmzzzzr vxxrr tmp is written in the directory specified by VMEM DIRECTORY The program exits with an error message if this file would exceed VMEM MAX Megabytes The 3 memory parameters are mostly hardware dependent It is 10 advised to set MEM_MAX to 50 100 of the actual amount of memory present in your machine If disk space is not the limiting factor VMEM MAX should be set to a higher value 2048 on a 32 bit machine or even more on a 64 bit machine The choice of the VMEM_DIRECTORY is critical First this write enabled directory must be large enough to contain each input image in floating point format Second it is strongly recommended to have the data on a fast disk Note that the default path for VMEM DIRECTORY is tmp which on many systems is on a partition simply not large enough to handle the typical quantities of data to process An alternative is to use the current directory at the expense of disk thrashing De projecting and co adding simultaneously all input images would frequently imply many files open at the same time and large amounts of virtual memory SWARP takes a more sequential approach each input image is mapped in a tightly fitting rectangular subsection of the output frame A
53. ually a size of 3 meshes the default is sufficient but it may be necessary to use larger dimensions especially to compensate in part for small background mesh sizes or in the case of large artifacts in the images Median filtering also helps reducing possible ringing effects of the bicubic spline around bright features In some specific cases it might be desirable to median filter only background meshes whose original values exceed some threshold above the filtered value This differential threshold is set by the BACK FILTERTHRESH parameter in ADUs The default is 0 By default the computed background map is automatically subtracted from the input image But there are some situations where it is more appropriate to subtract a constant from the image e g images where the background noise distribution is strongly skewed The BACK_TYPE configuration parameter set by default to AUTO can be switched to MANUAL to allow for the value specified by the BACK_DEFAULT parameter to be subtracted from the input image The default value is 0 As said before the background estimation procedure is used not only for background subtraction but also for other tasks in SWARP such as weight calibration Thus even if SUBTRACT_BACK is set to N or BACK_TYPE is in MANUAL mode reasonable values for other background parameters must be given to ensure proper working of the software Note that the present version of background subtraction doesn t work
54. ub from_users bertin swarp 4 3 Installation To install you must first uncompress and unarchive the archive gzip dc swarp x x tar gz tar xvf A new directory called swarp x x should now appear at the current position on your disk You should then just enter the directory and follow the instructions in the file called INSTALL The software is also available as a precompiled RPM for Linux systems with an x86 architecture The simplest way to install an RPM package is to log as root and use the following command rpm U swarp x x dist arch rpm 5 Using SWARP SWARP is run from the shell with the following syntax swarp Input_imagel Input_image 2 Image_list1 Image_list2 c configuration file Parameter1 Value1 Parameter2 Value2 The part enclosed within brackets is optional The file names of input images can be directly provided in the command line or in lists preceded with Lists are ASCII files containing the input file names one per line One should use lists instead of image file names if the number of input images is too large to be handled directly by the shell Any Parameter Value statement in the command line overrides the corresponding definition in the configuration file or any default value see below 5 1 The Configuration file Each time SWARP is run it looks for a configuration file If no configuration file is specified in the command line it is assumed to be called

User's guide - Astromatic.net

Contents

Download Pdf Manuals

Related Search

Related Contents