Home

Very Large Telescope Paranal Science Operations

1. dfits fits grep TPL ID Usually you want to get the value of a list of given FITS keywords in a list of FITS files fitsort reads the output from dfits classifies the keywords into columns and prints out in a readable format the keyword values and file names Example dfits fits fitsort NAXIS1 NAXIS2 BITPIX fitsort also understands the shortFITS notation where e g HIERARCH ESO TPL ID is short ened to TPL ID A classification example could be both commands are equivalent since fitsort is case insensitive dfits fits fitsort TPL ID DPR TYPE dfits fits fitsort tpl id dpr type The output from this combination is something like FILE TPL ID DPR TYPE ISAAC 1999 04 09T19 16 32 047 fits ISAACSW_img_cal_Darks DARK ISAAC 1999 04 09T19 56 51 011 fits ISAACSW_img_cal_Darks DARK ISAAC 1999 04 09T20 37 09 058 fits ISAACSW_img_cal_Darks DARK ISAAC 1999 04 09T21 00 49 003 fits ISAACSW_img_cal_Darks DARK ISAAC 1999 04 09T21 24 27 081 fits ISAACSW_img_cal_Darks DARK ISAAC 1999 04 09T21 37 26 086 fits ISAACSW_img_cal_Darks DARK This kind of table is useful in getting an idea of what is present in a directory or list of direc tories Loading such a summary table into a spreadsheet program also makes it conveniently readable INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 4 2 2 Visualizing FITS Tables Similarly a FITS table can be visualized on the command line without the need to call a full fledge
2. Illumination Correction Flagging bad pixels removing vignetted regions Sky subtraction Removing the 50 Hz pickup e OOo Pe OES ET E Gr Registration and stacking re gt Removal of residual bias variations Steps indicated with a star are supported by the jitter recipe 4 1 Darks and dark recipe Dark frames are exposures without detector illumination The dark current of the ISAAC SW detector is small so the dominant feature in these frames is the detector bias which is also called the zero level offset since it is not possible to take a zero second exposure with the SW array Usually one takes at least three darks and combines them with suitable clipping to create the dark frame that is subtracted from the science data As the bias is a function of the DIT the DIT of the science data and that of the dark must match Dark frames are acquired through a dedicated template which obtains usually at the end of the night at least three dark frames for each DIT that was used during the night The darks can be combined with the dark recipe This recipe will produce one dark frame for each DIT The readout noise is also measured The dark recipe actually consists of two separated sub recipes the first one creates the dark and the second one computes the readout noise The dark created by dark is nothing more than an average of the input files after having sorted them by identical DIT Cosmic ray rejection has
3. e Apply the formula listed in Sec 6 3 2 INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 30 6 3 4 Electronic ghosts For bright sources one can see electronic ghosts which are 8 16 24 etc rows away from the true source The amplitude of these ghosts depends on the brightness of the source their position relative to the central row and how fast the array is read The faster the array is read the stronger these ghosts are For very bright sources one may see a negative ghost which is four pixels away from the source Prior to March 2001 the array was read out more quickly than it is now so these ghosts were significantly stronger 6 3 5 Fringing The Aladdin array fringes in the M band We do not know how well the fringing can be removed with normal flats 6 3 6 Image quality It has taken us a long time to get acceptable image quality in the LW arm However the imaging data particularly in M band still have a distorted elongated PSF while most of the spectra show strongly tilted telluric absorption lines 6 3 7 Dead region The top left hand corner of the array is completely insensitive to flux 7 LW imaging Reducing science data LW imaging data can be taken with two different techniques The first one removes the sky by chopping the secondary mirror in phase with the detector readout The second one uses techniques which are identical to those used in the SW arm For frames taken without chopping the b
4. e Non linearity correction e Flat Fielding e A B Subtraction e Slit Curvature Correction and wavelength calibration e Registration and stacking e Extraction e Telluric line correction e Flux calibration Steps that are marked with an asterisk have not been thoroughly tested Steps which are very similar to the equivalent one described in the previous section are not discussed any further in the following subsection We invite the reader to refer to the previous sections as needed 8 1 First Sky subtraction non chopping only When the chopper is not used there are two techniques one uses in taking IR spectra There is the classical sequence where one observes the object at two slit positions i e ABBAABBA etc and there is the more complex case where one has observed the target along several slit positions In the classical case one simply subtracts frames taken at different slit positions So one needs to form A B B A etc This simple step removes the sky and results in an image with two spectra one positive and one negative In more complex cases one could build a sky frame from several spectra as one does when building the sky frame in imaging This results in an image with only one positive spectrum INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 34 8 2 A B Subtraction chopping only For templates that use chopping the typical observing sequence is ABBAA etc One first groups the frames into AB
5. which is usually better TEL AMBI FWHM END Astronomical Site Monitor seeing at start Note that this value might differ significantly from the ISAAC image quality which is usually better Instrument Keywords Here is a non exhaustive slit of instrument keywords INS MODE The instrument mode It is a label codifying the arm short or long and the mode imaging spectroscopy etc used for the current frame See the ISAAC User s Manual for more information about possible values of this keyword INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 5 2 7 INS FILT1 1D SW filter on filter wheel 1 INS FILT2 1D SW filter on filter wheel 2 INS FILT3 1ID LW filter on filter wheel 1 INS FILT4 1D LW filter on filter wheel 2 INS OPTI1 ID Mask or slit in position e g slit_1 INS GRAT NAME Grating in position LR or MR INS GRAT WLEN Central wavelength microns Note however that the value recorded here in spite of the number of digits is a rough estimate of the central wavelength INS GRAT ORDER Grating order INS OPTI2 NAME Name of the objective INS LAMP1 ST Argon lamp status INS LAMP2 ST Xenon lamp status DET DIT Detector Integration Time seconds DET NDIT Number of averaged DITs DET MODE NAME Detector readout mode INS PIXSCALE Pixel scale in arcseconds per pixel Observation Block Keywords OBS PROG ID Program ID OBS NAME Name of the OB as prepared with P2PP OBS TARG NAME Target package na
6. Abbreviations and acronyms o cesos LA Stylistic CONVeORHOne lt s c ek wean dw dee bea A ISAAC FITS Information 2 1 Extracting FITS Information nodo nicas 22 Visualizing FITS TABES obocia Be ORE DEORE HE Oa AR 23 File Names oe cea ee hee ee DEAE A A Sa 24 Telescope Keywords s sac ge da Mee hed ean ee eR E eR eS 2 5 Instrument Keywords ocios rss rre ER ee e 2 6 Observation Block Keywords o o ee of Template Keywords ated EE ewe ira A 2 7 1 Keywords common to all templates 2 7 2 Keywords common to the jitter templates me Pipeline Produs lt a kw dr ARA A 29 ummary LAO a EN a A A e An Overview of Short wavelength Data Sc EXPTINE NDIT and DIT a ceada isis rare eo 3 2 General Features of the ISAAC SW Infrared Detector oo BaS VISOS wee cee bee ein ded ee BEES SAK EY a Die DRI regibter BOW co eee ew ee ee ed BH Ee Re da ww g 3 2 3 Electrical ghosts lt p croco nespi te eee ra Eee BES Rs 322A Detector mon lwesrily o lt 4 542644 pos SS epee OL eee OL EE EW 3 20 Odd even column eflect ic dee bee eee ER EEE BS 32 0 Remnants irom bright SOUR fi eet ewe eh E eR CN 3 2 7 QE reduction from really bright sources 928 WO PICKUP sci ek RA di A KEG a ee EOI amp acer es ee ad A be ee ee ee a g SW Imaging Reducing the science data 4 1 Darke end dark recipe e bebe eee ewe eee eGR eee ees 411 1 Read out noise computation
7. computed zero point e mag is the known magnitude of the standard star in the observed band e flux is the measured flux in ADUs in the image e DIT is the detector integration time Note that neither the extinction nor the colour correction are included in the ZP The average airmass is given in the output result file together with individual airmass values for each frame The average extinction on Paranal for the J Js H Ks and NB_M filters is available from the ISAAC web pages The correspondence between the filter in which the observations were taken and the filter in the standard star table is listed below The two filters are reported in the QC parameters QC FILTER OBS and QC FILTER REF This correspondence completely ignores corrections due to filter mismatch and in some cases these corrections are substantial ISAAC filter Reference filter Z J SZ J Js J J J SH H H H K K or Ks Ks Ks or K L M L M INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 20 ISAAC narrow band filter Reference filter NB_1 06 J NB_1 08 J NB_1 19 J NB_1 21 J NB_1 26 J NB_1 28 J NB_1 64 H NB_1 71 H NB_2 07 Ks NB_2 09 Ks NB_2 13 Ks NB_2 17 Ks NB_2 19 Ks NB_2 25 Ks NB_2 29 Ks NB_2 34 Ks 4 8 1 Colour terms The match between the ISAAC filters J H and Ks and those used to establish the faint IR sta
8. curve is then multiplied into the object spectrum For solar type stars a blackbody curve is a good enough description of the spectral energy distribution above 1 6 microns Below 1 6 microns a more accurate description of the con tinuum is required The spectral energy distribution of a wide variety of stars are available through the ISAAC web pages The second step is absolute flux calibration If the magnitude of the target is known and if the spectra were taken with the LR grating a reasonably accurate calibration can be obtained by convolving the spectrum with the filter curve and determining the appropriate scaling If neither of the above are true then determining the absolute flux calibration is more difficult and less certain ISAAC standard star observations are done with the slit the object was observed In the case the telluric has been observed also with the 2 slit or slitless it is possible to estimate the slit losses Slit losses can also be roughly estimated from the image quality in the acquisition image Alternatively if the seeing did not change too drastically while the target and the standard were observed slit losses can be ignored 5 8 SPJITTER The pipeline recipe that reduces SW spectroscopic data is called spjitter The recipe uses an initialisation file which contains parameters that define how the recipe runs The initialisation file contains parameters which indicate e which table to use to correct for slit
9. in a table named outname _extracted tfits The spectra are traced with a 3rd order polynomial and the results are stored in outname _shapes tfits 4 A table with 2d polynomial modeling the slit tilt For each grating the spectral tilt is modeled with a 2d polynomial which is written to a table named outname poly2D_XR tfits This table is the one used by spjitter to correct the spectral tilt 5 10 Response Function On a regular basis the observatory observes in slitless mode a standard star that has a reasonably well characterised spectral energy distribution These data are used to calculate the instrument throughput It is used for quality control purposes and observers will have little use for this routine The recipe respfunc takes the FITS image created by spjitter extracts the spectrum correlates the RA and DEC in the FITS header with a star in the catalog and calculates the efficiency and the conversion curves The output consists of four files e outnamel_extr tfits is the extracted spectrum INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 26 e outnamej_back tfits is the background spectrum e outname _conversion tfits is the conversion curve e outname efficiency tfits is the efficiency curve INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 27 6 An Overview of Long wavelength Data The LW arm of ISAAC uses a Aladdin 1024x1024 array When reducing data from the LW arm you should keep
10. is_rename recipe to rename your files and set this keyword accordingly Nothing requires this name to be set it is only here for convenience e PRO DID contains the version number of the dictionary used for all product keywords e PRO TYPE contains the type of the data products as one of TEMPORARY PREPRO CESSED REDUCED or QCPARAM INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 7 PRO REDLEVEL contains the reduction status of the file its possible values are not clearly defined yet for infrared instruments PRO CATG is probably the most important product keyword since it labels each frame with a product ID unique to the recipe It qualifies files with hopefully understandable product labels PRO STATUS specifies whether the reduction performed OK or whether some warnings were issued by the algorithm PRO DATE gives the date and time of the pipeline execution for the frame creation PRO REC ID identifies the recipe that generated the file with a unique name PRO REC DRS ID identifies the Data Reduction System that was used to produce the file ISAAC pipeline for all ISAAC recipes PRO DATANCOM specifies the number of raw frames that were combined to generate the product Its exact meaning depends on the recipe see each recipe documentation to learn what it refers to ISAAC files from which the electrical ghost has been removed contain two additional key words e GHOSTREM is a FITS boole
11. pairs and subtracts one frame from another This will result in an frame with one positive spectrum and two negative spectra The positive spectrum will have a flux which corresponds to 2 DIT seconds 8 3 Slit curvature correction and wavelength calibration ISAAC spectra are strongly curved and tilted Before the 2d spectra are shifted and combined users should think about straightening them It is useful to do the wavelength calibration at the same time so that the horizontal axis is in wavelength units Wavelength calibration in the LW is more complicated The arc lines are fewer and they have to compete with the bright thermal background In the M band there are no arc lines at all so the wavelength calibration with arcs can only be done with the grating in third order For L band MR spectroscopy we provide arcs with the grating in the first and third order Users can choose which arc they wish to use For reasons which are not clear the measured wavelengths of the 3rd order lines are slightly different to what one gets by multiplying the wavelengths by 3 The difference has been measured in the L band since it is possible to see both 3rd order and 1st order lines For the medium resolution grating the difference is described by the formula Correct Wavelength 1 005 CalibratedWavelength 0 005 x GratingSetting The formula for the low resolution grating has not been determined Alternatively for bright targets one can use th
12. that a residual bias remains after the exposure with the lamp off is subtracted from the one with the lamp on This is a fundamental property of the array the bias is a function of the amount of flux that the array sees It may be possible to use the parts of the array that are not exposed to light to partially remove the residual bias however this is not yet part of the recipe At some wavelengths the spectral response of the lamp is a very strong function of wavelength and it is possible in extreme cases for the frame with the lamp off to have a higher level in some regions that the frame with the lamp on In these cases it is better to estimate the bias from the regions which do not have direct flux i e the 50 or so pixels at the top and bottom of the image INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 22 The recipe does not remove the wavelength dependence of the lamp from the flat field This is removed when the telluric standard is divided into the spectrum of the science target Implicit in this statement is that the same flat was used for both the target and the standard and that the target and the standard were observed at roughly the same place on the array If neither of these things are true then flux calibration relative and absolute will be less accurate 5 3 Slit curvature correction and wavelength calibration ISAAC spectra are strongly curved and tilted Before the 2d spectra are combined they need to
13. the flat field one describes it over two length scales on the scale of individual pixels and on the scale of the entire array As discussed above the pixel to pixel accuracy is around 0 2 0 3 On the scale of the entire array the accuracy of the flat field is of the order of 1 3 This is measured by placing a bright standard over several positions of the array and by measuring the flux of the standard after the data has been flat fielded this test is sometimes called the thousand points of light test Alternatively one can use the science data itself if dithering has been used and if there are enough sources that are sufficiently bright The large scale accuracy of the flat field can be improved to the 1 level by using an illumination correction INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 16 Scientific data should be normalised with flats that were taken within one week of the scientific data Beyond this the relative accuracy at which pixels can be normalised slowly degrades 4 3 Illumination Corrections The illumination correction takes into account low frequency differences between the true flat and the twilight flat These variations have an RMS of 2 If this is accurate enough then an illumination correction is not needed To compute these variations a dedicated calibration template is offered The procedure images a standard star over a grid of 17 positions 1 central and a 4x4 grid with the convention th
14. y 530 330 It is probably caused by imperfections with the 4 07 micron NB filter as it is not seen in other long wavelength filters In most cases the ghost is removed by nodding but some residuals can be seen when parallactic angle is changing quickly This occurs when the telescope crosses the meridian near Zenith Flats taken with the 4 07 micron NB filter will contain this ghost so it may be better to flat field with one of the other NB filters 7 6 Photometric Accuracy The information we have on the photometric accuracy of LW imaging data is limited but we do see systematic errors at the 0 03 magnitude level 8 LW spectroscopy Reducing science data LW spectroscopic data can be taken with two different techniques The first uses the chopping secondary and the second does not If chopping was not used then the most basic way of taking IR spectra is to observe the target along two slit positions The sky is then removed by a process which is sometimes called double sky subtraction The basic steps of how to reduce these type of data are e Dark subtraction e Ghost removal e Non linearity correction INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 33 e Flat Fielding e First sky subtraction e Distortion Correction and wavelength calibration e Registration and stacking e Extraction e Telluric line correction e Flux calibration For frames taken with chopping the basic steps are e Ghost removal
15. 4 with 7 09 in the above text INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 10 3 2 General Features of the ISAAC SW Infrared Detector 3 2 1 Bias variations The ISAAC SW infrared detector bias is a function of the detector integration time DIT and the detector illumination It also varies in time most notably at the rows where the reading of the detector starts that is rows 1 2 3 and rows 513 514 515 It is therefore common to observe bias variations from one image to the next This is particu larly the case for the first image in a sequence template of images These bias variations are non uniform across the array but are uniform along most rows These variations are usually not a serious problem and do not prevent one from using all the images but they may require some special treatment e g fitting all lines with rejection of positive signal so as to derive the vertical pattern of the bias or of its variation between images The effect is strongest in DCR mode and is more easily seen in low background conditions 3 2 2 Shift register glow The detector shift registers generate light which is in turn detected by the detector These glows are visible on 4 quadrants of the detector at the bottom and top of the image Although strongest in NDR they can also be seen in DCR The glow subtracts out perfectly when subtracting sky or dark frames that have been taken with the same DIT Moreover most of th
16. Analysis Facility PArameter File Right Ascension Unit Telecope Very Large Telescope 1 4 Stylistic conventions The following styles are used bold italic box teletype in the text for commands etc as they have to be typed for parts that have to be substituted with real content for buttons to click on for examples and filenames with path in the text Bold and italic are also used to highlight words 2 INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 3 2 ISAAC FITS Information Ancillary data attached to ISAAC files are all written into FITS headers The following chapter lists the most important keywords and describes commands on how to retrieve them For ease of reading keywords are shortened from HIERARCH ESO A B C to A B C shortFITS notation Notice that all of this information is present in the ESO dictionaries available from the ESO archive Web site The following chapter only tries to summarize the most important information 2 1 Extracting FITS Information There are many tools to extract and parse FITS headers One convenient way of extracting FITS information and displaying it on a terminal or re directing it to a text file is to use two stand alone programs called dfits and fitsort Both are included into the ISAAC pipeline distribution dfits dumps a FITS header on stdout You can use it to dump the FITS headers of many files to allow the parsing of the output Example
17. EUROPEAN SOUTHERN OBSERVATORY ES Organisation Europ ene pour des Recherches Astronomiques dans H misph re Austral Q Europ ische Organisation f r astronomische Forschung in der s dlichen Hemisph re ESO European Southern Observatory Karl Schwarzschild Str 2 D 85748 Garching bei M nchen Very Large Telescope Paranal Science Operations INSTRUMENT data reduction cookbook Doc No VLT MAN ESO 14100 4031 Issue 80 0 Date 11 10 2007 E Mason er EE tk OS iena paia ay EE Date Signature A Kaufer A Date Signature 0 Hainaut Released EE EECHER Date Signature INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 ii This page was intentionally left blank INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 ill Change Record Issue Rev Date Section Parag affected Reason Initiation Documents Remarks 80 11 0ct 2007 all cookbook creation and first archiving INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 iv Acknowledgment This cookbook is an extract and an updated of the ISAAC data reduction guide written by J G Cuby P Amico N Devillard and C Lidman They deserve the credit for all the precious information herein included INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 Contents 1 Introduction fat EEN 2 0 os d e he BO BI ie E ER et WEE ee E 1 2 Reference documents 22 bs kee eR ee eR RO eR ee A 1 3
18. ND AMBI FWHM START AMBI FWHM END ID NEXP EXPNO Grating order Grating central wavelength microns Status of the Argon lamp Status of the Xenon lamp Instrument acquisition mode Mask or slit in position Optical objective name Pixel scale in arcseconds per pixel Program ID Observation Block name Target package name File name given during acquisition File name as attributed by the pipeline Data product category Number of frames combined into this product Pipeline reduction date Dictionary version for PRO keywords Applied recipe ID Data Reduction System ID Data product reduction level Data product reduction status Data product type Right ascension J2000 in degrees Cumulated offset in X in pixels Cumulated offset in Y in pixels Cumulated offset in alpha in arcseconds Cumulated offset in delta in arcseconds Poisson homogeneity in jitter throw Relative offset in X in pixels Relative offset in Y in pixels Relative offset in alpha in arcseconds Relative offset in delta in arcseconds Airmass at start of exposure Airmass at end of exposure ASM seeing at start ASM seeing at end Template ID Number of exposures expected in the template Exposure rank in the template VLT MAN ESO 14100 4031 INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 9 3 An Overview of Short wavelength Data The SW arm of ISAAC uses a Hawaii 1024x1024 array from Rockwell When reducing data from the SW detect
19. R results in better noise performance For these reasons DCR is used when the background is high imaging and NDR is used when the background is low spectroscopy It is important to appreciate that different parts of the detector are read out at different times If we take DCR as an example set the DIT to the minimum i e 1 77 seconds and set the clock at 0 seconds when the first row is read out then row 1 is the difference of two reads that occur at 0 and approximately 1 77 seconds and row 512 is the difference between two reads that occur at approximately 1 77 and 3 54 seconds During the first 1 77 seconds row 512 is still collecting photons In other words different parts of the detector sample different parts of the integration ramp and because the integration ramp is non linear a source at row 500 will always have fewer counts than the same source at row 100 If the peak flux of source and sky is less than 10 000 ADUs then you do not need to worry about these effects unless you wish to obtain photometry which is better than 1 If it is greater then 10 000 ADUs and DITs are small then these effects become important 3 1 EXPTIME NDIT and DIT Up until March 2000 the EXPTIME keyword was defined as NDIT x DIT This caused some confusion as the frames are the average and not the sum of NDIT exposures After March 2000 the EXPTIME keyword is set to the DIT 2For data taken after 24 07 2001 replace 1 77 with 3 55 seconds and 3 5
20. S CEE E a ESS EE ERE EH 35 6 7 Removing telluric lites ec ss osie ee eae a EE RE KER RS 35 Ors Je CxO e s sret ea ee ee E eee He bee ee we Ee 35 8 9 Star trace recip s so kh ee OE Ae Da eR RE OES A BEE ES 36 8 10 Observation recipe chopping e 02 0 es ee eee eee 36 8 11 Response function recipe s seori DRE ER ew ERE Bo 36 INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 1 1 Introduction 1 1 Purpose The document is intended for astronomers who want to reduce ISAAC data It describes the various data formats delivered by ISAAC observational scenarios and reduction procedures This document concentrates on the methodology rather than individual routines that are available in either IRAF or MIDAS However from time to time we will mention some routines that the authors have found useful The document also describes the algorithms implemented in the ISAAC pipeline data reduction package Though the ISAAC pipeline is described in details in the ISAAC pipeline user manual see http www eso org projects dfs dfs shared web vit vit instrument pipelines html here we will list and present some of the ISAAC pipeline routines By presenting the shortcomings these routines have we allow the users to decide whether they need to reduce their data more carefully The current document is aimed at describing the algorithmic concepts present in the recipes and provide help for users who want to analyze data from ISAAC
21. The pipeline does produce reduced data however this is not meant to replace more general reduction packages such as IRAF MIDAS or IDL The pipeline does not replace interactive analysis and cannot make educated choices Thus the data that the ISAAC pipeline produces should be considered as a way of quickly assessing the quality of the data a quick look if you like or a first pass at the reduction of the data This document does not describe the ISAAC instrument its modes of operations how to acquire data the offered templates or the various issues attached to Phase IT Proposal Prepa ration The reader is assumed to have read the ISAAC User s Manual beforehand and have a basic knowledge of infrared data reduction in imaging and spectroscopy 1 2 Reference documents 1 ESO DICB Data Interface Control Document GEN SPE ESO 00000 0794 2 ISAAC User Manual 3 ISAAC Pipeline User Manual 1 3 Abbreviations and acronyms The following abbreviations and acronyms are used in this document INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 DICB DIT DPR SciOp ESO Dec eclipse Data Interface Control Board Detector Integration Time Data Products FITS identifier Science Operations European Southern Observatory Declination ESO C Library Image Processing Software Environment ESO MIDAS ESO s Munich Image Data Analysis System FITS IRAF PAF RA UT VLT Flexible Image Transport System Image Reduction and
22. acted images to a common reference it is necessary to precisely estimate the offsets between them jitter applies a 2d cross correlation routine to determine the offsets to an accuracy of 1 10th of a pixel There are other ways to find out offsets between frames with many point sources point pattern matching is a possibility Identifying the same objects in all consecutive frames would also yield a list of offsets between frames An initial estimate of the offsets between frames can be found in the FITS headers jitter assumes that the offsets found in the input FITS headers have a certain accuracy If there are no input offsets they are all initially estimated to be zero in both directions Registering the images is done by re sampling them with subpixel shifts to align them all to a common reference usually the first frame Re sampling can make use of any interpolation algorithm but be aware that using cheap and dirty algorithms like nearest neighbor or linear interpolation can degrade the images by introducing aliasing jitter offers many higher order interpolation kernels that introduce few or no artifacts however the noise high frequencies will be smoothed a little bit Stacking the resulting images is done using a 3d filter to remove outliers and jitter gives you a choice between 3 different filters Linear means that all frames are actually averaged without filtering pass all filter This is not recommended as this is likely to keep cos
23. an set to T rue if the electrical ghost removal procedure has been applied to the frame e GHOSTVER is an ID of the algorithm used for ghost correction 2 9 Summary Table The following table summarizes all keywords introduced in this chapter in alphabetical order FITS keyword ADA POSANG ARCFILE DEC DET DIT DET NDIT DPR CATG DPR TYPE DPR TECH GHOSTREM GHOSTVER INS FILT1 ID INS FILT2 ID INS FILT3 ID INS FILT4 ID INS GRAT NAME Meaning Position angle on sky degrees Archive file name Declination J2000 in degrees Detector Integration Time Number of averaged DITs Data product category Data product type Data product acquisition technique Flag for ghost removal Ghost removal algorithm ID SW Filter ID in first wheel SW Filter ID in second wheel LW Filter ID in first wheel LW Filter ID in second wheel Grating in position INSTRUMENT data reduction cookbook INS INS INS INS INS INS INS INS OBS OBS OBS GRAT ORDER GRAT WLEN LAMP1 ST LAMP2 ST MODE OPTI1 ID OPTI2 NAME PIXSCALE PROG ID NAME TARG NAME ORIGFILE PIPEFILE PRO PRO PRO PRO PRO PRO PRO PRO PRO RA SEQ SEQ SEQ SEQ SEQ SEQ SEQ SEQ SEQ TEL TEL TEL TEL TPL TPL TPL CATG DATANCOM DATE DID RECH ID RECi DRS ID REDLEVEL STATUS TYPE CUMOFFSETX CUMOFFSETY CUMOFFSETA CUMOFFSETD POISSON RELOFFSETX RELOFFSETY RELOFFSETA RELOFFSETD ATRM START ATRM E
24. asic steps are similar to those used for data taken in the SW arm e Dark subtraction e Ghost removal e Non linearity correction e Flat Fielding e Sky subtraction e Registration and stacking For frames taken with chopping the basic steps are e Ghost removal e Non linearity correction INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 31 e Flat Fielding e A B Subtraction e Registration and stacking Steps that are marked with an asterisk have not been thoroughly tested 7 1 Dark subtraction The dark frame and the frame from which it is subtracted should have the same DIT and should have been taken with the same readout mode In comparison to the SW arm the zero level offset of the array is almost independent of DIT and flux The dark frame should not be subtracted from chopped frames 7 2 Flat fielding At long wavelengths the sky is relatively insensitive to changes in the twilight sky so one cannot use the method used for SW data Instead one observes a region of the sky with the telescope pointing to the Zenith a region to the South with an airmass of 2 and a region that is even further South an airmass of 2 4 The flat can then be created by subtracting the images that were taken at either one of the higher airmasses from the image taken at Zenith The resulting image is then normalised to 1 Five images are taken at each position Since the position of these images is slightly different sta
25. at the first frame has the star at the center of the image center means up to 50 pixels from the true image center and subsequent positions are indicated through keywords in the FITS header Usually the procedure is run once every six months and after major instrument interventions These illumination frames need to be sky subtracted and flat fielded The easiest way to remove the sky and the dark is to subtract one frame from another For flat fielding one can either use the flat that are closest in time to the science frames or those that are closest in time to the illumination frames In any case it is important to use the same set of flat filed for the science and the illumination frame processing The next step is to compute the flux not magnitude of the stars through some fixed aperture and to fit a two dimensional 2nd or 3rd order polynomial to the flux values as a function of position on the array The IRAF routine surfit is particularly useful for doing this The surface which is fit is the illumination correction after it has been normalized to a mean of unity The ISAAC pipeline recipe illum also produce a 2D fit of the measured fluxes The illumination correction can be applied to either the data or the flat field itself however some care is required in determining if one needs to multiply or divide It depends on how the fit was done Note that the jitter recipe does not take into account illumination frames during the redu
26. ata in which nodding or the classical ABBA technique was used one should extract the spectrum without fitting the sky Fitting the sky only adds noise For more complex cases a low order fit to the sky may be required 8 7 Removing telluric lines The technique at long wavelengths is similar to that used at short wavelengths Please refer to the SW documentation 8 8 Flux calibration The technique at long wavelengths is similar to that used at short wavelengths Please refer to the SW documentation The only difference is that it is not possible take slitless spectra in the LW arm INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 36 8 9 Star trace recipe As for SW on a regular basis the observatory takes data that can be used to determine the tilt of spectra as a function of position along the slit and the relationship between the position of an object along the slit and the corresponding location of the spectrum on the array The data consists of n images of a bright star stepped along the slit and spectra of the star in the MR and LR gratings at each of these positions In total n images and 2 x n spectra are obtained The ISAAC pipeline recipe that processes this data is the same as for SW Please refer to Section 5 9 for more details 8 10 Observation recipe chopping The pipeline recipe that reduces LW spectroscopic data is called the same as for SW and works in a similar way The recipe first classifies the f
27. be complex As the flat is made from exposures of varying flux and since the zero level offset changes with flux we are not measuring the relative sensitivity of a pixel but more the relative sensitivity plus the change in the zero level offset At this point in time we cannot remove the change in the zero level offset and this ultimately limits the accuracy at which the relative sensitivity of pixels and hence the flat field can be measured Nevertheless we can quantify the effect this has on the flat field by dividing one flat by another The typical pixel to pixel RMS in the flat field is of the order of 6 The typical pixel to pixel RMS of an image which is the division of one flat by another is 0 4 0 6 This translates to an accuracy of 0 3 0 4 in the flat itself Saturation If the array is heavily saturated two affects occur Firstly a remnant whose timescale depends on the level of saturation appears and secondly the QE of the array decreases This has a much longer timescale and appears to be a strong function of wavelength One sees it strongest at J and weakest at Ks The solution to this problem is to not saturate the array but this will occur from time to time so flats need to be taken often enough so that the scientific data is not affected Since the timescale of the effect is long the current practice of taking flats as required is satisfactory 4 2 2 Accuracy and period of validity When describing the accuracy of
28. be straightened It is useful to do the wavelength calibration at the same time so that the horizontal axis is in wavelength units The wavelength scale can be calibrated with either arc frames or the OH lines that are im printed on each spectrum The advantage of the arcs is that there are lines covering the entire 0 9 to 2 5 micron range The disadvantage is that the arcs are taken separately and in most cases this means that the grating has moved between the time the target was observed and the arcs were taken One can use the OH lines to cross check and correct the zero point of the wavelength calibration which will be a necessary step in most cases The advantage of the OH lines is that they are numerous and that they lead to a slightly more accurate wavelength calibration The disadvantages are that in some regions particularly beyond 2 2 microns there may be too few lines to do a good fit in standard star observations where exposure times are short the OH lines may be too faint and in LR observations where the resolution is low the OH lines may be heavily blended For both arcs and OH lines a 3rd order Legendre 4 terms gives a good description of the dispersion 5 3 1 Arc recipe For a given instrument setting arcs consist of two or three exposures one with the arc lamps off and additional exposures with one or both of the arc lamps on The lamps are Xenon and Argon The arcs are used to model the slit curvature and to deriv
29. c tion process so you would have to correct the flat field beforehand to take this into account 4 4 Bad pixels and vignetted regions Before April 2001 the top 20 30 pixels of the ISAAC image were vignetted This vignetting was removed during the March 2001 intervention So for data taken before April 2001 this region can be trimmed without any loss of information There are a lot of bad pixels on the right hand edge of the array Users may want to trim the 10 last columns of the ISAAC image or alternatively flag them as bad The jitter command can replace bad pixels by an average of their valid neighbors It cannot trim whole regions 4 5 Sky subtraction This is the most important step and great care and a good understanding of the technique are necessary if good results are required This is particularly important for deep imaging as an error at the 0 01 level will significantly effect the photometry of the faintest sources There are two cases In relatively blank fields the sky is created from the object frames themselves For crowded fields or large objects the sky is created from frames that were INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 17 specifically taken to measure the sky For deep exposures the sky is computed from a subset of exposures and there will be one sky frame for each object frame For accurate photometry it is very important that the object frame is not included in the frames that are us
30. curvature e which table to use for wavelength calibration e which table to use to correct for spectral tilt the one produced by the startrace recipe and e which flat field to use The recipe starts by classifying the input images according to the cumulative offsets in the headers The classic way of taking IR spectroscopic data is to observe the target along two positions along the slit which we will call A and B An example may be the sequence AAABBBBBBAAA After flat fielding all the data the recipe will take the first three A frames and average them take the first three B frames and average them etc The recipe then subtracts one average from the other corrects for slit curvature spectral tilt and wave length calibrates If the tables for the wavelength calibration and the correction for slit curvature are missing the recipe will use the OH night sky lines If there are too few of these the recipe will use a model to do the wavelength calibration and will skip the correction for slit curvature If the flat field is missing the recipe skips the flat fielding step INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 25 The subtracted frames will contain positive and negative spectra The two spectra are com bined by multiplying the image by 1 and adding it to the original after a suitable shift The resulting frames are then added together to give the final result At the end a spectrum can be extracted Either the u
31. d FITS capable package The dtfits command has been written for precisely that purpose You will find it useful for spectroscopic data reduction if you need to check out the results of the pipeline recipes that produce FITS tables dfits and fitsort will help you classify tables and see ancillary data attached to them but dtfits will display all information contained in the table itself in ASCII format on the command line There are various options to help make the output readable on a terminal or by a spreadsheet program See the dtfits manual page to get more information 2 3 2 4 File Names PIPEFILE if set contains the name of the product using the official naming scheme for ISAAC products This name can be set using the renaming recipe is_rename ORIGFILE contains the name of the file on the instrument workstation ARCFILE is the archive file name Telescope Keywords Here is a non exhaustive list of telescope keywords 2 5 RA Right ascension J2000 in degrees Notice that the comment field indicates the value in hh mm ss s format DEC Declination J2000 in degrees Notice that the comment field indicates the value in hh mm ss s format ADA POSANG Position angle on sky as measured from North to East degrees TEL AIRM START Airmass at start TEL AIRM END Airmass at end TEL AMBI FWHM START Astronomical Site Monitor seeing at start Note that this value might differ significantly from the ISAAC image quality
32. delta Cumulative offsets are always relative to the first frame in the batch TPL EXPNO 1 Relative offsets are always relative to the previous frame TPL EXPNO 1 in the batch If the same guide star is used before and after an offset the offsetting accuracy is about 0 1 arc seconds All recipes looking for offset information take this into account and will use the header offset information as a first guess and will refine the offset through cross correlation techniques In AutoJitter mode the jitter offsets are generated using a Poisson distribution SEQ POISSON is an integer describing the Poisson homogeneity factor used for this distribution See the eclipse web page http www eso org eclipse for more information about this factor The jitter recipe from ISAAC pipeline always expects offsets to be given in pixels not in arcseconds If your headers do not mention the offsets in pixels you must translate arcseconds to pixels yourself and feed the information back into the jitter command The input offsets are then given by an ASCII file instead of being read from the FITS headers 2 8 Pipeline Products To allow identification of pipeline products some keywords are inserted in the output FITS headers e PIPEFILE is a standard 8 char FITS keyword which contains the name of the file as set by the pipeline when creating the product it is useful as a label to identify the file If this keyword is not set in your products you can use the
33. e glow is outside the useful part of the image in spectroscopy and the noise induced by the glow is negligible in the useful part 3 2 3 Electrical ghosts The HAWAII detector generates electrical ghosts which seem to have a reproducible behavior The effect basically consists of an additional signal which on one row is proportional to the sum of the intensity along this row and the row 512 rows away This effect is mostly an issue in spectroscopy of relatively bright sources e g standard stars as it might affect the level of the continuum and e g the line equivalent widths The procedure to remove the electrical ghost is to use a dedicated ISAAC pipeline recipe called ghost The ghost recipe writes into any processed file header two keywords GHOSTREM is a flag indicating that the process has been applied to the file and GHOSTVER indicating what the algorithm version ID is This way the algorithm cannot be applied twice to the same file unless you use the force option The implemented algorithm is the following e Create a 1d signal in which each element contains the sum of all pixels along a line of the input image In other words collapse the 2d image along the x axis e The signal in 1 512 is added to the signal in 513 1024 and visa versa e Multiply by 1 35e 5 e Subtract element N of this 1d signal from line N in the input image INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 11 3 2 4 Det
34. e recipe will produce the spectrum corrected for slit curvature 8 4 Combining 2d spectra non chopping For the classical ABBA technique one multiplies each image by 1 and adds it back to itself after a suitable shift This method of combining data is often called double sky subtraction as it effectively moves any residual sky that remains after the first subtraction It results in an image that has one positive spectrum and two negative spectra either side of the positive spectrum In the more complex cases one combines the individual spectra after suitable shifts have been applied 8 5 Combining 2d spectra chopping For templates that use chopping one registers the positive spectra Since there are many hot pixels it is very helpful if there are several spectra to register and if these spectra are offset from one another The final step is optional and depends on how crowded the field is One can use the two negative spectra that are either side of the positive spectrum to increase the S N ratio of the observations The end result will be a central positive spectrum with an exposure scaling of 4 DIT seconds two negative spectra with an exposure scaling of 3 DIT seconds and two additional positive spectra with an exposure scaling of DIT seconds This step should not be applied to images for which the non linearity correction was done on the chopped frames unless the negative beams have been treated correctly 8 6 Extraction For d
35. e the wavelength calibration The ISAAC pipeline recipe arc starts by classifying images based on instrument setting a setting is defined by the resolution the central wavelength and the slit The slit curvature is modeled with a bivariate 2 d polynomial If we let the distorted image be expressed in u v coordinates and the corrected image in x y the curvature is modeled with u atbet cy dr ey fry v y The dispersion relation 1st order is computed by matching a xenon and or argon atlas with the corrected spectra which can be produced with the c option This is a weakness of the current version of the routine as a 3rd order fit provides a better description of the dispersion Users familiar with IRAF and or MIDAS would notice that the correction for slit curvature and the wavelength calibration are done in two steps with this recipe whereas using IRAF or MIDAS users can do it in one step INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 23 5 4 Combining 2d spectra For the classical ABBA technique one multiplies each image by 1 and adds it back to itself after a suitable shift This method of combining data is often called double sky subtraction as it effectively removes any residual sky that remains after the first subtraction It results in an image that has one positive spectrum and two negative spectra either side of the positive spectrum In the more complex cases one combines the individual spectra after sui
36. e ubiquitous telluric absorption lines This has been tested on 1 d spectra only and has proved to be as accurate as using the arcs For faint targets or for the purpose of correcting the 2d slit curvature one can use the telluric emission features It is perfectly reasonable to use the arc lines to correct for the slit curvature and the telluric features to do the wavelength calibration of the extracted spectra For both arcs and telluric lines a 3rd order Legendre 4 terms gives a good description of dispersion 8 3 1 Arc recipe The recipe is the same as for SW The slit curvature and the wavelength calibration are computed in the same way The main difference lies in the way the frames are classified e The frames are classified by instrument setting e For each setting there should be an even number of frames In some cases there will be one pair of frames an arc with the grating at a single order and in other cases there will be two pairs of frames arcs with the grating in the first and then third order e In each pair there should be a dark a frame with the lamp off and a frame with the arc lamp on The lamp frame is dark corrected and then reduced and a table containing the results is written to disk INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 35 The procedure produces two output files a FITS table which contains the fit to the slit curvature and the linear dispersion relation With the c option th
37. ector non linearity Firstly you should try to set the exposure times so that the exposure level of the source and the sky is below 10 000 ADU If this is the case then you do not need to worry about non linearity If this is not the case then you should think about how non linearity may effect your data and if you really think that you should apply a non linearity correction then you are on your own at this point because we have not developed any software to do this The reason for this are twofold Firstly different parts of the array are read out at different times so the non linearity correction at row 512 is always going to be larger than the correction at row 1 Secondly since the bias is a function of the flux it is difficult to know the precise value of the signal to correct This second obstacle is not as severe as you may at first think as the error you would introduce by ignoring the zero level offset will in most case be much smaller than the correction itself If this does not deter you and you develop a way of correcting the non linearity then we d be happy to hear from you The non linearity of the array can be fitted with the following function Fue meas 4 75e 11 Fe dss Since we have not developed software to correct for non linearity this formula has not been tested 3 2 5 Odd even column effect This effect can be seen as an offset between the odd and even columns of the array It is a function of the flu
38. ed to compute the sky This is a weakness of the current ISAAC pipeline recipe jitter For H and K band observations the sky frame should be computed from frames that were taken 5 10 arc minutes either side of the object frame For J band observations these numbers can be doubled For conditions where the sky background is varying rapidly clouds or observations taken just after evening twilight a more rapid sampling of the sky is necessary All sky frames contain objects so one has to combine them with suitable clipping A robust method is to first scale frames to a common median and then remove objects by rejecting the highest and lowest pixels Rejecting the two highest and two lowest pixels would produce even more robust results The remaining pixels are then averaged the median can also be used but it is a noisier statistic The resulting sky frame is then scaled to match the median of the object frame and subtracted A more sophisticated approach is to do the sky subtraction in two steps The first step reduces the data as described above produces the combined image and then locates all the objects In the second step the data is re reduced with the knowledge of where the objects are These objects are then excluded when the sky is estimated in the second pass This is the approach used by the XDIMSUM package in IRAF and for very deep imaging it is the recommended package 4 6 Image registration and stacking To register the sky subtr
39. er to the description given for SW data as the steps are identical INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 32 For templates that use chopping the typical observing sequence is ABBAABBA etc One first groups the frames into AB pairs and subtracts one image from another This will result in an frame with one positive image and two negative images The positive image will have a flux which corresponds to 2 DIT seconds 7 4 Registration and stacking For templates that do not use chopping please refer to the description given for SW data as the technique is the same For templates that use chopping one registers the positive images Since there are many hot pixels it is very helpful if there are several images to register and if these images are offset from one another The final step is optional and depends on how crowded the field is One can use the two negative images that are either side of the positive image to increase the S N ratio of the central image The end result will be a central positive image with an exposure scaling of 4 DIT seconds two negative images with an exposure scaling of 3 DIT seconds and two additional positive images with an exposure scaling of DIT seconds This step should not be applied to images for which the non linearity correction was done on the chopped frames unless the negative beams have been treated correctly 7 5 Ghost with the NB 4 07 filter This filter has a ghost around x
40. es the object at two slit positions i e ABBA etc and there is the more complex case where one has observed the target along several slit positions In the classical case one simply subtracts frames taken at different slit positions So one needs to form A B B A etc This simple step removes the bias and results in an image with two spectra one positive and one negative In the more complex case one could build a sky frame from several spectra as one does when building the sky frame in imaging This results in an image with only one positive spectrum 5 2 Spectroscopic flats and flat fielding The flat is the normalised difference of two images one with the lamp on and another with the lamp off In general three pairs of images are taken The ISAAC pipeline recipe sp_flat starts by classifying images by instrument setting A setting is defined by the slit used the central wavelength and the grating name which are obtained from the FITS header Each pair of on off spectra are then reduced as follows subtract the off frame from the on e normalize the whole frame with the computed mean of the center part of the difference frame e fit a polynomial in the slit direction and divide the frame by the polynomial The aim of this step is remove the lamp shape in the slit direction since the illumination of the slit is not uniform e Average the results if there is more than one on off pair A weakness of the ISAAC pipeline recipe is
41. gt soes sos poea be bbe ba es 243 Twilight Flats md flat elding cocoa 40 an eR oR eee Rad es 220 LOMAS 0 o RR RS SE AS RM A 4 2 2 Accuracy and period of validity o o 43 Illumination Corrections o c s ee Abe eS AOR E ee E AE E 44 Bad pixels and vignetted regions 22 sek EN EN EE E EN E YX O O OU Ou On A V da PW D NOP RF k Fi wo INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 Aa 2 oon ht oe he SERED SE eee ee we Cee ES e 4 0 Image registration and stacking A A Ad E a cias Wee we ee Rw a 4 7 Removing the remaining bias residuals lt 025 4548 be a A8 260 PO ke Me tk ek ew ee ee eo eR ee ES AST A plour eS lt gt ds DE OOP RAO EGER A e A 5 SW Spectroscopy Reducing the science data Oil First Sun s A ve ee ERE ORS Ow ROE 5 2 Spectroscopic flats and flat fielding 5 3 Slit curvature correction and wavelength calibration Gok RO RAINS e sewa OE Go Ae AAAS DES REE DY Bo of Combine SA Speca cnt da ge e dng Me de dere Re ede ee eR eS 50 POCO ke ha Paw te E AE we A AA A 5 6 Removing telluric lines lt o lt 44 2 4464 24604 bee darra tad nacau 7 Te calibration s e esor ar 64 8 m oR A E DS OPIITTER re cadie ea a KG EE SE OR Eaa E EH RES 5 9 Star trace frames and recipe EE EEN Ree E ee 510 Response Function 2 e ces sree Vee ee Yee e e eee oS ewe ee 6 An Overview of Long wavelength Data GI Te e oi end Be E A A 6 2 File formats Chopping and Non c
42. he y dependence can be removed by masking additional pixels The routine that removes the effect in imaging data is called oddeven The routine works less well for spectroscopic data The effect is harder to remove from spectroscopic data and this is especially true for the flat fields We have retaken all flat fields for all the spectroscopic set ups that were used between May 12th and July 24th 2001 If you suspect that your spectroscopic flats are contaminated INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 12 by the odd even column effect then you should search the archive for data that was taken with the same instrument setup after July 24th 2001 3 2 6 Remnants from bright sources If the array is exposed to a bright source a remnant becomes visible The amplitude of the remnant and the time it lasts depend on the brightness of the source In broad band imaging these remnants are not a problem as they are much fainter than the background However in spectroscopy they are a nuisance and by Murphy s law they are bound to land on your spectrum We ask astronomers to check the brightness of their sources and in case where there are bright sources we will arrange the schedule so that these sources are observed at the end of the night However there is little we can do if the acquisition image contains a bright source 3 2 7 QE reduction from really bright sources If the source is really bright then the QE of the ar
43. hopping 6 3 General features of the ISAAC LW infrared detector Pat The vere level offset bias caw cea eee EES Ee REESE Ho 6 3 2 Detector non linearity 2 4226 244s dee ede ee da ede 6 3 3 Correcting frames that were taken without chopping Gad lk BOGS ec s e ee ee hee OX eae Se RA Do MOINS 2 be ee BE ert OER EE ERR SERRE eee eRe Be Goo Migs quay ca dei heey hee ae AE E A eR eS har Dead Pee s roroa e eRe ORD RE SER ES A KRESS 7 LW imaging Reducing science data cl Dark SAO lt a Ek A E OS Ee HR Ta MIOS Au sasara rs rr Tok DKY SONIC o os ec eee ee bee ee be DESY Oe ed ee eee ee TA Registration and stacking cross ee SR eA EES ee mo Ghost with the NES AH filter roe es bk ee Pad eee OER REE Ed wo TO Photometric ASUS sosa a Es ACA AA A 8 LW spectroscopy Reducing science data 8 1 First Sky subtraction non chopping only 2 4 8 2 A B Subtraction chopping only NEEN ENEE EN E ES vi 16 17 18 18 20 20 21 21 22 22 23 23 23 24 24 25 25 27 27 28 28 28 28 29 30 30 30 30 30 31 31 31 32 32 32 INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 vii 8 3 Slit curvature correction and wavelength calibration 34 Cal CO A nise E AEN E Re ed ee E 34 BA Combine 2d Speolra on Chopping sors hee see ee he ee Ree eA 35 8 5 Combining 2d spectra chopping 26s ee ee ee E 35 S0 Pieteaop mg oo i vi kee eee EEE PEER
44. hot pixels The readout modes with a low bias setting are used without chopping They have much fewer hot pixels but they have a smaller well depth In addition to these modes there are three modes that are used for a small number of calibra tion frames The modes their names the minimum integration time the recommended flux range and the saturation both in ADU are listed in the following table Detector Mode Name Current MINDIT Recommended Range Saturation UnCorrHighBias UCR 0 1073 0 28000 37000 DoubleCorrHighBias DCR HB 0 2771 0 28000 37000 DoubleCorrLowBias DCR LB 0 3451 0 16000 21000 UnCorrHighBiasCal 0 0733 0 28000 37000 DoubleCorrHighBiasCal 0 1419 0 28000 37000 DoubleCorrLowBiasCal 0 1419 0 16000 21000 Table 1 Current LW detector settings 6 1 Chopping The technique of chopping involves rapid sampling of the sky by moving the secondary in phase with the read out of the detector In what follows the ON beam refers to the positive INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 28 image the OFF beam refers to the negative image In addition to chopping the telescope nods so the ON beam at position A overlaps with the OFF beam of position B The typical nodding sequence is ABBAABBA etc See the ISAAC user manual for more details 6 2 File formats Chopping and Non chopping All LW modes can be used with the chopping secondary Only a subset of these modes MR spectroscopy and imaging with
45. if the spectral resolution of the instrument is a function of the wavelength which is the case for ISAAC as it means that the kernel for convolution also has to be a function of wavelength The arc spectra and the OH lines can be used to estimate what this function is Hot stars usually contain helium and hydrogen lines If the spectral regions around these lines are of interest then one should think carefully about using these type of stars If the resolution is high enough which is certainly the case for MR observations one can try to remove these lines by fitting them Alternatively one can use a second telluric standard that does not have helium or hydrogen lines so that these lines can be removed from the hot star The ISAAC FITS header does not always contain the full target name of the telluric standard and sometimes operators forget to include it in the night report To find out which telluric standard was used look at the RA and DEC of the target and consult the following link http www eso org paranal sciops catsearch html INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 24 5 7 Flux calibration The first step is to obtain a relative flux calibration The second step is to do absolute flux calibration If the telluric standard was a hot star then a blackbody curve can be used to model the continuum of the standard The spectral type of the star can be used to give an idea of what temperature to use The blackbody
46. in mind the following points e The array is simultaneously read out from 32 ports There are 8 ports per quadrant and the readout is done along detector rows e There is no shutter so the array is continuously exposed to light e The array is intrinsically more non linear that the Hawaii array and it is common to work in the regime where corrections of around 0 10 are required The ISAAC IRACE controller and the associated DCS software control the readout of the array They are extremely flexible and allow many different readout schemes to be used We offer three readout modes UCR Uncorrelated Read High Bias DCR HB Double Correlated Read High Bias DCR LB Double Correlated Read Low Bias UCR first resets the array and then performs a single read The minimum integration time of this mode is 0 1073 seconds Before March 2001 the minimum integration time of this mode was 0 0733 seconds DCR performs two reads and the difference between these two reads is the image DCR HB and DCR LB differ in the way array voltages are set DCR LB has a slower readout and a smaller well depth but far fewer hot pixels The minimum integration time for DCR HB is 0 2771 seconds and the minimum integration time for DCR LB is 0 3451 seconds Before March 2001 the minimum integration time for DCR HB was 0 1419 seconds and DCR LB was not offered The readout modes with a high bias setting are used with chopping They have a very large number of
47. ity The detector non linearity as measured over a representative region of the array can be fitted with the function fr futax fx b fM 1 where fm is the the measured flux and fr is the true flux For the readout modes which use the high bias voltage equation 1 gives a relatively poor description not better than 0 5 of the non linearity at low flux levels 4000 ADU The co efficients for the various readout modes are given in the table These co efficients have been measured over a region which is thought to be representative of the entire array In detail this will not be true as each pixel will behave differently To correct any given frame for non linearity one requires one or two additional frames For observations that do not use chopping just a dark frame is required For observations that use chopping both the dark frame and the half cycle frames are required These frames allow you to determine the measured flux in each pixel which can the be corrected for non linearity INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 29 Detector Mode Name a b Validity UnCorrLowBias UCR 2 04e 6 0 75e 10 Before March 2001 UnCorrLowBias UCR 6 28e 7 1 04e 10 After March 2001 DoubleCorrHighBias DCR HB 1 26e 6 1 00e 10 DoubleCorrHighBias DCR HB 1 05e 6 0 84e 10 DoubleCorrLowBias DCR LB 2 78e 6 1 65e 10 Before March 2001 After March 2001 Table 2 LW linearity The procedure depends on the data set If
48. ke dome flats ISAAC imaging data from the SW imaging arm are flat fielded with twilight flats The flats are derived by imaging a region of the sky relatively free of stars Between 10 to 25 exposures with constant DIT and NDIT are taken for each filter According to the twflat recipe for each pixel on the detector a curve is plotted of the median plane value against the individual pixel value in this plane This curve shows the pixel response from which a robust linear regression provides a pixel gain The image showing all pixel gains is normalized to have an average value of 1 i e the flat field One then divides all images with such a flat field This will result in photometry that is consistent to the 2 level over the ISAAC field of view If more accurate photometry is required then an illumination correction should be applied If one was to look at the flat fielded images one would see that they are far from flat One would see a jump between the two halves of the array and one would perhaps see structure at intermediate 5 10 pixels and large several hundred pixels scales These structures have a variety of causes The jump in the middle is caused by the fact that we have not removed the zero level offset perfectly The structures at intermediate scales are probably caused by pupil ghosts and by dust that has moved The structures at large scales are probably caused by scattered light Most of these features are additive so they are
49. me as prepared with P2PP Template Keywords 2 7 1 Keywords common to all templates TPL ID contains an unique identifier describing the template which was used to produce the data Frame selection in the pipeline is mostly based on this keyword value DPR CATG Data Product category SCIENCE CALIB DPR TYPE Data Product type OBJECT SKY DPR TECH Data Product acquisition technique e g IMAGE SPECTRUM TPL NEXP Number of scheduled exposures within the template Templates might have been aborted so it is possible to have fewer frames than TPL NEXP in a batch INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 6 e TPL EXPNO Exposure number within template A template may produce several different frame types Frames are discriminated by the value of the DPR keywords DPR CATG DPR TYPE and DPR TECH take different values depending on the observed frame type 2 7 2 Keywords common to the jitter templates The offsets sent to the telescope for jitter observations both in imaging and spectroscopy are stored into 8 keywords This applies to AutoJitter AutoJitterOffset and GenericOffset templates e SEQ CUMOFFSETX and SEQ CUMOFFSETY for cumulative offsets in pixels e SEQ CUMOFFSETA and SEQ CUMOFFSETD for cumulative offsets in arcseconds alpha delta e SEQ RELOFFSETX and SEQ RELOFFSETY for relative offsets in pixels e SEQ RELOFFSETA and SEQ RELOFFSETD for relative offsets in arcseconds alpha
50. mic rays and other outliers in the final frame Median means that the final frame is the median of all re sampled frames The last filter default scales all frames by their medians and removes the highest and lowest pixel values before taking an average See the jitter documentation for more information Notice that in versions later than eclipse version 4 jitter re samples and stacks in one step to speed up the process Also added since version 4 the final frame is a union of all input INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 18 images as opposed to an intersection for previous versions which means that it is bigger than any of the initial input frames 4 7 Removing the remaining bias residuals In most cases the final combined image will contain small but noticeable jumps in the vertical direction This is caused by the imprecise removal of the bias when the dark was subtracted The jumps can be removed very effectively by averaging the image along rows and subtracting the resulting one dimensional image from each column of the original image To make sure that objects do not bias the result one clips the 200 highest and lowest pixels from the computation of the one dimensional image 4 8 Zero Points Standard stars are observed every night in the J Js H and Ks filters For the NB filters standards are observed as required Standard stars are imaged over a grid of five positions one just above the center of
51. ndard star system of Persson et al AJ 116 2475 are quite good so we expect the colour terms to be small If the zero point is expressed in the form ZPx tisaac XLco a J K ico where tisaac is the ISAAC instrumental magnitude Xico is the catalog magnitude and for simplicity we have ignored extinction we expect that a will differ from 0 by less than 0 01 for X being one of J H or Ks The Js filter of ISAAC has a slightly redder effective wavelength than J so we expect a non negligible colour term Theoretically we expect a 0 04 However the colour transformation between ISAAC magnitudes and those of LCO have never been experimentally verified so the results quoted here should be considered with this in mind 5 SW Spectroscopy Reducing the science data The most basic way of taking IR spectra is to observe the target along two slit positions The sky is then removed by a process which is sometimes called double sky subtraction The basic steps of how to reduce these type of data are 1 Ghost Removal 2 First subtraction 3 Removal of the 50 Hz pickup 4 Flat fielding 5 Slit Curvature Correction and wavelength calibration INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 21 6 Combining spectra 7 Extraction 8 Telluric line correction 9 Flux calibration 5 1 First subtraction There are two techniques observers use in taking IR spectra There is the classic sequence where one observ
52. not been implemented INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 14 4 1 1 Read out noise computation This particular recipe is used for quality control and instrument monitoring The input list of frames is browsed in the order they are received For every pair of consecutive frames that share the same DIT NDIT and readout mode the readout noise is computed Notice that the recipe does not try to make all possible pairs from the input list so it is important to make sure that the files are correctly sorted If the frames are kept in the order they were acquired this should not be a problem For each pair the second frame is subtracted from the first The resulting image is then cut out into 4 quadrants and on each quadrant the following measurement is applied e Generate 100 9x9 windows on the input pixel surface These windows are optimally scattered using a Poisson distribution to make sure they sample the whole area with as little overlap as possible e Compute the pixel standard deviation in each window e The readout noise is the median of all these measured standard deviations e The error on the readout noise is the standard deviation of all measured standard devi ations Notice that the algorithm is identical in LW mode but the whole detector is considered instead of cutting it into four quadrants 4 2 Twilight Flats and flat fielding UT1 does not have a flat field screen so it is not possible to ta
53. or you should keep in mind the following points e The array is read out simultaneously in four quadrants e Within each quadrant the read starts in the bottom left hand corner reads along a single row before moving to the next row and then ends up in the top right hand corner e There is no shutter so the array is continuously exposed to light e The minimum DIT might vary between 1 77 sec and 3 55 sec depending on the odd even column effect see 3 2 5 For example for data taken before 24 07 2001 the detector read out time is 1 7728 seconds this also corresponds to the minimum integration time For data taken after 24 07 2001 the detector read out time is 3 5451 seconds This also corresponds to the minimum integration time The ISAAC IRACE controller and the associated DCS software control the readout of the array They are extremely flexible and allow many different readout schemes to be used Currently only two readout schemes are supported DCR Double Correlated Read and NDR Non Destructive Read DCR first resets the array and then performs two reads one at the beginning of the integration and one at the end The difference between these two reads is the image NDR first resets the array and then reads the array N 1 times where N depends on the DIT Longer DITs will result in more reads A line is fit to N reads the first read is discarded and the slope of the fit gives the image For a given DIT DCR is faster but ND
54. rames into the three frame types int hcyclel and hcycle2 It then classifies the int frames according to offsets that are recorded in the header i e frames of type A and type B and subtracts frames of type B from type A This removes the residual background very well Images are then flat fielded If the flat field is missing the recipe skips the flat fielding step The list of combined images are corrected for slit curvature wavelength calibrated shifted and recombined together to form the final combined 2d spectrum If possible the wavelength calibration is done with the sky lines in the half cycle frames Otherwise it uses a physical model Then at the end the spectrum is detected and extracted 8 11 Response function recipe The respfunc recipes can be used also on LW data to calculate the instrument throughput However regular standard star observations in the LW band are not part of the ISAAC calibration plan INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 37 oho
55. ray decreases We do not fully understand this effect but it would appear that the effect is strongest in the blue and that the only way of removing it is to warm up the detector 3 2 8 50Hz pickup The Hawaii array suffers from 50Hz pickup which appears as lines that are almost aligned with detector rows The strength of the pickup depends on how the array is readout and the cabling of the electronic It is a function of time When present it is strongest in DCR and for some DIT s in NDR Therefore for the NDR we have come up with a recommended list of DITs see the ISAAC User Manual As an example before April 2001 DITs of 60 and 180 seconds were strongly affected by the pickup After this date we tuned the number of reads so that the pickup with these DITs is weak Eventually after having replaced a fan in the IRACE cabinet in 2006 the 50 Hz pickup noise as almost completely disappeared In the cases where the background level is low it may be possible to see the 50Hz pickup noise In most cases the pickup will average out when the images are registered and stacked see 4 6 However should this be not the case and should you need to remove the pickup noise you might consider the following procedure It turns out that the periodicity of the 50Hz signal is very close to 6 5 pixels in the vertical direction To see how this is useful take an image where the 50Hz signal is strong shift it by 13 pixels in the vertical direction and subtrac
56. removed at the sky subtraction stage By products of this routine are a map of the zero intercepts and an error map of the fit Both can be used to visually check the validity of the fit A bad pixel map can also be produced by declaring all pixels whose value after normalization is below 0 5 or above 2 0 as bad INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 15 4 2 1 Limitations The method of using twilight flats to flat field the data suffers from several limitations Non linear change in the intensity of the twilight sky with time Since the sky is either decreasing in intensity evening twilight flats or increasing in intensity morning twilight flats since these changes are non linear in time and since the array takes some time to read out one can get a discontinuity in the flat field between the two halves of the array which has an amplitude of 1 with the current DIT and NDIT settings The effect can be removed rather well by averaging the morning and evening twilight flats as the effect inverts itself between the two times Odd even column effect The odd even column effect should be removed from the individual flat fields However this only needs to be done for flat fields that were taken when the effect was strong See Sec 3 2 5 The variable zero level offset The zero level offset of the Hawaii array is a function of the flux so when the flux of the array changes so does the offset The change appears to
57. rs can be removed before the frames are averaged In principle one should apply the non linearity correction to the flat field frames however in practice correcting for the non linearity in flat field frames will make very little difference to the final result as the pixel to pixel sensitivity variation in the Aladdin array is quite low about 1 As a comparison the pixel to pixel sensitivity variation in the Hawaii array is much higher about 7 If a non linearity correction is applied a dark frame with the appropriate read out mode and DIT should be subtracted from all frames before the non linearity correction is made If one divides one flat with another the pixel to pixel RMS is around 0 2 so the S N of the flats is high However there are other systematic effects which cause much larger differences For flats that are taken close in time these differences are caused by pupil ghosts at the level of 2 imperfections in the objective at the level of 1 5 and hot pixels at the level 10 20 For flats that are taken during different nights the differences caused by imperfections in the objective are larger and one starts to see differences which can only be described as smudges with a size of around 30 pixels Since the RMS of the flat is much lower than the typical deviation caused by hot pixels one should replace these hot pixels with the local average 7 3 Sky subtraction For templates that do not use chopping please ref
58. ser specifies the position of the spec trum they want to extract in the initialisation file or the spectrum of the brightest object is extracted 5 9 Star trace frames and recipe On a regular basis the observatory takes data that can be used to determine the tilt of spectra as a function of position along the slit and the relationship between the position of an object along the slit and the corresponding location of the spectrum on the array The data consists of n images of a bright star stepped along the slit and spectra of the star in the MR and LR gratings at each of these positions In total n images and 2 x n spectra are obtained The recipe that processes this data is called startrace The input frames can be corrected for slit curvature at the very beginning via a command line option In this case the user has to provide the ARC table containing the model of the slit curvature The recipe produces several files 1 Positions table The location of the stellar images and spectra are found and stored in a table named outname _positions tfits 2 Positions correspondence table The relationship between the position of the star on the slit and the location of the corresponding spectra are fitted with a polynomial There is one polynomial for LR and one for MR and the co efficients of both are stored in the table called outnamej_corresp tfits 3 Extracted spectra The spectra are traced extracted wavelength calibrated and stored
59. t it from itself The 50 Hz disappears This simple procedure does not account for objects so a more sophisticated procedure is required e Divide the array into four quadrants and treat each quadrant separately e For each row which we label n average rows n 52 n 39 n 26 n 13 n 13 n 26 n 39 n 52 with suitable clipping to remove objects and subtract this average from row n One may chose to average more rows that the row listed here This method is probably only applicable to images which already have been sky subtracted and special attention has to be paid when the row is near the quadrant borders For data taken after 24 7 2001 when the detector readout speed was changed the periodicity of the 50 Hz pickup is about 3 25 pixels The procedure to remove it remains unchanged INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 13 3 2 9 Fringing It is possible to see fringing on the detector at about the one percent level in spectra taken with the medium resolution grating The fringes can be partially removed by taking flats immediately after the observations and or by observing a telluric standard in the same place on the array as the science target If neither of these strategies were employed then fringing is difficult to remove 4 SW Imaging Reducing the science data The basic steps in reducing SW imaging data are 1 Removal of the odd even column effect Ghost removal Dark subtraction Flat Fielding
60. table shifts have been applied 5 5 Extraction For the classical ABBA technique one should extract the spectrum without fitting the sky Fitting the sky only adds noise For more complex cases a low order fit to the sky may be required 5 6 Removing telluric lines This is a critical step that requires some care and possibly some experimentation The aim is to remove the large number of telluric lines that appear in IR spectra This is done by dividing the object spectrum with that of a telluric standard Since this is a division of one spectrum by another it is important that the strength shape and centroid of the telluric lines match First and foremost the telluric standard and the object have to observed with the same instrument setup with roughly the same airmass and if possible consecutively Secondly the object and science data should be reduced in the same way and with the same calibration frames For the best results one may have to modify the spectrum of the telluric standard so that the center and strength of the telluric lines match those of the object spectrum The next step is to remove spectral features that have been imprinted onto the object spectrum from the telluric standard itself Telluric standards are either hot stars or solar type dwarfs Both types contain spectral features that should be removed For solar type stars one can use the observed solar spectrum to remove the features This can be tricky
61. the NB_3 21 and NB_3 28 filters can be used without chopping The format of data delivered by these modes is different For the data that is taken without chopping the files correspond to the average of NDIT exposures This is similar to the format delivered by the SW detector For the data that is taken with chopping one gets two sorts of frames chopped frames and half cycle frames Chopped frames correspond to the difference between frames taken with the secondary at the two positions i e ON and OFF beams In these frames the sky has already been subtracted and one has both positive and negative images Half cycle frames correspond to images taken at either of the these positions In these frames the sky has not been subtracted and images are always positive During the first year of operations half cycle frames were produced for each chopped frame Currently the half cycle frames are delivered for the first chopped frame only The chopped frame is not quite the difference between the two half cycle frames due to rounding errors and the way the half cycle and chopped frames are calculated The difference is at the level of 10e 4 ADU which is insignificant 6 3 General features of the ISAAC LW infrared detector 6 3 1 The zero level offset bias The bias of the LW detector behaves much better than the bias of the SW detector The LW dark is weakly dependent on the DIT and on the amount of flux on the array 6 3 2 Detector non linear
62. the array and one in each quadrant The recipe finds the standard it assumes that the star in the first image is near the center computes the instrumental magnitude and then uses the standard star database to determine the ZP which is uncorrected for extinction The standard star database contains about 1000 stars with magnitudes in the J H K Ks L and M bands although most stars only have magnitudes in a subset of these filters Stars are currently taken from the following catalogs e Arnica e ESO Van der Bliek e LCO Palomar e LCO Palomar NICMOS red stars e MSSSO Photometric e MSSSO Spectroscopic e SAAO Carter e UKIRT extended e UKIRT fundamental The implemented recipe does the following For any couple of consecutive images imagel image2 1 diff imagel image2 2 Locate in diff the star around the expected pixel position provided by the FITS header or by an external offset list 3 Compute the background around the star and the star flux INSTRUMENT data reduction cookbook VLT MAN ESO 14100 4031 19 4 Store the flux result in an output table Apply steps 2 to 4 to the inverted image image2 image1 This yields 2 N 1 measurements for N input frames From this statistical set the highest and lowest values are removed then an average and standard deviation are computed The conversion formula from ADUs to magnitudes is zmag mag 2 5 x logio flux 2 5 x log o DIT where e zmag is the
63. x of the array and it evolves with time The amplitude also depends on the quadrant and can be as large as 6 Before March 30th 2001 the effect was only visible when the flux on the array was above 10 000 ADUs It was possible to avoid it by ensuring that the flux was never above this However after this date and for reasons which are unknown the effect became visible at much lower flux levels On July 24th 2001 the read speed of the array was slowed down The odd even column effect was reduced to less than 1 over the flux range 0 20000 ADU As a direct consequence of this the minimum integration time of the array was set 3 55 seconds On August 24th 2001 the effect re appeared even with the slower readspeed On October 20th 2001 during an instrument intervention the problem disappeared On December 15th 2001 the effect re appeared and since the two interventions in 2005 has disappeared Since the effect depends on the quadrant each quadrant has to be treated separately A very effective way of removing the effect is to take a Fourier Transform of individual quadrants and mask the one pixel in Fourier Space that corresponds to a spatial frequency of one pixel The effect of applying the correction on the photometry is less that a hundredth of a magnitude This works very well for the lower right upper left and upper right quadrants It works less well for the lower right quadrant where there appears to be a y dependence to the effect T
64. you have half cycle frames for every exposure then it is simplest to correct the half cycle frames directly If you have only the half cycle frame of the first exposure then the correction for the non linearity is more complex Correcting the half cycle frames e Subtract the dark from each half cycle frame e Apply the formula listed in Eq 1 e Subtract the half cycle frames from each other to create a chopped frame Correcting the chopped frames Correcting the chopped frame for the array non linearity is more complicated and less accurate than correcting the half cycle frames e Determine the mean level of the half cycle frames in the first exposure Call this m1 e Determine the mean level of the dark frame with the same detector set up Call this m2 e Compute m1 m2 an artificial background and add it to all frames that were taken with the telescope at position A i e frame 1 4 5 8 9 etc e Apply the correction with Eq 1 to these frames e Multiply all frames that were taken with the telescope at position B i e frames 2 3 6 7 10 etc by 1 and add m1 m2 e Apply the correction with Eq 1 to these frames e Subtract these frames from each other to create a chopped frame This procedure corrects the central image only To correct the other images also one needs to process the ON and OFF beams separately 6 3 3 Correcting frames that were taken without chopping e Subtract the dark from each half cycle frame

Very Large Telescope Paranal Science Operations

Contents

Download Pdf Manuals

Related Search

Related Contents