Home

PAPI Documentation

1. 2 9 7 Data grouping Once the raw files are classified they are grouped into observing sequences taking into account the keywords added by the Observation Tool OT and finding out the dither sequences observed This way all files beloging to the same observing sequence will be processed together 2 10 Processing description This section gives a description of each step of the pipeline in a greater detail and list the parameters that can be changed if needed Next figure shows the main steps that are involved in the PANIC pipeline 64 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 PANIC Pipeline D PRNIC LEMON lt gt ii Science mode Quick mode or extended 2 10 1 Outline the non linearity is corrected a master flat field in computed by combining all the appropriate images without offsets if the mosaic is of an Sky Target type only the sky frames are used a bad pixel mask is computed by detecting in the master flat field the pixels with deviant values if provided an external bad pixel mask is also used adding the bad pixel in the previus one for each image a sky frame is computed by combining a certain number of the closest images this sky frame is subtracted by the image and the result is divided by the master flat bright objects are detected by SExtractor in these cleaned images to measure the offsets among the images
2. gainmap HERR HERE HEHE EEE EEE EEE EE EEE EE PEE EE H EE 4 EEE AA object_names in order to make it possible to work in batch mode is it possible to run the PANIC gainmap module in all the master flat images specifying this parameter which ones will be considered That is only those images whose object name matchs one of the names listed in this parameter will be considered when generating the gain map Note that if is contained in the list _all_ object names will be matched This symbol thus provides a way to easily specify all the images which is equivalent to saying do not filter images by their object names object_names MASTER_SKY_FLAT MASTER_DOME_FLAT MASTER_TW_FLAT 1000 length in pixels of the central area used for normalizatior image For example for suffix D and the imput file home images ferM 0720 o fits n in 42 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 mingain 0 1 pixels with sensitivity lt MINGAIN are assumed bad 0 7 maxgain 1 9 pixels with sensitivity gt MAXGAIN are assumed bad 1 3 nsigma 10 badpix if sensitivity gt NSIG sigma from local bkg 5 0 nxblock 16 image size should be multiple of block size 16 nyblock 16 16 normalize yes if yes apply a previous normalization to master flat images area_width 1000 area to use for normalization
3. k 0006 k 0007 k 0008 k 0009 k 0010 k 001 k 001 k 001 k 001 k 001 k 001 k 001 k 001 k 0019 k 0020 k 0021 k 0022 J 0 Ui ats CQ BJ H KY_F LAT FIL k 0023 k 0024 k 0025 k 0026 k 0027 k 0028 KY_F LAT FIL PAPI 2015 05 28 09 18 01 484 DEBUG reductionset 1150 Found 16 groups of files PAPI 2015 05 28 09 18 01 484 DEBUG reductionset 1157 PAPI 2015 05 28 09 18 01 484 DEBUG reductionset 1158 GROUPED SEQUENCES PAPI 2015 05 28 09 18 01 484 DEBUG reductionset 1159 PAPI 2015 05 28 09 18 01 484 DEBUG reductionset 1167 SEQUENC 0 TYPE D PAPI 2015 05 28 09 18 01 485 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 485 DEBUG reductionset 1170 data2 2015 03 10 domefl PAPI 2015 05 28 09 18 01 485 DEBUG reductionset 1170 data2 2015 03 10 domefl PAPI 2015 05 28 09 18 01 485 DEBUG reductionset 1167 SEQUENCE 1 TYPE S PAPI 2015 05 28 09 18 01 485 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 485 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 485 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 485 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 486 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 486 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05
4. 79 PAPI Documentation Release 1 2 20151020173439 80 Python Module Index A acknowledgments 71 B building 5 C calibration 51 command line 36 config 36 configuration 29 D dark 51 data 30 downloading 5 DRP 72 F flat field 52 53 focus 57 fwhm 57 IAA 72 installing 5 irdr 58 L license 71 M modules 47 O observing mode 72 off line 29 on line 29 options 46 OT 72 P PANIC 72 PAPI 72 papi 49 Pipeline 72 INDEX pipeline 5 prerequisites 5 quick look 7 quicklook 29 quickstart 29 H Recipe 72 reduce module 46 requirements 5 run 36 running 7 29 S seeing 57 sextractor 57 sky 58 sky background 58 source 5 super flat 52 53 T twilight 53 U uncalibrated 30 81
5. k check data if true check data properties matching type expt filter ncoadd mjd e Check Check if versions of PAPI modules are right Input FITS data files GEIRS is capable of saving the frames in different modes integrated FITS cubes MEF etc Next ones are available in the OT when the OP Observing Program is defined Multi Extension FITS MEF Integrated Multi Extension FITS MEF Cube Integrated All SEF Integrated FITS Cube SEF Cube Individual SEF Individual However PAPI does not accept any kind of FITS data files available in GEIRS only the configured in the OT except Individual As result PAPI accepts the next type of FITS files in order of preference Integrated Multi Extension FITS MEF a unique FITS file with four extensions MEF where each extension corresponds to one of the 4 images produced by the single detector chips If the number of coadd NCOADDS is 0 then they will be integrated arithmetic sum in a single image This is the default and more common saving mode used in fact it is the default and more wished saving mode This mode will also be used when the software or hardware sub windowing is set and the integrated option is selected Then there will be an extension for each sub window Non integrated Multi Extension FITS MEF a unique FITS file with four extensions MEF one per each detector or window having each extension N planes where N is the
6. Just be sure the number of parallel parameter is set to True on the PAPI_CONFIG file When parallel True the pipeline will reduce each detector in parallel using all the cores available in your computer How do report a issue Please submit issues with the issue tracker on github 2 3 PAPI 2 3 1 Purpose PANIC pipeline hereafter PAPI performs the automatic data processing both for quick look and for science quality of the data produced by PANIC The automated processing steps include basic calibration removeing instrumental signature dark and flat fielding cosmic ray removal treatment for electronic ghosts cross talk sky subtraction non linear count rate correction robust alignment and registration removing the field distortion 2 3 PAPI 29 PAPI Documentation Release 1 2 20151020173439 This chapter gives an introduction in how to get started with PAPI showing the steps that would normally be necessary to reduce a data set from PANIC In particular this example assumes that we have a series of FITS images from an observation run 2 3 2 Quickstart Running PAPI can be as simple as executing the following command in a terminal papi py s raw data d result Where raw data is the directory of the raw dataset uncalibrated having both science or calibration files and result is the path to the directory where the calibrated data produced by the pipeline will be saved Example pa
7. photometry py i data reduced fits o tmp calibration pdf correctNonLinearity HAWAII 2RG near IR detectors exhibit an inherent non linear response It is caused by the change of the applied reverse bias voltage due to the accumulation of generated charge The effect increases with signal levels so that the measured signal deviates stronger from the incident photon number at higher levels and eventually levels out when the pixel well reaches saturation The correctNonLinearity module corrects PANIC images for their count rate dependent non linearity It used the header keywords READMODE and DET ID to determine the correction It corrects the first image and in the case of a multi extension image the second image as well with the appropriate power law For details see PANIC detector non linearity correction data Usage 54 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 Options h help show this help message and exit m MODEL model MODEL FITS MEF cube file of polinomial coeffs c4 c3 c2 cl s SOURCE FILE LIST source SOURCE FILE LIST Source file list of FITS files to be corrected o OUT DIR out dir OUT DIR filename of out data file default tmp S SUFFIX suffix SUFFIX Suffix to use for new corrected files f force Force Non linearity correction with no check of headervalues NCOADD DAT solveAstrometry Perfor
8. PAT NEXP 5 PANIC Pattern total number of expositions IMAGETYP SCIENCE PANIC Image type 2 9 5 Data Raw images pixels are coded with 32 bit signed integers BITPIX 32 however final reduced images are coded with 32 bit single precision floating point BITPIX 32 The layout of each chip image in a raw image is de scribed above 2 9 6 Classification Any raw frame can be classified on the basis of a set of keywords read from its header Data classification is typically carried out by the Pipeline at start or by PQL that apply the same set of classification rules The association of a raw frame with calibration data e g of a science frame with a master dark frame can be obtained by matching the values of a different set of header keywords filter texp ncoadds itime readmode date obs etc Each kind of raw frame is typically associated to a single PAPI pipeline recipe i e the recipe assigned to the reduction of that specific frame type In the pipeline environment this recipe would be launched automatically In the following all PANIC raw data frames are listed together with the keywords used for their classification and correct association Type Description DARK Dark frame DOME FLA Dome flat field frame lamp on lamp off SKY FLAT Sky flat field frame FOCUS Focus frame of a focus series SCIENCE Science frame SKY Sky frame mostly clear used for extended object reduction
9. PERCT350 2730 PERCT375 2732 PERCT400 2733 PERCT425 2735 PERCT450 2736 PERCT475 2738 PERCT500 2739 PERCT525 2741 PERCT550 2743 PERCT575 2745 PERCT600 2746 PERCT625 2748 PERCT650 2749 PERCT675 2750 PERCT700 27534 YJ PERCT725 2754 PERCT750 2756 PERCT775 2758 PERCT800 2760 PERCT825 2763 PERCT850 2765 PERCT875 2768 PERCT900 DITA J PERCT925 2776 PERCT950 2780 PERCT975 2787 RA 332 367528 DEC 51 084307 PIXSCALE 0 45 CUNIT1 deg CUNIT2 deg i CTYPE1 RA TAN CTYPE2 DEC TAN CRVAL1 332 36752753434 CRVAL2 51 0843069975685 CDT 1 0 000124999996688631 CD2 2 0 000124999996688631 CD1 2 2 56379278852432E 14 CD2 1 2 56379278852432E 14 CRPIX1 81 CRPIX2 2132 DET ID SG1 COMMENT WCS assumes CHIPGAPX 167 BSCALE T BZERO 0 END ile A le AD ile A le AD ile A le AD ile A le AD ile A le AD ile A le AD ile A le AD 17 5 percent 20 percenti 22 5 percent 25 percenti 27 5 percent 30 percenti 32 5 percent 35 percenti 37 5 percent 40 percenti 42 5 percent 45 percenti 47 5 percent 50 percenti 52 5 percentile A 55 percentile AD 57 5 percentile A T i i i i T i 60 percentile AD ile A le AD ile A le AD ile A le AD ile A le AD ile A le AD ile A le AD 62 5 percen 65 p
10. default False t no type checking Do not make frame type checking default False v verbose verbose mode default Usage calDark py options argl arg2 Example calDark py s data PANIC VO0 dark seq txt o data out masterDark fits calDarkModel The calDarkModel module performs a dark model To do that a input dark series exposures with a range of exposure times is given Then a linear fit is done at each pixel position of data number versus exposure time A each pixel position in the output map represents the slope of the fit done at that position and is thus the dark current expressed in units of data numbers per second The dark model obtained will be a FITS files with two planes extensions plane 0 dark current in DN sec e plane 1 bias DARKCURRENT value The median dark current in data numbers per second found from the median value of the output dark current map Usage Usage calDarkModel py options argl arg2 Options h help show this help message and exit s SOURCE FILE LIST source SOURCE FILE LIST Source file listing the filenames of dark frames o OUTPUT FILENAME output OUTPUT FILENAME final coadded output image 5 show stats Show frame stats default False Example calDarkModel py s tmp darkModel txt o tmp darkModel fits calDomeFlat The calDomeFlat module creates a master flat field from dome
11. some of the master dark are not found then the procedure will fail The procedure to create the master sky flat is as follow 1 Check the TYPE sky flat and FILTER of each Flat frame If any frame on list missmatch the FILTER then the master twflat will skip this frame and continue with then next ones EXPTIME do not need be the same so EXPTIME scaling with mode will be done 2 Check either over or under exposed frames 10000 mean level lt 40000 ADUs 3 We subtract a proper MASTER DARK it is required for TWILIGHT FLATS because they might have diff EXPTIMEs 4 Make the combine median sigclip rejection the dark subtracted Flat frames scaling by mode 5 Normalize the sky flat wrt SG1 detector dividing by its mean value Build GainMap This command is used to produce a Master GainMap file from a set of files currectly selected in the Data List View It checks that all the selected files are compliant ie have the same FILTER NCOADD READMODE and shape You have to select at least three flat frames dome dusk or dawn For sky flats the procedure will look for the required master dark frames to subtract in the current output directory and in the external calibration directory If some of the master dark are not found then the procedure will fail Dome flat do not need dark subtraction The procedure to create the master sky flat is as follow 1 Check the TYPE sky flat and FILTER of each Flat frame If
12. the directory to which the resulting images will be save temp_dir data2 tmp the directory to which temporal results will be saved av If no outfile name is given None the result of each sequence reduced will be saved with a filename as PANIC DATE OBS fits where DATE OBS is the keyword value of the first file in the sequenc output file tmp reduced fits of each PANIC detector separatelly Otherwise False all be processed sequencially parallel True ncpus 8 4 Number of CPU s cores to used for parallel processing verbose True currently not used logfile tmp papi log to be implemented reduction_mode reduction mode to do with the raw science files reduction_mode quick default reduction mode quick science lemon quick lemon 14 detector detector to reduce process 01 02 03 04 all For O2k this parameter has no effect Ql ext1 0 2048 0 2048 SG4 for CAM DETROT90 2 the bad detector Q2 ext2 2048 4096 0 2048 SG1 Q3 ext3 0 2048 2048 4096 SG3 Q4 ext4 2048 4096 2048 4096 SG2 Since GEIRS r731M 18 version new MEF extension naming id trailing Decide if parallel processing capabilities will be activated True i e split th processing ab EXTNAME Qi j 2 3 PAPI 37 file with a PAPI Documentation Release 1 2 20151020173439 DET ID SGi j same ids as before and
13. the object mask are multiplied by the bad pixel mask to remove false detections a cross correlation algorithm among the object masks is used to measure the relative offsets It also works if no object is common to all the images the cleaned images are combined using the offsets creating the quick image to remove the effect of faint obejcts on the estimate of the sky frames SExtractor is used on the combined image to create a master object mask the object mask is dilatated by a certain factor to remove also the undetected object tails for each image a new sky is computed by taking into account this object mask if field distortion can be neglected these images are combined by using the old offsets creating the science image field distortion is removed from the cleaned images by using SCAMP computed distortion model the pixels containing deviant pixels are identified and flagged the old offsets could be effected by field distortion therefore new offsets are computed for the undistorted images 2 10 Processing description 65 PAPI Documentation Release 1 2 20151020173439 finally the cleaned corrected images are combined Main configuration file See Main config file Data set classification One of the main features of PAPI is that the software is able to do an automatic data reduction While most of the pipelines are run interactively PAPI is able to run without human interaction It is done bec
14. 0 032836 degrees E of N How to inspect the profile of the stars in an image You should follow the next steps 1 select in the Data List View the image to inspect double click to display the image into ds9 and zoom to the area you wish to inspect go to the tool bar or Tool menu and open an IRAF console type in the iraf console imexam Un A V N focus the mouse cursor on the ds9 display and type the imexam comand you wish for the inspection For example type r to show the radial profile of the selected star 6 once you have finished the inspection type q to exit from imexam How do quick reduce an observed sequence There are two options if you know the files that compose the sequence you can select them and then right click and run the Quick Reduction command e go to the List View Filter and select GROUP then look for the sequence you are looking for in the Data List View right click and select Reduce Sequece command For the quick reducion the pipeline will use the preferences established on Setup tab How do quick reduce an observed sequence using dark and flat master calibration files You should follow the next steps 1 Check your sequences are right ie they are well formed and there were no interruption It some sequece calibration or science is not well formed the you should use FITS gt Create DataSeq menu option in order to fix not well formed sequence 2
15. 144687 deg H A 21 04 34 72 T FOCUS 0 mm telescope focus CASSPOS 0 deg cassegrain position rel to NSEW OBJECT unknown telescope target POINT NO 0 ct pointing counter DITH NO 0 ct dither step EXPO NO 2 ct exposure read counter FILENAME test 0001 fits FILE ID Panic 2015 05 28T16 22 23 164 0001 001 instru time image windo PLNAME y macro template name IMERO 2740 ms 60 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 IMER1 2740 ms IMER2 0 us PTIME 2 pixel time base index PREAD 10000 ns pixel read selection PSKIP 150 ns pixel skip selection LSKIP 150 ns line skip selection READMODE line interlaced read read cycle type IDLEMODE wait idle to read transition IDLETYPE ReadWoConv idle cycle type SAVEMODE line interlaced read save cycle type NEXP 1 cycle repeat count CPAR1 1 cycle type parameter ITIME 2 739931 s on chip integration time CTIME 5 481201 s read mode cycle tim HCOADDS 1 ct of hardware coadds EMSAMP 1 ct electronic multi sampling PCOADDS 1 ct f of coadded plateaus periods SCOADDS 1 ct of software coadds SWMSA
16. 28 09 18 01 486 DEBUG reductionset 1167 SEQUENCE 2 TYPE S PAPI 2015 05 28 09 18 01 486 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 486 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 486 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 486 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 487 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 487 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 487 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 487 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 487 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 487 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 487 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 487 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 488 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 488 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 488 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 488 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 488 DEBUG reductionset 1170 data2 2015 03 10
17. Ctrl F 2015 03 11 2015 03 11 ec00000000 Build Master Twlight Flat Ctrl T Last file received darks March10 0255 fits Build Gain Map Ctri G Remove Clear All Build BPM Ctrl B Apply Dark FlatField BPM Ctrl A Event log Time UTC 20150520 16 01 58 Start of observing sequence Il 20150520 16 01 58 Detected end of observing sequenc 20150520 16 Start of observing sequence D Ctrl F 20150520 16 01 58 Detected end of observing sequel btrac Y Ctrl S 20150520 16 01 58 Start of observing sequence DAR der A 20150520 16 01 58 Detected end of observing sequ subtrac Ctri N 20150520 16 01 58 Start of observing sequence ick Ctrl Q 20150520 16 01 58 Detected end of observing sequenc ee SGA 20150520 16 01 58 Start of observing sequence D Hut 4 Shift A 20150520 16 01 58 Detected end of observing sequel J o b Shift P 20150520 16 01 58 Start of observing sequence D um F 20150520 16 01 59 Detected end of observing sequence Statistics Shift S FWHM mean estimation Ctrl W Background estimation Shift B Math FITS Apply Non Linearity Correction Shift N File Edit View Frame Zoom Color Region WCS Analysis Help File TiltProcedure_March10_0017 fits Object M Value o Physical 2685 000 Y 15 1 Image 2685 000 Y 1558 000 Frame 1 1000 0 000 fle edit view frame bin zoom scale color region wes h
18. EXPT NCOADD ITIME READMOD check_prop yes suffix the string if any to be added to the filename of each resulting the resulting image would be saved to home images ferM_0720_o_D fits de db de db d suffix D min frames minimun number of frames required to build a master dark min_frames 5 dflats object_names in order to make it possible to work in batch mode is it possible to run the PANIC flat module in all the images specifying in this parameter which ones will be considered That is only those images whose object name matchs one of the names listed in this parameter will be considered when generating the master dome flat equivalent to saying do not filter images by their object names object_names DOME_FLAT_LAMP_OFF DOME_FLAT_LAMP_ON E FE FEE FE HE FE AE FE FE FE FE FE FE FE FEAE FE 4 H H AE H AE AE HE EE EE EE EE EE EE EEE EE EH FE H FE H AE EH EEE FE FE FE FE FEAE FE EE H AE H HEE AE AE FE AE EE FE AE FE FE FE FE FE 4 FE FE EE EE FE 4 H AE FE AE AE FE AE E FE AE FE EAE Note that if is contained in the list all object names will be matched This symbol thus provides a way to easily specify all the images which is image For example for suffix D and the imput file home images ferM 0720 o f This parameter is optional as if nothing is specified nothing will be appended Note that if x is contained
19. M REDUCTION MODE red mode REDUCTION MODE Mode of data reduction to do quick science lab lemon quick lemon m OBS MODE obs mode OBS MODE Observing mode dither ext dither other S SEQ TO REDUCE Seq to reduce SEQ TO REDUCE Sequence number to reduce By default all sequences found will be reduced W DETECTOR window detector DETECTOR 30 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 Specify which detector to process Q1 SG1 Q2 SG2 Q3 SG3 Q4 SG4 Q123 all except SG4 all default all p print Print all detected sequences in the Data Set T SEQ TYPE Sequences type SEOQ TYPE Specify the type of sequences to show DARK FLAT all DOME FLAT SKY FLAT FOCUS SCIENCE CAL all default all b build calibrations Build all the master calibrations files C EXT CALIBRATION DB ext calibration db EXT CALIBRATION DB External calibration directory library of Dark amp Flat calibrations D MASTER DARK master dark MASTER DARK aster dark to subtract F MASTER FLAT master flat MASTER FLAT aster flat to divide by B BPM FILE bpm file BPM FILE Bad pixel mask file g GROUP BY group by GROUP BY kind of data grouping based on to do with thedataset files ot filter
20. PANIC output_dir data2 out the directory to which the resulting images will be saved temp_dir data2 tmp the directory to which temporal results will be saved 8 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 verbose True Run parameters run_mode Lazy default initial run mode of the QL it can be None Lazy Prereduce Although the user can edit these values in the config file some of them can be set easily on PQL s GUI For the complete list of the parameters available on the PAPI_CONFIG file see Main config file section 2 2 5 PQL s main window PQL Main window contains a Menu bar 1 Tool bar 2 four Tabbed panels 3 and an Event Log Window 4 Images are displayed in an external well known application ds9 Plots results are displayed in the additional windows usually generated by matplotlib than can be copied to the clipboard printed or saved Menu bar The menu bar provides acces to some PQL s capabilities 1 File View Settings Calibrations Tools ON ot BOO N Help Opens a web browser which shows an on line HTML version of this user s manual This will fail if the internet conection or proxy is not correctly configured 7 Exit Quit PQL application Tool bar The tool bar duplicates some of the options available from the menu bar or the pop up menu Currently there are several buttons which provide quick access to cha
21. VNBIA3 0 V det bias voltage VNBIAS 0 B VNBIA4 0 V det bias voltage VNBIAS 0 B VPBIA1 0 V det bias voltage VPBIAS 0 B VPBIA2 0 V det bias voltage VPBIAS 0 B VPBIA3 0 V det bias voltage VPBIAS 0 B VPBIA4 0 V det bias voltage VPBIAS 0 B VNCAS1 0 V det bias voltage VNCASC 0 B VNCAS2 0 V det bias voltage VNCASC 0 B VNCAS3 0 V det bias voltage VNCASC 0 B_VNCAS4 0 V det bias voltage VNCASC 0 B_VPCAS1 0 V det bias voltage VPCASC 0 B_VPCAS2 0 V det bias voltage VPCASC 0 B_VPCAS3 0 V det bias voltage VPCASC 0 B_VPCAS4 0 V det bias voltage VPCASC 0 B_VBOUB1 0 V det bias voltage VBIASOUTBUF 0 2 9 Data formats 61 PAPI Documentation Release 1 2 20151020173439 B_VBOUB2 0 V det bias voltage VBIASOUTBUF 0 B VBOUB3 0 V det bias voltage VBIASOUTBUF 0 B_VBOUB4 0 V det bias voltage VBIASOUTBUF 0 B REFSA1 0 V det bias voltage REFSAMPLE 0 B REFSA2 0 V det bias voltage REFSAMPLE 0 B REFSA3 0 V det bias voltage REFSAMPLE 0 B REFSA4 0 V det bias voltage REFSAMPLE 0 B_REFCB1 0 V det bias voltage REFCO
22. any frame on list missmatch the FILTER then the master twflat will skip this frame and continue with then next ones EXPTIME do not need be the same so EXPTIME scaling with mode will be done 2 Create the proper master dome sky flat Once the master dome flat is created the procedure will compute the gainmap as follow Build BPM TBC Apply Dark amp FlatField amp BPM This option subtracts a master dark file then divides by a flat field and finally mask the bad pixels on the current selected files The master dark and master flatfield files can be searched for automatically into the output and external calibration directories or can be selected manually by the user 20 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 If some of them dark or flat are not found or selected pressing Cancel in the file dialog then it will not be used or applied In the case of the bad pixel mask BPM it cannot be selected but specified in the PAPI config file However the user will be asked for about which action to do with the bad pixel mask whether set bad pixels as NaNs fix bad pixels with an interpolation algorithm or do nothing with BPM Apply Non Linearity Correction It applies the Non Linearity correction to the selected file or set of files in the Data List View and show the result in ds9 it also set bad pixels to NaN and will be displayed as green pixels or the default color configured in
23. data with PAPI you should use SEF of MEF non integrated FITS cube mode Beware that the order of the chip in the raw image produced is as described in next figure Next table shows the mapping of extension quadrant names and detectors Extension Name Q1 Q2 Q3 Q4 Detector Hw ID SG1 SG2 SG3 SG4 Note that the order of the extensions in the FITS file is Q1 ext 1 Q2 ext 2 Q3 ext 3 and Q4 ext 4 2 9 3 Headers The header keywords currently used in a raw PANIC FITS file is as shown bellow 2 9 Data formats 59 PAPI Documentation Release 1 2 20151020173439 Primary Header SIMPLE T BITPIX 32 AXIS 2 2 AXIS1 4096 AXIS2 4096 EXTEND T FITS dataset may contain extensions COMMENT FITS Flexible Image Transport System format is defined in Astronomy COMMENT and Astrophysics volume 376 page 359 bibcode 2001A amp A 376 359H BSCALE J BZERO 0 adu real bzero bscalex value BUNIT adu adu MJD OBS 57170 68257 d Modified julian date days of observation DATE OBS 2015 05 28T16 22 54 0402 d UT date of observation end DATE 2015 05 28T16 22 54 file creation date YYYY MM DDThh mm ss UT UT 58974 040247 s 16 22 5
24. dir R scie done Use a specific calibration directory for data reduction To reduce a complete directory using the calibration found in an specific directory master dark and flat field calibrations previously processed you have to use the C path option This way if PAPI cannot find the required calibrations into the input directory my raw directory will look for them into the external calibration directory provided my calibration dir Command papi py s my raw data directory d my output directory C my calibrations dir Enable the Non Linearity correction for the data processing If you need to enable to Non Linearity correction see PANIC detector non linearity correction data you only have to edit the PAPI_CONFIG file and set nonlinearity apply parameter to True Note Be ware that when using Non Linearity correction all the files used and calibrations must be non linearity corrected Otherwise you don t get an consistent result Reduce a single detector By default PAPI processes all the detector and builds the mosaic with the reduced detectors However if you do not need to reduce all the detectors but only one of them Q1 Q4 you can use the option W Qx 2 3 PAPI 35 nce PAPI Documentation Release 1 2 20151020173439 W DETECTOR window detector DETECTOR Specify which detector to process Q1 SG1 Q2 SG2 Q3 SG3 Q4 SG4
25. ds9 gt Edit gt Preferences gt General gt Color on the display The corrected image is saved in the output directory with a _ C suffix The master Non Linearity correction file used for the correction is defined in the configuration file Apply and show BPM This command can be used to apply the BPM to the selected file in Data List View the and show the results NaNs as green pixels or the default color configured in ds9 gt Edit gt Preferences gt General gt Color on the display The bad pixel masked image is saved in the output directory with a _BPM suffix The master Bad Pixel Mask file used is defined in the configuration file Focus evaluation The Focus evaluation procedure is based in the IRAF starfocus routine It only differs on the final plot that is obtained from non saturated stars and the best focus is computed computing the curve fit of these points The PSF size is measured with the the FWHM of the best fit Moffat profile MFWHM Once you have obtained a focus series using the Observation Tool the procedure to evaluate and get the best focus value for that serie is as follow Warning The input images of the focus series should be saved as SEF Single Extension FITS because IRAF starfocus does not works with MEF files However if your focus series was saved as SEF the routine will previously convert to SEF and then you should not have to do any other conversion 1 Select the files of the focus ser
26. flat division Sky subtraction using N nearest frames or own sky Field distortion removal mage align and stacking Preliminary astrometric solution Preliminary photometry In addition PQL allows you to execute manually in an interactive way some tasks with the data For example you will be able to select a file compute some statistics values background FWHM min max or ask for the sky subtraction looking for the nearest N frames around the selected one Other option available is to select a set of files and request to shift and align them PQL can be operated in both near real time mode during the observation and offline mode after the observation with all data files already stored in the disk however its functionalities have been provided mainly in near real time to check the status and progress of the observation during the night The visualization application used to display the images is SAOImage ds9 which supports FITS images multiple frame buffers region manipulation and many scale algorithms and colormaps 2 2 2 FITS files and headers PQL only supports FITS Flexible Image Transport System with two dimensional image formats Due PANIC has a FPA of four detector the FITS files can be Single Extension FITS SEF orMulti Extension FITS MEF however MEF are prefered The complete definition of the FITS headers can be found on the GEIRS documentation For general purpose such as view
27. in the list all object names will be matched This symbol thus provides a way to easily specify all the images which is the resulting image would be saved to home images ferM 0720 o D fits suffix F min frames minimun number of frames required to build a master dome flat min_frames 5 This parameter is optional as if nothing is specified nothing will be appended lts check_prop if true the frames used to build the master will be checkd to have the same acquisition properties EXPT NCOADD ITIME READMODE FILTER check_prop yes suffix the string if any to be added to the filename of each resulting image For example for suffix D and the imput file home images ferM 0720 o fits 2 3 PAPI 41 PAPI Documentation Release 1 2 20151020173439 area_width 1000 length in pixels of the central area used for normalization median_smooth median filter smooth of combined FF to reduce noise and improve the S N and preserve the small scale high frequency features of the flat median_smooth False H H H H 4 4 EEE REE ERE ERE ERE RE ERE HEE GEE HGH EE EGE EEE twflats FE H EE EEE AE FE FE AE FE FE AE FE FE AE FE EEE FE AE AE H AE FE EE E FE AE FE FE AE FE FE AE FE FE AE FE FE AE FE FE FE FE FE AE FE AE AE FE AE AE FE AE FE FE AE FE EE E EEH object_names in order to make it pos
28. model tal Calibs mNONLIN_L datal Calibs mNONL les scamp conf les swarp conf apply T HERE H ls ar ly good replaced wi for isolated no fix BPM but only set into account in GainMaps none grab file TH k TH ject names no action will BPMask gt Bad pixeles gt 0 FITS file containing the BPM datal Calibs bpm lir v01 00 fits datal Calibs mBPM LIR 01 01 mef fits datal Calibs master bpm lir ones join fits HARPER RET GREE EGE HEE HEE ERE GEE EEE EERE EE HE EE HE EE HEH in order to make it possible to work in batch mode possible to run the PANIC dark module in all the images this parameter which ones will be considered That is be done wi Good p ractor conv rue for correction IR 01 01 fits HERE HEE HEE FEAE AE TE AE AE TE Dilatation of the object mask Due to field distortion it is recommended to dilete the object mask in order to have a good object masking for the 2nd skysubtraction pass Next value is a mult scale factor to expand object regions default is 0 5 ie make 50 larger dilate 0 2 Mosaic engine tool to be used to build the final mosaic with the 4 detectors swarp use SWARP from Astromatic net not always work montage use Montage tool in principle the best option other no mosaic is built but a MEF with 4 extensions For more information see http www astro
29. several aspects of PAPI are not yet covered sufficient detail 2 8 1 Modules The PAPI pipeline consists of a set of processing modules which implement from basic calibration to generating final co added registered mosaics See table below These modules can be run as a stand alone routines depending 46 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 of your needs e g to create a master dark or flat field for calibration or you can use them as a pipeline calling the main script papi which will use each of the modules as they are needed in order to accomplish a complete data reduction of a set of raw images Main Modules Description papi Main pipeline script to start the entire data reduction process applyDarkFlat Finds out the best Focus value from a focus series astrowarp Creates final aligned and coadded frame using SEx SCAMP and SWARP CalBPM Creates a master Bad Pixel Mask from a set of darks and flats calibration files calCombineFF Combine a dome Flat field and a sky Flat field into a new Flat field calDark Creates a master dark by combination of a dark sequence calDarkModel Creates a master dark model from a dark series calDomeFlat Creates a master Dome Flat calSuperFlat Creates a master Super Flat from a set of object or sky frames calTwFlat Creates a master Twilight Flat ca
30. the order in the MEF file is 01 02 03 04 0123 all except 04 detector all if any default master calibration files to use master dark None master flat None master bpm None External calibration DataBase directory used as an external calibration database Then if during the reduction of a ReductionSet RS no calibration dark flat are found in the current RS then PAPI will look for them into this directory If the directory does not exists or no calibration are found then no calibrations will be used for the data reduction Note that the calibrations into the current RS have always higher priority than the ones in the external calibration DB ext calibration db data2 Masters2 check data True Remove crosstalk If True a procedure to remove the crosstalk will be executed just after the 1st 2nd sky subtraction both O2K or PANIC remove crosstalk True Cosmic Ray Removal If True a procedure to remove the CR will be executed just after the 2nd sky subtraction It has only sense for LEMON output because CR should be removed during the stack combine co adding with SWARP remove cosmic ray False Purge output If True a procedure to remove the temporal or intermediate files list objs ldac xml will be removed from the output directory just after the end of the RS reduction purge output True Estimate FWHM after reduction
31. the right bad pixel mask BPM First pass sky subtraction 2 10 5 Sky model TBC 66 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 Object detection Offset computation First pass coaddition Master object mask SExtractor is again used to find objects in this first pass coadded image in order to mask then during next sky estimation This time the parameters controlling the detection threshold should be set to have deeper detections and mask faint objects The parameters involved nad ther default values are mask_minarear 10 mask_thresh 1 5 The resulting object mask is extended by a certain fraction to reject also the undetected object tails Non Linearity HAWAII 2RG near IR detectors exhibit an inherent non linear response It is caused by the change of the applied reverse bias voltage due to the accumulation of generated charge The effect increases with signal levels so that the measured signal deviates stronger from the incident photon number at higher levels and eventually levels out when the pixel well reaches saturation The common approach is to extrapolate the true signal Si t from measurements with low values and fit it as a function of the measured data S t with a polynomial of order n For the correction PAPI uses a master Non Linearity FITS file that store the fit to be applied to the raw images There is file for each readout mode The filename is composed as mNONLIN_ lt r
32. tional settings have to be made You should check though that the images have correct header keys 2 11 References 2 11 1 Publications Ib ez Mengual J M Fern ndez M Rodr guez G mez J R Garc a Segura A J Storz C The PANIC software system Proc SPIE 7740 77402E 2010 Ib ez Mengual J M Garcia A J Storz C Fried J W Fern ndez M Rodr guez J F Advanced PANIC quick look tool using Python Proc SPIE 8451 2012 2 11 2 Webpages PANIC PAnoramic Near Infrared Camera Instituto de Astrof sca de Andaluc a IAA CSIC Max Planck Institute for Astronomy MPIA Calar Alto Observatory GEIRS Observation Tool OT LEMON Astromatic MultiDrizzle Handbook 2 12 PAPI FAQ 2 12 1 Installation Does PAPI work with python 3 It has not been tested yet but probably it does not work What GUI toolkit does PAPI QL use PAPI Quick Look uses Qt toolkit 68 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 Can PAPI QL work with PyQt5 It has not been tested yet but probably it does not work 2 12 2 Data reduction What is the best way to reduce PAPI data The recommend to use the OT and execute the OBs That way the headers will include meta data about the observation and thus the pipeline can group the data and find the required calibrations for a successful reduction Then you only have to type gt papi s datal PANIC my_program How does
33. v7 3b5 Montage v3 3 montage_wrapper 0 9 8 Additional packages are optionally required sphinx to build the documentation Note If you are using a SCAMP version lt 2 0 4 lastest stable version then you need to install the CDSClient Otherwise if you are using SCAM version gt 2 0 4 then you need libcurl Anycase if you are behind a proxy you need to set the proxy server in your system http proxy http your proxy your port export http proxy PAPI Documentation Release 1 2 20151020173439 2 1 2 Download The latest stable version of PAPI can be downloaded from GitHub repository 2 1 3 Building and Installation PAPI installation is thought to be done as a personal user non root however it should work under any system directory ie usr local 1 To install PAPI as a personal user non root follow the next steps Once you have installed the required packages described above you are ready to install PAPI for this follow the next steps git clone https github com ppmim PAPI git papi cd papi papi setup sh 2 To install PAPI as root on your system follow the next steps cd usr local git clone https github com ppmim PAPI git papi cd papi Edit the papi setup sh and set the right values to PAPI HOME and PAPI BIN variables and ther papi setup sh Warning The script papi setup sh is currently implemented only for the Bash s
34. with NCOADD 1 Of course this values will be specific for each detector and in case of a multi detector instrument should be the lowest value of all detectors satur level 50000 44 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 catalog 2MASS Catalog used in SCAMP configuration MASS USNO A1 USNO A2 USNO B1 SC 1 3 GSC 22 GSC 2 3 UCAC 1 UCAC2 UCAC 3 NOMAD 1 PPMX DENIS 3 SDSS R3 SDSS R5 SDSS R6 or SDSS R7 The pipeline is designed for the PANIC data files You should change this options in case you were going to work with images whose keywords are not the same object_name IMAGETYP Target description julian_date MJD OBS Modified Julian date x_size NAXISI Length of x axis y size NAXIS2 Length of y axis ra RA CRVAL1 Right ascension in decimal degrees The list defines the priority in which the values are read dec DEC CRVAL2 Declination in decimal degrees That is if DEC is not found CRVAL2 will be read and so on filter FILTER Filter name Next are some configurable options for the PANIC Quick Look tool some important di rectories source data O2K Feb 2012 120213 it can be a directory or a file GEIRS data log file source mnt GEIRS DATA source home panic GEIRS log save CA2 2m log source mnt tmp fitsfiles corrected source home panic tmp fitsfiles corrected source datal PANIC source home panic tmp fits
35. with thedataset files ot filter k check data if true check data properties matching type expt filter ncoadd mjd e Check Check if versions of PAPI modules are right PAPI creates a in memory SQLite database to store the uncalibrated input data fits headers and pipeline metadata Results FITS file s with coadd as result of the reduction and calibration of the specified sequences otherwise the error will be shown in the console and log file Examples 48 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 The following example reduce in quick mode all the sequences of the given directory Spapi py s my raw_data directory d my output directory M quick applyDarkFlat This module receives a series of FITS images and applies basic calibration subtract and divide by the given calibration files master dark and master flat field Options h help show this help message and exit s SOURCE FILE LIST source SOURCE FILE LIST Source file listing the filenames of raw frames d DARK FILE dark DARK FILE Master dark to be subtracted f FLAT FILE flat field FLAT FILE Master flat field to be divided by o OUT DIR out dir OUT DIR Directory where output files will be s
36. 1000 TEE EE EE EE HE RE EE HE EEE HE HE EE HE HE HE EE EE RE EE HE ERE EE EE ER EE HEH EH EE EE EE skysub TEE EE HE E FE FE FE EE HE HE E FE EE HE EEE FE FE HE E HE HE FE AE EE HE EE EE RE EE EE E EEE EE EE RE Used for createOb Mask skySubtraction object names in order to make it possible to work in batch mode is it possible to run the PANIC skysubtration module in all the images specifying in this parameter which ones will be considered That is only those images whose object name matchs one of the names listed in this parameter will be considered when generating the master dark Note that if is contained in the list all object names will be matched This symbol thus provides a way to easily specify all the images which is equivalent to saying do not filter images by their object names object names SKY SKY FOR check prop if true the dark frames used to build the master will be checkd to have the same acquisition properties EXPT NCOADD ITIME READMODE FILTER check prop yes Suffix the string if any to be added to the filename of each resulting image For example for suffix D and the imput file home images ferM 0720 o fits the resulting image would be saved to home images ferM 0720 o D fits This parameter is optional as if nothing is specified nothing will be appended suffix S min frames minimun number of frames required
37. 2015 03 10 Standard Star FS15 PAPI 2015 05 28 09 18 01 492 DEBUG reductionset 1170 data2 2015 03 10 Standard Star FS15 PAPI 2015 05 28 09 18 01 492 DEBUG reductionset 1170 data2 2015 03 10 Standard Star FS15 PAPI 2015 05 28 09 18 01 492 DEBUG reductionset 1170 data2 2015 03 10 Standard Star FS15 PAPI 2015 05 28 09 18 01 493 DEBUG reductionset 1170 data2 2015 03 10 Standard Star FS15 PAPI 2015 05 28 09 18 01 493 DEBUG reductionset 1167 SEQUENCE 8 TYPE SKY FLAT FIL PAPI 2015 05 28 09 18 01 493 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 493 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0001_ PAPI 2015 05 28 09 18 01 493 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0002_ PAPI 2015 05 28 09 18 01 493 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0003_ PAPI 2015 05 28 09 18 01 493 DEBUG reductionset 1170 data2 2015 03 10 skyflats dawn 0004 PAPI 2015 05 28 09 18 01 494 DEBUG reductionset 1167 SEQUENCE 9 TYPE SKY FLAT FIL PAPI 2015 05 28 09 18 01 494 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 494 DEBUG reductionset 1170 data2 2015 03 10 skyflats dawn 0005 PAPI 2015 05 28 09 18 01 494 DEBUG reductionset 1170 data2 2015 03 10 skyflats dawn 0006 PAPI 2015 05 28 09 18 01 494 DEBUG reductionset 1170 data2 2015 03 10 skyflats dawn 0007 PAPI 2015 05 28 09 18 01 494 DEBUG reductionset 1170 data2 2015 03 10 skyflats dawn 0008 PAPI 2015 05 28 09 18 01 494 DEBUG redu
38. 20150310 __ ja tne mi nato Input directory This is where you tell PQL where the data are or being saved by GEIRS This directory is specified at the beggining of the night on the Observation Tool PQL requieres all data to lie in some main directory not being required to distribute the files in individual sub directories for darks flats and science images It is advised that this directory follow the next format datal PANIC YYYYMMDD To set the value the user must push the Input Dir button Note that the value in this field has only effect when the checkbox on the right is clicked Output directory This is where you tell PQL where the data generated by PQL as result of some processing will be saved This directory must also be specified at the begining of the night and is advised to follow the next format 10 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 data2 out YYYYMMDD To set the value the user must push the Output Dir button Output Dir Note that the value in this field has only effect when the checkbox on the right is clicked Temporal directory This is where you tell PQL where the temporal files generated by PQL as result of some processing will be saved and probably deleted after at the end of that processing This directory must also be specified at the begining of the night and is advised to follow the next format da
39. 4 0402 UTC at EOread LST 30949 087329 s local sidereal time 08 35 49 087 EOread ORIGIN Centro Astronomico Hispano Aleman CAHA OBSERVER Mathar TELESCOP CA 2 2 FRATIO F 08 1 OBSGEO L 2 546135 deg telescope geograph longit 2015A8A 574A OBSGEO B 37 223037 deg telescope geograph latit 2015A amp A 574A OBSGEO H 2168 m above sea level 2015A amp A 574A 36R LAMPSTS calib lamp INSTRUME PANIC PAnoramic Near Infrared camera for Calar Alto CAMERA HgCdTe 4096x4096 IR Camera 4 H2RGS PIXSCALE 0 45 arcsec px EGAINI 4 84 ct electrons DN EGAIN2 4 99 ct electrons DN EGAIN3 5 02 ct electrons DN EGAIN4 5 45 ct electrons DN ENOISE1 16 ct electrons read ems 1 ENOISE2 16 6 ct electrons read ems 1 ENOISE3 18 5 ct electrons read ems 1 ENOISE4 17 9 ct electrons read ems 1 ROVER MPIA IR ROelectronic Vers 3 Version det electronics WPOS 5 ct number of GEIRS wheels W1POS Coldstop22 W2POS H y W3POS Ks y W4POS dummy W5POS Black FILTER NO J filter macro name of filter combinations STRT_INT 58943 164225 s 16 22 23 1642 start integration UT STOP_INT 58946 502476 s 16 22 26 5025 stop integration UT RA 172 8182 deg R A 11 31 16 4 DEC 33 088802 deg Dec 33 05 20 EQUINOX 2000 a Julian Epoch OBSEPOCH 2015 403645 a Julian Epoch AIRMASS 1 232127 1 airmass HA 316
40. 8 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 Statistics Shift S FWHM mean estimation Ctrl W Background estimation Shift B Math MEF2Single Single2MEF Split MEF Split Single Collapse Cube Create DataSeq 2 Create the output directory for the calibrations then create the calibration pushing Create calibrations button in the main panel 3 When Create calibrations have finished go to Calibration tab and select the directory having the master calibrations created just in setep 2 4 Go to the Setup gt Pre reduction Mode tab and check the option Dark Flat and select the detectors you want to process SG1 SG4 5 Finally select the sequence you want to reduce either selecting one by one the files in the Data List View or selecting the sequence with the Group classification then run Quick reduction from the Pop up menu How do make mosaics with PQL By default PQL proccess or pre reduce only the SG1 detector Q1 and then no mosaic is built However you can go to the Setup Tab and modify in the Detector to reduce combo box the detector s to reduce in case of selecting All or G123 all less SG4 the corresponding mosaic will be generated Currently PAPI aligns and coadds using SWARP or Montage see mosaic engine in config file the images as they are located on the sky to build the mosaic How do make use of parallelisation
41. 8 UT 1448 HIERARCH CAHA AMBI PRESS 778 hPa air pressure day 20150528 UT 1448 HIERARCH CAHA AMBI CLOUD 25 7 cloud sensor day 20150528 UT 1448 COMMENT Linux panic22 3 11 6 4 desktop 1 SMP PREEMPT Wed Oct 30 18 04 56 UTC 20 COMMENT 13 e6d4a27 x86 64 COMMENT Plx API Version 7 10 EOFRM000 58943 164227 s 16 22 23 1642 UTC past midnight EOFRM002 58944 177113 s 16 22 24 1771 1 01289 UTC past midnight END Extensions Header SG1 XTENSION IMAGE IMAGE extension BITPIX 32 number of bits per data pixel AXIS 2 number of data axes AXIS1 2048 length of data axis 1 AXIS2 2048 length of data axis 2 PCOUNT 0 required keyword must 0 GCOUNT 1 required keyword must 1 EXTNAME Q1 HDUVERS I DETSEC 2049 4096 1 2048 px section of DETSIZE DATASEC 5 2044 5 2044 px section of CHIPSIZ PERCT025 2688 2 5 percentile ADU PERCT050 2700 5 percentile ADU PERCT075 2705 7 5 percentile ADU PERCT100 2708 10 percentile ADU PERCT125 2712 12 5 percentile ADU PERCT150 2714 15 percentile ADU 62 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 PERCT175 2716 PERCT200 2718 PERCT225 2720 PERCT250 2723 PERCT275 2725 PERCT300 2726 PERCT325 2728
42. ANIC team and Calar Alto Observatory for their constructive input during the entire development process and the writing of this software 2 15 License Copyright c 2008 2015 IAA CSIC All rights reserved Author Jose M Ibanez Institute of Astrophysics of Andalusia IAA CSIC This file is part of PAPI PANIC Pipeline This program is free software you can redistribute it and or modify it under the terms of the GNU General Public License as published by the Free Software Foundation either version 3 of the License or at your option any later version This program is distributed in the hope that it will be useful but WITHOUT ANY WARRANTY without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE See the GNU General Public License for more details 2 14 Acknowledgments 71 PAPI Documentation Release 1 2 20151020173439 You should have received a copy of the GNU General Public License along with this program If not see lt http www gnu org licenses gt 2 16 Glossary DRP Data Reduction Pipeline IAA Instituto de Astrof sica de Andaluc a observing mode One of the prescribed ways of observing with an instrument OT Observation Tool PANIC Panoramic Near Infrared Camera PAPI PANIC Pipeline Pipeline An automated data reduction pipeline can be defined as a set of contiguous data processing operations designed to automatically transform data from one functional level to another Re
43. E datal PANIC 2015 03 10 Standard Star FS27 March10 0060 fits 6030 568 2377 875 datal PANIC 2015 03 10 Standard Star FS27 March10 0060 fits 3069 276 3096 073 datal PANIC 2015 03 10 Standard Star FS27 March10 0060 fits 3852 473 3223 324 datal PANIC 2015 03 10 Standard Star FS27 March10 0060 fits 3219 446 3060 269 datal PANIC 2015 03 10 Standard Star FS27 March10 0059 fits 6059 874 2386 128 datal PANIC 2015 03 10 Standard Star FS27 March10 0059 fits 3106 257 3151 27 datal PANIC 2015 03 10 Standard Star FS27 March10 0059 fits 3862 996 3222 919 datal PANIC 2015 03 10 Standard Star FS27 March10 0059 fits 3258 566 3099 714 E LI M H PWN H FWHM mean estimation This command computes the FWHM of the selected image using the FWHM_IMAGE value returned by SEx tractor For the computation only stars which fulfill the next requirements are selected 1 not near the edge of the detector 2 elliticiy lt ellipmax default 0 3 26 Chapter 2 Contents STDDEV 8704 104 866 066 4300 289 2335 363 8698 008 49 268 4270 374 2331 496 510 Z 4 PAPI Documentation Release 1 2 20151020173439 area gt minare default 32 pix snr gt snr_min default 5 sextractor flag O the most restrictive ON Un A WX fwhm in range 0 1 20 to avoid outliers For MEF files the application will ask you which detector you want to use for the FWHM estimation Note Itis wo
44. GeirsWritten output_dir data2 out the directory to which the resulting images will be saved temp_dir data2 tmp the directory to which temporal results will be saved verbose True Run parameters run_mode Lazy default initial run mode of the QL it can be None Lazy Prereduce Although all parameteres of the config file SPAPI CONFIG are important some of them have special relevance to the right execution and in the results obtained in bold are default values apply dark flat 01112 remove crosstalk True False nonlinearity apply True False ext calibration db path group by ot group bpm mode none fix grab mode None No BPM file will be read Bad Pixels will be determined using the GainMap obtained from the Superflat or Skyflat During sky filtering the computed bad pixels from gainMap will be replaced with NaNs However in science mode on the first pass of skyfilter and in order to get a good object mask the bad pixels will be replaced by the background level mode grab Read if exists BPM file read bad pixels will be added to bad pixels computed in the gainmap During sky filtering the bad pixels read computed will be replaced by NaN However in science mode on the first pass of skyfilter and in order to get a good object mask the bad pixels will be replaced by the background level mode fix Read if exists BPM file read bad pixels will be repl
45. I 2015 05 28 09 52 15 211 DEBUG reductionset 2415 reduceSet Files generated 1 d xx PAPI 2015 05 28 09 52 15 212 DEBUG reductionset 2416 data2 out mDark Xdb5bc PAPI 2015 05 28 09 52 15 212 DEBUG reductionset 2417 Sequences failed 0 34 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 Reduce all the sequences of a given directory Command Spapi py s my raw_data directory d my output directory With this command the pipeline will reduce all the detected sequences in the my raw_data directory using the default values set in the PAPI_CONFIG file and with the reduction mode specified in reduction mode quick science quick lemon lemon lab However you can specify the reduction mode quick science quick lemon lemon using the M option as follow papi py s my raw data directory d my output directory M quick Reduce all the sequences of a given set of directories If you need to reduce all the sequences of a given set of directories then you should create an script to do that for example see next bash script bin bash Script to reduce a set of directories PAPI SHOME bin papi py CONFIG_FILE PAPI_CONFIG MY DIRS JAN 2015 03 05 2015 03 06 2015 03 07 2015 03 08 2015 03 09 for dir in MY DIRS do if d data2 out dir then mkdir p data2 out dir fi S PAPI c CONFIG FILE s datal PANIC S dir g ot d data2 out
46. LBUF 0 B REFCB2 0 V det bias voltage REFCOLBUF 0 B REFCB3 0 V det bias voltage REFCOLBUF 0 B_REFCB4 0 V det bias voltage REFCOLBUF 0 EMP_A 79 068001 K oly frame 194 08 C EMP_B 79 999001 K Detector 193 C PRESS1 1 0E 05 Pa 1 020e 10 bar pressurel EMPMON 8 ct 4 of temp monitrd 2015 05 28 16 21 loc t EMPMON1 84 508003 K 188 64 C Cold plate EMPMON2 97 056 K 176 09 C Lens Mount 1 EMPMON3 85 961998 K 187 19 C Charcoal EMPMON4 75 846001 K 197 30 C LN2 detector tank EMPMON5 87 633003 K 185 52 C Filter wheel housing EMPMON6 94 026001 K 179 12 C Preamps EMPMON7 79 591003 K 193 56 C LN2 main tank EMPMON8 89 347 K 183 80 C Radiation shield CREATOR GEIRS trunk r737M 13 May 28 2015 16 17 00 Panic COMMENT no comment OBSERVAT CAHA Calar Alto Almeria Andalucia Spain panic ca OPCYCL 9 Operation cycle number OPDATE 2015 04 28T15 16 00 UT date of operation cycle start NTCYCL 7 Mounting cycle number NTDATE 2015 01 29T15 00 00 UT date of mounting cycle start HIERARCH CAHA AMBI WINSP 4 5 m s wind speed day 20150528 UT 1448 HIERARCH CAHA AMBI WINDIR 149 deg wind direction day 20150528 UT 1448 HIERARCH CAHA AMBI TEMP 15 2 C temperature day 20150528 UT 1448 HIERARCH CAHA AMBI HUMI 46 rel humidity day 20150528 UT 1448 HIERARCH CAHA AMBI DEWP 3 7 C dew point day 2015052
47. MP 1 ct software multisampling COADDS 1 ct effective coadds total EXPTIME 2 739931 s total integ time FRAMENUM 1 of 1 saved SKYFRAME unknown DETSEC 1 4096 1 4096 px xrange and yrange of window DATASEC y px xy range of science data DETSIZE 1 4096 1 4096 px full size of the 4 detector mosaic CHIPSIZX 2048 px single chip pixels in x CHIPSIZY 2048 px single chip pixels in y DETROT90 O ct 90 deg SW image cw rotations DETXYFLI O 1 SW image flip 1 RightLeft 2 UpDown B EXT1 2 679688 V external bias 2744 B EXT2 2 679688 V external bias 2744 B EXT3 2 679688 V external bias 2744 B EXTA 2 679688 V external bias 2744 B_DSUB1 1 569727 V det bias voltage DSUB 3420 B_DSUB2 1 569727 V det bias voltage DSUB 3420 B_DSUB3 1 569727 V det bias voltage DSUB 3420 B_DSUB4 1 569727 V det bias voltage DSUB 3420 B_VREST1 1 07999 V det bias voltage VRESET 2353 B_VREST2 1 07999 V det bias voltage VRESET 2353 B_VREST3 1 07999 V det bias voltage VRESET 2353 B_VREST4 1 07999 V det bias voltage VRESET 2353 B_VBIAG1 2 199707 V det bias voltage VBIASGATE 3604 B_VBIAG2 2 199707 V det bias voltage VBIASGATE 3604 B_VBIAG3 2 199707 V det bias voltage VBIASGATE 3604 B_VBIAG4 2 199707 V det bias voltage VBIASGATE 3604 B_VNBIA1 0 V det bias voltage VNBIAS 0 B_VNBIA2 0 V det bias voltage VNBIAS 0 B
48. PAPI Documentation Release 1 2 20151020173439 J M Ib ez Mengual IAA CSIC October 26 2015 CONTENTS 1 Caveat 3 2 Contents 5 2 1 Installation amp Contiguration ees 2 n noe A wa he ee a A 5 22 PANIC Quick Look Tool PQL 0 ee 7 2 9 PAPI duh ee eee Oe BAe OWS E a ee X be EA ee ox e UR ee PURA 29 2A lmaseSelectiont 4 5 24 4 6 dope aos Pee Peek bd baw be ee s abet de S BA 44 25 Datasset Classification lt sos eme wc A Ge we ee RR ee ee a EAE ROS ee Re 44 26 Jdmageselecton pe ud des Bed dete A ex se ede e d 44 2 7 DatassetclassificatlOn oum pois Roo mE SOR ex RR FOX OR ROR 8 16 180 QR E 44 2 00 JPAPIR fer nce 4 i212 4 4 54 49 see veux he eS DHE a Y oe aS 46 2 95 Data formats p ue uou det eos la Ert ERU A Buell aioe Seded eed A dd 58 2 10 Processing description vs lor eed a A S RE RR SE 64 2 HT SUD TP ew AG BS Ro ee BS A ee De 1A 68 2 12 PAPEBAQ lt cete baw whe Pe ee eRe bee ee A EE RES 68 2 13 Troubleshooting sa os a SOG Ee Gey SL Ee e e do a a 69 214 Acknowledgments 21k oe ox n om Ro pov x a ee baa Be al e OH 71 PEE D 71 2 16 Glossary e g he ko hE Pa else ee PS eR 18 96 28 dos edel ee ae RUNE E BUR 72 3 Citation 73 4 Indices and tables 75 Python Module Index 77 Python Module Index 79 Index 81 PAPI Documentation Release 1 2 20151020173439 Release 1 2 Date October 20 2015 Warning This Documentation is still a work in progress some of the material is not organized and s
49. PAPI treat bad pixels See Bad Pixel Treatment How can we reach hundredth of magnitude accuracy in photometry The best way to accurately photometrically calibrate PANIC images is to use 2MASS stars in the field itself to derive the photometric solution The accuracy strongly depends on the number of bright 2MASS stars within the filed of view but ranges from a few 1 100th of a magnitude to 0 1 magnitudes if only faint stars are contained in the field Additionally observing supplementary standard star fields can be asked for when preparing the observations To perform the 2MASS photometric calibration on an image you should use the photometry command as follow gt photometry i directory prereducedField fits o test pdf How good is PAPI astrometry and how are PSF variations corrected At present the pipeline applies a correction for PSF distortions based on a distortion map derived during the astrometric calibration done with SCAMP a software developped by Emmanuel Bertin and 2MASS 2 13 Troubleshooting This section gives a description of some problems that can be obtained and how they can be solved 2 13 1 Why do get a command not found error when trying to run one of the PAPI modules If you are trying to run the command from the same directory as the executable i e SHOME bin make sure that 9999 your path contains If you are trying to run the command from another directory add HOME bin to your p
50. Q123 all except SG4 all default all Example papi py s my raw data directory d my output directory W Q1 2 3 3 Reduction modes PAPI currectly supports next reduction modes quick default single pass for sky background subtraction science double pass for sky background subtraction quick lemon single pass for sky background and neither alignment nor coadd is done Jemon double pass for sky background subtraction and neither alignment nor coadd is done lab for laboratory purposes For more details see Processing description 2 3 4 How NOT to use PAPI PAPI uses a strictly linear approach for data reduction which makes for easy and transparent processing And you have to stick to that It is usually not possible to take data that has been processed half way by other software and do the rest in PAPI FITS headers will not be understood naming conventions not met and data structures totally incompatible 2 3 5 Configuration files PAPI has a set of configuration files required to run properly They are the next ones papi cfg main configuration file In addition to the command line options PAPI has a configuration file in which the user can set both the command line options and a wider set of additional ones This config file can be specified with the c option but by default it is looked for it in the config files directory defined by PAPI_CONFIG environment variable scamp cfg SCAMP confi
51. RCE FILE LIST list of input optionally corrected dome ON and OFF flat images o OUTPUT FILENAME output OUTPUT FILENAME The output bad pixel mask L LTHR lthr LTHR he low rejection threshold in units of sigma default 20 H HTHR hthr HTHR he high rejection threshold in units of sigma default 20 D MASTER DARK master dark MASTER DARK Optional Master dark frame to subtract S show stats Show statistics default False v verbose verbose mode default Example calBPM py s tmp domesF txt D tmp masterDark fits o tmp masterBPM pl calCombineFF Combine a master dome Flat field and a master sky Flat field into a combined master Flat field The procedure followed is The procedure for taking advantage of the facts that the large scale flat field variation of the dark sky flat match that of the program frames and the dome flats have very high S N in each pixel goes as follows a Median smooth the combined dark sky flat this improves the S N and preserves the large scale features of the flat b Median smooth the combined dome flats using the same filter size as was used for the dark sky flat c Divide the combined dome flat by it s median smoothed version The result is a frame that is flat on large scales but contains all the high spatial frequency flat field information d Now multiply the smoothed dark sky frame and th
52. SG4 SG123 All Calibrations panel This panel allows the user to set some of values for the search of master calibration files Main Setup Calibrations Logs SetCalbeDi raatazmastersz lt Select a directory or log file f d files location Master Dark l ut m TExp s Master Flat ta2 o Flat J ejbE o M Filter Master BEM data1 Calibs mBPM_UR_01 01 mef fi None Action L fit unknown Readout Mode Master NLC Set Calibs Dir Pushing this button the user select the additional external directory from which the QL will look for master calibration files Normally it is used to provide to the QL with additional calibrations dark flat from previous nights Master calibrations found in the output directory will have higher priority than those ones This directory is also called external calibration in PAPI command line C EXT CALIBRATION DB ext calibration db EXT CALIBRATION DB External calibration directory library of Dark amp Flat calibrations 16 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 Or ext_calibration_db in the config file Then if during the reduction of a ReductionSet RS no calibrations dark flat are found in the current RS then PAPI will look for them into this directory If the directory does not exists or no calibration are found then no calibrations will be used for the data reduction Note that the cali
53. a rotator angles for your observations The first effort at characterizing and removing the cross talks made use of the Medamp technique By this we mean isolating then subtracting what is common to all 32 amplifiers This effectively seems to remove the edge and negative cross talks which both affect all 32 amplifiers But it does not remove the positive crosstalk Note that the assumption is that the amplitude of the edge and negative cross talks is the same ona 11 32 channels We tried inconclusively to prove disprove that assumption If amplifier dependant the amplitude variations must be less than 10 2 10 Processing description 67 PAPI Documentation Release 1 2 20151020173439 We experimented doing the medamp at various stages of the processing and found the best results when removing the crosstalk as the very last step after sky subtraction Rigorously it should actually be the very first step since crosstalk effects are produced in the very last stages of image generation The module used to correct the crosstalk is dxtalk py py in adition the crosstalk correction can be enable in the configuration file PAPI_CONFIG setting in the general section the keyword remove_crosstalk True Extended Objects If your targets are really extended and or very faint then you should seriously consider observing blank SKY fields They will be recognized and automatically used in the correct manner once identified by PAPI No addi
54. aced by bi linear interpolation during dark and flat fielding and before sky filtering sky subtraction During sky filtering bad pixels computed in the gainMap will be replaced with the background level The prefered mode is grab whether you want to use a BPM file or none if you do not 2 7 1 Getting PAPI Data PAPI requires the full set of uncalibrated data products and best reference files for each observation in the input image set These files can be readily obtained through the CAHA archive When requesting data from CAHA you need to specify nstrument PANIC Science Files Requested Uncalibrated Raw 2 7 Data set classification 45 PAPI Documentation Release 1 2 20151020173439 Reference Files Advanced Data Products CALAR ALTO OBSERVATORY ARCHIVE Accessing to Calar Alto Observatory Archive Please use your account to log in Username Lagin gt gt Password New Account Forgot your account CAHA A E 2011 2 7 2 Caveats As we stated previously PAPI was developed primarily for reducing NIR imaging data of any kind of sources galactic extragalactic coarse or crowed fields and extended objects Here are some tips for reducing each types of data Coarse fields Crowded fields Extended objects Add tips here 2 8 PAPI Reference Release 1 2 Date October 20 2015 Warning This Reference is still a work in progress some of the material is not organized and
55. age list INPUT IMAGE LIST o OUTPUT FILE output file OUTPUT FILE output filename default tmp out fits input image list to collapse into a single 2D image Example collapse i data mycube fits o data anymore a cube fits collapse l data list txt o data anymore a cube fits checkQuality The checkQuality module computes some initial image quality estimations using SExtractor eval focus series The eval focus series module computes the best focus estimation for a focus exposure series It is done according to the FWHM value estimated for each frame fitting a curve the the values pair values FWHM focus and finding out the minimun Requirements T FOCUS telescope focus keyword value present in the header Raw Images with enought number of stars A series of images taken with covering a range of telescope focus values including the best focus value 2 8 PAPI Reference 57 PAPI Documentation Release 1 2 20151020173439 genLogsheet health imtrim modFits skyfilter The skyfilter module uses the external package irdr_skyfilter to perform the sky background sub traction from a dither sequence of science frames It works with almost all kind of dither sequences even with sequences used for extended objects T S T S T T S T T S T T For more details on skyfilter see the Appendix section skyfilter spatial noise 2 9 D
56. alog reference catalog used for the calibration 2MASS USNO B1 GSC 2 2 SDSS R5 2 Astrometric Engine which tool you want to use to the astrometric calibration SCAMP or Astrometry net 3 Detect threshold the SExtractor threshold to be used to detect sources Display Here you can select which files are displayed automatically in the DS9 You have next options Only results default only FITS files created in the output directory as result of some processing Only new files only new FITS files detected in the input directory 2 2 PANIC Quick Look Tool PQL 15 PAPI Documentation Release 1 2 20151020173439 All files both new files detected in the input directory and the results in the output directry None no files will be displayed Lazy mode Under this box the user can select the operations to be executed when the Lazy Mode is activated in PQL Cur rectly the available and exclusive operations are Apply Dark Flat BPM Subtract Last Frame science Subtract Nearest Sky Pre reduction Under this box the user can select the operations to be executed when the Pre reduction Mode is activated in PQL Currectly the available and exclusive operations are Apply Dark and FlatField Apply SuperFlat default Apply BPM Bad Pixel Map Correct for non linearity Select the number of frames to computer the sky bacground 1 5 default 4 Detector to reduce SG1 default SG2 SG3
57. ata formats This section gives a description of the raw data produced by PANIC and how they are organized However for a deeper description see the GEIRS manual 2 9 1 Detector The detector is an array FPA of four HAWAII 2RG detectors The inter chip gap between the detectors is only 167 pixels or 75 arcsec at the 2 2m telescope and is filled by dithering with sufficient amplitude For applications which image only 30x30 arcmin this design is ideal HAWAII 2RG detectors have an effective surface of 2040x2040 sensitive pixels A 4 pixel wide border is used as reference to correct for relatively slow bias drifts 2 9 2 FITS GEIRS the software part in charge of the data acquisition and saving is capable of saving the frames in different FITS Flexible Image Transport System formats integrated FITS cubes MEF etc Next ones are available in the Observation Tool OT when an OP Observing Program is defined Multi Extension FITS MEF Integrated Multi Extension FITS MEF Cube Integrated All SEF Integrated FITS Cube SEF Cube ndividual SEF Individual However PAPI does not accept any kind of FITS data files available in GEIRS only the configured in the OT except Individual As result PAPI accepts the next type of FITS files in order of preference Integrated Multi Extension FITS MEF a unique FITS file with four extensions MEF where each extension corresponds to one of the 4 images produ
58. ath 2 13 2 PAPI does not recognize a group of files as a well formed sequence It maybe because the sequence is missing some files of the sequence or some of them have no proper headers ie they were not observed with OT Solutions You can try to reduce the sequence using the groping mode filter instead of ot It means PAPI will not use any OT header information but RA DEC DATE OBS and FILTER 2 13 Troubleshooting 69 PAPI Documentation Release 1 2 20151020173439 Use the Create Data Seq option which will create fix a new sequence adding the required header keywords Statistics Shift S FWHM mean estimation Ctrl W Background estimation Shift B Math MEF2Single Single2MEF Split MEF Split Single Collapse Cube Create DataSeq 2 13 3 PAPI can not run the astrometric calibration Verify that You have cdsclient installed You have an internet connection and no firewall is blocking CDSClient You can config CDSClient to run with a proxy 2 13 4 PAPI can not run the photometric calibration Verify that You have an Internet connection It is used to query the on line 2MASS or USNO B catalog Check that the input reduced image is astrometrically calibrated and it the filter name FILTER keyword match some of the 2MASS or USNO B catalog 2 13 5 PAPI says some files in the input file list does not look a FITS file Verify that The input file is not ending wit
59. ause of the classificaton algorithm that is implemented in PAPI and that allow an automatic identification of the data sets grouping the files according to the observation definition with the OT 1 The data grouping algorithm 2 Sky finding algorithm for extended objects In case of not using the OT during the observation also a data grouping is possible althouth with some limitations Let s see how it works Data Preparation Firstly each FITS file is linearity corrected if it was enabled in the configuration file nonlinearity apply If integrations where done with repetitions gt 1 and saved as a cube with N planes then the FITS cube is collapsed doing a simple arithmetich sum of N planes Then the image is divided into the number of chips in the FPA which constitutes 4 chips in a mosaic From this step on the pipeline works on individual chips rather than whole images thereby enhancing the speed and enabling us to do multi chip processing on multi CPUs Calibrations In next sections we describe the main calibration to be done by PAPI 2 10 2 Computing the master dark TBC 2 10 3 Computing the master flat field TBC 2 10 4 Computing the Bad Pixel Mask The map of all bad pixels hot low QE are derived from the non linearity tests However also the nonlinearity analysis provides a list of non correctable pixels which always will be considered invalid So currently there is no procedure in PAPI to compute
60. aved astrowarp The ast rowarp module performs the alignment and warping of a set of images in principle previously reduced but not mandatory The module uses the Astromatic packages sextractor scamp and swarp to accomplish this task Usage Options h help show this help message and exit c CONFIG FILE config file CONFIG FILE config file s SOURCE FILE source SOURCE FILE Source file list of data frames It can be a file or directo o OUTPUT FILENAME output OUTPUT FILENAME final coadded output image v verbose verbose mode default Example astrowarp py c papi cfg s tmp test files txt o tmp astrowarp fits CalBPM This module creates a master Bad Pixel Map pl iraf file from a set of dome on and off flats The algorithm followed to create the BPM is the next 1 Classify split the frames in 3 sets DOME FLAT LAMP ON DOME FLAT LAMP OFF DARKS and and check whether there are enough calib frames Check the master dark Texp Subtract the master dark to each dome flat Combine dome dark subtracted flats on off Pu spe 03 BS Compute flat low flat high 6 Create BPM iraf ccdmask Usage 2 8 PAPI Reference 49 ry name PAPI Documentation Release 1 2 20151020173439 Options h help show this help message and exit s SOURCE FILE LIST source SOU
61. better com blog 2009 10 21 better ways t PHT HEE FE FE EE H H AE HEE EE HE AE E EE EE EE H AE AE H AE H HEHE FE H H AE HE 4 HEE FE AE EE EE EEE PEE FE AE H HOME environment variable File containing the File containing the File containing the SCAMP configuration SWarp configuration fi dd dd IN_RRR MPIA_01 01 fits badpixels th the BPM ixels 0 bad pixels gt 0 to NaN the bad pixels good_pixels FE EAE FE AE E FE AE FE FE AE FE FE AE FE FE AE FE FE AE FE FE AE FE FE AE FE AE FE FE AE AE FE AE AE FE AE FE 5555555555555 th a bi linear interpolation from nearby pixels It will be taken 0 REE FE H HE AE FE FE AE FE H FE FE H FE FE FEAE H H FE AE E FE HEE E H ER FE FE EE EEE EE EE EEE EE EEE ES it specifying in only those images SExtractor configuration file list of paran neutal netwot lter definj fil fil e e LE j make larg heters that w rk weights fo ition 40 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 whose object name matchs one of the names listed in this parameter will be considered when generating the master dark equivalent to saying do not filter images by their object names object_names dark check_prop if true the dark frames used to build the master dark will be checkd to have the same acquisition properties
62. brations into the current RS have always higher priority than the ones in the external calibration directory Load last When this button is pushed the most recent master calibration files found in output directory and external cali brations are shown in the fields below If Use as default is click checked then the displayed files will be used as default calibrations when Apply Dark_FlatField_BPM is run Otherwise Apply Dark_FlatField_BPM routine will ask the user for the master calibration files to be used Log panel It is an extension or duplicate of the Even Log window of the main panel but with a wider area for messages 2 2 6 Event log window The event log window shows important events and results generated by PQL For example the filename of the new files generated are shown or the error produced while the processing of some sequence This window is used only as output and you cannot type any command on it Event log Time UTC 2 2 7 Pop up menu It is a context pop up menu that appears when the user select a file or a set of them in the Data List View and click the right mouse button Next figure shows the options of that pop up menu 2 2 PANIC Quick Look Tool PQL 17 PAPI Documentation Release 1 2 20151020173439 Display Image Ctrl D Image Info Ctrl H Copy files to clipboard Ctrl C Copy files to text file Shift T Show Dither pattern Ctrl P Build Master Dark Ctrl M Build Master Dome Flat Ctrl F Bui
63. calTwFlat This module receives a series of FITS images twilight flats and a master dark model and creates the master twilight flat field Usage Options h help show this help message and exit s SOURCE FILE LIST source SOURCE FILE LIST Source file list of data frames It can be a file or directory name d MASTER DARK master dark model MASTER DARK Master dark model to subtract each raw flat it will be scaled by TEXP o OUTPUT FILENAME output OUTPUT FILENAME final coadded output image b MASTER BPM master bpm MASTER BPM Bad pixel mask to be used optional n normalize normalize master flat by median If image is multi detector then normaliz m median smooth Median smooth the combined flat field default False L MINLEVEL low MINLEVEL flats with median level bellow default 1000 are rejected H MAXLEVEL high MAXLEVEL flats with median level abov default 100000 are rejected v verbose verbose mode default Example 52 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 calTwFlat py s tmp twflats txt d tmp darkModel fits o tmp masterTF fits n calGainMap The calGainMap module creates a master gain map from a master flat field dome twilight or superflat NOT normalized and previously created The flatfield will be nor
64. ced by the single detector chips If the number of coadd NCOADDS is gt 0 then they will be integrated arithmetic sum in a single image This is the default and more common saving mode used in fact it is the default and more wished saving mode This mode will also be used when the software or hardware sub windowing is set and the integrated option is selected Then there will be an extension for each sub window 58 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 Non integrated Multi Extension FITS MEF a unique FITS file with four extensions MEF one per each detector or window having each extension N planes where N is the number of coadds NCOADDS ie a cube of N planes This mode will be also used when the software or hardware subwindowing is set up and the no integrated option is selected Single integrated FITS file the four detectors are saved in single file and in a single exten sion FITS image SEF If the number of coadds NCOADDS is gt 0 then they are integrated arithmetic sum in a single frame Single non integrated FITS cube the four detectors are saved in a single extension FITS SEF file and each individual exposition in a plane layer of a cube It means N planes where N is the number of coadds or expositions Note Currently PAPI is not working with non integrated individual files of an exposition In case you are interested in no integrated files and wish to reduce the
65. cipe A software object that processes the data obtained with a given observing mode of the instrument 72 Chapter 2 Contents CHAPTER THREE CITATION If your research uses PAPI we d appreciate it if you could acknowledge the fact by including the following citation This research made use of PAPI the pipeline of PANIC instrument It is funded by the Spanish Ministry of Econ omy and Competitiveness with funds from the European Union FEDER and the Spanish national budget through the grants ICTS 2006 15 ICTS 2007 10 ICTS 2008 24 ICTS 2009 32 and the project Intramural 200450E458 of the Spanish National Research Council Ib ez Mengual J M Fern ndez M Rodr guez G mez J R Garc a Segura A J Storz C The PANIC software system Proc SPIE 7740 77402E 2010 e Ib ez Mengual J M Garcia A J Storz C Fried J W Fernandez M Rodriguez J F Advanced PANIC quick look tool using Python Proc SPIE 8451 2012 73 PAPI Documentation Release 1 2 20151020173439 74 Chapter 3 Citation CHAPTER FOUR INDICES AND TABLES genindex modindex search Documentation last updated on October 20 2015 75 PAPI Documentation Release 1 2 20151020173439 76 Chapter 4 Indices and tables PYTHON MODULE INDEX r reduce 46 77 PAPI Documentation Release 1 2 20151020173439 78 Python Module Index PYTHON MODULE INDEX r reduce 46
66. ctionset 1167 SEQUENCE 10 TYPE KY FLAT FI PAPI 2015 05 28 09 18 01 494 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 495 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0009_ PAPI 2015 05 28 09 18 01 495 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0010_ PAPI 2015 05 28 09 18 01 495 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0011_ PAPI 2015 05 28 09 18 01 495 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0012_ PAPI 2015 05 28 09 18 01 495 DEBUG reductionset 1167 SEQUENCE 11 TYPE KY_FLAT FI PAPI 2015 05 28 09 18 01 495 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 495 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0013_ PAPI 2015 05 28 09 18 01 495 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0014_ PAPI 2015 05 28 09 18 01 496 DEBUG reductionset 1167 SEQUENCE 12 TYPE KY_FLAT FI PAPI 2015 05 28 09 18 01 496 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 496 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0015_ PAPI 2015 05 28 09 18 01 496 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0016_ PAPI 2015 05 28 09 18 01 496 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0017_ PAPI 2015 05 28 09 18 01 496 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0018_ PAPI 2015 05 28 09 18 01 496 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0019_ PAPI 2015 05 28 09 18 01 497 DEBUG reductionset 1170 data2 2015 03 10 sky
67. cumentation Release 1 2 20151020173439 Setup Panel This panel allows the user to set some of the parameters used for the processing It is divided into six group boxes as shown in next figure Main Setup Calibrations Logs Common Settings Y Data Grouping Astrometry X Check header RA Dec offsets Jarcsecs Catalog Detect Threshold X Create Log file Time offsets secs 2MASS EE Temporary Dir Max num files files Astrometric Engine AstrometryNet x OT Keywords 1 Display ds9 c Pre Reduction Mode Apply DARK FLAT x Make Astrometry x Apply SuperFlat x Quality measurements background psf Lazy Mode Apply BPM x Build stacked image o Correct for non linearity Apply DARK FLAT BPM sky nearest frames Detector to reduce Subtract Last Frame science 4frames z SG1 c Subtract Nearest Sky Common Settings In this group you can set the next parameters 1 Check header 2 Create log file 3 Temporary directory Data grouping It contains some parameters used for the data grouping when any OT keywords are present in that case PQL will try to group the files follwing the near in sky and time criterion 1 RA Dec offsets 2 Time offsets 3 Max number of files If OT keywords are present then check box OT should be ckecked default mode Astrometry In this group you can set some parameters related with the astrometric calibration done during the processing 1 Cat
68. dian Method used to compute the dither offsets only for 1st pass wcs using the astrometric calibration and coordinates of the center of the images cross correlation no astrometric calibration required use irdr offsets cross reference offset algorithm For big offsets and sparse poor fields it not recommended Note for the object mask registering in the 2nd pass of skysub wcs is the method always used hard coded method wcs method cross correlation single_point If true means that the SEextractor objmask will be reduced to a single point centroid to run the cross reference offset algorithm i e each object is represented by a single one valued pixel located at the coordinates specified by its X_IMAGE and Y_IMAGE parameters in the SExtractor catalog It is done mainly to avoid problems with large object masks extended objtects satured objects etc that make the cross reference algorithm too slow and even might with wrong results single_point False Object mask mask_minarea 10 sex DETECT_MINAREA min of pixels above threshold mask maxarea 0 sex DETECT MAXAREA SExtractor gt 2 19 5 max of pixels above threshold O unlimited mask thresh 2 5 5 0 sex DDETECT THRESH used for object masking Sex SATUR LEVEL level in ADUs for a single exposure image at which the pixel arises saturation Note than that value should be updated with NCOADDS or NDIT keyword
69. e reduction but the processing stops after the 2nd sky subtraction and no final co added image is produced It is useful for LEMON processing for light curves Last file received This field shows the last file received detected by PQL Buttons Subract last2 button It will produced a new image as result of the subtraction of last two images received Create calibrations button This button will start the processing of all the calibration sequences received in the input directory As result a list of master calibrations combined darks or flats will be generated in the output directory START button This button starts the processing of all the sequences received You will be asked whether to proccess all the current images or only the new ones As result a list of master calibrations and science calibrated images will be generated in the output directory Add button This button allows to add manually a single file to the Data List View from wherever the file is Remove button This button removes manually from the Data List View the currently selected file but it does not remove neither from the local database nor the file system Clear All button It removes all the current files from the Data List View but they will not be removed from the file system As result it will empty the Data List View until a new input directory is selected or a new file is detected in the current one 14 Chapter 2 Contents PAPI Do
70. e result of the division in the previous step As result a flat field with the low spatial frequency properties of the dark sky flat combined with the high S N high spatial frequency properties of the dome flat is obtained Usage calCombineFF py options argl arg2 Module to combine a dome Flat field and a sky Flat field Options version show program s version number and exit h help show this help message and exit d DOMEFF domeFF DOMEFF input dome Flat Field s SKYFF SkyFF SKYFF input sky Flat Field o OUTPUT IMAGE output OUTPUT IMAGE output filename of combined Flat Field default combinedFF fits Example calCombineFF py d data masterDF fits s data masterSF fits o data masterFF fits calDark The calDark module receives a series of FITS images master darks and create the master dark and computer several statistics 50 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 Usage Usage calDark py options argl arg2 Options h help show this help message and exit s SOURCE FILE LIST source SOURCE FILE LIST Source file listing the filenames of dark frames o OUTPUT FILENAME output OUTPUT FILENAME final coadded output image n normalize normalize master dark to 1 sec default False e scale scale raw frames by TEXP default False S show stats Show frame stats
71. e subtracted it is supposed to be done by the skysubtraction some people think they are not required apply_dark_flat 1 some other values really required Maximum seconds max_mjd_diff 900 10min 600secs aprox superflat of temporal distant allowed between two consecutive frame allowed for two consecutives frames into a max_ra_dec_offset 2602 Maximum distance arcsecs max num files 50 Maximum number of files allowed in a sequence only for filter pix scale 0 45 4 default pixel scale of the images equinox 2000 equinox in years radecsys ICRS reference system pattern x fits if specified only those images that match the pattern accordj considered when autodetecting FITS images in directories no ranges expressed with will be correctly matched NOTE it i or discarded specify its type at the beginning of filter name Z Z the key stored in the FITS header when the filter is Z filter name Y Y filter name J J filter name H H Filter H admits list of strings if multiple values are possi filter name K K filter name Ks KS Coadd mode 2nd pass ie final coadd dithercubemean swarp dithercubemean it uses the irdr dithercubemean routine and then requires image offsets computed with offsets c swarp it uses the astrometric calibration to register the images with SWARP it is more time consuming than cubemean due to it
72. eadmode gt _ lt version gt fits The FITS file has a primary header with no data and two data extensions for each detector They are labeled LINMAX lt i gt and LINPOLY lt i gt with i 1 4 being the quadrant index numbered similar to the scheme for MEF data files from GEIRS Note that the indices do not necessarily correspond to SG hardware IDs which are written in the header instead The extension LINMAX lt i gt is a 32bit float 2048x2048 data array containing the maximum correctable signal for each detector Uncorrectable pixels have a NaN instead of a numerical value The extension and LINPOLY lt i gt is a 32bit float 2048x2048x4 data cube containing the polynomial coefficients c 1 4 in reverse order The first slice in the cube is c 4 the second c 3 etc The module used to correct the non linearity is correctNonLinearity py in adition the non linearity correction can be enable in the configuration file PAPI_CONFIG setting in the nonlinearity section the keyword apply True Crosstalk HAWAII2 sensors with multiple parallel readout sections can show crosstalk in form of compact positive and negative ghost images whose amplitude varies between readout sections PAPI has a optional de crosstalk module that assumes that the amplitude is the same therefore the correction will only partially remove the effect if at all If you know in advance that this will be a problem for your science case then consider choosing different camer
73. elp open save header page setup print exit NOAD IRAF PC IRAF Revision 2 16 1 EXPORT Mon Oct 14 21 40 13 MST 2013 This is the EXPORT version of IRAF V2 16 supporting PC systems Welcome to IRAF To list the available commands tupe or To get detailed information about a command type comand or load a package type its name Type package or logout to get out of the CL Type news to find out what is neu in the version of the system you are using Visit http iraf net if you have questions or to report problems Using global login file home panic iraf login cl Initializing SAMP No Hub Available The following commands or packages are currently defined c Select stars to mensure uith h and Hue with q Additional options are g 4 When you have finished of selecting all the stars you want for the focus evaluation type q 5 Then an IRAF interactive graphics with the first fit will appear and the best focus obtained On that 22 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 graphics you should remove the images stars focus points that you consider are not good for the focus evaluation outliers for this type x and then i s f p Type u to undo the removing of the outliers If you need more info about this commands see starfocus Starfocus Cursor Commands When selecting objects with the image cursor the following comma
74. em for your science case then consider choosing different camera rotator angles for your observations It can be activated or deactivated in the Main config file remove crosstalk TruelFalse Usage Options h help i INPUT IMAGE o OUTPUT IMAGE O overwrit show this help message and exit input image INPUT IMAGE input image to remove crosstalk output OUTPUT IMAGE output filename default dxtalk fits overwrite the original image with the corrected on Example dxtalk py i dxtalk py i tmp pruebaDC fits O tmp pruebaDC fits o tmp pruebaDC dx fits makeobjmask Creates object masks SExtractor OBJECTS images for a list of FITS images or a single FITS image Expects the command sex SExtractor Version 24 in path If weight maps exist they will be used assume weight map 2 8 PAPI Reference 53 name requi t 5 the gainmap PAPI Documentation Release 1 2 20151020173439 filename given by replacing fits with weight fits The module can produce single poing masks i e a single pixel set to 1 per each detected object if single poing option is true Usage Options h help show this help message and exit s INPUTFILE file INPUTFILE It can be a source f
75. ercent 67 5 percen 70 percent 72 5 percen 75 percent 77 5 percen 80 percent 82 5 percen 85 percent 87 5 percen 90 percent 92 5 percentile A 95 percentile AD 97 5 percentile ADU deg R A 22 09 28 2 deg Dec 51 05 04 arcsec px WCS units along axis 1 WCS units along axis 2 WCS axis 1 WCS axis 2 deg RA in mosaic center deg DEC in mosaic center deg px WCS matrix diagonal deg px WCS matrix diagonal deg px WCS matrix outer diagonal deg px WCS matrix outer diagonal px RA and DEC center along axis 1 px RA and DEC center along axis 2 lower right SW chip CHIPGAPY 167 north 90 deg Cc Y c Oa OU U aoe U c QU c U G IJ QU G UG Uc QU c U aU ou 2 9 4 Observation Tool keywords Next keywords are automatically added to the FITS header by the PANIC Observation Tool OT as each file is created If these are not saved neither PAPI nor PQL will work correctly OBS TOOL PROG ID VOT Visi Y i PANIC Observing Tool Software version PANIC Observing Program ID 2 9 Data formats 63 PAPI Documentation Release 1 2 20151020173439 OB_ID 6 PANIC Observing Block ID OB_NAME OB CU Cnc Ks 2 PANIC Observing Block Name OB PAT 5 point PANIC Observing Block Pattern Type PAT NAME OS Ks 2 PANIC Observing Secuence Pattern Nam PAT EXPN 1 PANIC Pattern exposition number
76. everal aspects of PAPI are not yet covered with sufficient detail Welcome This is the Documentation for PAPI the Data Reduction Pipeline of PANIC instrument and for PANIC Quick Look Tool PQL the GUI of PAPI used during the observation all processing routines used by PQL are implented in PAPI PAPI is the automatic image processing pipeline for data taken with the PAnoramic Near Infrared Camera PANIC for the 2 2m and 3 5m Telescopes at Calar Alto Observatory CAHA The pipeline is written in Python and developed at the Institute of Astrophysics of Andalusia CSIC The automated processing steps include basic calibration removeing instrumental signature cosmic ray removal treatment for electronic ghosts cross talk sky subtraction non linear count rate correction robust alignment and registration This manual is a complete description of the data reduction recipes implemented by the PAPI pipeline showing the status of the current pipeline version and describing data reduction process of the PANIC data using PAPI Although PAPI was developed for the PANIC camera its development was initiated using data from Omega2000 camera at the 3 5m CAHA telescope and HAWK I camera at the VLT thus in principle it should work with data these instruments although it is not optimized for them In addition to this html version of the manual there is also a pdf version to download Development Jos Miguel Ib tiez Mengual IAA CSIC Contr
77. flat observations a bad pixel map an various statistics Usage 2 8 PAPI Reference 51 PAPI Documentation Release 1 2 20151020173439 Options h help show this help message and exit s SOURCE FILE LIST source SOURCE FILE LIST Source file list of data frames It can be a file or directory name o OUTPUT FILENAME output OUTPUT FILENAME final coadded output image n normalize normalize master flat by median If image is multi detector m median smooth Median smooth the combined flat field default False v verbose verbose mode default Example calDomeFlat s tmp domeFlats txt o tmp masterDF fts n calSuperFlat The calSuperFlat module creates a master super flat field from science observations a bad pixel map an various statistics Usage Options h help show this help message and exit s SOURCE FILE LIST source SOURCE FILE LIST Source file list of data frames It has to be a fullpath fil name o OUTPUT FILENAME output OUTPUT FILENAME output file to write SuperFlat b BPM bpm BPM bad pixel map file default none N norm normalize output SuperFlat If image is multi chip normalization wrt chij m median smooth Median smooth the combined flat field default False Example calSuperFlat py s tmp test files txt o tmp superFlat fits N
78. flats_dawn_0020_ PAPI 2015 05 28 09 18 01 497 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0021_ PAPI 2015 05 28 09 18 01 497 DEBUG reductionset 1167 SEQUENCE 13 TYPE KY_FLAT FI PAPI 2015 05 28 09 18 01 497 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 497 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0022_ PAPI 2015 05 28 09 18 01 497 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0023_ PAPI 2015 05 28 09 18 01 497 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0024_ PAPI 2015 05 28 09 18 01 497 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0025_ PAPI 2015 05 28 09 18 01 498 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0026_ PAPI 2015 05 28 09 18 01 498 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0027_ PAPI 2015 05 28 09 18 01 498 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dawn_0028_ PAPI 2015 05 28 09 18 01 498 DEBUG reductionset 1167 SEQUENCE 14 TYPE DARK FILTER 2 3 PAPI 33 PAPI Documentation Release 1 2 20151020173439 Show grouped files per filter and coordinates of a raw directory Command papi py s my raw data directory g filter p Reduce quick a specificied number of sequences of the group list To reduce the sequneces from N1 to N2 from the group list obtained with a p command you have to use the S parameter with two val
79. get field FWHM of given stars getDarks Gives the unique values of read mode itime ncoadd save mode of a set of files of a given directory Used to know the DARKS required from them getImageOffsets Gives the image offsets arcsecs based on the WCS of the image headers papi Description The papi module see PAP is the main PAPI module to run the data reduction It starts by creating a subdirectory in the output dir directory using the name give on the command line or in the PAPI_CONFIG file Within the run directory a Q1 Q4 subdirectories one for each detector will be created The temporal files will be saved and deleted at the end in the temp dir directory Syntax Usage papi py This is PAPI OPTION the PANIC PIpeline data reduction system DIRECTORY IAA CSIC Version 1 2 2015 2 8 PAPI Reference 47 508064845 PAPI Documentation Release 1 2 20151020173439 Options version show program s version number and exit h help show this help message and exit c CONFIG FILE config CONFIG FILE Config file for the PANIC Pipeline application If not specified config files papi cfg is used s SOURCE source SOURCE Source file list of data frames It can be a fileor directory name d OUTPUT DIR out dir OUTPUT DIR Output dir for product files o OUTPUT FILE output file OUTPUT FILE Final reduced output image t TEMP DIR
80. guration file swarp conf SWARP configuration file sextractor sex SExtractor configuration file sextractor conf sextractor cong sextractor nnw sextractor param 2 3 6 Examples TBD 36 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 2 3 7 Main config file This file has a structure similar to that of Microsoft Windows INI files It is divided into sections each of which has a number of name value entries The order in which sections appear is not important Any plain text editor can be used to do edit the file If some section or keyword is missing the application will fail and inform about that File papi cfg Default configuration file for PAPI 1 3 updated 24 Jul 2015 HEHE HEE EH EE EH FE FE AE FE FE AE FE FEAE H AE H FE EEE EEE AE FE FE FE FE FE FE FE FE FE FE FE AE FE FE AE FE EE EEE EE EE general FE H FE H AE FE H FE FE AE FE FE AE FE FE AE FE FEAE FE 4 FE AE 4 H AE 4 H 4 FE AE 4 FE AE AE H AE FE AE FE FE FE 4 FE 4 FE FE 4 FE FE AE FE H 4 EE 4 AE 4 AE FE AE Instrument pani o2k hawki if INSTRUME keyword does not match an error will be throw Letters not case sensitive instrument PANIC Some important directories nOTE oUTPut dir must be different from Quick Look source home jmiguel DATA SIMU_PANIC_3 q1 txt it can be a directory or a text source datal PANIC output_dir data2 out
81. h a blank empy line The file has Unix text format Text files created on DOS Windows machines have different line endings than files created on Unix Linux DOS uses carriage return and line feed rn as a line ending which Unix uses just line feed n 2 13 6 What is the best way to reduce PAPI data The recommend to use the OT and execute the OBs That way the headers will include meta data about the observation and thus the pipeline can group the data and find the required calibrations for a successful reduction Then you only have to type gt papi s datal PANIC my program 70 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 2 13 7 How can we reach hundredth of magnitude accuracy in photometry The best way to accurately photometrically calibrate PANIC images is to use 2MASS stars in the field itself to derive the photometric solution The accuracy strongly depends on the number of bright 2MASS stars within the filed of view but ranges from a few 1 100th of a magnitude to 0 1 magnitudes if only faint stars are contained in the field Additionally observing supplementary standard star fields can be asked for when preparing the observations To perform the 2MASS photometric calibration on an image you should use the photometry command as follow gt photometry i directory prereducedField fits o test pdf 2 13 8 How good is PAPI astrometry and how are PSF varia
82. he closest in time images to the currectly selected Once the close images have been found PQL asks the user to confirm about them to proceede to the background computation and subtraction Quick reduction It allows you to perform a quick reduction of the selected files at least 5 files are required on the Data List View If you only select one file then the PQL will look for the nearest in time files and ask you to confirm about them and the desired name for the final coadd For the quick reducion the pipeline will use the preferences set up on Setup tab 2 2 PANIC Quick Look Tool PQL 25 PAPI Documentation Release 1 2 20151020173439 Once the quick reducion is done the filename will be written in the Event Log Window and if selected it will be display on DS9 display Asirometric calibration Note Although the input FITS file does not need to be calibrared it is recommended The astrometric calibration is built on top of Astrometry net tool The command asks you about which detector to use of the calibration SG1 Q1 SG2 Q2 SG3 Q3 or SG4 Q4 The new astrometrically calibrated file will be created in the output directory speficied earlier and will have the same name as the original input file but ending with the ast fits suffix Once the astrometric calibration is done you could look into the header keyword ROTANGLE which gives you the rotation angle of the image It can be useful to check whe
83. hell and will modify your bashrc file adding a new line at the end 2 1 4 Building the documentation The PAPI documentation is base on sphinx With the package installed the html documentation can be built from the doc directory cd papi doc make html The documentation will be copied to a directory under build sphinx The documentation can be built in different formats The complete list will appear if you type make 2 1 5 Bug reports Please submit issues with the issue tracker on github 2 1 6 Release Notes 1 2 x Support for new MEF structure Qi old format SGi_1 also supported Bug Fixes 1 0 x First version 6 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 2 2 PANIC Quick Look Tool PQL 2 2 1 Purpose PANIC Quick Look hereafter PQL performs some on line data processing for quick look or quality check of the data being acquired taking a close look at a raw near infrared image and getting a quick feedback of the running observation PQL is an application with a graphical user interface which monitors the GEIRS data output waiting for new FITS files coming from GEIRS When a new file is detected it is added to the file list view in the main panel and then PQL will perform the task previously specified by the user in the setup configuration Some of the available tasks are Only display the FITS image with no processing Dark subtraction
84. ibution PANIC Team CONTENTS 1 PAPI Documentation Release 1 2 20151020173439 2 CONTENTS CHAPTER ONE CAVEAT Currently PAPI it is able to reduce data taken with the Observing Tool OT defining the required observing blocks OB or manually through GEIRS scripts PAPI was primarily developed and optimized for reducing broad band imaging data of extragalactic sources such as imaging data taken for field galaxy surveys and galaxy cluster surveys Other types imaging data have been reduced with PAPI but results can not be as good as desired See Troubleshooting for tips PAPI is not designed to reduce any kind of field taken with PANIC PAPI Documentation Release 1 2 20151020173439 4 Chapter 1 Caveat CHAPTER TWO CONTENTS 2 1 Installation amp Configuration 2 1 1 Requirements and Supported Platforms Because PAPI is written mostly in Python and ANSI C it can run on any platform that has the required Python modules and GCC compilier However it has been developed and deeply tested under openSuSE 12 x 13 x x86_64 Linux OS Python 2 7 x or higher and the following packages are required NumPy v1 6 SciPy gt v0 12 2 Astropy v0 3 1 Matplotlib gt v1 3 0 PyQt4 IRAF with STSDAS and MSCRED v2 16 xl liraf for xgterm stsci python v2 14 CDSClient SExtractor gt v2 8 6 SCAMP gt v1 7 0 SWarp gt v2 19 1 Astrometry net with 42xx index files SAO DS9 and XPA gt
85. ies from the Data List View 2 Right click and select Focus evaluation An IRAF console and ds9 windows will bring up and the first file of the focus series will be displayed on ds9 3 Focus the mouse over the stars you think are nice for the evaluation and type m or g give the profile of the selected star 2 2 PANIC Quick Look Tool PQL 21 PAPI Documentation Release 1 2 20151020173439 PANIC QuickLook File View Settings Calibrations BO H Main Setup Calibrations Logs Tools Help Use GEIRS file Current night Check Input Dir datal PANIC 2015 03 10 x fits Set Filename Filter E Output Dir data2 out 0 Tasks in the queue PRNIC List View Filter QL Mode Create Calibrations Data List View Filename datal PANIC 2015 03 10 TiltProcedure_March10_0022 fits FOCUS pate 11 datal PANIC 2015 03 10 TiltProcedure March10_ 0021 fits FOCUS 5 2015 03 11 datal PANIC 2015 03 10 TiltProcedure March 0 002 Display image datal PANIC 2015 03 10 TiltProcedure March10 001 dd CANA SO Image Info Ctrl H datal PANIC 2015 03 10 TiltProcedure_March10_001 Copy files to clipboard Ctrl C datal PANIC 2015 03 10 TiltProcedure_March10_001 Ma 3 q a A OPE COPY files to text file Shift T MAA EMITE Show Dither pattern Ctrl P Idatal PANIC 2015 03 10 TiltProcedure_March10_001 A AE emcees Build Master Dark Ctri M datal PANIC 2015 03 10 TiltProcedure_March10_001 Build Master Dome Flat
86. ile listing data frames or a single FITS file to proc o OUTPUTFILE output OUTPUTFILE Output text file including the list of objects mask files created by SExt m MINAREA minarea MINAREA SExtractor DETECT MINAREA default 5 t THRESHOLD threshold THRESHOLD SExtractor DETECT THRESH default 2 0 l SATURLEVEL saturlevel SATURLEVEL SExtractor SATUR LEVE default 300000 1 single point Create a single point object mask default False Example makeobjmask py s tmp reduced SEQ fits o tmp obj_mask txt makeobjmask py s tmp reduced SEQ fits o tmp obj mask txt 1 1 100000 m 10 photometry This module receives a reduced image of any known NIR filter and match to 2MASS catalog performing a fit in order to get a estimation of the Zero Point It is based on the method followed here http www ast cam ac uk ioa research vdfs docs reports 2masscal pdf Usage Options h help show this help message and exit i INPUT IMAGE input image INPUT IMAGE Input image to calibrate to do photometric comparison with c BASE CATALOG base catalog 2MASS USNO B BASE CATALOG Name of base catalog to compare with 2MASS USNO B not used def S SNR snr SNR Min SNR of stars used for linear fit default 10 0 z ZERO POINT zero point ZERO POINT Initial Magnitude Zero Point estimation 25 0 used for SExtractor o OUTPUT FILENAME output OUTPUT FILENAME Output plot filename default photometry pdf Example
87. ing and simple analysis only minimal headers keywords are required However and in order to group and reduce observing sequences the following header keywords are also required OBS TOOL OT V1 1 PANIC Observing Tool Software version PROG ID PANIC Observing Program ID OB ID 6 PANIC Observing Block ID OB NAME OB CU Cnc Ks 2 PANIC Observing Block Name OB PAT 5 point PANIC Observing Block Pattern Type PAT NAME OS Ks 2 PANIC Observing Secuence Pattern Name PAT EXPN 1 PANIC Pattern exposition number PAT NEXP 5 PANIC Pattern total number of expositions IMAGETYP SCIENCE PANIC Image type These keywords are automatically added to the FITS header by the PANIC Observation Tool as each file is created If these are not saved PQL will not work correctly 2 2 PANIC Quick Look Tool PQL 7 PAPI Documentation Release 1 2 20151020173439 2 2 3 Starting PQL To start PQL GUI you can lauch it from the PANIC computer panic22 panic35 once you are logged as obs22 obs35 user Thus as any one of the workstations of the observing room open a X terminal window and log into the PANIC computer as follow for 2 2m ssh X obs22 panic22 ask Calar Alto staff for password for 3 5m ssh X obs35 panic35 ask Calar Alto staff for password Once you are logged into the PANIC computer to launch PQL GUI type next com
88. ions with the FITS files selected on the Data List View 1 MEF2Single converts a MEF file to SEF file Single2MEF converts a SEF file to MEF file Split MEF extracts the extension one per each detector of the MEF file to individual files Split Single extracts the extension one per each detector of the SEF file to individual files Collapse Cube sums arithmeticly the planes of the given cube single plane 2D image D nn A U N Create DataSeq modifies headers of the set of selected FITS files to create a new Data Sequece compliant with PAPI as they would be observed with the OT This command can be usefil to fix or re order broken sequences observation was interrupted or to remove or add files to a observed sequence You will be asked for the type of sequence DARK DOME_FLAT SKY FLAT FOCUS or SCIENCE you want to create 2 2 PANIC Quick Look Tool PQL 27 PAPI Documentation Release 1 2 20151020173439 2 2 8 How dol How to determine the telescope focus To determine the telescope focus you should run a OT focus serie around the guest value and then run the Focus Evaluation How to determine the field rotation To determine the field rotation firstly you should observe a enough crowded field and then run the astrometric calibration on it for each detector Once you have the new FITS astrometrically calibrated you have to look for the ROTANGLE keyword in the new header For example ROTANGLE
89. ionset 1170 data2 2015 03 10 domefl ts 0002 fits PAPI 2015 05 28 09 18 01 499 DEBUG reductionset 1170 data2 2015 03 10 focus 0016 fits typ PAPI 2015 05 28 09 18 01 500 DEBUG reductionset 1170 data2 2015 03 10 focus 0012 fits typ PAPI 2015 05 28 09 18 01 500 DEBUG reductionset 1170 data2 2015 03 10 domefl ts 0001 fits PAPI 2015 05 28 09 18 01 500 DEBUG reductionset 1170 data2 2015 03 10 domefl ts 0044 fits PAPI 2015 05 28 09 18 01 500 DEBUG reductionset 1170 data2 2015 03 10 focus 0013 fits typ PAPI 2015 05 28 09 18 01 500 DEBUG reductionset 1170 data2 2015 03 10 focus 0015 fits typ PAPI 2015 05 28 09 18 01 500 DEBUG reductionset 1170 data2 2015 03 10 focus 0014 fits typ PAPI 2015 05 28 09 18 01 500 DEBUG reductionset 1170 data2 2015 03 10 domefl ts 0041 fits PAPI 2015 05 28 09 18 01 500 DEBUG reductionset 1170 data2 2015 03 10 Standard Star FS15 PAPI 2015 05 28 09 18 01 501 DEBUG reductionset 1170 data2 2015 03 10 GJ623 Test Photom M PAPI 2015 05 28 09 52 15 122 DEBUG calDark 283 Saved master DARK to data2 out mDark Xdb5b PAPI 2015 05 28 09 52 15 122 DEBUG calDark 284 createMasterDark finished Elapsed time s PAPI 2015 05 28 09 52 15 123 DEBUG reductionset 2596 OUTPUT file generated data2 out mDar PAPI 2015 05 28 09 52 15 179 DEBUG reductionset 3099 Inserting result in DB data2 out mD PAPI 2015 05 28 09 52 15 211 DEBUG reductionset 2414 reduceSet All sequences processed PAP
90. lGainMap Creates a Gain Map from any master flat correctNonLinearity Corrects the images pixel values for non linearity dxtalk Removes cross talk spots from input images makeobjmask Creates a objects mask SExtractor OBJECTS images for a list of FITS im ages photometry Performs a photometric calibration comparison with 2MASS solveAstrometry Performs a astrometric calibration using Astrometry net and 42xx index files remove_cosmics Detects and clean cosmic ray hits on images based on Pieter van Dokkum s L A Cosmic algorithm eval_focus_serie Estimates the best focus value of a focus exposures cleanBadPix Cleans masked bad pixels from an input image Utilities Description createDataSeq Modifies headers of a set of FITS files to create a Data Sequece compliant with PAPI getBPM Creates the BPM file from the NonLinearity correction MEF file The bad pixels will be saved as 1 s mef Tool to convert from SEF to MEF and viceversa also allows to give splits of the extensions or join SEFs collapse Collapses add them up arithmetically each cube of a list files into a single 2D image genLogsheet Generates a text file as a log sheet from a set of images imtrim Crops cuts the input image edges modFITS Allows to perfom the modification of any FITS keyword runStarfocus Run IRAF starfocus for a focus sequece and return the best focus value and a plot of the fit runPsfmeasure Run IRAFpsfmeasure for a focus sequece and
91. ld Master Twlight Flat Ctrl T Build Gain Map Ctrl G Build BPM Ctrl B Apply Dark FlatField BPM Ctrl A Apply Non Linearity Correction Shift N Apply and Show BPM Shift B Focus evaluation Ctrl F Subtract own sky Ctrl S Subtract near Sky Ctrl N Quick Reduction Ctrl Q Astrometric Calib Shift A Photometric Calib Shift P Statistics Shift S FWHM mean estimation Ctrl W Background estimation Shift B Math FITS Some actions in the menu could be disabled and greyed out if they are not availabe or applicable to the selected files Display image It displays the currect selected image in the SAOImage ds9 display it will launch the ds9 application if it is not opened yet Image info It is a quick way to see some basic information of the selected image The information is mainly concerning the FITS structure and exposition times used The information will be shown in the Event Log Window as follow datal PANIC 2015 05 19_SS_zenith_Ks_1_3 SS_Ks_SG1_4_0024 fits SEF Filename Image Shape 32 32 Filter Ks ITIME 0 045000 NCOADDS 1 EXPTIME 0 045000 TYPE FOCUS OT keywords True Of course if you need any other information of the file you can find it using the ds9 gt File gt Display Header option Copy files to clipboard It copies the current selected files to the clipboard This way you could paste the full pathnames to any other file It is quite useful whe
92. malized to make a gainmap and set bad pixels to 0 Usage Options h help s SOURCE FILE source SOURCE FILE o OUTPUT FILENAME L MINGAIN low H MAXGAIN high MAXGAIN x NXBLOCK nx NXBLOCK y NYBLOCK ny NYBLOCK n NSIGMA nsigma NSIGMA N normal show this help message and exit Flat Field image NOT normalized It has to be a fullpath file Ooutput OUTPUT FILENAME output file to write the Gain Map MINGAIN pixel below this gain value are considered bad default 0 5 pixel above this gain value are considered bad default 1 5 X dimen pixels to compute local bkg even default 16 Y dimen pixels to compute local bkg even default 16 number of stddev from local bkg to be bad pixel defaul if true the input flat field will be normalized before build Example calGainMap py s tmp masterTF fits o tmp masterGain fits calGainMap py s tmp masterTF fits o tmp masterGain fits L 0 7 H 1 2 dxtalk PANIC HAWAII 2RG sensors with multiple parallel readout sections show crosstalk in form of compact positive and negative ghost images whose amplitude varies between readout sections PAPI has a optional de crosstalk module that assumes that the amplitude is the same therefore the correction will only partially remove the effect Gf at all If you know in advance that this will be a probl
93. mand start ql amp The next figure shows a snapshot of the main window of PQL GUI that will bring up the start gl command File View Settings Calibrations Tools Help HQem Main Setup Calibrations Logs Use GEIRS file Current night Check batenranc20isos2s jw sms ia daazot Cid Co Tasks in the queue PANIC List View Filter QL Mode Muss le crete catbratiors Data List View Rename SS COTO Ninos reris pstsobs ES th N T datal PANIC 2015 06 25 skyflats_dawn_0005 fits SKY FLAT Last fle received daks otsas O J Event log Time UTC 2 2 4 Configuration files The configuration files used by PQL are located in the PAPI_LHOME config_files The main config file is the same file used by PAPI ie SPAPI_CONFIG and usually called papi cfg This file includes a lot of parameters used by PAPI and therefore by PQL during the processing however at the end of the PAPI_CONFIG file there is section called quicklook where the user can set some specific parameters for PQL TEE EE HE E FE FE FE TE FE FE HE TE FE FE HE E FE FE FE TE FE FE HE TE FE FE HE E FE FE HE TE EEE EERE EEE EEE EE EE E E E E EE HH EH HE HE EE quicklook H HE ae H FE FE TE FE HE H FE H FE FE H FE HE E FE HE EE H FE HE E H HH Next are some configurable options for the PANIC Quick Look tool some important directories source datal
94. ms the astrometric calibration of a set of images in principle previously reduced but not mandatory this routine is built on top of Astromety net tool Usage Options h help show this help message and exit s SOURCE FIL E source SOURC o OUTPUT DIR Source E FIL file output dir OUTPUT li DIR st of data frames It can be a file or directory name Place all output files in the specified directory default tmp p PIXEL SCAL r recursi E pixel scale PIX EL SCALE Pixel scal ve Recursive subdirectories only first level of the images remove cosmics Remove the cosmic ray hits in the input image it is built on top of Pieter van Dokkum s L A Cosmic algorithm Usage Options h help i INPUT IMAGE input image INPUT I input image to remove cosmics output OUTPUT IMAGE output filename default without cosmics fits te overwrite the original image with the corrected on o OUTPUT IMAGE O overwri m mask show this help message and exit If true AGE the mask with cosmics detected and removed is written into a FITS 2 8 2 Utilities Besides the modules for data reduction PAPI has a set of utilities than can be used as tools for preparing the data reduction execution they are 2 8 PAPI Reference 55 PAPI Documen
95. n using the PAPI commands on the command line to run some operation that is not available 18 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 on PQL Copy files to text file If copies the current selected files into the specified text file It is quite useful when using the PAPI command line to run some operation that is not available on PQL Show Dither pattern It brings up a plot of the full FOV and with the dither offsets obtained from the RA Dec coordinates found in the FITS header You have to select a set of images in the Data List View and then right button and Show Dither pattern Ditther offsets magnification 1 0 pix_scale 0 23 T T T T T T T 600 9 1 N e o L Dec arcsec o 200 400 4 600 ti i i i i 600 400 200 0 200 400 600 RA arcsec Calibrations Next options allow you to build the master calibration files from a given set of selected files Build Master Dark This command is used to produce a master DARK file combining the set of files currectly selected in the Data List View It checks that all the selected files are compliant ie have the same EXPTIME NCOADD ITIME READMODE and shape You only have to give the name of the master dark file to be created The master dark is computed using an average combine with a minmax rejection algorithm Build Master Dome Flat This command is used to produce a Master DOME FLAT file combini
96. nds are available Page cursor command summary g Measure object and graph the results m Measure object q Quit object marking and go to next image At the end of all images go to analysis of all measurements show Show current results When in the interactive graphics the following cursor commands are available All plots may r Page cursor command summary Toggle magnitude symbols in spatial plots Size and ellipticity vs radius from field center at one focus Undelete all deleted points Delete nearest point star or focus selected by query Zoom to a single measurement space gt Step through different focus or stars in current plot type a Spatial plot at a single focus b Spatial plot of best focus values d Delete star nearest to cursor e Enclosed flux for stars at one focus and one star at all focus f Size and ellipticity vs focus for all data i Information about point nearest the cursor m Size and ellipticity vs relative magnitude at one focus n Normalize enclosed flux at x cursor position o Offset enclosed flux to by adjusting background P Radial profiles for stars at one focus and one star at all focus q Quit r Redraw S E u x Z lt beta val Beta parameter for Moffat fit level val Level at which the size parameter is evaluated overplot y n Overplot the profiles from the narrowest profile radius val Change profile radius show file Page all information for the c
97. ng the set of files currectly selected in the Data List View It checks that all the selected files are compliant ie have the same FILTER NCOADD READMODE and shape You have to select at least one DOME_FLAT_LAMP_OFF and one DOME_FLAT_LAMP_ON image and then provide the name for the master dome flat to create The procedure to create the master dome flat is as follow 1 Check the EXPTIME TYPE dome and FILTER of each Flat frame 2 Separate lamp ON OFF dome flats 2 2 PANIC Quick Look Tool PQL 19 PAPI Documentation Release 1 2 20151020173439 3 Make the median combine sigmaclip of Flat LAMP OFF frames 4 Make the median combine sigmaclip of Flat LAMP ON frames 5 Subtract lampON lampOFF implicit dark subtraction 6 optionally Normalize the flat field with median robust estimator Note that we do not need to subtract any MASTER_DARK it is not required for DOME FLATS it is done implicitly because both ON OFF flats are taken with the same Exposition Time Build Master Twlight sky Flat This command is used to produce a Master SKY FLAT file from a set of files currectly selected in the Data List View It checks that all the selected files are compliant ie have the same FILTER NCOADD READMODE and shape You have to select at least three SKY FLAT images dusk or dawn The procedure will look for the required master dark frames to subtract in the current output directory and in the external calibration directory If
98. nge the most frecuently used PQL actions add a file to the current view change the source input directory the same that Input directory display the current selected image the same that Display open an IRAF console open Aladin tool quit PQL on the right border HOe m g Main panel This tab panel contains the following controls Input directory Ouput directory Filename filter Current night 2 2 PANIC Quick Look Tool PQL 9 PAPI Documentation Release 1 2 20151020173439 Use GEIRS file Data list view List view filter QL mode Subract last 2 button START processing button Create Calibrations button Main Setup Calibrations Logs Use GEIRS file Current night Check datai Panic 2015 0625 X fts Set Filename Fiter M 3 dxaat JO 0 Tasks in the queue PANIC List View Filter QL Mode m INPUTS B Lazy zj Create Calibrations Data List View Subtractiasta Renme 7 masse iter wmes r mrie Dateobs obje E Total integration time of the file Nimg TEXP datal PANIC 2015 06 25 skyflats_dawn_0005 fits SKY FLAT 2 16 24 2015 06 26 04 21 36 3937 EN Data directories In the Main tab panel of POL main window the fitst thing to set up are the data directories Setup Calibrations Use GEIRS file Current night Check input Dir fatan Panic2o150310 __ gt Output Dir 1dats2
99. nput directory OUTS Files of the ouput directory DARK Files marked IMAGETYP as DARK images DOME FLAT Files marked as DOME FLAT image FOUCS Files marked as FOCUS image from a focus series SKY FLAT Files marked as SKY FLAT images SCIENCE Files marked as SCIENCE image or with unknown type MASTERS Files marked as MASTER calibration files produced by PAPI REDUCED Files marked as calibrated by PAPI GROUP Special case that show all the files groupped as observed sequences OBs ALL Show all the files not matter the type of it 12 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 List View Filter INPUTS INPUTS OUTS 1 DOME_FLAT FOCUS SKY_FLAT SCIENCE MASTERS REDUCED GROUP QuickLook mode The quick look mode combo box allows you to select the mode in which PQL will be run when the START processing button is pushed The current modes are QL Mode Pre reduction Quick LEMON Full LEMON Full Reduction None No processing action is done Lazy default If the end of a calibration DARK FLAT sequence is detected the master file is built Otherwise the SCIENCE files are processed as specified in the Setup gt Lazy Mode Apply DARK FLAT BPM Subtract Last Frame Science Subract Nearest Sky Pre Reduction If the end of observing sequence is detected it is processed in a quick mode single pass for sky subtraction For calibration sequences
100. number of coadds NCOADDS ie a cube of N planes This mode will be also used when the software or hardware subwindowing is set up and the no integrated option is selected Single integrated FITS file the four detectors are saved in single file and in a single exten sion FITS image SEF If the number of coadds NCOADDS is gt 0 then they are integrated arithmetic sum in a single frame Single non integrated FITS cube the four detectors are saved in a single extension FITS SEF file and each individual exposition in a plane layer of a cube It means N planes where N is the number of coadds or expositions 2 3 PAPI 31 PAPI Documentation Release 1 2 20151020173439 Note Currently PAPI is not working with non integrated individual files of an exposition In case you are interested in no integrated files and wish to reduce the data with PAPI you should use SEF of MEF non integrated FITS cube mode Show grouped files in a raw directory For the grouping the application uses the keywords written by the OT during the observation Command Spapi py s my raw_data directory Example papi py s data2 2015 03 10 p by ot ME FLAT FI ats 0042 fits ats 0043 fits KY FLAT FIL k 0001 k 0002 k 0003 k 0004 k 0005 KY_F LAT FIL
101. of each sequence stimate fwhm Fals min frames minimun number of frames required to reduce a sequenc min frames 5 group by the pipeline will try to group the data files in two main ways 38 Chapter 2 Contents obs mode dither default observing mode of input data files to reduce dither ext dither other check data integrity It consists in checking if TEXP NCOADD FILTER and READMODE match properly PAPI Documentation Release 1 2 20151020173439 OT following the specific keywords provided by the OT as OB_ID OB_PAT and then different observing sequences could be grouped and reduced or FILTER only group by filter band and then only one observing sequence NONE No grouping criteria will be taken force only one group with all ot group_by OT or FILTER or NONE The ABOVE option values can be modified at the invokation time of the pipeline in IMAGETYP F should be pr the files command 1 apply_dark_flat Neither dark nor flat field will be applied The pipeline will look for a master dark and master flat field to be applied to the raw science frames Both master DARK and FLAT are optional i e each one can be applied even the other is not present It no DARK FLAT are found the reduction continues without apply them but implicity due to skysubtraction Master flat will be looked for to be applied AFTER skysubtraction but no DARK will b
102. pi py s my raw data directory d my result directory Optional Arguments For most image sets PAPI can be run in the default configuration with no additional interaction required If the default settings are insufficient for processing a particular data set there are a number of run time options which may be applied to help improve the reductions The next command will show some of the available options papi py help Then the listing of the PAPI command line options Usage papi py OPTION DIRECTORY This is PAPI the PANIC PIpeline data reduction system IAA CSIC Version 1 2 20150508064845 Options version show program s version number and exit h help show this help message and exit c CONFIG FILE config CONFIG FILE Config file for the PANIC Pipeline application If not specified config files papi cfg is used s SOURCE source SOURCE Source file list of data frames It can be a fileor directory name d OUTPUT DIR out dir OUTPUT DIR Output dir for product files o OUTPUT FILE output file OUTPUT FILE Final reduced output image t TEMP DIR temp dir TEMP DIR Directory for temporal files r ROWS rows ROWS Use only files of the source file list in the rangeof rows specified 0 to N both included R recursiv Does recursive search for files in source directory i list Generate a list with all the source files read fromthe Source and sorted by MJD
103. rth mentioning that SExtractor does a background subtraction when looking for objects and that the FWHM value is rather imperfect and overstimated compared with IRAF imexam values E Bertin There are currently 2 ways to measure the FWHM of a source in SExtractor Both are rather imperfect FWHM IMAGE derives the FWHM from the isophotal area of the object at half maximum FLUX RADIUS estimates the radius of the circle centered on the barycenter that encloses about half of the total flux For a Gaussian profile this is equal to 1 2 FWHM But with most images on astronomical images it will be slightly higher A profile fitting option will be available in the next version of SExtractor I am currently working on it Background estimation This command shows the background image of the currently selected image using the SExtractor feature CHECKIMAGE_TYPE BACKGROUND Math operations This option allows the next basic operations with the FITS files selected on the Data List View 1 Sum images it allows the selection of two or more images single arithmetic sum will be done 2 Subtract images only two images can be selected 3 Divide images only two images can be selected 4 Combine images median sigmaclip 1t allows the selection of two or more images If FITS files are cubes with the same dimension then the math operation will be done plane by plane FITS operations This option allows the next conversion operat
104. runs Astrometry Net SEx S Note for the lst coadd for object mask dithercubemean is x xalways used to avoid the distortion correction coadd mode dithercubemean coadd mode swarp they fi grouping ing to the ru tilde expansi 5 because thi lename vamos ple AMP SWARP 2 3 PAPI 39 PAPI Documentation Release 1 2 20151020173439 mosa HEE HET AE E H AE 4 HEE con HEHEHE H EE HEH E Next paths are relative to PAPI_ irdr sext sextractor_param sext ic_engine fig_files _bin irdr ractor conf ractor nnw sextractor conv Scamp conf co Sswarp conf co HEHE HE HEE EAE EE EH non HEHE HE HEE AE AE FE EAE Non Linearity correction appl FI model model bpm Ba mode bpm dar ob bpm file bpm file linearity y False TS file con da _rrrmpia lir 4 HH d Pixel Mas Fix Probably on grab Bad Pix montage bin con HEH ERE H HEHE HE H fig files sext config files sextractor param TH ractor sex config_files sextractor nnw con nfig_fi 19 V ga E e PENN fX k mode fig files sext FE TE HEE TE taining the NL
105. s when present in the header So the value specified here is for a single image with NCOADD 1 Of course this values will be specific for each detector and in case of a multi detector instrument should be the lowest value of all detectors satur level 55000 Minimun overlap correlation fraction between offset translated images from irdr offset c min corr frac 0 1 Maximun dither offset in pixels allowed to use a single object mask In order to know if a sin gle common object mask deeper or multiple individual object masks for each sky subtracted file is needed For values gt max dither offset and due to the optical distortion multiple individual masks are used otherwise a common object mask got from the first coadd max dither offset 200 Astrometric engine SCAMP or AstrometryNet engine AstrometryNet engine SCAMP Object mask mask minarea 20 sex DETECT MINAREA min of pixels above threshold mask maxarea 0 sex DETECT MAXAREA SExtractor gt 2 19 5 max of pixels above threshold O unlimited mask thresh 3 5 sex DETECT_THRESH used for object masking expand mask 0 5 amount to expand the object mask regions sex SATUR_LEVEL level in ADUs for a single exposure image at which the pixel arises saturation Note than that value should be updated with NCOADDS or NDIT keywords when present in the header So the value specified here is for a single image
106. sible to work in batch mode is it possible to run the PANIC flat module in all the images specifying in this parameter which ones will be considered That is only those images whose object name matchs one of the names listed in this parameter will be considered when generating the master twflat Note that if is contained in the list _all_ object names will be matched This symbol thus provides a way to easily specify all the images which is equivalent to saying do not filter images by their object names object names TW FLAT DUSK TW_FLAT_DUSK SKY FLAT check prop if true th frames used to build the master will be checkd to have the same acquisition properties EXPT NCOADD ITIME READMODE FILTE check prop yes suffix the string if any to be added to the filename of each resulting the resulting image would be saved to home images ferM 0720 o D fits This parameter is optional as if nothing is specified nothing will be appended suffix F min frames minimun number of frames required to build a master twlight flat min_frames ll Co area width median smooth median filter smooth of combined FF to reduce noise and improve the S N and preserve the large scale features of the flat median smooth False 4 H H H 4 ARA 4 H 4 H 4 H H H H H 4 H
107. skyflats dus PAPI 2015 05 28 09 18 01 488 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 488 DEBUG reductionset 1167 SEQUENCFE 3 TYPE S PAPI 2015 05 28 09 18 01 489 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 489 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 489 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 489 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 489 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 489 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 489 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus PAPI 2015 05 28 09 18 01 489 DEBUG reductionset 1167 SEQUENCFE 4 TYPE S PAPI 2015 05 28 09 18 01 490 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 490 DEBUG reductionset 1170 data2 2015 03 10 skyflats dus 32 Chapter 2 Contents k 0029 PAPI Documentation Release 1 2 20151020173439 PAPI 2015 05 28 09 18 01 490 DEBUG reduc
108. t view control displays all the files found in the input directory or in the output directory if the check box at the right of output directory is checked Additionaly the use can add any other FITS file The control is a multicolum table with the next fields Data List View a datal PANIC 2015 06 25 skyflats_dawn_0005 fits SKY FLAT Filename Full path name of the file found in the Image type The type of the FITS file detected DARK DOME_FLAT SKY_FLAT FOCUS SCIENCE Nimgs Number of images layers of the cube if image is integrated no cube then 1 TExpFile Total Exposition time of the file Nimgs EXPTIME Thus EXPTIME TExpFile Nimgs Date Obs Observation data of the file DATE OBS keyword Object Object name OBJECT keyword RA Right ascention of center of the image Dec Declination of the cener of the image You can sort the list by any column filename image type exptime filter date obs object right ascension declination by clicking on their headers as usual by default the list is sorted by the Date Obs field showing the most recect file at the top A double click on any row displays all its file into SAOImage ds9 For further details of any of the files you can also look at the header of a fits image using ds9 using the File Display Fits Header menu option List view filter It allows you to select the type of files to be shown in the data list view The options are INPUTS Files of the i
109. ta2 tmp YYYYMMDD To set the value the user must push the Temporary Dir button than appears on the Setup tab instead the Main tab used for input and output directory Main i Calibrations Logs Common Settings Data C x Check header RA Det x Create Log file Time o Temporary Dir Max n Current night checkbox When you click this checkbox the Input directory and Output directory fields will be automatically filled with the currect night date If the current night Input Ouput directories donot exist PQL will ask you if you want to create them The currect night is supposed to start at 8 am UTC and to end at 8 am UTC of next day Use GEIRS file When this checkbox is clicked PQL will use the tmp fitsGeirsWritten file to detect the new files created by GEIRS Files older than 1 day will no be considered This detection method for FITS files is not frecuently used but can be useful whether some problem arise reading files just after they have been written by GEIRS Filename filter In this box the user can filter the name of the files should appears on the data list view from the input directory output files are not filtered The filter can contains and wildcards For example March10_00 1 0_00 1 Set Filename Filter 2 2 PANIC Quick Look Tool PQL 11 PAPI Documentation Release 1 2 20151020173439 Data list view Tha data lis
110. tation Release 1 2 20151020173439 Utilities Description check_papi_modules Check whether all python modules required by PAPI are installed checkQuality Computes some quality values from the image FWHM STD RMS collapse Collapse sum each cube of a list of files into a single 2D image eval_focus_serie Finds out the best Focus value from a focus series genLogsheet Creates a log sheet from a set of FITS files health Compute the Gain and Noise from a set of flat images grouped in packets and with increased level of Integration Time imtrim Cut crop edges of the input image modFits Modifies a keyword inside a FITS header skyfilter Subtracts sky background to a dither sequence of frames spatial_noise Compute the Spatial Noise from a set of dark images grouped in pairs with the same Integration Time check_papi_modules Check whether all Python modules required by PAPI are installed The modules currently required are Module Version Numpy 1 6 PyRaf 1 1 PyFITS 3 0 Matplotlib 0 98 1 Scipy 0 10 PyQt4 QtCore 4 8 PyWCS 1 11 vo 0 7 Atpy 0 95 Example check papi modules py PAPI Python checking tool Checking Python Version PAPI needs Python Version 2 Y with Y 2 7 Your Python version 2 7 3 is fine Testing Python module installation for module atpy PAPI needs at least version 0 9 5 Your version 0 9 6 of atp
111. temp dir TEMP DIR Directory for temporal files r ROWS rows ROWS Use only files of the source file list in the rangeof rows specified 0 to N both included R recursiv Does recursive search for files in source directory l1 list Generate a list with all the source files read fromthe Source and sorted by MJD M REDUCTION MODE red mode REDUCTION MODE Mode of data reduction to do quick science lab lemon quick lemon m OBS MODE obs mode OBS MODE Observing mode dither ext dither other S SEQ TO REDUCE Seq to reduce SEQ TO REDUCE Sequence number to reduce By default all sequences found will be reduced W DETECTOR window detector DETECTOR Specify which detector to process Q1 SG1 Q2 SG2 Q3 SG3 Q4 SG4 Q123 all except SG4 all default all p print Print all detected sequences in the Data Set T SEQ TYPE Sequences type SEOQ TYPE Specify the type of sequences to show DARK FLAT all DOME FLAT SKY FLAT FOCUS SCIENCE CAL all default all b build calibrations Build all the master calibrations files C EXT CALIBRATION DB ext calibration db EXT CALIBRATION DB External calibration directory library of Dark amp Flat calibrations D MASTER DARK master dark MASTER DARK aster dark to subtract F MASTER FLAT master flat MASTER FLAT aster flat to divide by B BPM FILE bpm file BPM FILE Bad pixel mask file g GROUP BY group by GROUP BY kind of data grouping based on to do
112. the master file will be built and for science sequences a quick reduction will be done using options configured in the Setup gt Pre Reduction Mode and the calibrations found in local database output directory and external calibration directory Note that the pre reduction options configured in the config file will be overwritten Pre Reduction Mode e Apply DARK FLAT x Make Astrometry Apply SuperFlat x Quality measurements background psf Apply BPM x Build stacked image Correct for non linearity sky nearest frames Detector to reduce Astrometric Engine 4frames SGl ly AstrometryNet 2 2 PANIC Quick Look Tool PQL 13 PAPI Documentation Release 1 2 20151020173439 Quick LEMON The same as Pre reduction but the processing stops after the 1st sky subtraction and no final co added image is produced It is useful for LEMON processing for light curves Full Reduction If the end of observing sequence is detected it is processed in a science mode double pass for sky subtraction For calibration sequences the master file will be built and for science sequences a science reduction will be done using options configured in the Setup gt Pre Reduction Mode and the calibrations found in local database output directory and external calibration directory Note that the pre reduction options configured in the config file will be overwritten Full LEMON The same as Pr
113. ther the instrument rotator is set properly at the telescope Photometric calibration Note Your data is assumed to be calibrated Dark subtraction flat fielding correction and any other necessary steps should have been performed before any data is fed to the photometric calibration We need to first distinguish between absolute and relative photometric calibration Absolute photometric cali bration would be required to determine the system throughput and or the true magnitude of our stars Relative photometry is a simpler task that would allow us to measure the uniformity and linearity of response across the detector This section refers to absolute photometry The photometric calibration involves taking sufficiently long integrations with PANIC to get good a good SNR The night must be photometric and the integration time and zenith angle need to be recorded To reduce the dependence on zenith angle it would be best to take images within 30 of zenith The photometric calibration can be performed using the saved images The photometric calibration will be useful for validating our throughput calculations Using the photometric calibration to determine the true magnitudes of stars is more challenging Statistics It gives some statistics mean mode stddev min max values of the currently selected image s If the image s is are MEF then the command shows the stats of each extension 1 4 as shown in next example FILE MEAN MOD
114. tions corrected At present the pipeline applies a correction for PSF distortions based on a distortion map derived during the astrometric calibration done with SCAMP a software developped by Emmanuel Bertin and 2MASS 2 13 9 Is there a way to look at or edit FITS headers The PAPI package includes the WCS library and tools including a program called edhead This is built automat ically when you build PAPI and it is installed into the HOME bin directory To run edhead simply type edhead filename fits It will strip the header from the FITS file and open it for editing using the program defined in your environment Make the changes save and exit the editor and the header is re attached to the image 2 13 10 Does PAPI generate a log file of the processing Yes it can be configured in the PAPI_CONFIG file with the parameter logfile tmp papi log For each exe cution the log filename will have an suffix with the timestamp of the data and time i e tmp papi Y Y Y Y MM DDTHH MM SS ss log 2 14 Acknowledgments This software has been developed in the scope of the PANIC project The project has been funded by the Span ish Ministry of Economy and Competitiveness with funds from the European Union FEDER and the Spanish national budget through the grants ICTS 2006 15 ICTS 2007 10 ICTS 2008 24 ICTS 2009 32 and the project Intramural 200450E458 of the Spanish National Research Council We also thank all our colleagues from P
115. tionset 1170 data2 2015 03 10 skyflats_dusk_0030_ PAPI 2015 05 28 09 18 01 490 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dusk_0031_ PAPI 2015 05 28 09 18 01 490 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dusk_0032_ PAPI 2015 05 28 09 18 01 490 DEBUG reductionset 1170 data2 2015 03 10 skyflats_dusk_0033_ PAPI 2015 05 28 09 18 01 490 DEBUG reductionset 1167 SEQUENCE 5 TYPE SKY FLAT FIL PAPI 2015 05 28 09 18 01 491 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 491 DEBUG reductionset 1170 data2 2015 03 10 skyflats dusk 0034 PAPI 2015 05 28 09 18 01 491 DEBUG reductionset 1170 data2 2015 03 10 skyflats dusk 0035 PAPI 2015 05 28 09 18 01 491 DEBUG reductionset 1170 data2 2015 03 10 skyflats dusk 0036 PAPI 2015 05 28 09 18 01 491 DEBUG reductionset 1170 data2 2015 03 10 skyflats dusk 0037 PAPI 2015 05 28 09 18 01 491 DEBUG reductionset 1167 SEQUENCE 6 TYPE SKY FLAT FIL PAPI 2015 05 28 09 18 01 491 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 491 DEBUG reductionset 1170 data2 2015 03 10 skyflats dusk 0038 PAPI 2015 05 28 09 18 01 492 DEBUG reductionset 1170 data2 2015 03 10 skyflats dusk 0039 PAPI 2015 05 28 09 18 01 492 DEBUG reductionset 1170 data2 2015 03 10 skyflats dusk 0040 PAPI 2015 05 28 09 18 01 492 DEBUG reductionset 1167 SEQUENCE 7 TYPE SCIENCE FILT PAPI 2015 05 28 09 18 01 492 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 492 DEBUG reductionset 1170 data2
116. to build a master super flat min frames 5 half width of sky filter window in frames hwidth 2 area width 1000 length in pixels of the central area used for normalization Object mask mask minarea 10 4 sex DETECT MINAREA min of pixels above threshold mask maxarea 0 4 sex DETECT MAXAREA SExtractor gt 2 19 5 max of pixels above threshold mask thresh 3 5 sex DETECT THRESH used for object masking 1 5 dexpand mask 0 5 amount to expand the object mask regions sex SATUR_LEVEL level in ADUs for a singl xposure image at which the pixel arises saturation Note than that value should be updated with NCOADDS or NDIT keywords when present in the header So 2 3 PAPI 43 PAPI Documentation Release 1 2 20151020173439 2 4 Image selection 2 5 Data set classification One of the main featthe value specified here is for a single image with NCOADD 1 Of course this val ues will be specific for each detector and in case of a multi detector instrument should be the lowest value of all detectors satur_level 55000 2 6 Image selection 2 7 Data set classification One of the main feat skymodel sky model used used during the sky subtraction It will be a parameter for the IRDR skyfilter executable median the normal way for coarse fields default min suitable for crowded fields skymodel me
117. ues N1 and N2 where NI number of the first sequnece to reduce N2 number of the last sequence to reduce Command papi py s my raw data directory S N1 N2 Example papi py s data2 2015 03 10 S 14 20 By default PAPI process the files in quick mode single pass for sky subtraction however if you can use the science mode double pass for sky subtraction adding the M science when you run PAPI Example papi py s data2 2015 03 10 S 14 20 M science If you only want to reduce a specific sequence for example number 14 you should type papi py s data2 2015 03 10 S 14 14 PAPI 2015 05 28 09 18 01 498 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 498 DEBUG reductionset 1170 data2 2015 03 10 darks March10 0251 PAPI 2015 05 28 09 18 01 498 DEBUG reductionset 1170 data2 2015 03 10 darks March10 0252 PAPI 2015 05 28 09 18 01 499 DEBUG reductionset 1170 data2 2015 03 10 darks March10 0253 PAPI 2015 05 28 09 18 01 499 DEBUG reductionset 1170 data2 2015 03 10 darks March10 0254 PAPI 2015 05 28 09 18 01 499 DEBUG reductionset 1170 data2 2015 03 10 darks March10 0255 PAPI 2015 05 28 09 18 01 499 DEBUG reductionset 1167 SEQUENCFE 3 5 TYPE UNKNOWN FIL PAPI 2015 05 28 09 18 01 499 DEBUG reductionset 1168 PAPI 2015 05 28 09 18 01 499 DEBUG reductionset 1170 data2 2015 03 10 PANIC 2015 03 10T21 PAPI 2015 05 28 09 18 01 499 DEBUG reduct
118. urrent set of objects Size type Size type Radius FWHM scale val Pixel scale for size values xcenter val X field center for radius from field center plots ycenter val Y field center for radius from field center plots The profile radius may not exceed the initial value set by the task parameter 2 2 PANIC Quick Look Tool PQL 23 PAPI Documentation Release 1 2 20151020173439 irafterm NOAO IRAF V2 16 1 panic panici Wed 16 26 57 20 May 2015 Average best focus of 32 8816 with GFWHM of 2 72 E 1 il i p t f p 1 i 4 irafterm NOAD IRAF V2 16 1 panic panicl Wed 16 26 57 20 May 2015 Average best focus of 32 9219 with GFWHM of 3 02 Ctr ORGCD Reem 24 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 5 Once you have removed the outliers type q with the focus on the plot window and you will get the final plot with the fit of the values and the estimation for the best focus of the telescope FWHM pixels 32 6 32 7 32 8 32 9 33 0 33 1 33 2 T FOCUS mm 6 Finally the best focus obtained will be sent to the OT which will ask you for confirmation for setting the new telescope focus Subtract own sky It subtracts the background to the current selected image the background computation is done using the own image For this BACKGROUND option of SExtractor is used Subtract near sky It subtracts the background to the currect selected image using t
119. y is fine Testing Python module installation for module scipy PAPI needs at least version 0 10 Your version 0 10 1 of scipy is fine Testing Python module installation for module pywcs PAPI needs at least version 1 11 Your version 1 11 4 10 of pywcs is fine Testing Python module installation for module PyQt4 QtCore PAPI needs at least version 4 8 Your version 4 9 1 of PyQt4 QtCore is fine Testing Python module installation for module vo PAPI needs at least version 0 7 Your version 0 8 of vo is fine 56 Chapter 2 Contents PAPI Documentation Release 1 2 20151020173439 Testing Python module installation for module numpy PAPI needs at least version 1 6 Your version 1 6 2 of numpy is fine Testing Python module installation for module pyraf PAPI needs at least version 1 1 Your version 2 0 of pyraf is fine Testing Python module installation for module matplotlib PAPI needs at least version 0 98 1 Your version 1 1 0 of matplotlib is fine Testing Python module installation for module pyfits PAPI needs at least version 3 0 Your version 3 1 0 of pyfits is fine collapse Sum the planes of each cube of a list files into a single plane 2D image Usage collapse py options argl arg2 Options h help show this help message and exit i INPUT IMAGE input image INPUT IMAGE input cube image to collapse into a 2D image 1 INPUT IMAGE LIST input im

PAPI Documentation

Contents

Download Pdf Manuals

Related Search

Related Contents