Extending Depth of Field in Microscopy
Contents
1. A simple extension of this technique can be used to combine the in focus parts of any number of photographs The Practice Micrometrics Multi Focus Composition MFC is the software tool built upon the info softinstrument com Scientific Instrument Software Corporation Limited above ideas Understandably for MFC to be useful in practical situations and scientific researches more sophisticated formulations have to be employed so as to guarantee a correct reconstruction Fortunately MFC is extremely simple to learn and convenient to use due to a very clear and in depth understanding of the image formation process In order to extend the depth of field you need to acquire a series of images at successive heights covering the whole axial extent of the object space in steps smaller than depth of field of the optics In order to use MFC to create a wholly focused image you need to store these images in a folder with their file names in dictionary order corresponding to sequence of focusing It is required that these images are of the same width and height and in the same format although the format can be any of the most popular ones such as JPEG or TIFF MFC implements two major functions image fusion and registration Image fusion is the core of the quite involved procedure of merging focuses Of course MFC keeps all the complexities to itself and requires no interference from user side That is in the context of metallograp2. Displacemen Image Plane Object Position Objec Object Position Illustration 2 Limitations on the depth of field should not be a problem for routine observation as you may adjust the focus knob to examine various parts of the specimen It can be a problem however when you need to record your findings unless we save a stack of images acquired at various heights where for each part of the specimen we wish to look at we have it focused within at least one of the images among the stack Needless to say the stack is burdensome to store and cumbersome to recall In this case we will need a software tool to fuse these images into a single picture that is properly focused at every part of it To work with stereomicroscopes images have to be registered before they are info softinstrument com Scientific Instrument Software Corporation Limited subject to fusion The Principle The core of image fusion is a proper selection of best focused pixels from all available images The technical note written by Paul Haeberli in October 1994 A Multi Focus Method for Controlling Depth of Field explains the approach which we reproduce below with Paul s kind permission Although we illustrate with an example from macro photography the same idea also applies to microscopy Introduction When a photograph is taken with a camera the lens is focused at a particular distance Objects nearer or farther than this focal di
3. Image 17 A bolt from an aircraft info softinstrument com Scientific Instrument Software Corporation Limited Image 18 The bolt documented with MFC info softinstrument com
4. MFC accepts images of very large size and in many formats facilitating interfacing to a very wide variety of cameras To master MFC a microscopist needs to understand only one controlling parameter Maximum Step Displacement which refers to the greatest shift between adjacent images when working with stereomicroscopes MFC is otherwise fully automatic keeping all the algorithmic complexities to itself To use MFC 1 Capture a stack of images and put them in a previously empty folder Do not alter any settings of the microscope and camera during the process except the focus Save the images in one format with names in dictionary order For example Image 01 bmp Image 02 bmp and so on 2 Specify an integer that is the upper bound for the possible shifts between adjacent images If such a value is not known initially experiment from small values to larger ones The program will give the detected shifts during computation process Use them as feedback for the training Use the default value 0 for compound microscopes 3 Open the image files to begin depth of field extending the rest needs no user input More details follow info softinstrument com Scientific Instrument Software Corporation Limited Operations Scenario 1 Micrometrics Multi Focus Composition Maximum Step Displacement E Index File Horz Vert Specify Maximum Step Displacement This is the only controlling parameter for extending de
5. Windows File Open dialog box where you browse to the folder which contains the images of the specimen that you have collected at various focus settings as is shown below 10 2 em Ba B20 Filename BZ05 jpq BZ06 ipg BZ07 ipg BZ08 jpg Open Files of type JPEG ipa v C Open as read only Illustration 4 After a file filter Files of type has been chosen all files of the specified type in the folder will be listed As is typical Windows practice you can check Thumbnails in View Menu for a preview of the images You can also press CTRL A to select all listed files for maximum simplicity When you have selected all of the images you wish to include in this computation and pressed Open button MFC will merge them into a totally focused image The final image is stored by the name MM3D BMP in the same folder as input images and is opened automatically by the default Bitmap viewer or editor for viewing and editing There is nothing else you need to do info softinstrument com Scientific Instrument Software Corporation Limited Stereomicroscope Imaging with stereomicroscopes requires an additional step Before pressing the Start button you need to specify the MSD in the edit box to the left of Start button as shown below Micrometrics Multi Focus Composition Maximum Step Displacement 12 Index File Horz Ver
6. edit box for each direction The check box is used to indicate to MFC if this particular setting will be used at all and in the case it will be used the edit box supplies the value Remember in the main program there is also the parameter MSD hereafter referred to main MSD the final parameter setting is as the following 1 If neither Vert nor Horz is checked main MSD will be used for aligning in both horizontal and vertical directions 2 If either Vert or Horz but not both are checked main MSD will be used in place of the unchecked direction 3 If both Vert and Horz are checked main MSD will not be used Of course in the degenerated case where zero is specified for a direction no registration is attempted for that direction This may happen when the camera is neatly mounted where at most lateral translation of one direction can occur You may wish to exploit this to speed up the registration process However the algorithm we used for MFC is highly efficient so your saving of time by doing so is not significant info softinstrument com Scientific Instrument Software Corporation Limited If you have a motorized stage attached to your stereomicroscope you may bypass the registration process In this situation it is simple to adjust the focus of your stereomicroscope in uniform steps and the lateral translation between adjacent images should also be uniform For each stepping configuration o
7. hundreds of images Because MFC loads one image at a time total number of images does not matter MFC is able to process images of millions of pixels With reference to optical resolution cameras of 1 6 million pixels are the right choices for microscopy MFC is designed to work with images of various resolutions Image 3 The above image is created from 75 sections each having more than 5 35 million pixels info softinstrument com Scientific Instrument Software Corporation Limited 2600x2060 Note that the nail is extremely demanding since it significantly extends the object space Blur Removal The primary and most annoying blurs in microscopic imaging are out of focus blur and spherical aberration While MFC mainly means to remove out of focus blur by merging in focus parts of a stack of partially focused images it also provides an equally striking functionality to correct spherical aberrations The two functions work in a row to deliver crystal sharp pictures Spherical aberration correction called DEHAZE in MFC is an advanced option and can be accessed in control menu Settings As with the overall design of MFC DEHAZE is very easy to use one controlling parameter suffices for the very complicated process of image restoration The following three images serve as an illustration of DEHAZE Image 4 is the best we could capture directly from a stereomicroscope Image 5 is result of depth of field extend
8. on the PC a message window will pop up at the lower right corner of the screen as shown below i Found New Hardware Senselock SENSES v1 70 gt gt 9 55am The system has now detected the presence of the dongle It will then prompt to install the driver for the dongle with a dialog box like the following Found New Hardware Wizard Welcome to the Found New Hardware Wizard This wizard helps you install software for Senselock SENSES v1 70 If your hardware came with an installation CD lt 6 or floppy disk insert it now What do you want the wizard to do Install from a list or specific location Advanced Click Next to continue The driver has already been copied to hard disk by the setup program so please select the first option Install the software automatically Recommended After that a warning message may appear like this info softinstrument com Scientific Instrument Software Corporation Limited Hardware Installation The software you are installing for this hardware Senselock SENSE3 v1 70 has not passed Windows Logo testing to verify its compatibility with Windows XP Tell me why this testing is important Continuing your installation of this software may impair or destabilize the correct operation of your system either immediately or in the future Microsoft strongly recommends that you stop this installation now and contact the hardware vendor for software
9. FC begins to detect the possible lateral shifts between the first and second image if MSD has not been set to 0 Image fusion follows Detected shifts between successive images are provided in the third and fourth columns starting from the second rows as depicted below info softinstrument com Scientific Instrument Software Corporation Limited Maximum Step Displacement 12 Index File Horz Vert BZ05 1280 1024 BZ06 B20 B2Z08 B209 B2Z10 BZ11 BZ12 BZ13 B214 0 1 2 3 4 5 6 7 8 9 MMM amp MPN W ocoosacocooOo ono Illustration 8 The detected shifts are mainly useful to imaging with stereomicroscopes If you merge images captured from metallographic or biological microscopes and specify 0 for MSD MFC will not attempt to detect the shifts since there is none In this case the third and fourth columns will be filled with zeros and serve only as an indicator of fusion progress For stereomicroscopy the utility of detected shifts is twofold If any of the detected shifts is greater than the specified MSD you probably need to adjust the focus knob of the stereomicroscope in finer steps or specify a greater MSD You can of course press Abort button to interrupt the current computation or allow it to be finished for a fuller idea on all focusing steps On the other hand if none of the given shifts exceeds the MSD you most likely will arrive at a correct registration In rare cases where the image regis
10. Please enter your information nter your name and the name of the company for which you work User Name Company Name ACCU SCOPE INC InstallShield The second place asks where to create the program folder as shown in the next screenshot info softinstrument com Scientific Instrument Software Corporation Limited fame s Composition InstallShield Wizard x Choose Destination Location ct folder where setup will install files Install Multi Focus Composition to C Micrometrics Multi Focus Composition InstallShield The program folder may be anywhere on hard disks Press Change button to browse to a new location Scientific Instrument Software Corporation Limited The third place prompts for a restart of computer as illustrated in the following Multi Focu s Composition InstallShield Wizard InstallShield Wizard Complete The InstallShield Wizard has successfully installed MultiFocus Composition Before you can use the program you must restart your computer Yes want to restart my computer now No will restart my computer later Remove any disks from their drives and then click Finish to complete setup InstallShield Reboot is NOT necessary unless a complete refreshing of system is desired Scientific Instrument Software Corporation Limited 4 Plug in the dongle If it is the first time that Micrometrics Multi Focus Composition is being installed
11. Scientific Instrument Software Corporation Limited Extending Depth of Field in Microscopy Tools for this purpose are available as both a command in the standard microscopy software Micrometrics SE SE Premium and the standalone program Micrometrics Multi Focus Composition MFC In SE the images are stored in an internal buffer Field Group whereas in MFC the images are loaded directly from disk files The focus enhancement technique is otherwise identical among Micrometrics family of software products The following discussion is based on the standalone program MFC info softinstrument com Scientific Instrument Software Corporation Limited Installation Guide System Requirements Memory 512MB or More Operating System Microsoft Windows Vista XP or 2000 Installation 1 Remove the dongle from the PC if it is currently attached to the PC 2 Download www micrometrics net download mfc rar which resolves to autorun inf and mfc exe when unzipped To create an installation CD burn these files onto a CD 3 Insert the installation CD into the computer or double click mfc exe and follow the instructions on screen There are three places that might need user input The first place is for the user name and company name as shown below info softinstrument com Scientific Instrument Software Corporation Limited Multi Focus Composition InstallShield Wizard Customer Information
12. ates the situation with microscopic imaging but the same problem also exists in macro photography Objective Reflective Light Space Object Depth of Field SC rere gt Specimen Illustration 1 info softinstrument com Scientific Instrument Software Corporation Limited Axial resolution of microscope like horizontal resolution is determined by numerical aperture alone In microscopy the depth of field is very shallow and usually measured in units of microns The following table presents variations in depth of field of a series of objectives with increasing numerical aperture and magnification Depth of Field Magnification Numerical Aperture x fits 02s a Table 1 The case for stereomicroscope is slightly more complicated where depth of field is strongly influenced by total magnification including contributions of both objective and auxiliary attachment lenses A typical scenario is given below Objective Zoom Factor Numerical Aperture Depth of Field Micrometers x ody m 0s fo Bs fons fs na Yous e info softinstrument com Scientific Instrument Software Corporation Limited Table 2 Another complicating factor of imaging with stereomicroscopes is the lateral displacements of images as recorded at different focus settings This is due to the common practice that camera port shares the same oblique light path as one of the eyepieces which is illustrated below
13. be applied If you wish to restore the default settings just delete the file engine mfc and it will be re created with default values info softinstrument com Scientific Instrument Software Corporation Limited Align the Input Images Settings Registration Smat Search 64 Margin O Classic Quality Dehaze Maximum Step Displacement JHorz 12 C Vert Resolution Denoise O Fixed Step Displacement 2 Delatitude Redetail Crop Brightness Left Fight Top Bottom Direct O Linear C Manual 0 Maximum C Automatic 0 lo Output File C MM3D BMF When imaging with stereomicroscope the images may shift with respect to each other due to the fact that the video port shares one of the two oblique light paths of the eyepieces This is the only case where lateral displacement of images has to be taken care of MFC is designed primarily for users who do not have a motorized stage that is focus will be adjusted manually In this situation the shift of image may vary MFC attempts to determine the amount of lateral translation between adjacent images from the images themselves This process of image alignment is called registration Registration can be based on the whole images or smaller regions of images Registration based on whole images is called Classic method in MFC while registration based on smaller regions of images is called Smart met
14. c shift detecting operation mode of MFC helps to achieve sub pixel accuracy in aligning images info softinstrument com Scientific Instrument Software Corporation Limited The Program MFC has a very simple structure and consists of four files in a folder an executable Df exe a dynamic link library Scopex dll a user s manual in Compiled HTML format Micrometrics Multi Focus Composition chm and a temporary configuration file Engine mfc The installation program will create a new folder called program folder on the computer to hold these files The architecture is illustrated below Program Folder Micrometrics Scopex dll Multi Focus Composition chm Illustration 10 Df exe is responsible for user interface i e the dialog It transmits user specifications i e file names MSD and so on to Scopex dll and displays feedbacks from Scopex dll For the vast majority of imaging works you will be operating from this simple dialog alone However MFC does provide a number of advanced options You can access these options by the augmented control menu which can be brought up by either click on program icon or right click on caption bar The control menu is shown below The control menu contains besides the usual Move and Close another four items Settings Manual Online Help and About Multi Focus Composition Settings shows a dialog box hosting all the ad
15. d with such a detail you may use the crop functions of MFC There are two crop settings Automatic mode means to have MFC to handle the cropping operations In this case the output image is smaller than input image since only the region where all images overlap will be saved Upon completion of each computation the crop values will be given for reference purpose Manual mode means to cut the outer rows and columns as directed by the four parameters Left Top Right and Bottom The two modes can both be turned on in this case Automatic crop is done first and Manual crop is done for the output of Automatic crop info softinstrument com Scientific Instrument Software Corporation Limited Set an Alternative Output Image File Settings Displacement Registration ae Smat Search 64 Margin O Classic Quality Dehaze Maximum Step Displacement Horz 12 C Vert Resolution O Denoise Fixed Step Displacement Delatitude Horz 2 Redetail Crop Brightness Left Fight Top Bottom i Direct O Linear C Manual O Maximum C Automatic Output File C MM3D BMF The default output image file of MFC is named MM3D BMP in the same folder as the input image files However you are free to set a different path and name for the output image file To do so just check and fill Output File You may use the button to brow
16. e image by 2 bits On the other hand we can artificially raise the high frequency part of an image to reveal the details In signal processing jargon this is called high boost filtering Here we also need to take care of visual psychology which says variations in dim regions of an image are often interpreted as noise For this reason we have introduced the projecting operator p which modified the high frequency pixel values according to spatially corresponding low frequency pixel values before they are added together to create the enhanced image r x y As with feature equalization MFC works in hundredth To further simplify the use of MFC only two parameters have to be specified Relatitude is a in hundredth for example if you wish to shrink the dynamic range of the output image by 1 bit set Relatitude to 50 Redetail refers to p but here you just need to specify an integer and MFC will handle the selection of projecting operator Valid values for Relatitude are integers between 1 and 99 Valid values for Redetail are integers greater than 100 Setting Relatitude and Redetail to 100 will have no effect on the output image info softinstrument com Scientific Instrument Software Corporation Limited Adjust Brightness of the Output Image Settings Displacement Registration ae Smat Search 64 Margin 32 O Classic Quality Dehaze Maximum Step Displacement Horz 12 Cvet 12 Resolutio
17. e and is otherwise identical to the info softinstrument com Scientific Instrument Software Corporation Limited situation when the dongle is present Apparently you can copy the whole program folder to another computer for demonstration purpose After you have done with the demonstration just remove it and no fingerprint will be left The Performance Fidelity and Accuracy Extending of depth of field is needed in microscopy as another image forming process instead of electronic montage performed by computer artists We naturally ask if the resulted image faithfully reproduces the object had we had the optics of greater depth of field The crucial part of multi focus composition is the way focus is determined Apparently the degree of being focused for a pixel cannot be decided by this pixel alone As explained in Paul s paper we need some neighboring pixels to comprise a contrast MFC extends this idea further to exploit neighboring pixels in three dimensions i e adjacent planes As a consequence to focus determination we need ways to group pixels and to reconstruct an image from these groups so that no part is missing in the final composition MFC is an analytic approach guaranteeing correctness of focus inference and completeness of composition It does not depend on nor contain ad hoc means to achieve these ends MFC performs focus finding on sub pixel level During the process it attempts to split unresolved pixels until thei
18. er method for easy understanding although we are not using linear shift invariant filters for MFC An image f x y can be separated into two parts by passing it through a low pass filter g x y f x y h x y t x y f x y g x y 1 x y a g x y p g xy ty In the above g x y is the low frequency part of f x y and t x y is the high frequency part of f x y stands for convolution h x y is a low pass filter such as a box filter or a Gaussian filter a is a constant to be multiplied to the low frequency part of f x y p is the a one dimensional function called projecting operator and finally r x y is the enhanced image we wish to derive from f x y The low frequency part g x y defines the dynamic range of the original image f x y The greatest value of g x y is called the latitude of f x y To shrink the dynamic range info softinstrument com Scientific Instrument Software Corporation Limited of the resulting image r x y we just need to multiple g x y with a number smaller than 1 i e set a lt 1 The reason why we need to reduce the latitude of an image is that sometimes the dynamic range of an image exceeds that of the display device Even if the dynamic range of the image matches that of the display device we still have to consider the perception capability of human eyes Setting a 0 5 is equal to reducing the dynamic range of the image by 1 bit Setting a 0 25 is equal to reducing the dynamic range of th
19. f the motorized stage sometimes simply a z stepper you may first use the above registration method to determine the shift but then just select Fixed and specify the detected values and registration will be performed no more To summarize in Displacement section of the advanced options select Registration and specify the needed parameters if you do not use a motorized stage or select Fixed and specify the constant values of shift if you do have a motorized stage Crop the Output Image Settings Displacement Registration Method Smat Search 64 Margin O Classic Quality Dehaze Maximum Step Displacement C Horz 1 2 C Vert Resolution Denoise O Fixed Step Displacement Delatitude Hoz 2 Redetail Brightness Direct O Linear O Maximum Output File CAMM3D BMAy info softinstrument com Scientific Instrument Software Corporation Limited The output image of MFC corresponds spatially to the first input image The parts of the other input images not overlapping with the first input image will be cropped quietly This rule of cropping suggests the best sequence of image acquisition For example if we know that images will shift to the bottom right direction it is best to start the image capture from an image which is perfectly focused in the upper left part When you do not have much control over the image acquisition process or do not want to be bothere
20. he images within the collection Appropriate numerical methods may then be employed to extract best focused parts of each image and reconstruct a totally focused image from these parts as illustrated below The upper half of the object is in focus The lower half of the object is in focus The whole object is in focus When imaging with stereomicroscopes it is necessary to register the images before performing the depth of field extending This is due to the practice of attaching the camera to one of the oblique eyepiece bore of stereomicroscope The image shifts when focus knob is adjusted If a motor stage is employed to aid image acquisition the step info softinstrument com Scientific Instrument Software Corporation Limited size can be guaranteed to be uniform so is the shift between adjacent images If the focus is adjusted manually the step size and shift may vary Micrometrics Multi Focus Composition MFC is an easy to use software tool for extending depth of field in optical microscopy It is designed to complement existing microscopic imaging systems For this reason it operates on disk files and does not depend on the presence of any special hardware or software In particular a motor stage is not needed although it certainly simplifies the work In fact during the process MFC reveals the spatial translations detected which helps a microscopist to train himself toward a more uniform focus adjustment
21. hic and biological microscopes or repro stands with macro or zoom lenses perpendicularly mounted depth of field extending with MFC is fully automatic You just need to make sure to adjust focus knob in steps smaller than the depth of field of the optical system you are using Image registration is a preprocessing to image fusion to correct for translations of images in x and y directions and is invoked only when imaging with stereomicroscopes If we have a mechanism to adjust the focus in uniform steps we will be able to deal with shifts in many ways If we have to adjust the focus manually we may have to determine the shifts from the very images we use to extend depth of field It then becomes necessary for practical efficiency reasons only to specify a controlling parameter Maximum Step Displacement MSD which means to be the biggest possible shift in pixels between adjacent images within the stack If you specify 0 for MSD no image registration will be performed If you specify a value greater than 0 MFC will interpret this value as the greatest possible shift in both x and y directions and will search within the range for a best estimate of the actual shift Alignment of images is subsequently handled by MFC according to shifts detected info softinstrument com Scientific Instrument Software Corporation Limited Metallographic and Biological Microscope From a user s perspective MFC is simply a dialog box Upon start MFC a
22. hod since here info softinstrument com Scientific Instrument Software Corporation Limited MFC is responsible for selecting corresponding regions within each two adjacent input images In Smart registration method you will need to specify two controlling parameters Search and Margin Search means the size of a square area of image measured in pixels which contains a representative feature of the image For example if you set Search to 64 then MFC will search each pixel array of 64x64 pixels within each image to determine the degree of correspondence between the two adjacent images The default value 64 usually works well however you are free to experiment for an optimal value Margin means the number of rows and columns counting from each border of the image to be excluded from registration The reason for excluding outer regions of each image is that uneven illumination at the border of field of view may interfere with the registration process Again the default value 32 should be applicable in most cases Usually you don t have to spend a lot of time on deciding this value Registration trials will be performed in both directions horizontally and vertically For both methods of image registration you must specify Maximum Step Displacement MSD This parameter places an upper limit on the registration trials for each direction Note in the dialog box there are both a check box and an
23. ile operations in a separate thread avoid blocking user interface A user program may also link to Scopex dll for the added functionality of field depth extending info softinstrument com Scientific Instrument Software Corporation Limited Settings Displacement Registration ee Smat Search 64 Margin O Classic Quality Dehaze Maximum Step Displacement Horz 12 C Vert Resolution Denoise O Fixed Step Displacement Horz 2 i Redetail Delatitude Crop Brightness Right Top Bottom Na nt O Direct O Linear C Manual Maximum C Automatic C Output File D S1 MFC BMP Illustration 12 MFC does not contain intrinsic links to bookmarks of the manual You are free to translate the content into a language other than English or to create a totally new one As long as you keep the file name to Micrometrics Multi Focus Composition chm MFC is always able to load it by the Manual command MFC enforces the so called green installation in that it does not modify the registry of the host computer except for setting up the dongle and it does not copy files to folders other than its program folder You can completely remove the program by simply deleting its folder In fact Scopex dll only checks the presence of the dongle upon completion of a field depth extending task If the right dongle is not found it simply stamps a few lines of text on the final imag
24. ing with 28 partially focused images Image 6 is the output of DEHAZE info softinstrument com Scientific Instrument Software Corporation Limited Image 4 info softinstrument com Scientific Instrument Software Corporation Limited info softinstrument com Scientific Instrument Software Corporation Limited info softinstrument com Scientific Instrument Software Corporation Limited Latitude Reduction and High boost Filtering Some specimen may have critical information in both very dark and bright areas While we can invest on cameras with very large dynamic ranges to capture the right image we still have difficulties in display the obscured details The two advanced options of MFC DELATITUDE and REDETAIL are helpful in this regard DELATITUDE shrink the dynamic range of the image without sacrificing local contrast so that all features of interest are perceived in normal display print range REDETAIL goes a step further to boost the previously buried details Again DELATITUDE and REDTAIL each requires only one controlling parameter Noise Suppression In fluorescent microscopy and low light photography images may appear grainy Another advanced option of MFC DENOISE might be of help in reducing noise DENOISE attempts to determine the statistics of noise from all images in the stack and operate in an optimal manner to suppress the noise in the reconstructed image Again DENOISE is configurable with a si
25. master the peculiarities of the ways a particular program drives cameras One the other hand all camera manufacturers supply software for capturing images onto disk files In the case of digital photograph camera storing acquired images onto dictionary order files into a single directory is the industrial standard on Windows XP Speed MFC is very fast In fact more time is spent on reading files than computation This is primarily because the algorithms for image registration and fusion are highly advanced For the sake of precision images in MFC are represented by floating points However the codes are highly optimized to conform to or exploit the computer architecture and operating system Image 2 It takes less than 3 minutes to synthesize the above image from 46 sections each having info softinstrument com Scientific Instrument Software Corporation Limited more than 3 million pixels We were using a consumer PC for the task The configuration was Intel Pentium IV CPU 2 0GHz 1GB DDR Memory and 5400RPM hard disk If you are to use RAID or SCSI disks you will save considerable time as file reading and writing consumes significant portion of the total time needed by MFC Handling Large Number and Size of Images MFC accepts unlimited number of input images Normally a few dozens of sections work fine for our purpose However in demanding situations such as imaging thick specimen under high magnification we may arrive at
26. n Denoise O Fixed Step Displacement Delatitude Horz 2 Redetail Crop Brightness Right Top Bottom Direct O Linear C Manual O Maximum C Automatic Output File C MM3D BMP There is chance to adjust the overall brightness of the MFC output image with this option If you select Direct no adjustment will be applied and the intensity of output image will be the average of those of the input images If you select Linear the dynamic range of the output image will be extended to 8 bit if it is not already If you select Maximum the smallest intensity value of the output image will be mapped to zero Note that it is best not to set this option for color images since color may change after the mapping info softinstrument com Scientific Instrument Software Corporation Limited Technical Note The Problem While there are many causes to focusing difficulty limited depth of field is of a fundamental nature and cannot be worked around by optics alone Depth of field refers to the axial resolving power of an optical system which may be conveniently identified as the distance between the nearest object plane in focus and the farthest object plane also simultaneously in focus When the axial extent of the object space exceeds the depth of field of the optics it is impossible to straightforwardly capture an image that is in focus everywhere The following figure illustr
27. ngle parameter info softinstrument com Scientific Instrument Software Corporation Limited The Proof Metallographic Imaging Tilted Sample t go te we To pi f ps ay es 4 ta by i Rk Image 7 The left part is MFC output The right part is one of the input images Scientific Instrument Software Corporation Limited Jagged Surface Image 8 The left part is MFC output The right part is one of the input images Scientific Instrument Software Corporation Limited Pathology Image 9 This is one of the directly captured images of a thick specimen Scientific Instrument Software Corporation Limited Image 10 MFC output for imaging thick specimen under biological microscope Scientific Instrument Software Corporation Limited Fossil Image 11 The best we see under stereomicroscope Image 12 MFC used in archeology info softinstrument com Scientific Instrument Software Corporation Limited Plant Image 13 One of the stereoscopic images for documentation of herb medicine Image 14 MFC helps to generate a more focused image info softinstrument com Scientific Instrument Software Corporation Limited Insect Imaging Image 15 An Aphid examined under stereomicroscope at 60X Image 16 MFC employed at documentation stage info softinstrument com Scientific Instrument Software Corporation Limited Fracture Analysis
28. nslations between successive images and to fuse the most focused parts of each image one by one The caption bar indicates the portion of the whole task that has been finished and the time needed to complete the remaining work After each image has been processed the detected shift relative to the previous image is given in the third and fourth columns of the information window When the computation ends the resulted image is stored by the file name MM3D BMP in the same directory as input images and is opened automatically by the default Bitmap viewer or editor Click on Abort button to cancel the computation In this case the state returns to starting up except the content of information window remains Refer to Scenario 1 info softinstrument com Scientific Instrument Software Corporation Limited Scenario 5 MM3D BMP Paint File Edit View Image Colors Help a S0R0SSeec SSC ee 2 oe For Help click Help Topics on the Help Menu k Preview and Edit The final image may then be saved under a new name in another folder It is also convenient to edit the image such as placing text and graphical overlays info softinstrument com Scientific Instrument Software Corporation Limited Advanced Options Configure the Computation Engine Behind the apparent simplicity of MFC is a rather sophisticated computation engine For the vast majority of applications it is not necessary to set the advanced option
29. oftinstrument com Scientific Instrument Software Corporation Limited impossible to do so due to the following three reasons 1 Imprecise modeling of image degradation process 2 Noise in images 3 Finite word length of our computers We usually can set our goal at some time around 0 8 MFC works with the parameter in hundredth so this time translates to 80 in Dehaze Valid values for Dehaze are integers greater than 0 Note setting Dehaze to 100 means doing nothing A value greater than 100 will lead to blur A value less than 100 will attempt to correct the spherical aberrations but too small a value may cause the resulting image look faked Resolution Settings Displacement Registration ene Smat Search 64 Margin O Classic Quality Dehaze 100 Maximum Step Displacement C Horz 12 C Vert Resolution 6 P O Denoise Fixed Step Displacement Horz 2 Redetail Delatitude Crop Brightness Right Top Direct O Linear C Manual O Maximum C Automatic Output File C AMM3D BMF info softinstrument com Scientific Instrument Software Corporation Limited This is a quite involved concept It defined the way in general how MFC handles contrast Valid values for Resolution in the advanced options are integers greater than 0 and less than the greatest integer which when raised to power of 2 is less than the width and height of the images as mea
30. oftware Corporation Limited Scenario 3 6 Initializing Maximum Step Displacement 12 Index File Horz Vert B205 1280 1024 BZ06 B20 B208 B209 B210 B211 BZ12 BZ13 B214 0 1 2 3 4 5 6 7 8 9 Initialize The computation engine is set up after the images have been loaded The state is indicated on the caption bar Note the former Start button changes its text to Abort The information window i e the list is then stuffed with status of the computation The first column gives the indices of the images The second column gives their corresponding file names The first row of the third and fourth columns gives the width and height of the images while the rest rows are to display detected shifts in horizontal and vertical directions respectively once the initialization finished Click on Abort button to cancel the computation In this case the state returns to starting up except the content of information window remains Refer to Scenario 1 info softinstrument com Scientific Instrument Software Corporation Limited Scenario 4 Micrometrics MFC 10 28 finished 1 24 left Maximum Step Displacement 12 Index File Vert B205 1024 BZ06 B20 B208 B209 B210 B211 BZ12 BZ13 B214 0 1 2 3 4 5 6 7 8 9 MMM amp MPM Nw oocea cc ooo Align and Fuse Images Once initialized the computation engine proceeds to detect and accommodates for possible spatial tra
31. ppears as below Micrometrics Multi Focus Composition Maximum Step Displacement Index File Illustration 3 There are only two places on the dialog which might need user input The first is the edit box to the right of the text Maximum Step Displacement where you specify a controlling parameter for image registration when imaging with stereomicroscopes You can ignore this parameter when imaging with metallographic or biological microscopes The second is the Start button to the right of edit box You press this button to load image files and initiate depth of field extending The table at the lower half of the dialog box is for information on input images and progress of computation It has four columns Index File Horz and Vert Upon the loading of the image files the first and second columns will be stuffed with indexes and file names of the input images The first rows of the third and fourth columns will contain the width and height of the images Once the computation starts the rest of the rows of the third and fourth columns will give detected shifts in horizontal and vertical directions for each image relative to the previous one Caption bar of the dialog box will also inform the progress of computation and the time info softinstrument com Scientific Instrument Software Corporation Limited needed to complete the task You press Start button to bring up the standard
32. pth of field which stands for the greatest possible spatial translation between any two adjacent images measured in pixels within the stack Type an appropriate integer in the edit box Specify a small value such as 12 when imaging with stereomicroscopes Always experiment from small values to large ones until the reconstructed image looks plausible Specify 0 in all other forms of optical microscopy Click on Start button to load a stack of images Refer to Scenario 2 info softinstrument com Scientific Instrument Software Corporation Limited Scenario 2 File name BZ05 ipg BEZOS ipg B20 jpg BZ08 jpg Files of type JPEG jpa Ma C Open as read only Choose Image Files This is the standard Windows File Open Dialog Select at least 2 files All images must be identical in width height and number of color channels For maximum efficiency press CTRL A to select all listed files and click left mouse button while holding down CTRL to deselect unwanted individuals All files to load will be sorted automatically into dictionary order independent of order of selection The output image will be stored by the name MM3D BMP in the same directory as input images Click on Open button to begin extending depth of field Refer to Scenario 3 Or click on Cancel button to return to initial state Refer to Scenario 1 info softinstrument com Scientific Instrument S
33. r depths are found MFC also simulates the behavior of objectives of lower numerical apertures for increased depth of field In this way even the blurred parts of images contribute to the total resolution of output image Simply put it MFC guarantees an image fusion free of artefacts and a resolution no less than that of all originals We give an example below The left part of the image is a reconstruction by MFC and the right part is result created with a well known program for the same purpose It is very easy to tell the difference Obviously the many arteifacts appearing on the right part are due to an improper approach to focus evaluation info softinstrument com Scientific Instrument Software Corporation Limited Image 1 Ease of Use Learning MFC is trivial It operates like filling a simplest dialog box If you are imaging with metallographic or biological microscopes or repro stands no control parameter needs be specified at all Extending depth of field from a stack of images is equivalent to opening a few files with standard Windows File Open Dialog If you are imaging with stereomicroscopes all you need to know is the simple fact that the images may info softinstrument com Scientific Instrument Software Corporation Limited shift laterally One controlling parameter Maximum Step Displacement suffices for this work Routine work with MFC is convenient MFC accepts files as input You don t have to
34. s In rare cases where you wish to optimize MFC or wish to exploit many additional functionalities built into MFC you may do so by fill a dialog box To bring out the dialog box first click on the program icon or right click anywhere else on the caption bar in the control menu that appears select Settings as shown below Micrometrics Multi Focus Composition Alt F4 Manual Online Help About Multi Focus Composition The dialog box where all advanced options can be set is as the following info softinstrument com Scientific Instrument Software Corporation Limited Settings Displacement Registration pee Smat Search 64 Margin O Classic Quality Dehaze Maximum Step Displacement Horz 12 C Vert Resolution Denoise O Fixed Step Displacement Delatitude Horz 2 Redetail Crop Brightness Left Fight Top Bottom i Direct a l O Linear C Automatic 0 o L Manual O Maximum Output File C MM3D BMP All settings will take effect immediately after you dismiss the dialog box with OK button When the program exits all valid parameters and selections will be written to the disk file engine mfc in the program folder i e the directory where Df exe resides When the program starts up it will look for the file engine mfc in the program folder If the right file can be found it will be loaded and the settings recorded therein will
35. s CTRL CLICK to add or remove a particular image from selection You can also SHIFT CLICK to select a subset of all images according to some ordering The reasons to exclude some images from participating computation can be that too closely spaced images contribute little to field depth augmentation or you are not so sure of their focus sequence After you press Open button MFC reads in the first image and initializes the computing engine as shown below info softinstrument com Scientific Instrument Software Corporation Limited 6 Initializing Maximum Step Displacement 12 Index File Horz Vert B205 1280 1024 BZ06 B20 B2Z08 B209 B2Z10 BZ11 BZ12 BZ13 B214 0 1 2 3 4 5 6 7 8 9 Illustration 7 The table in the dialog is now staffed with information of input images The first column lists the serials of the images as sorted in dictionary order the second column gives the corresponding file names the first rows of the third column and the fourth column present the image width and height respectively Note the width and height might be slightly different from the original as MFC requires that they be multiples of 8 for performance reasons Note that the Start button has changed its text to Abort to indicate computation in progress From now on you can press this button to terminate the computation for any reasons As soon as the necessary resources have been allocated M
36. se to a new destination Note the output image has to be in Bitmap 3 format info softinstrument com Scientific Instrument Software Corporation Limited Feature Equalization Settings Displacement Registration ae Smat Search 64 Margin O Classic Qual Dehaze 100 Maximum Step Displacement Horz 12 C Vert Resolution Denoise O Fixed Step Displacement Delatitude Horz 2 Redetail Crop Brightness Left Fight Top Bottom Direct O Linear C Manual 0 O Maximum C Automatic ae E Output File C MM3D BMP Even if the microscope is of perfect workmanship the image may still appear hazy especially under high magnification This is because of the spherical aberrations inherent in any optical system where a small dot may be imaged as a large dot not proportional to the total magnification and nearby pixels tend to blur each other Feature Equalization is a technique to remove haze from images To understand the technique we can visualize the degrading of image as a time process At time 0 we have a perfect image At time 1 we have the observed image i e the degraded image If we can go back to any time between 0 and 1 we would arrive at an image that is better in quality than the observed one Feature Equalization is a technique of inverting the time process It would be ideal that we go back to time 0 where there is no degradation However it is info s
37. stance will appear blurred By changing the focus of the lens near objects or distant objects can be made to appear in sharp focus If you want to create an image where distant objects as well as close objects are in focus two or more images can be merged together to make an image with increased depth of field The Technique Here are two images of the same scene one focused close and the other focused at a distance We can combine the in focus parts of both photographs using the following procedure First each input image is blurred info softinstrument com Scientific Instrument Software Corporation Limited Next we subtract the blurred image from the original above it and create an image that shows the magnitude of the difference This image will be dark where the original image is smooth and will be bright where the original image has edges The strength of the edge information maps directly into the brightness of these edge images Now we compare the two edge images and make an image that is black where the left image has more edge information and is white where the right image has more edge info softinstrument com Scientific Instrument Software Corporation Limited information Finally this is used to create an image with the best parts of each original image Where the image above is black pixels from the left image are used Where the image above is white pixels from the right image are used
38. sured in pixels 4 5 and 6 are the most frequently used settings and there is usually no reason to go for other values Noise Suppression Settings Displacement Registration mare Smat Search 64 Margin O Classic Quality Dehaze Maximum Step Displacement Horz 12 C Vert Resolution O Fixed Step Displacement Delatitude Horz 2 Redetail Crop Brightness Left Fight Top Bottom a l Direct O Linear C Manual 0 0 O Maximum C Automatic 0 0 Output File C MM3D BMFI This is only useful to very noisy input image As with most noise suppression method spatial averaging happens The uniqueness of MFC s approach is to limit the spatial averaging to a small support and vary the spectral response of the filter for better effects Valid values for Denoise are integers greater than 100 When it is set to 100 nothing is attempted info softinstrument com Scientific Instrument Software Corporation Limited Latitude Reduction and High Boost Filtering Settings Displacement Registration ae Smat Search 64 Margin O Classic Quality Dehaze Maximum Step Displacement Horz 12 C Vert Resolution Denoise O Fixed Step Displacement Horz 220 Delatitude Crop Brightness Bottom Direct O Linear C Manual 0 Maximum C Automatic lo Output File C MM3D BMP We will illustrate with the more familiar Fouri
39. t Illustration 5 MSD may be determined by a few trials or set according to experience As was previously discussed depth of field in microscopy is very small so you would be adjusting the focus knob in very fine steps hence MSD should be a small value In most cases you may safely experiment staring from value 12 During the computation process MFC will give the detected shift values If a particular value is greater than the specified maximum you probably need to restart with a greater MSD Detected shift values serve as a feedback to focus adjustment training Note that MSD needs not be an exact value and MFC only uses it to limit its search range The rest of the procedure is no difference from the case of metallographic or biological microscopes Since a large number and size of images are usually involved in stereomicroscopy we will discuss the operation of MFC in more detail below After having specified the MSD you can press Start button to load the files as show info softinstrument com Scientific Instrument Software Corporation Limited below Look in BeeZoom m ezos ez17 I w ezis w ezig mezzo E ezz1 E Bz22 File name BZ26 jpg BZ07 jpg BZ08 jpg BZ09 jpg Files of type JPEG ipg v C Open as read only Illustration 6 MFC does not required the file names of the input images be continuously numbered which means in this stage you can alway
40. tes between positive and negative values in large magnitude for example 4 and 5 you probably have specified a too small MSD Sometimes a sufficiently large MSD might also lead to incorrect registration which could only be discovered only when seeing the composed image S MM3D BMP Paint Efx File Edit View Image Colors Help SESS ee OPE ETEN E EEE For Help click Help Topics on the Help Menu Illustration 9 As soon as MFC begins to register and merge the input images it reports the progress info softinstrument com Scientific Instrument Software Corporation Limited on its caption bar including the percentage of task that has been finished and time needed for remaining computation The detected shifts in the table also indicates files that have been processed the file being processed and files yet to be processed Camera MFC does not depend on the presence of any particular camera Instead it optimizes its file operations since all cameras support a mechanism to transfer the captured images into PC in the form of disk files On Windows XP platform as soon as a digital still camera is connected to computer most often via USB cable the user is prompted to download all the images on the camera to a newly created folder on computer The file names are automatically numbered according to the sequence of acquisition This interfacing protocol is the starting point of MFC design You may even carry out serious
41. that has passed Windows Logo testing Continue Anyway Please click on Continue Anyway Finally a message window will pop up again at the lower right corner of screen to report that the driver has been installed and the dongle is ready to use as illustrated below i Found New Hardware x Your new hardware is installed and ready to use At this point Micrometrics Multi Focus Composition is fully functioning Select Start All programs Micrometrics Multi Focus Composition Multi Focus Composition to begin depth of field extending An icon has also been placed on desktop for quick access Note Installation of the dongle i e step 1 and 4 is not needed to run Micrometrics Multi Focus Composition for evaluation or demonstration purpose If the dongle is not found output image will be watermarked but is otherwise identical to the licensed case info softinstrument com Scientific Instrument Software Corporation Limited Introduction In transmission microscopy the thickness of specimen may exceed the depth of field of optics where we are able to capture partially focused images at best The same is also true for reflected and stereo microscopy and indeed macro photography The current practice is to acquire multiple images at various heights In this case although we don t have an image that is focused everywhere for each small region or feature we can always find it in focus on one of t
42. tration might produce wrong results you could experiment with a smaller or greater MSD In all cases the detected shifts will aid in training focus adjustment toward more suitable and uniform stepping sizes Remember that you need move the microscope stage in steps smaller that depth of field of the current objective Ultimately the image registration has to be judged by the final composed image MFC image fusion algorithm is based up the most rigorous mathematical principles and proven signal processing techniques and is guaranteed to produce the right result On the other hand image registration part of MFC although highly efficient and highly reliable could make mistakes occasionally When MFC finished processing a stack of info softinstrument com Scientific Instrument Software Corporation Limited input images it will open the resultant image in default Bitmap viewer as is shown below If the image looks plausible you can be reasonably assured of the success of image registration since the fusion part will never make a mistake If the final image looks absurd i e too edgy there must be something wrong with image registration Usually you don t have to wait until the final image has been created As was previously discussed if a detected value is greater than the specified MSD you normally would cut out of the remaining computation and restart with a new MSD or re capture the input images If you notice the detected shifts alterna
43. vanced options Manual loads user s manual Online Help invoke Internet Explorer to browse to Micrometrics resource website info softinstrument com Scientific Instrument Software Corporation Limited About Multi Focus Composition lists copyright notice version number and contact information for technical support and purchase Micrometrics Multi Focus Composition Move Close Alt F4 Settings Manual Online Help About Multi Focus Composition Illustration 11 Advanced options fall into three categories The first category is for customization and optimization of image registration including algorithm selection controlling parameters whether or not to exploit camera geometry and motorized stage and so on The second category modifies the way the resulted image is stored and cropped The third category contains image post processing operations including restoration enhancement and noise suppression Upon exit Df exe will store the user specification to the file Engine mfc Upon start Df exe will look for Engine mfc in its program folder If it finds it all advanced options are automatically applied The following is a snapshot of the options It is Scopex dll that actually does the real work in a separate thread Such a design intends to take advantage of the hyper threading capability of modern CPU Image registration and fusion are relatively lengthy operations computation and f
44. work with very low cost consumer digital cameras Mainstream microscopy cameras are usually more flexible and usually provide a number of ways for sequencing captured images which MFC is certainly able to accept as input When the camera is attached to the stereomicroscope in a tidy manner i e rows or columns of pixels on image sensor are perpendicular to the direction of stage movement MFC should detect 0 or very small values of lateral displacement in either x or y direction MFC can be instructed to take advantage of this property to speed up the computation However the registration algorithm of MFC is so smart and fast that the gain of speed by exploiting camera configuration is negligible Nevertheless the relationship between camera orientation and lateral shift can be employed to verify the validity of registration Motorized Stage Motorized stages are not a must for extending depth of field with MFC When you use a motorized stage for adjustment of focus the step size hence the lateral shift between adjacent images can be made more or less uniform and fine You can use MFC as usual to discover the shift and then configure MFC to bypass the registration and merely apply the given shift in aligning the images in subsequent tasks Again because of the intelligence and efficiency of MFC image registration algorithm the gain in speed is not significant Besides lateral displacement may not be exact multiples of pixel size The automati
Download Pdf Manuals
Related Search