Home

attached - Robotics and Mechatronics

1. 1 05 ION Z SUDd JO ON uunuluunio 3ODIg TIVRBI3IVW s3903 Q S 0 0 ZU Or D 0 80819 475 vo NMV3IQ 3M VNOIS 3WVN AVINONY 80 HSINH 32 vans S8I3I3WITIIW NI SNOISNAWIG HSINH 1931512345 3SIM3IBHIO SSNAN 8l Or D 0S 0S D S OXEW GL N 5 wy M 26 x0 5 UNLESS OTHERWISE SPECIFIED DIMENSIONS ARE IN MILLIMETERS DO NOT SCALE DRAWING REVISION SURFACE FINISH TOLERANCES LINEAR ANGULAR Part 045 Spacer ElectroMagneticSystem UNLESS OTHERWISE SPECIFIED FINISH DIMENSIONS ARE IN MILLIMETERS SURFACE FINISH TOLERANCES LINEAR ANGULAR DRAWN CHK D APPVD MFG Q A NAME SIGNATURE DATE MATERIAL WEIGHT DEBUR AND BREAK SHARP EDGES TITLE DWG NO SCALE 1 2 DO NOT SCALE DRAWING REVISION ReservoirHolder SHEET 1 OF 1 A4 141 4 eL CDS UNLESS OTHERWISE SPECIFIED FINISH DIMENSIONS ARE IN MILLIMETERS SURFACE FINISH TOLERANCES LINEAR ANGULAR NAME DRAWN CHKD APPV D MFG QA SIGNATURE
2. d 19 5 0 e 19 N e 1 m 1 0 1 z mm x mm x mm YZ plane YZ plane gt cmi T _ 19 5 gt gt gt k gt gt E 0 gt gt gt 19 N gt gt gt 1 gt gt gt gt 1 0 gt gt 1 1 0 1 z mm mm y mm fl Figure 3 4 Magnitude left and direction right of the magnetic field on the x y xz and yz plane in the workspace when a magnetic field in y direction is created The applied current vector is 0 6528 0 06854 0 6357 0 03481 0 4904 0 4797 0 4758 0 5045 The images show that the magnetic field is close to uniform in the workspace The standard deviation of the magnetic field and its direction in the workspace is 0 24 mT and 0 28 respectively 15 3 2 MAGNETIC SYSTEM XY plane XY plane e e e 1 18 4 e E EM e e E 18 2 E Oe e e m e 18 l e e 1 0 1 y mm x mm XZ plane XZ plane T 18 4 p Oq O RTE TJ E T 183 B o ES 18 1 1 1 1 0 1 z mm x
3. UNLESS OTHERWISE SPECIFIED FINISH DIMENSIONS ARE IN MILLIMETERS SURFACE FINISH TOLERANCES LINEAR ANGULAR NAME SIGNATURE DRAWN CHKD APPV D MFG QA DATE MATERIAL No of Parts 1 200 190 DEBUR AND BREAK SHARP EDGES Plexiglass DO NOT SCALE DRAWING REVISION TITLE EmloDriversPlexiGlassCover DWGNO A4 Part_037_ElmoDriverBase_3_ElectroMagneticSystem SCALE 1 5 SHEET 1 OF 1 130 1 133HS 1 1 91 05 5 JO 91 9 0 pezipoup yv OMS DII vo GAddv Jejdopy 8604s 3a vNOIS 3WVN 1 AVANT S3ONV331O1 3903 HSINI3 JOVIINS 3WOSION OG SEN 8 E TT mi ES To EN 4 ma 4 o Y MA ON TN 9G EL Y f aN 4 BN la H GL H 4 5 I Soe 006 X 0076 AES Tiv 0087 X S L 1 l n GeV des N Se 88 6 oL 15 9 x D 42 Y 12 400 M5x0 8 6H 10 7 4 0 010 56 96 0 029 UNLESS OTHERWISE SPECIFIED FINISH
4. DATE MATERIAL WEIGHT DEBUR AND BREAK SHARP EDGES TITLE DWG NO SCALE 1 2 DO NOT SCALE DRAWING Part 001 Helmet REVISION SHEET 1 OF 1 A3 130 1 133HS UIWOS 1HOI3M V vo GAddv 020 ate 3a vNOIS 3WVN avInONV AVANT S3ONVAHTIOL s3903 9 JOYAANS dAVHS SS3IJWITIIW NI SNOISN3WIG SSFINN LOZ 10 i 2 Ly L E 4 B o SECTION B B SCALE 2 1 N e SO M NE LH H 4 4 Jy D 12 Dera 17 11 Cit AR SURFACE FINISH EDGES TOLERANCES LINEAR ANGULAR NAME SIGNATURE DATE TITLE DRAWN en ingHolderU RIngHolderUpper MFG QA MATERIAL DWG NO A4 Black anodized aluminum Part_023_RingHolder_ElectroMagneticSystem No of Parts 1 SCALE 1 2 SHEET 1 OF 1 130 1 133HS T1 31V9S L SHDd JO ON 720 3ODIg ON OMG vo osog JopjoH ETT NMV3IQ uL 3M VNOIS 3WVN AVANT S3ONV331O1 s3903 9 30 v4ains NOISIA3N dAVHS SS3IJWITIIW N
5. gt gt gt gt 1 ai 17 8 gt gt gt gt gt gt gt g 0 gt gt gt gt gt 17 4 1A N gt gt gt gt 17 2 gt gt k gt 1 l gt gt 1 0 1 2 mm mm YZ plane YZ plane e 17 8 1r e zu g 17 6 e E 174 E 0 e _ N 17 2 1 1 0 e k l 0 1 z mm mm Figure 3 3 Magnitude left and direction right of the magnetic field on the x y xz and yz plane in the workspace when a magnetic field in x direction is created The applied current vector is 0 05195 0 6196 0 03537 0 6695 0 4163 0 3964 0 4284 0 4025 The images show that the magnetic field is close to uniform in the workspace The standard deviation of the magnetic field and its direction in the workspace is 0 22 mT and 0 32 respectively 14 CHAPTER 3 SETUP DESIGN XY plane XYplane 1 T T gt gt gt gt gt 1 19 6 gt gt gt gt gt gt gt H 19 4 gt gt gt 0 19 2 gt gt gt gt gt gt 19 _1 gt gt gt gt d gt gt gt 1 0 1 y mm x mm y mm XZ plane XZ plane e 1
6. Khalil et al 2013b 1 2 Thesis organization This thesis will start with modeling the magnetic properties of the microrobots in chap ter 2 The design of the setup and its components is covered in chapter 3 Next in chapter 4 some experiments which show the functionality of the setup are discussed The thesis is concluded with conclusions and recommendations in chapter 5 1 2 THESIS ORGANIZATION CHAPTER 2 MODELING OF MICROROBOTS Chapter 2 Modeling of microrobots Modeling of the magnetic properties of microrobots microparticles microjets and MTB is important for the design of magnetic systems and the control of the micro robots Based on the magnetic properties of the microrobots they can be divided into two categories force and torque controlled microrobots Microrobots which do not possess self propulsion capabilities require an external force to pull them through a fluid Therefore these microrobots are force controlled In our setup the microparti cles are part of this category Microrobots which use self propulsion to move through a fluid only require a magnetic torque to control their direction These types of mi crorobots are part of the torque controlled microrobots In our setup microjets and magnetotactic bacteria MTB are controlled using external magnetic torque 2 1 Modeling of force controlled microrobots The equation of the magnetic force F p R acting on a magnetic dipole is given by F
7. of coils in the same direction The lower set is supplied with 0 A of current The magnitude of the magnetic field is linear over the complete range of applied currents The saturation at 0 A and 1 9 A is attributed to limitations of the current source b The coils of the setup are equipped with metal cores The current is applied to the eight coils in the same direction The magnetic field is linear up to approximately 1 A of applied current At higher applied currents the current field relation is non linear and the magnetic field saturates at approximately 65 mT at 2 A figure 4 2 the results of the comparison are shown In figure 4 2 a the ratio between the measured and calculated magnitude of magnetic field By and Bg respec tively is shown for the 125 point in the workspace The average ratio is 0 984 with standard deviation of 0 041 The angle between the and is also calculated for every point in the workspace The results are shown in figure 4 2 b The aver age angle between the calculated and measured magnetic field is 2 9 with a standard deviation of 1 8 The deviation in magnitude and angle is attributed to the expected discrepancy between an ideal model and a practical implementation Also the initial position and orientation of the probe of our magnetometer is set by hand which affects the actual coordinates at which the magnetic field is measured 4 2 Experiment autofocus An important feature of th
8. 1999 URL http www sciencedirect com science article pii 8000634959977 4850 Dreyfus J Baudry M L Roper M Fermigier Stone and J Bibette Mi croscopic artificial swimmers Nature 437 7060 862 865 2005 URL http www nature com nature journal v437 n7060 abs nature04090 html L Firestone K Cook K Culp N Talsania and K Preston Comparison of autofocus methods for automated microscopy Cytometry 12 3 195 206 2005 URL http onlinelibrary wiley com doi 10 1002 cyto 990120302 abstract D Glozman and M Shoham Image guided robotic flexible needle steering 23 3 459 467 2007 doi 10 1109 TRO 2007 898972 URL http ieeexplore ieee org stamp stamp jsp arnumber 4252165 R Hartley and A Zisserman Multiple view geometry in computer vision volume 2 Cambridge Univ Press 2000 Intuitive Surgical Inc da vinci surgical system 2011 URL http www intuitivesurgical com 91 BIBLIOGRAPHY BIBLIOGRAPHY K Ishiyama M Sendoh and Arai Magnetic micromachines for medical applica tions Journal of Magnetism and Magnetic Materials 242 41 46 2002 URL http www Sciencedirect com science article pii S0304885301011817 J D Keuning J de Vries L Abelmann and S Misra Image based magnetic control of paramagnetic microparticles in water In Proc IEEE RSJ Int Intelligent Robots and Systems IROS Conf pages 421 426 2011 doi 10 1109 IROS 2011 6095011 URL http ieeexplore ieee o
9. 20 kg over travel ranges to 200 mm They are designed with high value components and feature a precision machined high density stress relieved aluminum base for exceptional stability with mini mum weight The highly precise 403 drive includes a preloaded lead screw providing a minimum incremental motion of 0 2 um For higher velocities and long lifetime the M 404 versions feature a low friction ball screw Application Examples m Automation m R amp D m Semiconductor technology m Metrology m Quality assurance testing offering a minimum incremen tal motion down to 0 1 Three motor drive options allow easy adaptation to different au tomation applications Five travel ranges from 25 to 200 mm are offered The stages can carry up to 20 kg and push pull up to 50 N Special versions for vacuum applica tions are also available see or dering information Maintenance Free High Guid ing Precision All models are equipped with high precision linear guiding rails and recirculating ball bear ings The recirculating ball bear ings are maintenance free and immune to cage migration The choice of components and care ful mounting guarantees high load capacity longer lifetime and high guiding accuracy Ad ditionally in the M 404 series the bearings are polished to achieve the optimum guiding accuracy Ordering Information 80 mm wide 80 mm wide M 40 3 Linear Transl
10. 24 V power supply Vacuum Compatible to 105 hPa DC Motor Gearhead Vacuum Compatible to 106 hPa VG M 40x xDG models are equipp ed with a DC motor and a shaft mounted optical encoder providing a minimum incremen tal motion of down to 0 1 uim M 40x x2S models feature a cost effective direct drive 2 phase stepper motor provid ing very smooth operation and a resolution of 0 16 um Limit and Reference Switches For the protection of your equip ment non contact Hall effect limit and reference switches are installed The direction sensing reference switch supports ad vanced automation applications with high precision Other Family Members The M 403 M 413 and M 404 M 414 series of linear stages form a modular system The M 403 is the basic family pro viding travel ranges from 25 to 200 mm M 413 is designed for higher loads with travel ranges from 100 to 300 mm The M 404 and M 414 stages have the same travel ranges and load ca pacities but offer higher preci sion and more speed Moving the NanoWorld www pi ws Technical Data Model Motion and positioning Travel range Integrated sensor Sensor resolution Design resolution Min incremental motion Backlash Unidirectional repeatability Pitch Yaw Max velocity Origin repeatability Mechanical properties Spindle Spindle pitch Gear ratio Motor resolution Stiffness in motion direction Max load Max push pull force
11. K S e LQR A B Q R The optimal input u can be found using the following equation u t R t K t x t G 16 90 BIBLIOGRAPHY BIBLIOGRAPHY Bibliography M Abayazid R J Roesthuis R Reilink and S Misra Integrating deflection mod els and image feedback for real time flexible needle steering In Press doi 10 1109 TRO 2012 2230991 URL http ieeexplore ieee org stamp stamp jsp arnumber 6392299 Early Access J J Abbott O Ergeneman M P Kummer A M Hirt and B J Nelson Modeling mag netic torque and force for controlled manipulation of soft magnetic bodies 23 6 1247 1252 2007 doi 10 1109 TRO 2007 910775 URL http ieeexplore ieee org stamp stamp jsp arnumber 4392562 Avago Technologies ASMT Ax3x 3W Power LED Light Source Data Sheet 09 2011 Basler AG Basler Aviator users manual for camera link cameras 7th edition 12 2010 URL http www baslerweb com media documents AW00083007000AviatorCameraLinkUsersManual pdf E Carpi and C Pappone Magnetic maneuvering of endoscopic capsules by means of a robotic navigation system 56 5 1482 1490 2009 doi 10 1109 TBME 2009 2013336 URL http ieeexplore ieee org stamp stamp jsp arnumber 4760277 Y R Chemla H L Grossman T S Lee J Clarke M Adamkiewicz and B B Buchanan A new study of bacterial motion superconducting quantum interference de vice microscopy of magnetotactic bacteria Biophysical journal 76 6 3323 3330
12. achieve better contrast images and to be able to view bacteria To avoid the disadvantages of commercial available system a custom build illumination system will be used This sys tem will use the more simple critical illumination principle In figure 3 11 a schematic 23 3 3 MICROSCOPIC SYSTEM Sample holder Microscope Focus lens Collector lens Figure 3 11 Components and light path of an critical illumination system The light from the LED is collected by the collector lens and focused into the sample by the focus lens The microscope objective receives the light and projects it onto the sensor of the camera The angle should at least match the angle 0 to fully illuminate the sensor representation is shown of the illumination system A power light emitting diode LED will be the light source The light of the LED is collected by the collector lens This lens will converge the light to a more or less parallel beam This parallel beam is then focused by another lens into the sample The light then passes to the microscope ob jective which will project the image onto the sensor of the camera For the sensor to be fully illuminated the angle a in figure 3 11 should match the angle 0 of the microscope objective To make sure the sensor is indeed fully illuminated the angle can be chosen slightly larger than 0 It should be noted that when a is larger than 0 it results in a lower overall illumination intensity o
13. board up to 4 axes M 403 xDG Rotary encoder 2000 0 018 0 2 10 1 200 200 2 5 Leadscrew 1 28 44444 1 3500 200 50 100 DC motor gearhead 0 12 2 5 3 Hall effect 20 to 65 C 863 single axis C 843 PCI board up to 4 axes 2 phase stepper motor 24 V chopper voltage max 0 8 A phase 400 full steps rev motor resolution with C 663 stepper motor controller For travels gt 100 mm the pitch yaw value is valid for every 100 mm Data for vacuum versions may differ Moving the NanoWorld www pi ws M 403 x2S 0 16 Leadscrew i 6400 3500 200 50 100 2 phase stepper motor 24 4 8 200 Hall effect 20 to 465 C 663 single axis Units Cts rev um um um um urad urad mm s um steps rev N um N N N V w Ncm APPENDIX C PI M 404 DATASHEET 78 APPENDIX D VIBRATION DAMPER DATASHEET Appendix D Vibration damper datasheet In this appendix the datasheet of the Paulstra RADIAFLEX vibration damper is shown In the setup the single stud fixing damper with reference number 511312 is used 79 RADIAFLEX DESCRIPTION Metalwork Mild steel plated Natural rubber bonded cylindrically shaped Welded fixings 5 styles single sided threaded stud single sided threaded hole double threaded stud double threaded holes combination fixing In Europe we often use different screw standards
14. by the camera This position is used by the P and PI controllers to point to point position control the microparticles and microjets in 3D space The position controller autofo cusing controller and feature tracking algorithms are implemented in real time Cur rently the control platform runs at 50 fps It is experimentally shown that the system is capable of achieving point to point position control of microparticles and microjets in 3D space The system is designed to control MTB in 3D space The magnitude of the magnetic fields are sufficient to control MTB and also the feature tracking algorithm is capable of tracking MTB 5 2 Recommendations The recommendations are divided into two categories which are discussed in separate sections First improvements to the design of the system and the software are discussed in section 5 2 1 Next future research is discussed in section 5 2 2 5 2 1 Design improvements The position accuracy of the system in controlling microrobots can highly be improved by using different controllers than the P and PI controllers that are currently used in the setup A good option would be the utilization of an optimal controller that 49 5 2 RECOMMENDATIONS minimizes the applied electrical currents to the coils By reducing the input currents the heat dissipation in the setup is minimized Heat dissipation is always an issue in systems which use electromagnets as actuators In appendix G the theory of an
15. is able to control paramagnetic microparticles in two dimensional 2D space Keun ing et al 2011 This system is also used to characterize and control self propelled microjets Khalil et al 2013a and magnetotactic bacteria Khalil et al 2012b Fur thermore interaction force estimation during manipulation of microparticles is imple mented on this system Khalil et al 2012a The goal of this Master s thesis project is to develop a testbed to perform 3D closed loop control of microrobots Therefore a system is realized that uses electromagnets to control different types and sizes of microrobots in 3D space The microrobots are optically tracked using microscopes with cameras attached The camera images are also used as feedback for the control system Furthermore an autofocus system is implemented to provide focused images when the microrobots move in the fluid CHAPTER 1 INTRODUCTION 1 1 Contributions During this master thesis the following contributions are made Realization of a magnetically actuated system which is capable of controlling mi croparticles microjets and MTB in 3D space Implementation of autofocusing on the setup for control of microrobot Implementation of a real time magnetic based control system Implementation of interaction force estimation during manipulation of micropar ticles Khalil et al 2012a Implementation of an optimal motion controller for paramagnetic microparticles in 3D space
16. is an iterative process In the microrobotic test setup we what to have a continuous flow of images aim is 100 fps so there is no time to focus using an active focusing system we will use a passive system While we are using standard cameras which lag additional phase detection hardware we also cannot use phase detection and therefore we will use contrast detection The contrast will be calculated in a ROI around the robot A good measure for the contrast of an image can be determined by calculating the sum of the absolute differences SAD of the grayscale pixel values For an image of size m n pixels and grayscale pixel value p the SAD equation becomes m 1 contrast bas Pajo Pai jl 3 25 i 1 j 1 where i and j determine the row and column of the pixel currently being processed respectively The m and n are the maximum column and row of the ROI respectively By optimizing the value of the SAD over time the contrast is optimized which yields an in focus image To optimize the contrast the slope of the contrast curve the contrast values of the sequence of images needs to be determined This is done by applying a least squares polynomial fit of a first order polynomial to the last n values of the contrast signal This yields a polynomial which can be described by y 3 26 35 3 6 CONTROL SYSTEM where the c coefficient can be used to determines if the contrast is increasing or de creasing Therefore
17. not for tracking microrobots in 3D space This makes the integration of such an autofocusing system in our setup very hard or even impossible Therefore we decided to design our own autofocusing system This way the problem of integrat ing a commercial system into our control system is nullified 26 CHAPTER 3 SETUP DESIGN The autofocusing system can be implemented using two different approaches mov ing the objective of the microscope and hereby changing the focal distance or moving the complete microscope assembly and hereby keeping the distance between object and microscope fixed while the focus distance is retained The method of moving the objective of the microscope will yield faster autofocusing than moving the microscope assembly because less mass has to be moved The downside is that the microscope has to be equipped with internal focus correction when using no infinity corrected lenses When using infinity corrected objective the magnification will be changed be moving the objective because also infinity corrected objective will not yield a perfect parallel beam Also while the magnification is not fixed it is very hard to calibrate the system The method of moving the microscope assembly does not have these disadvantages and with a powerful stage also the slower autofocusing can be fixed Moving the microscope assembly seems to be the best option for the microrobotic setup The microscope assembly is moved by a linear stage This
18. of 0 1 mm x 0 1 mm 1 mm x 1 mm and 2 mm x 2 mm Furthermore autofocus system is required to keep the microrobots in focus while moving in the fluid The last requirement is that the control system is implemented on a real time platform The requirements can be summarized as follows Gradients of the squared magnetic field B p B p 2 B p B p and 2 of 15 6 mT mm 15 6 mT mm and 49 7 mT mm are re quired respectively A magnetic field B p with magnitude of 19 1 mT is required The microscopic system should provide FOV of about 0 1 mm x 0 1 mm 1 mm x 1 mm and 2 mm x 2 mm when controlling the MTB microjets and microparti cles respectively An autofocus system is required The control system should be implemented on a real time platform 11 3 2 MAGNETIC SYSTEM Figure 3 1 Schematic representation of the cross section of a coil configuration A Helmholtz configuration which can generate uniform magnetic fields in the center c between two coils can be realized by placing the coils at a distance d equal to the radius of the coil r The electrical current should be applied in the same direction in the coils A Maxwell configuration which can generate uniform magnetic field gradient in the center c between the coils can be realized by placing the coils at a distance d equal to times the radius of the coils r The electrical curr
19. optimal controller is provided Another part of the setup that should be improved is the autofocusing system The current autofocusing system is not robust which leads to problems in tracking the mi crorobots Improving the current algorithm for contrast calculation or using a different algorithm to determine if a microrobot is in focus could make the autofocusing system more robust Furthermore the position of the microrobot in known in 3D space The position of the microrobot can be used as feed forward to the autofocusing algorithm to determine the direction in which the microscope should move to keep the microrobot in focus Calibration of the system is highly recommended when using position feed forward It is worth noting that an improved autofocusing system is also beneficial for the position accuracy of the controller The illumination can be improved In the current implementation the illumination module is not coupled to the microscope directly A direct coupling could prevent misalignment between the illumination module and the microscope which leads to more uniform illuminated images Uniform illuminated images are beneficial for the feature tracking algorithm and therefore for the accuracy of the position tracking of the microrobot It is expected that implementation of K hlen illumination will improve the tracking and focusing on the MTB The last part that should be optimized is the magnetic system The generated fields are uniform
20. p V m p B p 2 1 where m p R is the induced magnetic dipole moment of the microparticle and B p is the induced magnetic field at point R in the workspace The microparticles used in our setup have a spherical geometry According to Carpi and Pappone the magnetic dipole moment of a spherical object can be determined as the volume integral of the induced magnetization M p R Carpi and Pappone 2009 m p M p dV 2 2 V 4 3 77 2 3 In 2 2 is the volume of the spherical microparticle with radius rp The magne tization of the microparticle is related to the magnetic field strength H p R by M p x H p 2 4 where y is the magnetic susceptibility constant McNeil et al 1995 The induced magnetization vector is always aligned with the induced magnetic field because of the isotropic properties of a spherical object As a consequence zero torque can be applied to the microparticles and the microparticles are only subjected to pure forces The 2 2 MODELING OF TORQUE CONTROLLED MICROROBOTS magnetic field strength H p is related to the magnetic field B p by B p uH p where is the permeability coefficient given by uo 1 with uo the magnetic constant with value of 47 x 10 7 TmA 1 Substituting B p in 2 4 yields the following equation which relates the magnetization to the magnetic field M p TEB p 2 5 W
21. provides an switch to turn on and off the sys tem two potentiometers for dimming the illumination and a display and LEDs which can be used in future to show the status of MARS An emergency stop switch 7 in implemented which can be used to switch off the system instantly in case of an ac cident or unsafe situation The inside of the sphere is shown in figure 3 20 c and figure 3 20 d In figure 3 20 c a reservoir holder 8 with reservoir 9 is placed on the lower sphere half In figure 3 20 d one illumination module 10 is switched on Also the lower set of coils with cores 11 is visible Furthermore system specifications are provided in table 3 2 38 CHAPTER 3 SETUP DESIGN Figure 3 20 The realized Magnetically Actuated Robot System MARS a Render of the SolidWorks Dassault Syst mes SolidWorks Corp Waltham Massachusetts USA 3D model b Photograph of MARS c Photograph of the lower part of the spherical structure with a reservoir holder d Photograph of the lower part of the spherical structure with a illumination module switched on 1 Microscope 2 Camera 3 Linear motion stage 4 Magnetic system 5 Motor drivers 6 Control panel 7 Emergency stop switch 8 Reservoir holder 9 Reservoir 10 Illumination Module 11 Ser of coils with cores 39 3 7 REALIZED SYSTEM Item Stages Maximum velocity Travel range Maximum push pull force Microscopes Maximum resolving power Minim
22. spatial dimensions can be written as F p Fax mx G 1 F p Fp mi G 3 where F F and F are the magnetic force acting on the particle Fy is the drag force and is the buoyancy force While the drag force is a drag coefficient a times the velocity the equations can be rewritten F p ax mx G 4 F p ay G 5 F az mi G 6 These equations can be transformed into state space equations by choosing the states as x X X9 X Xg y X4 y Xg Z Xg Z The state space equations then become Xi X gt G 7 a 1 X2 mis 8 X4 G 9 a 1 X6 G 11 a 1 1 X6 X6 F F p G 12 m m m 89 APPENDIX G OPTIMAL CONTROLLER These state space equations can be rewritten into standard matrix form x Ax Bu The equation then becomes x 0 1 0 0 9 9 9 0 0 I o 0 2 0 0 x 0 0 0 F 00 1 000000 0 o 9 0 2 0 o 0 o t 0 0 F Xs 0 0 0 0 0 1 Xs 00 0 0 0 0 0 0 0 0 2 G 13 The continuous time algebraic Riccati equation CARE B K Q 0 G 14 The discreet time algebraic Riccati equation DARE ATKB R B KB B KA Q G 15 The optimal gain matrix K can be calculated using the Matlab function linear Quadratic Regulator design The syntax is
23. stage needs to ful fill some requirements concerning load capacity push pull force applicable moment backlash and control interface First of all the microscope assembly is estimated to have a mass of 2 kg amp 20 This yields a minimum load capacity of 20 for the horizontally mounted stage For the vertically mounted stage this yields a minimum push pull force of 20 N The center of mass of the load will be at a distance from the stage which yields a moment on the stage The distance between the stage and the center of mass is about 63 mm This yields a minimally allowable moment on the stage of 20 x 63 1260 Nmm Besides load capacity and so forth a very important feature is the backlash of the stage Especially for the bacteria because of their very small size this can be a critical requirement In a situation where a bacterium is being tracked when swimming away for the camera and suddenly changes direction 180 the microscope has to follow the bacterium to keep it in focus but the backlash in the drive chain could make that the bacterium is outside the focus depth of the microscope before the stage travel direction is reversed To prevent this the backlash should be limited The depth of focus of the Mitutoyo M Plan 10X Apo objective Mitutoyo Cor poration Kawasaki Japan we use in our setup is 3 5 um Practically the bacterium is still visible if it is within a range of approximately 5 times the depth of focus which is 18 um
24. supported by the motor controllers Elmo Whistle we are using The maximum applicable moment on the stage is not specified but the supplier assured us that the expected load will not cause any problems 3 4 2 Vibration isolation table The robots which will be used in the setup are very small microparticle are about 100 um microjet about 50 um and bacteria are the smallest with length of about 5 The small size makes the use of microscopes necessary The design of the microscopic system is discussed in section 3 3 When using microscopes it is important to have a stable platform for the microscope and the sample The high magnification causes even small vibrations from the surrounding environment to influence the images acquired by the cameras on the microscopes A good solution to reduce unwanted vibrations is placing the microscopic system on an optical table with vibration isolating supports actively or passively damped Unfortunately there is limited space available in the medial robotics lab so an optical table assembly is not an option for the moment Therefore we decided to use a smaller optical breadboard which has some build in vibration damping and place this on small vibration isolators The necessary stiffness k of the vibration isolators is calculated by describing the system as mass spring system The first eigenfrequency can be written as k 3 23 m It is expected that vibrations with a frequ
25. system 4 4o sd d os d oe NUR PUR ERG Ye 20 3 3 1 MICFOSCOD6S ai unan Minas escent Rr dos ees 20 3 3 2 Gamera S i re X xa ux S Wor aleve isle Oe Res 21 3 33 ll miinatioti cs DURE ew Due ES 23 3 4 Mechanical system 26 BAL AUEtOfOGUS e mate do S o S GUB es Ha 26 3 4 2 Vibration isolation table 28 3 5 Electricalsystemi 4 4v da ad ee ia tete ae 28 3 6 Control system 2 aou he xo eR RR Rer o do Hum 29 3 6 1 Real time software environment 31 3 6 2 Real time hardware environment 31 3 6 3 Feature tracking software 32 3 6 4 Autofocus implementation 35 3 6 5 Magnetic based control system 36 3 7 Realized system 38 4 Experimental results 41 4 1 Experiments on the magnetic system 41 4 2 Experiment autofocus 42 4 3 Experiments motion control 43 4 3 1 Motion control of a microparticle 44 4 3 2 Motion control of a microjet 48 5 Conclusions 49 5 41 Goncluslons ace a guy Ai tant e Vota 49 5 2 Recommendations 49 5 2 1 Designimprovements 49 5 2 2 Future res
26. system starts with defining requirements The requirements concern ing the magnetic system are important because this provides the main functionally of our system The microparticles move in the fluid by the magnetic force This force is provided by the magnetic system and has to overcome the drag force acting on the microparticle The equation of motion for a microparticle in three dimensional 3D space can be written as F p Fa p mx 3 1 F p Fp Fa p mz 3 3 where F p F p and F p are the magnetic force acting on the particle along x and z axis respectively Fay p and are the drag forces on the microparticle along x y and z axis respectively is the buoyancy force which only acts along the z direction Further m is the mass of the microparticle The drag force depends on the type of flow of the fluid around the microparticle The flow type is determined by the Reynolds number ee FE 3 4 n where p is the density of the fluid v the velocity of the microparticle r the radius of the microparticle and n the dynamic viscosity of the fluid Using 3 4 for a micropar ticle with a diameter of 100 um submerged in water with density of 998 2 kgm and furthermore using a dynamic viscosity of 1 mPa s Assuming the velocity of the mi croparticle will not exceed 1 mms the Reynolds number is calculated to be less than 3 1 REQUIREMENTS 0 1 This Re
27. the reservoir holder 3 is positioned between the coils 4 b Finite element FE model in Comsol Multiphysics COMSOL Inc Burlington U S A This model is used to perform finite element calculations on the magnetic fields between the upper and lower set of coils At a 90 angle in the horizontal plane the reservoir holder can be placed It is desirable to keep the distance of the coils to the center of the workspace small since magnetic fields are stronger close to the coils than further away from the coils However besides the coils also other components like two microscopes two illumination modules and a reservoir holder have to be placed in the same workspace Therefore the coils are placed at a larger distance from the center of the workspace The tradeoff is that the fields are weaker in the center of the workspace By using more coils and placing cores in the coils the combined magnetic fields are still sufficient to provide the required magnetic fields and gradients In figure 3 2 b a finite element FE model made in Comsol Multiphysics COM SOL Inc Burlington U S A is shown The FE model allows us to do calculations on the magnetic fields and therefore it is used to check the uniformity magnitude and gradient of the magnetic fields These properties are checked by applying currents to the coils and plotting the magnetic fields In figure 3 3 figure 3 4 and figure3 5 the magnitude and direction of the magnetic field i
28. 00 times magnification TIMM400 from SPI GmbH Op penheim Germany This microscope is only 22 mm in diameter 155 mm long and weights 100 2 In the basic configuration it has a variable magnification up to 400X which can be extended with additional modules like lenses and spacers The system 3 19 20 CHAPTER 3 SETUP DESIGN also incorporates a 5 megapixel MP camera with an analog or USB interface which can achieve up to 25 fps Unfortunately after a short testing period it is concluded that this microscopic system is not sufficient to be used in the microrobotic setup the frame rate is too low the image quality is poor at high magnification and the price is too high 4000 In appendix A a test report is included Another system that does meet our requirements is the Qioptiq Optem Zoom125C microscope system Qioptiq Luxembourg Luxembourg This is a modular system which can be composed to the needs in the project with a basis part that allows up to 6X magnification It can be combined with Mitutoyo M Plan Apo objectives Mitu toyo Corporation Kawasaki Japan and c mount cameras can be connected to proved video feedback to the control system By combining different objectives with the zoom basis part and an additions 2X magnifying part the total zoom range becomes 4X up to 120X which should be sufficient to image microparticles as well as bacteria The FOV which can be attained which this microscope are experimentally dete
29. 1910 Threaded hole fixing on request See current price list for availability of items except 12 5 35 The shear characteristics are measured under Axial Load See VIBRACHOC elastomer range references E3RP APPENDIX D VIBRATION DAMPER DATASHEET 82 APPENDIX E ELECTRONICS Appendix E Electronics In this appendix the interface connection of the Elmo drivers used to control the stage is shown in Table E 1 and the schematics of the power circuit is shown in Figure 3 14 Pin sub D Signal Function Elmo function connector 1 Not connected 9 MOT Motor connection M2 K1 3 red 2 MOT 4 Motor connection 4 M3 K1 4 yellow 10 PGND Power ground PE K1 1 blue 3 Not connected 11 n c Not connected 4 5V 5V input for encoder and logic 5V K10 2 white 12 NLIM Negative limit signal active high TTL DI 1 K6 1 white 5 PLIM Positive limit signal active high TTL DI 2 K6 2 black 13 REFS Position reference signal TTL DI 3 K6 3 brown 6 GND Logic ground GND 5 K10 3 grey 14 A Encoder signal A TTL ENCODER CHA K10 5 blue 7 Encoder signal A dash TTL ENCODER CHA K10 6 green 15 BG Encoder signal B TTL ENCODER K10 7 yellow B Encoder signal B dash TTL ENCODER CHB K10 8 orange Table E 1 Connection Interface panel Elmo stage 83 APPENDIX E ELECTRON
30. 30 1 133HS Uca3vos Z JO ON AUISIDEHPETISMOUOIDUILUNI wnujwnjo p zIpouo ON OMG TIVRBI3I VW s3903 NOISIA33 1 05 ION vo NMV3IQ 3M VNOIS 3WVN AVINONY YVAN S3ONV2331O1 HSINI4 3Ov433nS S3a31BWIT NI SNOISN3WIQ HSINH 1931512345 3SIM3IBHIO SSNAN L dIVOS 9 9 NOILO3S Sr i es 019 Z JO ON unutunjp p zIDOuD 32D g 298 130 133HS om d md gt i i S i e 14 N SH SINN SVs RSS NAS SERRA Wy MAMAN NNNM na VIVIV VV VM MU a BES JIWHHIWWWWWWIWWW AAA AIR NNI NI DRAN NAAR EUM A A A A AAA 2 z 2 5 S OXOEN 6 130 1 133HS JOPUI ADUOHOUILUNII 080 Od ON
31. Compression Shear OA T Max Max load Deflection Ref mimm OC load Deflect Deflect Ref daN mm dan Em mm 12 2 511110 10 8 6 10 1 6 125 09 m 12 8 2 5 511128 12 8 M3 6 12 12 15 0 75 m E ELS 10 2 5 821293 511125 12 5 15 M5 10 3 2 5 2 521128 20 35 25 4 521295 pe fg 5 511298 20 M5 12 4 28 4 B21296 15 511200 25 5 2 5 521298 4 511215 8 5 0 6 5 521178 5 511220 15 3 8 521249 5 5 511225 20 120 16 5 45 5 521297 1 511230 25 30 55 45 5 821299 30 25 1 45 5 521319 2 511265 255110 80 15 8 5 521340 T 311210 15 60 25 8 25 521341 55 511275 22 20 20 4 8 4 921281 25 50 55 8 4 5 521342 6 511280 30 50 1 8 8 6 521343 8 511285 40 50 10 6 5 6 521344 0 511290 3 1 2 5 821308 3 5 511308 5 1 4 521310 6 511310 30 8 6 521312 8 511312 9 1 1 5 521314 9 511314 4 3 521450 40 6 5 5 8 6 5 10 1 5 1 9 521456 6 45 521580 50 8 1 521581 1 9 521582 25 400 5 30 45 521601 60 36 10 25 300 8 30 1 521603 45 250 1 30 9 521641 35 450 8 6 8 821105 10 50 10 25 350 1 11 821710 10 300 14 15 821711 30 45 950 7 5 521803 30 35 950 7 5 521840 40 M14135 600 9 7 521841 70 35 500 17 15 521842 80 35 450 19 11 1521843 40 1100 8 60 7 521908 100 55 Mie 47 900 12 60 10 521909 80 750 19 60 I7 52
32. DIMENSIONS ARE IN MILLIMETERS SURFACE FINISH TOLERANCES LINEAR ANGULAR NAME SIGNATURE DRAWN CHKD APPV D MFG QA 80 DEBUR AND BREAK SHARP DO NOTSCALE DRAWING REVISION EDGES DATE TITLE PI BaseStageAdapter MATERIAL DWG RO A4 Black anodized aluminum Part 029 PlBaseCoarsesatgeAdapter ElectroMagneticSystem 7075 16 No of Paris 2 SCALE 1 SHEET OF 130 1 133HS JOAISSSIQS LZ 31V9S 1 SHDd JO ON SSDJOIXSId ONOMA SSDIOIX dIION9SS1QS 1 05 ION s3903 vo 3M VNOIS 3WVN SVIDONV AVANT S3ONV2331O1 HSINI 3OV33nS S3313WITTIW NI SNOISN3WIQ HSINH 1931512345 3SIM3IBHIO SSNAN 81 81 47 UNLESS OTHERWISE SPECIFIED FINISH DEBUR AND DIMENSIONS ARE IN MILLIMETERS BREAK SHARP DO NOT SCALE DRAWING REVISION BEEN UO Part 032 CoreMetal NND SCALE 1 1 SHEET 1 OF 1 UNLESS OTHERWISE SPECIFIED FINISH DEBUR AND DIMENSIONS ARE IN MILLIMETERS ERN BREAK SHARP DO NOT SCALE DRAWING REVISION BEEN UO SIGNATURE DATE ___ Part 036 CoreMetalLowerSet e d qu eo __ _ RS jJ j RS j WEIGHT SCALE 1 1 SHEET 1 OF 1 UNLESS OTHERWISE SPECIFIED FINISH DI
33. I SNOISN3WIG HSIN 3SIMYSHLO SSFINN cl i L 31VOS V V NOILO3S 802 0 004 4 910 0 052 Knurling 7 oZ 93 2 UNLESS OTHERWISE SPECIFIED FINISH DEBUR AND DIMENSIONS ARE IN MILLIMETERS BREAK SHARP SURFACE FINISH EDGES TOLERANCES LINEAR ANGULAR NAME SIGNATURE DATE DRAWN CHK D APPV D MFG QA MATERIAL Black anodized aluminum 7075 T6 No of Parts 4 DO NOTSCALE DRAWING REVISION TITLE RingHolderColumn DWG NO A4 Part 025 RingHolderColumn ElectroMagneticSystem SCALE 1 1 SHEET 1 OF 1 130 133HS S U31v2S L JO ON SSOGIOAUGOWIF ZEO WNUN PEZIPOUD 9 s3903 1 05 LON 3WVN S8I3I3WITIIW NI SNOISN3WIQ 1931512345 3SIM3IBHIO 5531 ssouxolu 12945 GL 9v o CM 00006 Sy 00006 GL SC C 4 00006 o 0 9 9v LS EE 06 440
34. ICS 12V Main switch 24V 24V Main relay Emergecy break Relay 1 hi Elmo 14 Backup power Elmo 13 Elmo 12 Elmo 11 Elmo 3 Elmo 2 Elmo 1 Figure E 1 Schematic representation of the electric power circuit of the system 84 APPENDIX E REAL TIME SOFTWARE ENVIRONMENTS Appendix F Real time software environments In this appendix several real time platforms are shortly described These platforms are MathWorks xPC Target dSPACE National Instruments LabVIEW Real Time 20 sim 4C and Real Time Linux The platforms will be evaluated on several topics like available computation power ease of use ease of system setup available hardware and costs xPC Target The real time platform from MathWorks is called xPC Target This system consists of a host and a target PC The host PC is your regular workstation which runs Simulink and is connected to the target PC by an Ethernet connection The host will not run in real time the target PC will The target PC can be a specialized system ordered from MathWorks or an consumer PC with additional interface cards The interface cards have to be supported by drivers from the Simulink library which limits the diversity of cards to be used Building your own system with consumer electronics can significantly reduces the costs Besides h
35. It is expected that when the backlash of the stage is lower than this 18 um it should be able to catch up with the bacterium Another requirement is the travel range of the stage While the field of view of the camera at the lowest magnification of the microscope is 2 5 mm by 2 5 mm the stage should be able to travel at least 2 5 mm so the complete workspace can be covered Obviously a larger travel range is preferred to make it easier for the user to find the center of the workspace without having to reposition the microscope assembly by hand Therefore the minimal travel range is thought to be 10 mm The last requirement is that the stage interface is compatible with the motor controllers we are aiming to use or that the supplied stage controller is compatible with control hardware MathWorks xPC Target The requirements of the autofocusing system are summarized as follows A minimal load capacity 20 is required A moment of 1260 Nmm should be applicable A minimal travel range of 10 mm is required e A maximal backlash of 18 um is required The stage should be compatible with motor control and its interface A linear stage that needs these requirements is the PI M 404 2DG Physik Instru mente PI GmbH amp Co KG Karlsruhe Germany The datasheet can be found in 27 3 5 ELECTRICAL SYSTEM appendix C This stage has a travel range of 50 mm a load capacity of 200 N and a backlash of 2 um The control interface is
36. MENSIONS ARE IN MILLIMETERS SURFACE FINISH TOLERANCES LINEAR ANGULAR DRAWN CHK D APPVD MFG Q A NAME SIGNATURE 39 5 26 9 MATERIAL WEIGHT 72 85 DEBUR AND BREAK SHARP EDGES TITLE DWG NO SCALE 1 2 2 500 DO NOT SCALE DRAWING REVISION DisplayCover SHEET 1 OF 1 A4 APPENDIX C PI M 404 DATASHEET Appendix C PI M 404 Datasheet In this appendix the datasheet of the PI M 404 linear precision stage is shown In the setup stage M 404 2DG is used 75 Physik Instrumente GmbH amp Co KG 2009 Subject to change without notice All data are superseded by any new release The newest release for data sheets is available for download at www pi ws R2 09 12 07 PI Piezo Nano Positioning M 403 M 404 Precision Translation Stage Cost Effective Large Choice of Drives amp Travel Ranges W For Cost Sensitive Precision Positioning Applications E Travel Ranges 25 to 200 mm E Resolution to 0 012 um E Min Incremental Motion to 0 1 um E Preloaded Precision Leadscrew or Recirculating Ball Screw Drives Provide High Speeds amp Long Lifetimes E Stress Relieved Aluminum Base for Highest Stability E Vacuum Compatible Versions Available E M 413 and M 414 Versions for Higher Load Requirements The M 403 and M 404 linear translation stage series provide cost effective solutions for pre cision positioning of loads up to
37. Max lateral force Drive properties Motor type Operating voltage Electrical power Torque Limit and reference switches Miscellaneous Operating temperature range Material Mass depends on dimensions travel range Recommended controller driver Max recommended velocity M 404 xPD Rotary encoder 4000 0 25 0 25 0 5 0 5 Recirculating ballscrew 1 3500 200 100 100 ActiveDrive DC Motor 24 26 50 Hall effect 20 to 65 C 863 single axis C 843 PCI board up to 4 axes M 404 xDG for all models 25 50 100 150 200 mm see Order Information Rotary encoder 2000 0 012 0 1 Recirculating ballscrew 1 42 92063 1 3500 200 100 100 DC motor gearhead 0 12 2 5 3 Hall effect 20 to 65 M 404 x2S 0 16 0 2 2 1 75 Ww i Recirculating ballscrew 1 6400 3500 200 100 100 2 phase stepper motor 24 4 8 200 Hall effect 20 to 65 Piezo Nano Positioning M 403 and M 404 dimensions in mm Sub D connector 15 pin 3 m cable M 403 xPD Rotary encoder 4000 0 25 0 25 6 1 200 200 10 Leadscrew 3500 200 50 100 ActiveDrive DC Motor 24 26 50 Hall effect 20 to 65 for all models Aluminum black anodized 1 7 1 8 2 1 2 2 2 5 kg C 863 single axis C 663 single axis C 843 PCI board up to 4 axes C 863 single axis C 843 PCI
38. University of Twente EEMCS Electrical Engineering Robotics and Mechatronics Realization of a Three Dimensional Magnetically Actuated Microrobotic System R M P Roel Metz MSc Report Committee Prof dr ir S Stramigioli Dr S Misra 5 Khalil PhD Dr ir L Abelmann April 2013 Report nr 008RAM2013 Robotics and Mechatronics EE Math CS University of Twente P O Box 217 7500 AE Enschede The Netherlands lt gt Abstract Minimal invasive surgery MIS aims to reduce patient trauma and recovery time One field of research within MIS is the utilization of microrobots to diagnose and deliver drugs at hard to reach regions within the human body These microrobots could have magnetic properties which allow us to control them by applying magnetic fields The goal of this Master s thesis project is to develop a testbed to perform three dimensional 3D closed loop control of microrobots Control of microrobots in 3D space is implemented using the developed Magnetically Actuated Robotic System MARS MARS has eight electromagnets which are used to generate almost uniform magnetic fields of 64 5 mT in magnitude magnetic field gra dients of 1 52 Tm and gradient of the squared magnetic fields of 127 1 mT mm These magnetic fields and gradients allow us to control paramagnetic microparticles microjets and magnetotactic bacteria within a 3D space of maximum 2 4 mm Feed back for the position controllers is
39. When also two microscopes illumination modules and reservoir holder have to be placed in the same space as the coils there is practically no space for Helmholtz and or Maxwell configuration of coils in our setup However reducing the non linearity in the mag netic fields by arranging the coils in an optimal way is still of great interest Therefore we design the configuration of the coils in a way that the fields and gradients are close to the preferred uniform magnetic fields and gradients The coils arrangement we designed is shown in figure 3 2 a The eight coils are placed in an upper and lower set of both four coils The upper and lower set of coils are placed at 45 and 45 with respect to the horizontal plane respectively In this configuration of coils the magnetic fields of the four upper coils can be combined to provide sufficient gradients to lift the microparticles while also sufficient space is created to position a microscope at the top The side view microscope is positioned 12 CHAPTER 3 SETUP DESIGN b Figure 3 2 Design of the configuration of the magnetic system a Model in Solid Works Dassault Syst mes SolidWorks Corp Waltham Massachusetts USA This model is used to design the physical dimensions of the magnetic system In the model eight coils with cores are used On the left and the top microscopes 1 are positioned Opposing these microscopes illuminations modules 2 are placed In the center
40. activated at 6 5 seconds The autofocusing system moves the microscope to achieve and maintain maximum contrast At maximum contrast the microparticle is in focus 4 3 1 Motion control of a microparticle In this experiment a microparticle is position controlled in 3D space by the imple mented PI position controller as described in section 3 6 5 controller scheme in fig ure 3 18 This controller uses the difference between the actual position and the setpoint as an input The output is a force vector of length 3 This force vector is mapped to currents for the 8 coils using the actuation matrix Because of the limited force the system can provide and the fact that gravity acts in the z direction the gains and saturation levels of the controller are chosen differently per spatial dimension In table 4 1 the gains and saturation levels are provided In z direction the saturation levels are chosen higher than the other two axis to be able to provide enough force to lift the particle The combined saturation levels are chosen in such a way that after mapping into currents the current limit of the coils is not exceeded A position control experiment is performed with this controller In this experiment a 100 um microparticle is point to point position controlled in the workspace by manu ally applying setpoints In figure 4 4 the first 20 seconds of the experiment are shown in a 3D plot The solid line represents the trajectory the particle has take
41. actual position and the setpoint as an input This input is normalized and multiplied by gain K after which the controller output is mapped to electric currents The controller gain K is set to 10 which implies a magnitude of the magnetic field in the setup of 10 mT Due to the fixed gain and the normalized error the magnitude of the magnetic field does not change during the experiment Only the direction of the magnetic field is altered by the controller The results of the motion control experiment are shown in figure 4 7 In this figure the position is represented by the blue solid line and the setpoint by the red dashed line The graph shows that the controller is capable of controlling the microjet to the setpoints While the microjet can not stop due to its self propulsion there will always be overshoot Furthermore the large peaks in the graphs are attributed to the feature tracking algorithm which loses track of the microjet The position accuracy during this experiment is approximately 157 um and the average velocity of the microjet is 121 ums 1 1 500 1 000 500 x position um 0 l 0 10 20 30 40 50 60 70 1 500 1 000 500 y position um 1 500 1 000 500 z position um 0 10 20 30 40 50 60 70 Time s Figure 4 7 Position of a microjet while it is position controlled in three dimensional space The used proportional controller scheme is presented in fi
42. and the contrast of the image The two position signals are used to calculate the position of the robot in 3D space This position and the position setpoint is used in the magnetic controller to calculate an error signal which is used to calculate the desired magnetic field force and or torque The desired magnetic field is mapped to currents for the eight coils which are driven by the current controllers The contrast from both images is used to maximize the contrast of the image Maximizing the contrast is done in the autofocus controller which sets the motor controllers with an desired position or velocity Camera H Camera V Image Image Processing Processing Contrast H 2D Position V 2D Position H Contrast V 3D Position Autofocus Calculation Control 3D Position Motor Controllers Position Setpoint Magnetic o p Control Desired Magnetic Field Mapping Current Controllers Figure 3 14 Software flowchart of the main processes The two cameras deliver an image to the image processing blocks The image processing blocks calculate the con trast of the images and the position of the robots in the images The contrast is used for autofocusing The position signals are combined in a 3 dimensional position This 3D position is used in the magnetic controller to calculated the desired magnetic field The magnetic field is mapped to currents for the eight electro
43. ards are available off the shelf This makes it a challenge to use cameras with the dSPACE system The dSPACE system uses dedicated hardware supplied and tested by dSPACE This makes the system relatively costly the costs of the complete control system are estimated to be over 15k The processor boards are programmed using Simulink and dSPACE Real Time Interface RTI from the host operating system A Simulink license is available at the university When the board is programmed it can run the model s in real time The RTI on the host shows parameters progress and simulation data during simulation and can apply chances to the model LabVIEW Real Time The LabVIEW Real Time system can be used to download real time application to which will run in real time The hardware for a real time system can be implemented in the users PC for less intensive applications or in a standalone solution which can be customized to the users needs using interface cards The standalone solution will be the best choice for our system because of the higher flexibility and computation power The hardware base of the standalone solution is a controller unit Other interface cards will be connected to the controller unit using a PXI connection in the chassis Vision applications can also take advantage of the specialized vision modules These vision modules are standalone image acquisition and processing units Combining a standalone controller solution with one or more v
44. ardware a Matlab and Simulink license is required with additional tool boxes These licenses are available at the university without any further costs When the hardware is setup and connected the software can be programmed The software is built as a model in Simulink which is downloaded to the target PC On the host PC another model can be run in non real time which communicates with the target PC using the network connection This way commands and parameters can be sent to the target and data is received on the host PC to show to the user dSPACE The dSPACE hardware can be partitioned into two categories the single board hard ware and modular hardware The single board hardware consists of an expansion card which can be mounted in a host PC This card has a microprocessor for running real time applications and I O capabilities The modular hardware consist of a base processor board which is mounted in a host PC and extension cards I O cards which can be connected to the processor board to extend I O capabilities The single board hardware has not enough processor power to do image processing at high rate like required in the microrobotics setup so the modular hardware has to be used An big advantage of the modular system is that also the computation power can be extended 85 APPENDIX E REAL TIME SOFTWARE ENVIRONMENTS by adding processor boards to achieve a multi processor system A major disadvantage is that no camera interface bo
45. ation Stage with lead screw M 4 Precision Linear Stage with ball screw m PD ActiveDrive DC Motor includes 24 V power supply DG DC Motor Gearhead 1 Travel Range 25 mm 2 Travel Range 50 mm 4 Travel Range 100 mm 6 Travel Range 150 mm 8 Travel Range 200 mm Ask about custom designs Low Cost of System Ownership The combination of these stages with the networkable single axis C 863 Mercury see p 4 114 and C 663 Mercury Step see p 4 112 controllers offers high performance for a very competi tive price in both single and mul tiaxis configurations Alter natively the C 843 motion troller PCI card with on board servo amplifiers is available Three Motor Drive Options The top of the line M 40x xPD high speed versions come equipped with the high perfor mance ActiveDrive system The ActiveDrive design de veloped by PI features a high efficiency PWM pulse width modulation servo amplifier mounted side by side with the DC motor and offers several advantages W Increased efficiency by eliminating power losses between the amplifier and motor E Reduced cost of ownership and improved reliability because no external driver is required H Elimination of PWM amplifier noise radiation by mounting the amplifier and motor together in a single electrically shielded case 25 VP 2 Phase Stepper Motor ActiveDrive DC Motor includes
46. ber no condenser lenses are available Also no power LEDs with different emitting angle can be found The conclusion that can be drawn is that the combination of LED and condenser lens cannot be optimized Therefore the condenser lens will be chosen according other requirements in this case the maximum dimensions of the illumination system This yields an aspheric condenser lens with outer diameter of 24 mm and an f number of 0 8 The position of the power LED with regard to the collector lens has to be determined experimentally The optimal position should yield a parallel beam of light after passing through the collector lens When this position is determined an enclosure can be designed with these dimensions The lens to focus the light can be chosen by NA of the microscope objective The only limitation that the lens should also fit in the illumination module which limits its 25 3 4 MECHANICAL SYSTEM diameter to about 25 mm The NA of the microscope objective M Plan Apo 10X Mi tutoyo Kawasaki Japan is 0 28 Different lenses are available with an outer diameter of 25 mm and a NA of 2 0 31 at a reasonable price Construction The schematic representation of the illumination system of figure 3 11 is the basis for the design of the illumination system After preliminary test to determine the optimal LED position for the specific condenser lens in cooperation with Islam Khalil an ad justable illumination module is designed see fig
47. ctivily are required The microjets and MTB require a torque to control their orientation The torque is applied using magnetic fields which can be expressed with 2 11 The magnetic dipole moment of the microjets and MTB is known Therefore we can calculate the magnetic torque for a certain magnetic field using 2 11 The magnetic torque has to overcome the drag torque to be able to rotate the microjet or MTB The drag torque T4 can be expressed as ac 3 14 where a is the rotational drag coefficient and w is the angular velocity of the micro robot The rotational drag coefficient is given by Chemla et al 1999 092 d 0 662 3 15 a 3 n a L E 5 10 CHAPTER 3 SETUP DESIGN where L and d are the length and diameter of the microrobot respectively Substitution of 3 15 in 3 14 and equate the result to the magnetic torque of 2 11 yields E m p x B p 0 92 0 662 3 16 While we are only interested in the magnitude of the magnetic field 3 16 can be rewritten as Tin 4 0 92 4 0 662 IIBCp II 3 17 Abbott et al showed that for low magnetic fields the maximum torque is obtained when the angle between the applied magnetic field and the magnetic dipole moment of the microrobot is 45 Abbott et al 2007 For the microjets we can now substi tuting the boundary frequency 25 rad s the length 50 um and width 5 u
48. current mapping The validity of our FE model is investigated by comparing a measured magnetic field with a calculated magnetic field under the same conditions In our setup a constant magnetic field is created by applying a current of 0 5 A to the upper set of coils and a current of 0 5 A in opposite direction to the lower set of coils The magnetic field is measured in a grid with coordinates 6 3 0 3 6 along the x y and z axis This brings the total amount of measurement points in the workspace of 12 to 125 The measurement is preformed with a three axis Hall magnetometer Sentron AG Digital Teslameter 3MS1 A2D3 2 2T Switzerland The FE model is supplied with the same input as the setup and the same data point are extracted from the model as are being measured In 41 4 2 EXPERIMENT AUTOFOCUS 3 J 60 H 2 E 40 1 20 00 0 5 1 1 5 2 00 0 5 1 1 5 2 I A I A a b Figure 4 1 The linearity of the current field relation is determined experimentally The current I is increased from 0 A to 2 A with steps of 0 1 A and applied to system This current creates a magnetic field with increasing magnitude The magnetic field B is measured in the center of the workspace at every current step with a three axis Hall magnetometer Sentron AG Digital Teslameter 3MS1 A2D3 2 2T Switzerland a The coils of the setup have air cores and the current is applied to the upper set
49. cy of 2 2 Hz which is also acceptable 3 5 Electrical system The electrical system of our setup consists of several key components We need drives for the coils and stages drives for the LED s of the illumination and power supplies 28 CHAPTER 3 SETUP DESIGN For safety reasons we also installed an emergency break switch The schematics of the power circuit of the system can be found in appendix The coils of our system are driven by a current supply the stages require a motor controller Elmo Whistle motor drives can act as motor controllers and current supplies depending on the mode they are put in Using Elmo Whistle drives allows us to control the stages as well as the current for the coils with the same type of driver Using only one type of drive in the system is an advantage because only one type of communication has to be implemented The DC motors in the stages will draw a maximum of 0 49 A so the smallest Elmo Whistle drive can be chosen the Elmo Solo Whistle 1 60 Elmo Motion Control Ltd Petach Tikva Isreal This drive accepts input voltages between 7 5 V and 60 V and can deliver up to 1 A Also it has multiple digital in and outputs and CAN interface for communication The coils require higher currents and therefore a different type of Elmo drive is chosen All the energy supplied to the coils is dissipated as heat therefore it is expected that the maximum current for the coils is 2 A In future these coils might be
50. d therefore direct illumination left is the best option for our setup In direct illumination different types can be distinguished like critical illumination and the K hler illumination K hler 1893 Also more advanced techniques are avail able like phase contrast and differential interference contrast These techniques are too complex to use in a custom build system like the microrobotic system and there fore will not be discussed here Critical illumination is the simplest type an image of the light source is projected on to the sample using a lens This lens concentrates all the light emitted by the source in the field of view which yields a very bright illuminated sample A downside of focusing the light is that it can cause an uneven illumination of the sample when not aligned or positioned properly Another disadvantage is that often the light source itself is visible in the image Realizing a highly even illumination of the sample can be achieved by an inventive system introduced by August K hler K hler 1893 This system uses multiple lenses and diaphragms to create a parallel beam of light which yields an even illuminated sample without a visible projection of the source in the image Several system are commercial available but they all have the same disadvantages it is hard to fit them in our system design and they are relatively expensive Also a lot of these systems use indirect lighting while we want to use direct lighting to
51. ding the grayscale image a binary image is obtained In this image see figure 3 15 b the particle is a white circle binary 1 the background is black binary 0 From this binary image all kinds of properties can be calculated under which the position coordinates a b Figure 3 15 Image processing sequence for determination coordinates of the center of microparticle The width of the image is 270um a Original image b Binary image after thresholding c Original image with position marker 32 CHAPTER 3 SETUP DESIGN a b c d e Figure 3 16 Image processing sequence for determination of the coordinates of the end of microjet The width of the image is 250um a Original image b Median filter applied c Binary image after thresholding d Rotated image to coincide with horizontal axis e Original image with position marker Tracking of microjets The microjets are harder to track than the microparticles This is due to their visual appearance The microjets in the images consist of two parts the body of the robots and the trail of bubbles The bubbles have a higher contrast with the background than the body Also the bubbles have a larger diameter than the body of the robot This makes that when applying the same tracker as with the microparticles the tracker often tracks the trail of bubbles instead of the robot itself Another problem is that the microjets are smaller than the micropar
52. direction is created The applied current vector is 0 009157 0 0 004817 1 113 0 7172 0 0 0 7071 The images show that the magnetic field gradient is not uniform in the workspace In the x y plane the magnetic field gradient is constant along the x axis and increasing with a constant slope along the y axis The direction of the magnetic field gradient is along the x axis In the xz plane the magnetic field gradient is increasing along x and z axis The direction of the magnetic field gradient is in x and z direction which could be beneficial while this provides additional lift in z direction 17 3 2 MAGNETIC SYSTEM I I A 1 E m v 3 a x x x E E E OF gt gt x ws EA gt Ss le 4 C ss mus d e EL 1 0 1 y mm YZ plane YZ plane m LT uw ge Ae ox _ 2 E E 0 P FT A N A T o a a a A 1 PIE 8 gt 1 0 1 z mm y mm y mm Figure 3 7 Magnitude left and direction right of the magnetic field gradient in x direction on the x y and yz plane in the workspace when a magnetic field gradient in y direction is created The applied current vector is 0 0 1 095 0 0 0 0 7461 0 7429 The images show that the magnetic field gradient is not uniform in the workspace In the x y plane the
53. e setup is the autofocus system this system keeps the robot in focus while it is moving in 3D space To determine if an image is in focus the autofocus system uses the contrast value of the image when the contrast is maximum the image is in focus To verify that the autofocus system is indeed capable of automatically bringing an out of focus object into focus an experiment is performed This experiment consists of two parts in the first part the maximum contrast value is determined in the second part the autofocus system is activated The first part was performed as follows a 42 CHAPTER 4 EXPERIMENTAL RESULTS 1 1 10 OT o g p 990 amp 8 m O V o 1 Cd CE 6r uz 25 e SE po RE 20 o8 Pi E TN d 09 4 2 d BE s 9 of 0 o 98 0 50 100 0 50 100 Point index Point index a b Figure 4 2 Finite element FE model validation A constant magnetic field is created in the setup as well as in the FE model by applying a current of 0 5 A to the upper set of coils and 0 5 A in opposite direction to the lower set The magnetic field is measure in the setup and extracted from the model in a grid of 125 points with coordinates 6 3 0 3 6 along the x y and z axis a The ratio between the magnitude of the magnetic field i
54. e yields non uniform magnetic fields in the workspace Also a solution to the mapping of gradients of the squared magnetic field has to be found to be able to con trol paramagnetic microparticles in a more controlled way Finally the currently used self propelled microrobots require hydrogen peroxide to propel themselves There fore these robots cannot be used ex vivo or in vivo Microrobots which use a different propulsion mechanism or chemical reaction should be designed for future use in the human body Another option is to also use biological microrobots for ex vivo and in vivo experiments 50 APPENDIX A TEST REPORT TIMM400 Appendix A Test Report 400 In this appendix a test report of the TIMM400 miniature microscopic system SPI GmbH Oppenheim Germany is included 51 Test report 400 microscopic system SPI GbmH www spi robot de Test if we can see the different particles at all colloids self propelled bacteria gold dust The 100um colloids can be seen very nicely At a working distance of 5cm an area of 2 by 2mm can be observed Smaller particles can also be seen but these are harder to focus and are visible as very small dots or blurs on the screen When looking at a bacteria sample small dots blurs can be seen but it is not for sure that these are indeed bacteria Image quality for different particles The image quality for the largest particles 100 colloids is acceptable t
55. earch 242 etant aras ee a deu 50 A Test Report TIMM400 51 iii CONTENTS CONTENTS B CAD Drawings 55 C PIM 404 Datasheet 75 D Vibration damper datasheet 79 E Electronics 83 F Real time software environments 85 G Optimal Controller 89 CHAPTER 1 INTRODUCTION Chapter 1 Introduction In the last 25 years a major change in surgery has occurred endoscopic surgery has revolutionized medicine by enabling surgeons to operate inside the human body with out making large incisions Morgenstern 2008 Endoscopic surgery also called min imally invasive surgery MIS has the advantage over open surgery that only small incisions have to be made Small incisions reduce patients trauma hospitalization and recovery time Over the years many techniques of MIS have evolved Surgical robots were developed which give the surgeon the ability to operate via small incisions on the patient An example of a widely used surgical robot is the da Vinci Surgical Sys tem da Vinci Surgical System Intuitive Surgical Sunnyvale California USA This system is shown in figure 1 1 In this system the movements made by the surgeon at the master device are recorded scaled and send in real time to the slave device The slave device operates the patient and sends a video stream back to the master device Another application of MIS which uses even smaller incisions is the use of flexible needles Abayazid et al In Press Glozman and Shoham 2007 Roes
56. ency of 2 Hz are acceptable in the system higher frequency vibrations should be damped Therefore we can state that the first eigenfrequency should approximately 2 Hz Furthermore it is estimated that the setup including breadboard has a mass m of about 70 kg According to 3 23 a total stiffness of 280 Nm is required for the vibration isolators Four vibration isolators supported the setup so the stiffness of the individual vibration isolator can be added This makes that a single vibration isolator should have a stiffness of about 70 Nm Supplier Paulstra has a variety of vibration dampers in the Radiaflex series Paulstra SNC Levallois Perret Cedex France In appendix D the datasheet is included In this datasheet the stiffness of the dampers is not specified only the maximum load and the corresponding deflection are mentioned When assuming a constant stiffness of the damper the stiffness k can be calculated using the load F and deflection u F k 3 24 u Calculating the stiffness of multiple dampers in the Radiaflex series one of the dam pers reference number 511312 has a stiffness closed to the required stiffness and its dimensions will fit nicely in the setup With a specified load of 70 daN and correspond ing deflection of 8 mm the stiffness becomes 87 5 Nm 1 This is more than the required 70 Nm 1 Recalculating the eigenfrequency of the system using 3 23 and this higher stiffness yields a first eigenfrequen
57. ent should be applied in the opposite direction in the coils 3 2 Magnetic system The magnetic system provides magnetic fields and gradients necessary to control the microrobots In general magnetic fields and gradients generated with electromagnets have the disadvantage of non linearity One way to reduce the effects of non linearity is to create uniform fields and gradients Yesin et al 2006 Uniform fields and gradi ents can be generated by special configurations of the coils Such a configuration is the Helmholtz configuration which generates uniform fields in the center of the Helmholtz configuration between two identical coils In figure 3 1 a schematic representation of a coil configuration is shown The coils are placed on the same axis and at a distance d equal to the radius of the coil r Applying currents to the coils in the same direction generates a uniform field close to the center c A similar configuration is the Maxwell configuration which generates uniform gradients close to the center of the Maxwell configuration between two coils In the Maxwell configuration the coils are separated d with time the radius of the coil r The currents are applied in opposite di rections in the coils A major disadvantage of these configurations occurs when using them for control in 3D space In this case the distance of a second set of coils equals the diameter of the coils in the first set the third set will become even larger
58. er Limited Singapore This driver can provide dimming of the light intensity by using a circuit like described in its datasheet The drivers are powered by the 12 V power supply An emergency switch is installed to manually shutdown the system to prevent dam age to components and or personal injury The currents in the system can be too high for a single switch to make or break the circuit Therefore a relay circuit is imple mented The emergency switch breaks the circuit that powers the main relay The main relay powers the Elmo drives of the stages and two high current relays These high current relays power the Elmo drives of the coils The Elmo drives also have a backup power supply which is not cut when the emergency switch is pressed The backup power supply only powers the electronics of the drive so communication will still be available when the emergency switch is pressed The emergency switch is im plemented as normally closed and the relays are implemented as normally opened 3 6 Control system A control system is needed for the microrobotic system to function properly This con trol system will be implemented in software A global overview of the software is visu alized in the flowchart in figure 3 14 Both cameras will provide the image processing 29 3 6 CONTROL SYSTEM block with an image In the image processing block first a Region of Interest ROI is determined This ROI is used to calculate the position of the robot
59. erface and connection type are according our require ments 22 CHAPTER 3 SETUP DESIGN 3 3 3 Illumination In the setup an optical microscope is used which requires a light source to be able to visualize the sample Illumination of microscopic systems can be divided into two major classes direct and indirect illumination Direct illumination are also called background illumination uses absorption of light by the sample to make the sample visible in the camera image figure 3 10 left The indirect illumination principle uses the reflection of light at the sample the make the sample visible in the camera image figure 3 10 right The indirect illumination can be implemented using ring illumina tion which is placed around the microscope objective or by using coaxial illumination Coaxial illumination is implemented by a semi transparent mirror inside the micro scope which reflects the light from the source directly out of the objective towards the sample In general the advantage of indirect illumination is that also surfaces of solid objects can be made visible A disadvantage is that samples with low reflection like transparent tissue are less visible than sample which are very reflective Microscope objective Sample holder v Figure 3 10 Schematic representation of two types of illumination principles Left direct illumination Right indirect illumination In our system also less reflective microrobots will be use
60. es and runs standard at 3 5 GHz with a turbo function to 3 9 GHz A variable clock frequency of the CPU can cause instability of the real time system Therefore the clock frequency is fixed The fre quency is overclocked to 4 5 GHz to gain even more computation power The system is tested to run stable at this higher clock frequency of the CPU The CPU is installed on an ASUS P8Z68 Deluxe GEN3 mainboard ASUSTeK Computer Inc Taipei Taiwan This mainboard is compatible with the CPU has plenty of extension slots and has a network chip that is compatible with xPC Target drivers The mainboard is equipped with 8 GB of DDR3 memory of which 2 GB can be used by the real time application Furthermore the system is equipped with two Neon CLD framegrabbers from BitFlow BitFlow Woburn Massachusetts United States The Neon CLD framegrabber has a PCI E x4 interface and a dual channel CameraLink interface design the card ac cepts two CameraLink cameras Unfortunately due to the drivers in xPC Target 31 3 6 CONTROL SYSTEM only one CameraLink channel can be used Communication with the Elmo drivers is done using a CAN bus The interface for the CAN bus is provided by a Softing CAN AC2 PCI Softing AG Haar Germany dual channel CAN interface card which can be installed in a PCI slot The main components of the real time controller hardware can be summarized as follows An Intel i7 2700K 4 5 GHz is used as CPU The mainboard
61. etic force acting on a microparticle can be summarized as follows 41 3 T TT YmV B p B for B 79 9 mT pt IIBCp II 2 10 Ps m V B p for B p gt 79 9 mT This equation is used in section 3 2 to design the magnetic based control system for microparticles 2 2 Modeling of torque controlled microrobots Microjets and MTB have self propulsion systems which converts chemical energy into kinetic energy Solovev et al 2009 Martel et al 2009 This propulsion system pro vides a thrust force which allows the microrobots to move in the fluid The microrobots can be steered by controlling their orientation The orientation can be controlled by ap plying magnetic torque on the microrobot This magnetic torque T p R acting on a magnetic body is given by x B p 2 11 CHAPTER 2 MODELING OF MICROROBOTS a b Figure 2 1 a A microjet with trail of bubbles b An magnetotactic bacterium In the inset the magnetic crystals can be seen as a chain of small black dots where m p is the magnetic dipole moment of the microjet MTB and B p is the applied magnetic field The magnetic dipole moment can be deter mined experimentally or it can be calculated using 2 2 The structure of the microjets is formed by rolling up multiple layers of different materials titanium chromium iron and platinum The exact dimensions of the different layers afte
62. ferent materials at 100um colloids a reservoir the working distance is about 5cm When looking through 1mm thick dirty glass no notable distortion will occur Also when using 2mm thick dirty Plexiglas no distortion is noticed and focusing is not affected When using thicker bmm Plexiglas the focus needed to be adjusted This is probably due to the diffraction caused by the thick plastic The image quality and focus were not notably affected Influence of magnetic fields No influence is determined 100um particles backlight high magnification Fr d E I E particles backlight particles direct illumination am Bacteria backlight Microscale 1mm 100lines backlight APPENDIX A TEST REPORT TIMM400 54 APPENDIX B CAD DRAWINGS Appendix B CAD Drawings In this appendix the CAD drawings of the 3D printed and machined parts are shown 55 I M30X0 5 zm VS SECTION B B SCALE2 1 UNLESS OTHERWISE SPECIFIED ARE IN MILLIMETERS ANGULAR NAME DRAWN CHKD APPV D MFG QA FINISH SIGNATURE 1 2 gt N N 1 de 7 PE nae DO NOT SCALE DRAWING REVISION EDGES DATE TITLE IlluminationSystemLensesHolder MATERIAL DWG NO A4 Black anodized aluminum lluminationSystemLensesHolder No of Parts 2 SCALE 1 1 SHEET 1 OF 1 1
63. field in the workspace By solving 3 27 for I the currents for a required magnetic field can be found I B 3 28 37 3 7 REALIZED SYSTEM The actuation matrix B p can be found by using 3 27 and a set of known currents and corresponding magnetic fields While the magnetic field can be assumed constant in the workspace the magnetic field in only one point is considered per current set This reduces the complexity of the calculation drastically The actuation matrix is found as a least squares solution of the used sets Therefore the more current sets are used the more accurate the actuation matrix becomes Mapping the magnetic fields gradients can not be done in a similar way as mapping the magnetic fields Substitution of 3 27 in 2 9 yields F p 808p 329 The force is quadratic in I and therefore an inverse mapping of force to electric current cannot be generated It is shown that the magnetic field gradients can be considered constant in the workspace of our system This simplifies 3 29 since v is a constant matrix and can be calculated offline 3 29 has to be solved online but since the magnetic force is quadratic in I no unique solution can be found A different approach is used to be able to provide controlled magnetic field gradients in our setup We assume that the magnetic field gradient is directed in the direction of the magnetic field We use the magnetic field
64. gure 3 19 with gain K 10 The blue line represents the position while the red dashed line represents the applied setpoint The large peaks in the graphs are attributed to the feature tracking algorithm which loses track of the microjet The position accuracy during this experi ment is approximately 157 and the average velocity is 121 ums 1 48 CHAPTER 5 CONCLUSIONS Chapter 5 Conclusions This chapter presents the conclusions and the recommendations 5 1 Conclusions The magnetically actuated system for controlling microrobots in 3D space is realized The magnetic system is capable of generating almost uniform magnetic fields in the workspace with a maximum magnitude of 64 5 mT gradients of the magnetic field of maximum 1 52 Tm and gradients of the squared magnetic field of maximum 127 1 mT mm These magnetic fields and gradients are sufficient to control mi croparticle microjets and MTB in 3D space Visual feedback for the controllers and users is obtained by two microscopes with cameras attached The resolving power of the microscopes is approximately 1 um and a field of view which ranges from 0 1mm x 0 1mm to 2 4mm x 2 4mm Furthermore the system is equipped with an aut ofocusing system which is capable of focusing on microrobots and maintaining focus while the microrobots move in 3D space The position of the microrobots is obtained from feature tracking algorithms that track the microrobots in the images provided
65. h cameras this can be used to combine the two sets into x yz space by simply adding the z coordinate to the x and y position 3 6 4 Autofocus implementation Autofocusing system can be partitioned into two classes active and passive Levoy etal 2012 An active focusing system sends some kind of radiation towards the scene from the captured reflection the distance of objects in the scene can be calculated using triangulation In passive focusing systems the scene is captured without sending any additional radiation In passive focusing system two common types can be distinguished phase detec tion and contrast detection Phase detection detects a phase difference of a point or multiple points in an image From this phase difference the distance from the object to the camera can be calculated When the object distance is known the position of the lens of the camera can be calculated By repositioning the lens to the calculated position the scene will be in focus This is also the main advantage of phase detec tion only one measurement is needed to focus an image The disadvantage is that additional hardware is needed to detect phase differences The contrast detection like the name suggests detects the contrast of an image When an image is in focus the contrast will be maximal So by finding the maximum contrast of an image the image will be in focus This can be done by repositioning the camera lens until the contrast is maximal this
66. he applied setpoints The arrows indicate the direction of the trajectory The motion of the microparticle is better shown in figure 4 5 where the x y and z positions are plotted on separate axis In this figure the full 60 seconds of the experiment are shown The position is represented by the solid line and the setpoint by the dashed line In the graphs the behavior of the system on a setpoint change can be seen almost instantly after the setpoint is changed the particle starts moving and with a little overshoot it reaches the demanded position It can also be seen that there is cross influence between the different axis For example at 25 seconds a small deviation in x position can be seen which is caused by a setpoint change in the yz plane On the z axis the influence of a setpoint change in the x y plane is even more noticeable In this experiment the average velocity is 367 ums 1 and the maximum velocity is 2 mms Another important property of the system is position stability around a setpoint This data is acquired by performing an experiment in which the setpoint is not altered 45 4 3 EXPERIMENTS MOTION CONTROL 2 000 1 500 1 000 500 x position um 2 000 1 500 1 000 500 y position um 1 500 1 000 500 z position um Time s Figure 4 5 Position of a microparticle while it is position controlled in three dimensional space The used proportional
67. he images be used for object tracking in software For the smaller particles 3um particles and bacteria the image quality is poor and is probably not usable for object tracking in software This is probably due to the too low magnification and or microscope quality Test focus in water and other fluids The microscope was tested for focusing in water The microscope system was set to a working distance of 5cm The focus was tested at 1cm 3cm and 5cm depth in the water Despite of disturbances caused by vibration of the setup and the water the image quality and focus were good See how the motorized scope performs This test cannot be performed because no motorized scope is available This type of scope is only made on request See what frame rates are achievable At a resolution of 2592x1944 pixels and enough light a frame rate of about 6 fps is achievable When the resolution is set to 1296x972 the frame rate increases to 22 fps which seems to be the maximum frame rate of the camera Performance with different lightning conditions direct light backlight When using only ambient light the scope will produce a black screen By applying backlight good bright images can be recorded Also when using direct light the microscope produces good images but it is harder to direct the light into the camera than when using backlight Performance with different reservoir materials glass plastics etc This test is done by looking through dif
68. hen the applied magnetic field reaches a certain magnitude the magnetization of the particles material saturates the magnetization becomes constant m The manufac turer micromod Partikeltechnologie GmbH Rostock Germany of the microparticles specifies the saturation mass magnetization to be 6 6 x 1073 Am g and the density to be 1 4 x 10 gm 3 Therefore the saturation magnetization is calculated to be 9 24 x 10 Am Using a susceptibility of 0 17 Khalil et al 2012a and the calculated saturation magnetization of m 9 24 x 10 Am in 2 5 yield the magnitude of the magnetic field at which the magnetization of the material saturates B p 79 9 mT When the magnetization saturates and becomes constant also the magnetic dipole mo ment becomes constant In this case 2 3 can be rewritten as 4 m p yr m gt 79 9 mT 2 6 Consequently the magnetic force of 2 1 can be rewritten as 4 _ B p 2 79 9 mT 2 7 At magnetic fields B p less than 79 9 mT the magnetization of the material of the particles is not saturated Therefore the magnetization depends on the magnetic field B p Substitution of 2 5 in 2 3 yields the expression for the magnetic dipole moment 41 m p 3077 Z2mB p IIB p lt 79 9 mT 2 8 Substitution of 2 8 into 2 1 yields the following force field map 41 F p 3 B p B lt 79 9 mT 2 9 The magn
69. icroscope to the maximum contrast value and oscillates around this value The final contrast value is in the same range as the maximum value found in the first part of the experiment From this experiment it can be concluded that the autofocus system is indeed capable of focusing on a out of focus robot in the work space 4 3 Experiments motion control The overall functionality of the system is shown by performing motion control experi ments in the realized system These experiments consist of controlling the position of a microparticle and microjet in 3D space 43 4 3 EXPERIMENTS MOTION CONTROL Contrast SAD value lo 0 2 4 6 8 10 12 14 16 Time s Figure 4 3 Experiment to show the functionality of the autofocus system The blue line shows the contrast value of the image which is calculated according to the sum of abso lute differences SAD principle The red dashed line represents the contrast value for the in focus image In this experiment a microparticle is put on the water surface and the microscope in positioned out of focus In the time span between 1 and 5 seconds the microscope makes a sweep with constant velocity in the z direction During this sweep the microparticle goes from out of focus to focused and to out of focus again This part of the experiment is to determine the maximum contrast and is only per formed for demonstrative purposes Next the autofocus is
70. in the workspace but the gradients of the magnetic field are less uni form By optimizing the system and the coil design also the gradients of the mag netic field can become uniform The cores used in the coils of the system should be redesigned The current cores are made from steel which require more energy to mag netize than specialized materials It is expected that using special core material for example VACOFLUX VACUUMSCHMELZE GmbH amp Co KG Hanau Germany and design the cores with a tapered end could generate magnetic fields with higher mag nitude at lower electrical currents 5 2 2 Future research The future research should be focused on medical implementation Therefore the next step in our research should be the modification of the setup to be suitable for clinical use It implies a couple of things that should be modified First the the microscopic system should be replaced with a system that is capable of visualizing microrobots in an opaque environment like the human body A clinical image modality that could be used is ultrasound imaging Tests have shown that microparticles of 100 um can be seen in ultrasound images Second the workspace of our setup is limited to the field of view of the microscope This workspace should be extended to be usable in a clinical environment When the workspace is extended also more research has to be done in the mapping of the magnetic field It is to be expected that an extended workspac
71. ing xPC Target and LabVIEW Real Time both systems deliver high performance have a large variety of libraries at their disposal and licenses are available at the university Furthermore it is estimated that the LabVIEW system will be more expensive than the xPC Target system Also the users have more experience with Simulink and xPC Target which makes setting up and configuring this system easier and less time consuming Therefore xPC Target is thought to be the best choice for the control system of the microrobotics setup 3 6 2 Real time hardware environment In section 3 6 1 the real time software environment is chosen The best choice for our system is xPC Target from MathWorks The xPC Target system can run on a con sumer PC with additional interface cards For our application a PC with high clocked CPU is required because of the image processing that is done at high frame rates Also the image processing steps have to be done in a specific order and therefore cannot be processed in parallel While two image streams have to be processed a multicore CPU can be beneficial the two image streams can be processed at the same time Other task could be executed in parallel with the image processing like autofocusing and communication with the host PC therefore a quadcore CPU will be the best choice A CPU that fulfills our needs is the Intel i7 2700K Intel Corporation Santa Clara California United States This processor has four cor
72. integral controller scheme is presented in figure 3 18 with gains K 0 14 0 14 0 20 K 0 08 0 08 0 10 The upper and lower saturation levels are set to 10 and 10 for the x and y axis and to 80 and 30 for the z axis The blue line represents the position while the red dashed line repre sents the applied setpoint The average velocity during the experiment is 367 ums and the maximum velocity is 2 mms 1 The setpoint is set to 258 420 540 pixels and the system controls the particle to stay at this setpoint The position of the particle is shown in figure 4 6 The position is represented by the solid line and the setpoint by the dashed line As can be seen in the graphs the control system keeps the microparticle close to the setpoint a deviation of about 2 um can be seen In table 4 2 more details about the position stability of the controlled microparticle are shown The first column of values shows the deviation between the mean posi tion of the particle and the setpoint in pixels The second column shows the standard deviation and in the third column the minimum position value is subtracted from the maximum position To provide a better understanding of these numbers the values are also transformed into micrometers under the assumption that the system operates in air Under this assumption every pixels corresponds with 2 33 um This makes the total windows size of the camera about 2 4 mm and the diameter of the used mic
73. is the ASUS P8Z68 Deluxe GEN3 The system is equipped with 8 GB of DDR3 memory of which 2 GB is available for the real time application The cameras are connected by two BitFlow Neon CLD framegrabbers Communication with the Elmo driver is provided with a Softing CAN AC2 PCI CAN adapter 3 6 3 Feature tracking software The position of the microrobots should be known in three dimensional space to be able to control them properly The position of the robots can be determined by using a tracker which is build in software This tracker uses the information in the two camera images to locate and track the microrobots while they move through the workspace The acquired coordinates from the two two dimensional images can be combined to obtain the coordinates of the position of the robot in three dimensional space While the three types of microrobots have different properties concerning their visual ap pearance also three different trackers are designed Every tracker uses a ROI to reduce computation time This ROI is first specified by the user and when a microrobot is found the position of the ROI is moved to coincide with the center of the microrobot The user can always overrule this calculated position of the ROI by specifying new coordinates for the ROI Tracking of microparticles The microparticles can be tracked relatively easy they have a well defined shape and a very high contrast with the background see figure 3 15 a By threshol
74. ision modules could be an interesting solution for our system The real time application can be developed on a separate PC an downloaded to the standalone real time hardware Another possibility is to use a hypervision module which runs a Windows operation system OS in a virtual environment on the stan dalone solution This OS can be used to develop the real time application and also provide an user interface which communicates with the real time application The costs of a complete solution that will fulfill our needs is estimated to be over 10k 20 sim 4C The 20 sim 4C system allows users to execute models as real time C code on dedicated hardware like a PC or an ARM 9 based processor board In this target machine ad dition interface cards can be installed to provide more I O functionality The choice of hardware is only restricted to the used RTAI Linux operating system and therefore consumer electronics can be used which can reduce overall costs of the system The target and host are connected via Ethernet using the TCP IP protocol The target machine will run a RTAI Linux operating system In this operating system the C code is downloaded and will be executed in real time Any 20 sim model can be used in 20 sim 4C When the model in downloaded to the target machine the user can start and stop the execution from the host PC Also other commands and parameters can be send to the target machine while execution the real time code The 20 si
75. le tissue interaction In Proc IEEE RSJ Int Intelligent Robots and Systems IROS Conf pages 2557 2563 2011 doi 10 1109 IROS 2011 6094969 URL http ieeexplore ieee org stamp stamp jsp arnumber 6094969 R J Roesthuis Abayazid S Misra Mechanics based model for predicting in plane needle deflection with multiple bends In Proc 4th IEEE RAS amp EMBS Int Biomedical Robotics and Biomechatronics BioRob Conf pages 69 74 2012 doi 10 1109 BioRob 2012 6290829 URL http ieeexplore ieee org stamp stamp jsp arnumber 6290829 M S Sakar S Schurle S Erni E Ullrich J Frutiger Ergeneman Kratochvil and B J Nelson Non contact 3d magnetic biomanipulation for in vivo and in vitro applications In Optomechatronic Technologies ISOT 2012 Interna tional Symposium on pages 1 2 IEEE 2012 URL http ieeexplore ieee org xpls abs all jsp arnumber 6403292 Solovev Y Mei E Berm dez Ure a Huang and Schmidt Catalytic micro tubular jet engines self propelled by accumulated gas bubbles Small 5 14 1688 1692 2009 URL http onlinelibrary wiley com doi 10 1002 smll 200900021 full K Berk Yesin K Vollmers and B J Nelson Modeling and control of untethered biomi crorobots in a fluidic environment using electromagnetic fields The International Journal of Robotics Research 25 5 6 527 536 2006 URL http ijr sagepub com content 25 5 6 527 short 93
76. m the dynamic viscosity 1 mPa s the magnetic dipole moment 1 4 x 10 1 and an gle between B p and is 45 in 3 17 which results in a required magnetic field BCp I of 19 1 mT The experimentally determined boundary frequency of MTB is 9 5 rad s Khalil et al 2012b When we substitute this and the length 5 um and width 0 2 um of the the dynamic viscosity 1 mPa s the magnetic dipole moment 3 59 x 10719 Am and angle between B p and is 45 in 3 17 the re quired magnetic field B p is calculated to be 1 9 mT The magnetic field required to rotate a MTB is lower than the magnetic field required for the microjets Therefore the magnetic field for the microjets is dominant and will be used as design input for the magnetic system of our setup Besides the requirements for the magnetic system other requirements need to be set The magnification of the microscopes can be specified by the field of view FOV It is devised that a FOV of approximately 20 to 25 times the size of the microrobot is sufficient A larger FOV results in a lower magnification of the microrobot which can make the image processing more difficult On the other hand a smaller FOV results in a larger microrobot in the image which makes the image processing easier but limits the workspace of the microrobot The size of the bacteria microjets and microparticles is 5 um 50 um and 100 um respectively This results in FOV
77. m program is very nice to design controllers and do simulations but currently it lags computer vision support This means that all the image processing and camera input should be programmed by hand Real Time Linux The real time Linux operating system can be installed on any PC hardware The only limitation can be the available drivers for hardware By installing interface cards which 86 APPENDIX E REAL TIME SOFTWARE ENVIRONMENTS are supported by Linux drivers into a consumers PC the hardware for the real time system is completed In the Linux operation system the real time application can be programmed in for example the C or C language This gives the user a lot of flexibility in how to implement functionality into the application On the other hand this flexibility leave the user also in control of timings and data integrity which could lead to instability when done improperly good knowledge of the programming language and Linux is essential 87 APPENDIX E REAL TIME SOFTWARE ENVIRONMENTS 88 APPENDIX G OPTIMAL CONTROLLER Appendix G Optimal Controller In a magnetically actuated control system heat in the electromagnets will always be an issues One way to prevent the coils from overheating is actively cooling the coils A better solution would be to generate less heat in the coils This can be done with an optimal controller that optimizes the current in the coils The equations of motion for the different
78. magnetic field gradient is increasing in y direction with a constant slope and its direction is primarily along the y axis In the yz plane the magnetic field gradient is increasing along y and z axis The direction of the magnetic field gradient is in y and z direction which could be beneficial while this provides additional lift in z direction 18 CHAPTER 3 SETUP DESIGN XZ plane XZ plane Ae E x d o f N 1 Poi T d a LITTA ola zm 1 0 2 mm x mm x mm YZ plane YZ plane qur 1 T d ow ow ct of f N A de E dee olg Ae qs 1 0 z mm y mm y mm Figure 3 8 Magnitude left and direction right of the magnetic field gradient in z direction on the xz and yz plane in the workspace when a magnetic field gradient in z direction is created The applied current vector is 0 9475 0 9859 0 9684 0 9894 0 0 0 0 The images show that the magnetic field gradient and its direction are close to uniform the workspace The magnetic field gradients are around 45 mT mm which shows that configurable design is capable of creating sufficient magnetic field gradients in the z direction to provide lift for the microparticle 19 3 3 MICROSCOPIC SYSTEM MTB the current coil configuration is used to provide a
79. magnets This software structure has to be programmed in some kind of platform We have 30 CHAPTER 3 SETUP DESIGN decided to implement a real time platform The available real time platforms are discussed in section 3 6 1 The image processing algorithms are worked out in sec tion 3 6 3 Robot tracker and section 3 6 4 Autofocus 3 6 1 Real time software environment For this project we decided to use a real time platform There are several platforms available MathWorks xPC Target dSPACE National Instruments LabVIEW Real Time 20 sim 4C and Real Time Linux In appendix F these platforms a shortly descibed and evaluated on several topics like available computation power ease of use ease system setup available hardware and costs The control system for the microrobotics setup should be an easy to implement sys tem and off the shelf available so Real Time Linux and 20 sim 4C will be discharged The other three system can perform at a similar high level The modular system from dSPACE can extent its computation power by adding additional processor boards but it is also more expensive Furthermore it is expected that curtain tasks cannot be done multithreaded which makes a high clocked singlecore CPU more beneficial than a multicore CPU at lower clock frequency The clock frequency of the dSPACE pro cessor boards lack behind the available processors in the market which makes them less beneficial for our setup Compar
80. mapping of 3 28 and limit the currents to positive values only Negative current values are set to zero The method creates magnetic field gradients in the direction of the applied magnetic field In section 3 2 it is shown that our method results in directional controlled magnetic field gradients Also the experimental results show the validity of our method A disadvantage of our method is that it is not a direct mapping of force to magnetic field gradient Consequently it is not possible to apply a specified force to a microparticle However the applied force can be calculated with the direct force current map of 3 29 and the applied currents 3 7 Realized system The Magnetically Actuated Robot System MARS is designed as a 3D model in Solid Works Dassault Syst mes SolidWorks Corp Waltham Massachusetts USA This de sign is shown in figure 3 20 a A photo of the MARS is shown next to it in fig ure 3 20 b The visual feedback of MARS is provided by a horizontal and vertical microscope 1 The microscopes are equipped with cameras 2 and mounted linear stages 3 which are used for autofocusing capabilities The magnetic system is placed in a spherical structure of 3D printed plastic 4 The sphere consists of two parts which can be taken apart to gain access to the lower set of magnets and the illumination mod ules The magnets and stages are driven by motor drivers 5 which are mounted on the control panel 6 The control panel
81. mm YZ plane YZ plane 4 x 18 4 1 4e qo OR e 18 2 T T TT E E eae e um N 18 TUTTI 1 cbe p Ae pd 0 0 1 1 1 0 1 z mm y mm Figure 3 5 Magnitude left and direction right of the magnetic field on the x xz and yz plane in the workspace when a magnetic field in z direction is created The ap plied current vector is 0 3353 0 3846 0 3237 0 3641 0 358 0 4154 0 3401 0 4217 The images show that the magnetic field is close to uniform in the workspace The stan dard deviation of the magnetic field and its direction in the workspace is 0 07 mT and 0 20 respectively 16 CHAPTER 3 SETUP DESIGN X plane XY plane T T T gt gt gt gt gt gt gt T 10 T gt gt gt gt g 0 gt gt gt 4 5 lt gt gt gt gt N 0 E ue dE 1 1 of gt gt 0 0 1 1 0 1 mm x mm x mm XZ plane XZ plane PPP Z E a ue 17 L _ oP PHF ET 0 A p F FS f ES N gt 68 f T SES d A BOT T l 1 0 1 z mm x mm x mm Figure 3 6 Magnitude left and direction right of the magnetic field gradient in x direction on the x y and xz plane in the workspace when a magnetic field gradient in x
82. n the image so it will be removed from the image and the algorithm looses track of the robot Another way to improve the algorithm is by applying erosion to remove background objects which are smaller than the robot itself When using erosion it has to be taken into account that not all robots have the same size by eroding the image also smaller robots could be removed Tracking of bacteria The magnetotactic bacteria are the hardest to track First of all their small size 5 um requires a high magnification of the microscope The higher magnification makes that more background noise becomes visible in the image Secondly the bacteria have a lower contrast with the background than the microparticles or microjets which makes the thresholding harder Thirdly when the bacteria is above or below the focus plane 33 3 6 CONTROL SYSTEM their appearance compared to the background becomes much darker or much lighter This makes that the tracker should find a dark or light spot in the image dependent on the position of the bacteria NOM gt re b d e Figure 3 17 Image processing sequence for determination coordinates of the center of a bacteria The width of the image is 20um a Original image b Contrast enhancement c Background subtraction d Contrast enhancement e Median filter applied f Binary image after thresholding g Original image with position marker First the contra
83. n the model and in the setup is calculated The mean ratio is 0 984 with a standard deviation of 0 041 b The angle between the measured and calculated magnetic field in every point is calculated The mean angle is 2 9 with a standard deviation 1 87 The deviation in magnitude and angle is attributed to the expected discrepancy between an ideal model and a practical implementation Also the initial position and orientation of the probe is set by hand which affects the actual coordinates at which the magnetic field is measured microparticle is put in the reservoir on the water surface The microscope is placed out of focus and then moved with constant velocity towards the microparticle and beyond The contrast value was logged during this motion In figure 4 3 the solid line in the time span between 1 and 5 seconds represents the contrast value of this part of the experiment In this graph the maximum contrast value is indicated with the dashed line this is the contrast value at which the microparticle is in focus The second part of the experiment is to verify that the autofocus system is capable of finding the same maximum contrast value by moving the microscope automatically In figure 4 3 this part of the experiment starts at 6 5 seconds when the autofocus system is activated The autofocus system is first searching the correct direction of motion to maximize the contrast When the direction is found the system moves the m
84. n the sensor because the light is only partially projected on the microscope objective Light source The light source for the illumination system will be a 3 W power LED Major advantages of a power LED over other light sources like halogen light bulb are the lower heat dissipation and light emission in a specific frequency range The lower heat dissipation results in less unwanted heating of the sample While in the setup a fluid reservoir is used the fluid could heat up and therefore unwanted currents in the fluid could occur So a low heat dissipation is preferred In section 3 3 2 the to be used cameras are discussed and in figure 3 9 the light sensitivity of the camera is shown The cameras are most sensitive to green light and therefore the LED will have to match this property to yield an optimal camera LED combination A power LED which has the desired properties is the Avago 3 W Green Power LED Light Source from the ASMT Ax3x series Avago Technologies San Jose California United States In figure 3 12 a the intensity of the illumination is shown Comparing figure 3 9 and 3 12 a it can be seen that the wave length ranges match Furthermore in figure 3 12 b the light intensity over the emitting angle of the LED is shown The intensity is relatively constant over the emitting angle which yields a uniform illumination of the sample when combined with a collector lens The LED will be driven by a special driver which can vary the light i
85. n the x y xz and yz plane are shown when a magnetic field is created along the x y and z direction The magnetic fields are almost uniform The standard deviation of the magnetic field magnitude in the workspace is 0 22 mT 0 24 mT and 0 07 mT when creating a magnetic field in x y and z direction respectively The standard deviation of the direction of the magnetic fields is 0 32 0 28 and 0 20 when a magnetic field is created along x and z direction respectively Furthermore the magnitude of the magnetic fields is close to the required 20 mT while the applied currents lower than 0 7 A Preliminary simulations of the magnetic field gradients showed that in the currently designed system the gradients do not meet the requirements Especially the gradients in z direction are not sufficient to provide the required lift to overcome gravity While the current design of the magnetic system is capable of providing close to uniform fields we decided to design a configurable system In the case of using microjets and 13 3 2 MAGNETIC SYSTEM XY plane XY plane I gt 1 17 8 gt gt gt gt gt d 17 6 gt m E 0 cor 17 4 gt gt gt gt gt gt gt gt 17 2 1 l gt 1 1 0 1 y mm x mm x mm XZ plane XZ plane I I I
86. n to move to the next setpoint which is represented by the solid dots The experiment starts at the most left dot in the diagram and the coordinates change according to the graphs in figure 4 5 As shown in figure 4 4 the system is capable of moving a microparticle 44 CHAPTER 4 EXPERIMENTAL RESULTS P gain Lgain Saturation max min x axis 0 14 0 08 10 10 y axis 0 14 0 08 10 10 z axis 0 20 0 10 80 30 Table 4 1 Gains and saturation of the proportional integrating PI position controller in 3D space The trajectory between two successive is often no straight line which is probably due to the imperfection of the mapping of the magnetic fields the very simple controller and maybe some fluid flow in the reservoir Also some overshoot can be seen when the particle arrives at the setpoint g 1 000 8 e N 500 1 800 xl 1 600 1 400 L 1 400 i 1 200 1 000 y position um x position um Figure 4 4 A microparticle is position controlled to several setpoint is the three dimensional workspace The used proportional integral controller scheme is presented in figure 3 18 with gains K 0 14 0 14 0 20 K 0 08 0 08 0 10 The upper and lower saturation levels are set to 10 and 10 for the x and y axis and to 80 and 30 for the z axis The trajectory of the microparticle is represented by the blue line and starts in the most left point in the diagram The red dots represent t
87. nearity of the current field relation and the validity of the FE model The linearity of the current field relation is checked by applying a range of currents to the coils of the setup and measuring the generated magnetic field The currents range from 0 A to 2 A with steps of 0 1 A The magnetic field was measured in the center of the workspace with a three axis Hall magnetometer Sentron AG Digital Teslameter 3MS1 A2D3 2 2T Switzerland The current field relation is measured in two configurations of the setup First the experiment is performed with air core coils The current is applied to the upper set of coils The lower set is supplied with 0 A The results of this experiment are shown in figure 4 1 a The current field relation is linear over the complete current range from 0 A to 2 A The saturation at 0 A and 1 9 A is attributed to limitations of the current source In the second experiment the coils are equipped with metal cores The current range is applied to the upper and lower set of coils in the same direction which generates a magnetic field along the z direction The results are shown in figure 4 1 b The magnetic field is linear up to approximately 1 A of applied current At higher applied currents the current field relation is non linear and the magnetic field saturates at approximately 65 mT at 2 A The FE model is an important tool in the design of our system It is used to design the magnetic system and to calculate the field
88. ntensity 24 CHAPTER 3 SETUP DESIGN 0 9 GREEN e 0 8 BLUE 0 8 E 07 ROYAL BLUE a 0 6 06 z 05 8 05 z 04 04 2 03 03 0 2 02 0 1 0 1 0 0 380 405 430 455 480 505 530 555 580 605 630 90 60 60 90 WAVELENGTH nm ANGULAR DISPLACEMENT DEGREES a b Figure 3 12 Specification of the Avago 3W Green Power LED Light Source from the ASMT Ax3x series Avago Technologies San Jose California United States Images from datasheet Avago Technologies 2011 a Emitting wavelength b Light inten sity over emitting angle Microscopic lenses Two different lenses are needed for the illumination system The first is to collect the light from the LED and the second to focus the light into the sample see also figure 3 11 For collecting the light a condenser lens can be used This lens can be chosen by f number written as f or This f number is defined as the ratio between focal length f and diameter of the entrance pupil D m N D 3 20 The f number also defines the angle at which light can be accepted From the geometric relation the angle can be written as 0 3 21 arctan 2f 3 21 By rewriting this equation it can be applied to equation 3 20 which yields an equation for the f number expressed in the angle 1 2tanO 3 22 While the LED is emitting light at approximately 120 see figure 3 12 b the f number is calculated to be 0 29 For this f num
89. or magnetic field can be a proportional P controller The schematic representation of the controller is shown in figure 3 19 The controller takes the position error vector as input The position error vector is calculated as the difference between the reference position and the measured position of the robot This error vector is normalized to obtain its direction The direction of the error vector is also the direction in which the magnetic field should be applied to orient the microjets and MTB towards the reference position The proportional gain K determines the magnitude of the applied magnetic 36 CHAPTER 3 SETUP DESIGN Force current Microrobotic Vref mapping system 2 ref Feature tracking Figure 3 18 Schematic representation of the magnetic field gradient controller This proportional integral PI controller takes the position error as input The position error is calculated as the difference between the reference position and the measured position of the microrobot This position error is multiplied by the proportional gain K and added by the sum of the position error multiplied with the integral gain K The output of the controller is mapped to currents These currents are limited to positive values and applied to the microrobotic system The limitation of the currents leads to the generation of magnetic field gradients X ref ES Torque current Mic
90. p resentation of the controller implementation is shown in figure 3 18 The controller takes the position error signal as input The error signal is calculated as the difference between the reference position and the measured position of the microparticle Fur thermore the controller is of the proportional integral PI type The integral term is required to provide the necessary force in z direction to lift the microparticle the inte gral term can have an output when the error is zero The output of the PI controller is limited in magnitude to prevent the current drivers from saturation The proportional gain K and integral gain K are both vectors are in This allows us to set the gains of each of the spacial dimensions separately Also the output limit can be set separately for each spatial dimension The output of the PI controller is a force vector IR This force vector is mapped to a current vector R8 1 by the actuation matrix These currents are applied to the coils of the setup Magnetic torque controller The microjets and MTB are actuated by magnetic torque The magnetic torque makes the microjets and MTB to rotate towards the applied field their will align themselves with the magnetic field The controller mainly has to control the direction of the mag netic field The magnitude of the magnetic field can be constant and only has to over come a curtain lower limit as discussed in section 3 1 Therefore the controller f
91. pace Another type of microrobots have self propulsion capabilities Ex amples of this type of microrobots are microjets Solovev et al 2009 and spiral type magnetic micromachines Ishiyama et al 2002 Microjets figure 1 2 b are small hollow tubes in which a catalytic reaction with the fluid they are submersed in pro duces air bubbles These air bubbles are used to propel the microjet through the fluid The magnetic properties of microjet allow for the control by applying magnetic fields Spiral type micromachines have a spiral shape which screws the microrobot through the workspace by magnetically rotating the microrobot around its axis Furthermore biological microrobots are of interest for medical applications Examples for these types of microrobots are magnetotactic bacteria MTB figure 1 2 c and red blood cells with artificial tails MTB propel themselves by using their flagella and can be mag netically controlled because of the magnetic nano crystals in their membrane Martel et al 2009 Khalil et al 2012b Red blood cells with artificial tails are propelled and controlled by applying an oscillating magnetic field Dreyfus et al 2005 200 nm Flagella Magnetic crystals a Figure 1 2 Microrobots which are used in our setup a Paramagnetic particle b Self propelled microjet Solovev et al 2009 c Magnetotactic bacterium Keuning et al have developed a small sized magnetically actuated system which
92. pproximately uniform fields When high gradients of the magnetic field are required to control the microparticles the magnetic system is slightly modified The cores in the upper four coil are placed closer to the center of the workspace and a spacer of 5 mm is added to lift the upper four coils The magnetic field gradients in the workspace are shown in figure 3 6 figure 3 7 and figure 3 8 when a magnetic field gradient is created along the x y and z axis respectively The images show that the magnetic field gradients are not uniform when a magnetic field gradient is created in x and y direction In z direction the magnetic field gradient is close to uniform It can also be seen that because of the configurable design the system is capable of creating the required magnetic field gradient in z direction In the x and y direction the magnetic field gradients can be increased by increasing the current to meet the requirements In addition a better gradient current mapping can improve the magnitude and direction of the magnetic field gradients 3 3 Microscopic system The goal of the setup is to control microrobots microparticles microjets and MTB while they move in fluid Controlling the microrobots requires a position feedback An option is to use visual feedback The size of our robots especially the bacteria requires high magnification of the workspace This feedback can be achieved by the use of a microscopic system This system consis
93. provided by two microscopes with attached cameras which provide images at 50 frames per second The images provided by the cameras are used by feature tracking algorithms to determine the position of the microrobot Furthermore MARS is equipped with an autofocusing system The position and auto focusing controllers and feature tracking algorithms are implemented on a real time control platform It is experimentally demonstrated that MARS is capable of achieving point to point position control of paramagnetic microparticles in 3D space The position accuracy during the experiment is approximately 5 3 um the average velocity is 367 ums 1 and the maximum velocity is 2 mms 1 Also a self propelled microjet is position controlled in 3D space The position accuracy during this experiment is approximately 157 um and the average velocity is 121 ums To our knowledge we have for the first time demonstrated the closed loop control of microjets in 3D space ii CONTENTS CONTENTS Contents 1 Introduction 1 LX Contributions we is RR ex e EE ET 3 1 2 Thesisorganization 3 2 Modeling of microrobots 5 2 1 Modeling of force controlled microrobots 5 2 2 Modeling of torque controlled microrobots 6 3 Setup design 9 3 1 Requirements 32 03 saa eee 9 3 2 Magnetic SYST MES sre web e e eve Box EUR Bee Le BA 12 3 3 Microscopic
94. r the rolling process are unknown which makes the calculation of the integral 2 2 inaccurate Khalil et al have determined the value of the magnetic dipole moment of the microjets experimen tally Khalil et al 2013a They found a magnetic dipole moment of 1 4 x 10 P Am at 2 mT 100 ums and 25 The size and quantity of magnetic crystals in MTB see figure 2 1 b differs per bacterium which makes calculating the magnetic dipole moment via the volume integral inaccurate Khalil et al have characterized MTB us ing a flip time technique rotating field technique and u turn technique Khalil et al 2012b The average magnetic dipole moment was calculated to be 3 59 x 10 19 Am at 2 mT 2 2 MODELING OF TORQUE CONTROLLED MICROROBOTS CHAPTER 3 SETUP DESIGN Chapter 3 Setup design The setup consists of different components such as motion stages microscopes illumi nation systems cameras electromagnets and drivers The overall performance of the system relies on the quality and synchronization of these components Therefore all components have to be designed or selected properly to fulfill the requirements of the system and to work together in an efficient way This way it can be assured that we obtain maximum results of the system In this chapter the design of the different parts of the system will be discussed The chapter is concluded with the final design of the setup 3 1 Requirements The design of the
95. rd edu courses cs178 applets autofocusPD html S Martel M Mohammadi O Felfoul Z Lu and P Pouponneau Flagellated mag netotactic bacteria as controlled mri trackable propulsion and steering systems for medical nanorobots operating in the human microvasculature The International journal of robotics research 28 4 571 582 2009 URL http ijr sagepub com content 28 4 571 short R G McNeil R C Ritter B Wang M A Lawson G T Gillies K G Wika E G Quate M A Howard III and M S Grady Characteristics of an improved magnetic implant guidance system 42 8 802 808 1995 doi 10 1109 10 398641 URL http ieeexplore ieee org stamp stamp jsp arnumber 398641 L Morgenstern An unsung hero of the laparoscopic revolution Eddie joe reddick md Surgical innovation 2008 URL http sri sagepub com content early 2008 10 22 1553350608325119 full pdf B J Nelson LK Kaliakatsos and J J Abbott Microrobots for minimally invasive medicine Annual review of biomedical engineering 12 55 85 2010 URL http www annualreviews org doi abs 10 1146 annurev bioeng 010510 103409 92 BIBLIOGRAPHY BIBLIOGRAPHY C Pawashe S Floyd and M Sitti Modeling and experimental characterization of an untethered magnetic micro robot The International Journal of Robotics Research 28 8 1077 1094 2009 URL http ijr sagepub com content 28 8 1077 short R J Roesthuis Y R J van Veen A Jahya and S Misra Mechanics of need
96. replaced so to be future proof we decided to use the Elmo Solo Whistle 5 60 Elmo Motion Control Ltd Petach Tikva Isreal This drive can deliver up to 5 A at 60 V which is thought to be enough for present and future application in the setup The coils require 24 V when operated at 2 A While there are 8 coils in the system the total current the power supplies have too deliver is 16 A A single 24 V power supply that can deliver 16 A at a reasonable price is hard to find Therefore we decided to combine two Elektro Automatik EA PS 524 11 T power supplies Elektro Automatik GmbH amp Co KG Viersen Germany These power supplies are capable of delivering 10 5 A at 24 V The power supplies are connected in parallel to make one virtual power supply that can deliver 21 A at 24 V Each power supply is connected in series with a high current diode to prevent the power supplies from feeding each other Note that when the coils are used at higher currents than 2 A also higher voltages are required which can not be delivered by these power supplies Other components in the system like the illumination and the stages require lower voltages Therefore an Elektro Automatik EA PS 512 21 T power supply Elektro Automatik GmbH amp Co KG Viersen Germany is also used This power supply can deliver 21 A at 12 V which is plenty for our system The illumination system is designed to use power LED s These LED s are driven by a driver the XP SF 10008 48 XP Pow
97. rg stamp stamp jsp arnumber 6095011 5 Khalil R M P Metz L Abelmann and S Misra Interaction force estimation during manipulation of microparticles In Proc IEEE RSJ Int Intelligent Robots and Systems IROS Conf pages 950 956 2012a doi 10 1109 IROS 2012 6386184 5 Khalil M P Pichel Zondervan Abelmann and S Misra Characterization and control of biological microrobots 2012b 5 Khalil V Magdanzy S Sanchezy Schmidtyz and S Misra Magnetotactic bacteria and microjets A comparative study under review 2013a 5 Khalil R M P Metz Reefman and S Misra Optimal motion control of paramagnetic microparticles in three dimensional space under review 2013b A K hler Ein neues beleuchtungsverfahren f r mikrophotographische zwecke Z Wiss Mikr 10 433 440 1893 B E Kratochvil M Kummer S Erni Borer D R Frutiger S Schuerle and B J Nel son Minimag a hemispherical electromagnetic system for 5 dof wireless microma nipulation Proceedings of IEEE ISER New Delhi India 2010 M P Kummer J J Abbott B E Kratochvil Borer A Sengul and Nelson Oc tomag An electromagnetic system for 5 dof wireless micromanipulation Robotics IEEE Transactions on 26 6 1006 1017 2010 URL http ieeexplore ieee org xpls abs all jsp arnumber 5595508 Marc Levoy Nora Willett and Andrew Adams Autofocus phase detection 02 2012 URL http graphics stanfo
98. rmined and shown in table 3 1 Microscope with 2X objective Zoom value of the microscope 0 6 1 0 2 0 3 0 4 0 5 0 6 0 um pixel 2 33 1 40 0 69 0 47 0 35 0 28 0 23 Image height um 2385 3 1434 0 481 0 481 0 360 4 288 3 240 2 Microscope with 10X objective Zoom value of the microscope um pixel 0 97 0 58 0 29 0 20 025 012 0 10 Image height um 991 7 595 7 299 9 200 0 149 8 120 1 100 1 Table 3 1 Field of view of the microscope at different setting The resolution of the system is checked to ascertain that it is sufficient for our appli cation Equation 3 19 is used in combination with a Mitutoyo M Plan Apo 10X objective Mitutoyo Corporation Kawasaki Japan and a green LED light source to calculate the resolution of the system Due to the long working distance of the objective the numer ical aperture is relatively low NA 0 28 In section 3 3 3 the light source is discussed the power LED emits light in a range around 520 nm Using these values in 3 19 a maximum resolution of the system of d 0 9 um is found It is presumed that this is high enough to view 5 um bacteria and therefore no problems are to be expected with viewing the larger microrobots 3 3 2 Cameras In our setup cameras are required to provide the visual feedback from the microscopes to the control system The frame rate of the cameras determine the bandwidth of our controller Keuning et al 2011 have built a system that is capable of controlling micropa
99. ropar ticle about 75 um 46 CHAPTER 4 EXPERIMENTAL RESULTS 604 602 600 i V Wm ry x position um 598 982 980 978 y position um 976 1 260 1 258 1 256 z position um 1 254 0 2 4 6 8 10 12 14 16 18 20 Time s Figure 4 6 Position stability of a microparticle over time A microparticle is position controlled in three dimensional space at a fix point The used proportional integral controller scheme is presented in figure 3 18 with gains K 0 14 0 14 0 20 K 0 08 0 08 0 10 and saturation of 10 10 10 10 80 30 The blue line represents the position the red dashed line represents the setpoint deviation std max min deviation std max min pixels pixels pixels um um um x axis 0 05 0 45 2 29 0 11 1 05 5 33 y axis 0 04 0 39 1 96 0 09 0 92 4 58 Z axis 0 38 0 38 1 66 0 88 0 89 3 86 calculated by using pixel to um mapping obtained from calibration in air Table 4 2 Microparticle position stability errors 47 4 3 EXPERIMENTS MOTION CONTROL 4 3 2 Motion control of a microjet In this experiment a microjet is point to point position controlled in the workspace by manually applying setpoints The used position controller is a proportional controller which is described in section 3 6 5 controller scheme in figure 3 19 This controller uses the difference between the
100. rorobotic Vref C Normalize mapping system Z 2 ref Feature tracking Figure 3 19 Schematic representation of the magnetic field controller This propor tional P controller takes the position error as input The position error is calculated as the difference between the reference position and the measured position of the mi crorobot This position error is normalized to gain its direction This direction signal is multiplied by the proportional gain K The output of the controller is mapped to cur rents These currents are applied to the microrobotic system to generate the required magnetic fields to provide torque to the microrobots field The resulting magnetic field vector R3 1 is mapped to a current vector R8 1 by the actuation matrix These currents are applied to the coils of the setup Actuation matrix The eight coils of our setup contribute to the total magnetic field in the workspace In section 3 2 it is shown that the magnetic fields and gradients are almost uniform in the workspace and therefore we can assume that they are constant in the workspace The relation between the magnetic field and the electric current in the coils can then be written as B p B p L 3 27 where B p R is the magnetic field in the workspace B p R is a constant actuation matrix and I R is the vector of applied currents The controller of our system calculates a desired direction and magnitude of the magnetic
101. rticles in 2 dimensional space The performance of this system is limited by the bandwidth of the controller which is 10 Hz We want to avoid this limitation and therefore aim for a bandwidth of 100 Hz This means that the cameras should have a frame rate of at least 100 fps Furthermore the resolution of the used cameras partially determine the position accuracy of the controlled microrobots The system described by Keuning et al 2011 uses a camera with resolution of 768 x 1024 For our system we want at least to match this resolution and therefore aim for cameras with a resolu tion of round 1 MP While there is hardly any color in the scenery of the microrobots 21 3 3 MICROSCOPIC SYSTEM 0 6 05 c 04 03 2 o 02 5 v lt 0 1 0 0 350 400 450 500 550 600 650 700 750 800 850 900 950 1000 1050 1100 Wave Length nm Figure 3 9 Basler Aviator light sensitivity from user manual Basler AG 2010 The camera is most sensitive to light with a wave length around 500 nm which corresponds with green light By designing an illumination system with a green light source the camera will produce bright images there is no need for color cameras monochrome cameras are sufficient This also de creases the data throughput by a factor 3 which is beneficial for the control system Another requirement for the cameras is that they should have an interface which is compatible with the control system In sec
102. st of the image is enhanced This is done by rescaling all pixel values to the maximum range The result is shown in figure 3 17 b Next to reduce the background noise of the image the tracking algorithm applies background subtraction to the ROI In background subtraction the previous image is subtracted from the present image This makes all the changes in the succeeding image visible While bacteria can not hold still they are always moving and so they will become visible when applying background subtraction see figure 3 17 c Again the contrast is enhanced To reduce noise left in the image a median filter is applied see figure 3 17 e Now the image can be threshold to get a binary image see figure 3 17 f In this image the overlapping part of the bacteria in both images is visible as white binary 1 the background is black binary 0 From this binary image all kinds of properties can be calculated under which the position coordinates The calculated position is shown with a black marker in figure 3 17 g As can be seen in the last image the calculated position is not in the center of the bacteria This is due to the background subtraction only the overlapping part of the bacteria in both images is visible after subtraction and of this overlap the position is calculated While this error is always the same it will not influence any measurements Furthermore it has to be noted that background subtraction can induce problems when the robo
103. t bacteria moves directly towards or away from the camera In that case the shape and position does not change and consequently the robot is not visible in the image anymore after background subtraction 3 dimensional coordinates reconstruction The two sets of 2D coordinates have to be combined into a 3D set of coordinates in a legit way Therefore the system needs to be calibrated A way to do this is described in Hartley and Zisserman 2000 This method uses an object with known dimensions which can be seen by both cameras In these images corresponding points are selected The set of coordinates of the corresponding points can be used to calculate the funda mental matrix of the camera system At the moment of writing this thesis a bachelor 34 CHAPTER 3 SETUP DESIGN student is working on implementing a calibration method for the 3D setup While this is not yet finished a temporarily solution is used This method assumes a perfect system no lens deformation of the microscopes no refraction of light in the fluid mi croscopes perfectly aligned with the reservoir and perpendicular to each other In this case the two sets of coordinates can be combined using the redundant coordinate this coordinate can stitch the two sets together From the vertically mounted camera the robot position in the x y plane is obtained and from the horizontally mounted camera the position in the yz plane is obtained While the y coordinate is obtained with bot
104. than our french standard To better satisfy this need Paulstra has created a new range TRadiaflex Europe This range is available with the 4 usual welded fixings and with a new fixing the threaded hole stop CHARACTERISTICS The design of the RADIAFLEX mount gives the following basic characteristics Radial elasticity greater than axial elasticity The rubber works in compression axial shear radial compression shear according to the fixing method Advantages Simple to fix Simple and economical Extensive range 11 stud diameters Several heights for each diameter 5 methods of fixing Recommendations Operation in shear is very useful for vibration isolation provided that the radial forces are not too great 34 DIMENSIONS AND COMPRESSIVE LOADS SINGLE STUD FIXING DOUBLE STUDS FIXING OA See Vibrachoc elastomer range Threaded studs 5 511450 40 50 6 511401 20 8 511452 20 10 511454 20 1 511456 300 6 511525 50 250 9 511535 90 1 511545 350 3 513601 E 400 6 511625 511635 511645 511735 10 350 12 511750 300 14 511770 1100 6 513801 950 8 511830 80 600 10 511840 500 17 511870 450 19 511880 Compression
105. this parameter can be used as input in a controller The controller consists of truth table which has the c parameter and the present rotation direction of the stage as inputs The output is the new rotation direction to be send to the motor controller The autofocusing on biological tissue like bacteria can be improved by using other methods for focusing Firestone et al 2005 compare nine different methods currently used in automated microscopic systems Examples of these methods are spectral anal ysis variance entropy and histogram measures 3 6 5 Magnetic based control system The microrobots used in our setup have different magnetic properties The microparti cles are actuated by magnetic forces which require magnetic field gradients while the microjets and MTB are actuated by magnetic torque which are induced by magnetic fields These differences in magnetic properties make the use of different controllers necessary The controller used for the microparticles controls the magnetic force on a microparticle The controller used to control the microjets and MTB calculates the next magnetic torque output The calculated magnetic force and torque are mapped to electric currents by the use of a so called actuation matrix Kummer et al 2010 Magnetic force controller The microparticles are actuated by magnetic forces The magnitude and direction of these magnetic forces are controlled by the magnetic force controller A schematic re
106. thuis et al 2011 2012 These flexible needles have a beveled tip which makes the needle bend when it is inserted into soft tissue By controlling the orientation of the needle it can be steered inside the soft tissue The control of the trajectory of the needle enables us to steer the needle around obstacles or organs Figure 1 1 The da Vinci Surgical System Image courtesy of Intuitive Surgical Inc Intuitive Surgical Inc 2011 A disadvantage of medical robots and flexible needles is that certain hard to reach areas within the human body are not accessible This limitation is overcome by using microrobots as medical instrument The use of microrobots as surgical instrument is still in an early conceptual stage However medical microrobots have the potential to revolutionize medicine by drug delivery or diagnose hard to reach areas in the human body Nelson et al 2010 These used microrobots have magnetic properties which enables us to control them by applying magnetic fields and gradients Research has been done in modeling magnetic properties of these microrobots Abbott et al 2007 Pawashe et al 2009 Also different systems have been realized which are capable of controlling microrobots in three dimensional 3D space More systems are provided in Kratochvil et al 2010 Sakar et al 2012 The microrobots used in these system consist of para magnetic bodies which require magnetic field gradients to move in the 3D works
107. ticles which makes that the microscope is set to a higher magnification Consequently more debris in the fluid becomes visible in the background The tracking problems can be overcome by designing a more sophisticated tracker this tracker will provide the coordinates of the front part of the robot First a median filter is applied see figure 3 16 b to the grayscale image to reduce noise and small artifacts in the background Next thresholding is applied to obtain a binary image see figure 3 16 c From this binary image the orientation of the major axis of the blob is calculated This orientation is then used to rotate the blob so it will coincide with the horizontal axis see figure 3 16 d This makes it easier to perform template matching to find the tip of the robots body From the template matching algorithm the coordinates of the tip of the robot in the rotated image are know By applying a rotation on the coordinates they transform back to the original image see marker in figure 3 16 e This algorithm can be improved by applying background subtraction This way all the stationary objects in the image are removed When there is some kind of flow in the reservoir there will not be many stationary objects so background subtraction will have little effect Another problem that could arise with background subtraction is when a robot moves directly towards a camera In this scenario the robot will appear as a circle and will look stationary i
108. tion 3 6 1 the realtime control platform is chosen to xPC Target This system can only be used in combination with USB and CameraLink cameras The high resolution and frame rate requirement reject the use of USB cameras so a CameraLink interface is required A final requirement is the mechanical interface type In section 3 3 1 the microscopes are described which have a C mount interface for the camera the cameras should be compatible with this interface Requirements of our system are summarized as follows A minimal frame rate of 100 fps is required Acamera resolution of 1 MP is required The cameras should provide monochrome images The cameras should have CameraLink interface The cameras should be compatible with the C mount connection The xPC Target system drivers support only one framegrabber for CameraLink cameras which are from the brand Bitflow The supplier of these framegrabbers sug gested to use Basler cameras because these are very compatible with their framegrab bers The Basler camera which is closest to our needs is the Basler Aviator A1000 120km Basler AG Ahrensburg Germany This camera has a 1 MP monochrome sen sor and can deliver 120 fps The monochrome sensor makes the camera most sensitive to green light wave length around 500 nm as shown in figure 3 9 The sensitivity for a specific frequency range can be very beneficial when designing the illumination accordingly Finally the data int
109. ts of a lightweight microscope with an attached camera and an illumination system These components are discussed in this section 3 3 1 Microscopes An optical microscope is a device which uses a light source and a set of lenses to visualize a sample at a magnified level The magnification depends on the kind of lenses objectives used in the microscope An important property of an objective be sides the magnification is the Numerical Aperture NA The numerical aperture is a dimensionless number that describes the maximum acceptance cone of an objective and can be calculated as follows 3 18 where n is the index of refraction of the medium which the system operates The angle 0 is the half angle of the top of the maximum cone at which the objective can accept light it is the angle between the focal length and half the clear aperture Another important property is the resolving power of the microscope This is the ability of the microscope to distinguish between two adjacent structural details There fore the resolving power is a measure for the size of details that can be made visible with the microscope The resolving power is also called the resolution of the micro scope The resolution d depends on the wave length A of the used light and the NA of the objective and is given by A 2 commercial available miniature microscopic system is the Technisch Industrielles Miniatur Mikroskop with 4
110. um field of view h x w Maximum field of view h x w Illumination Wave length light green Power output Cameras Resolution Maximum framerate Magnetic system Amount of coils Maximum current per coil Maximum magnetic field x direction Maximum magnetic field y direction Maximum magnetic field z direction Maximum gradient magnetic field x direction Maximum gradient magnetic field y direction Maximum gradient magnetic field z direction Maximum gradient squared magnetic field x direction Maximum gradient squared magnetic field y direction Maximum gradient squared magnetic field z direction measured quantity calculated as the average in the workspace Value 1 5 50 30 0 1 x 0 1 2 4 x 2 4 520 3 1024 x 1024 120 8 2 39 4 38 2 64 5 490 370 1520 23 4 18 1 127 1 Table 3 2 Specification of our system 40 pixels CHAPTER 4 EXPERIMENTAL RESULTS Chapter 4 Experimental results The functionality of our setup is shown by performing experiments First the magnetic field are measured to investigate the linearity of the current field relation and to verify the validity of our FE model Second the capability of the autofocusing system is demonstrated Finally a microparticle and a microjet are controlled in 3D space and the results of the control experiment are discussed 4 1 Experiments on the magnetic system The magnetic system is tested on two major subjects the li
111. ure 3 13 This module consists of a threaded cylinder In this cylinder the power LED is mounted on a heat sink and the lenses are fixed at seats in the cylinder wall The threaded cylinder sits in a larger nut The nut is connected via ball bearing to the mounting ring The mounting ring will be fixed to the magnetic system and also constrains the rotating motion of the cylinder by a plug By rotating the nut by hand the threaded cylinder will make a linear motion This makes it possible to adjust the focus point of the illumination The computer aided design CAD drawings can be found in appendix B b Figure 3 13 Adjustable illumination module a Photograph of the module b Cross section view of computer aided design model 3 4 Mechanical system The setup consists of lots of mechanical components These components have to be bought machined or 3D printed Most of these parts don t have special design issues and therefore will not be discussed in this section The technical drawings can be found in appendix B However two parts need special attention These parts are the autofocus mechanism and the table damping and will be discussed hereafter 3 4 1 Autofocus Commercially available autofocusing systems for microscope often use advanced mi croscopes with internal focus correction or use external devices which track video signals or use reflected laser beams Also these systems are designed for normal mi croscope use and
112. ynolds number yields a laminar flow type In laminar flow conditions Stokes law can be used to calculate the drag force According to Stokes law the drag force is a drag coefficient a times the velocity of the microparticle The Reynolds number of our microparticle allows us to neglect inertial terms We can now rewrite the equations of motion as F p ax 0 3 5 ay 0 3 6 F p F a 0 3 7 where is given The buoyancy force is given by V p p g where is the volume of the displaced water volume of the microparticle and are the densities of the microparticle and water respectively g is the gravitational constant Assuming low magnetic fields we can substitute 2 9 and the expressions for F and in 3 5 3 6 and 3 7 CENE P 5 p B p 0 3 8 M 6 0 3 9 41 4 TT V po Pw g 671 0 3 10 Oz Rewriting for magnetic field yields 5x E PBP 3 nud Xm 3 11 41 a Pw g gt E PBP i 41 3 3u Am gt 3 12 a 3 13 Designing a magnetic system for a velocity of 1 mms B p B p 2 B p B p and 2 B p B p of 15 6 mT2mm 15 6 mT mm and 49 7 mT mm respe

attached - Robotics and Mechatronics

Contents

Download Pdf Manuals

Related Search

Related Contents