WECAM USER MANUAL 1
Contents
1. 1 eene 16 ATI Overview othe GCU sese mtem Urn e 16 Aa TTE mair mEn D EE 17 4 2 1 mi mm 17 4 2 2 EE ER T vr 19 4 2 3 PCT ONG core EN A A E NT IA E T 20 4 2 4 IDE GE MEN m 22 4 2 5 HE E EEO 23 4 2 6 A E EE EEEE EAEE E E EAEE E A A E O EE 23 4 2 7 Ae E RR 24 4 2 8 hic puoi T 24 4 2 9 aE 28 2 3 TOOlIDaElSiisseiodenttetine e aa idee teile ieu deti teer pME 28 4 3 1 IVS AV toolbar sts e o NM 28 4 3 2 The SD VIEW TOOID AI sei ices A 29 4 3 3 TNE playback tool bI sse tns R rete EE REO bo e E PUER QU NH IU SEQ RIBUS 29 vale WN lg 2181 9 e Bae eee Men ore meet mer vom Rare ree Na ete eer Don eevee eer ran ee pre 30 45 ThE mall WINGOW sacs oe ers ma ee eee 31 46 ThedrPoperty WIHdOWausu deu eh Mp tt petia R 32 4 6 1 The project property Dagger iie ote qiu out qutd cette aoa dieto do tue eo d beue deus 32 4 6 2 Thesequence DrODertV DdBO csicscieo M epdaciivexide PieneenNoeiee Musee conniexdruiio netus 33 4 6 3 The track property paf Gon edu tie ee uen Oe Cu ied Ruta el ecoute Ca et Qu Gb Sa auos 33 4 6 4 The camera property pag Geirio a a nennen enne nennen 34 4 6 5 Ihe 3D object property PAGS aces seit Matar trs i bebat eut aetate Piet i 36 4 7 THectlYielihieiscceitinesiedu uiris e a ae buie ei bte acu ned 36 4 8 The parameters graplissssoode seen eee e ect iet teo be bas 37 49 Jst oft TOU Su cete ile sss audet ek chats ame i2. Transactions on Pattern Analysis and Machine Intelligence TPAMYIJ 31 6 974 988 2009 2 Guofeng Zhang Zilong Dong Jiaya Jia Liang Wan Tien Tsin Wong and Hujun Bao Refilming with Depth Inferred Videos Transactions on Visualization and Computer Graphics TVCGJ 15 5 828 840 2009 There are three steps contained in VDR listed as follows gt Depth Initialization initialize the depth map for each frame independently Bundle Optimization iteratively refine the depth maps associating multiple frames gt Depth Level Expansion further improve the depth precision by increasing the number of depth levels B Click the above items or Run All to open the setting dialog for depth recovery There are various parameters in the following dialogs Depth Recovery E ratio General Start o End 140 Bo Pipeline lw Initialization W Bundle Optimization w Depth Level Expansion Advanced Parameters v Output Tmp Data W Estimate Dsp Range Automatically Start End specifying the range of the frames that are needed to compute the depth maps Passes the pass number of Bundle Optimization and Depth level Expansion Output Tmp Data output the intermediate data to TMP directory for debugging or visualization Estimate Dsp Range Automatically Automatically Estimate the disparity range according to the recovered sparse 3D points Depth Recovery Advanced Parameters Block Resample x
3. 2 Y Overlap 8 Resample t Neighbor Frames Selection Min Dist B Init 5tep 2 Init Count 4 Normal Step 2 Max Count E Mean Shift Segmentation Spatial 5 Color 5 Minsize 50 Plane Fitting Size gt 300 Energy Rate Threshold 2 5 Min Dsp 1e 007 Max Dsp 0 01 Color Sigma 5 Dsp Sigma 0 03 Min Disparity Sigma 0 001 Projection Sigma 2 Min Projection Sigma 9 5 Depth Expansion Geometric Coherence Measure Sub Levels 10 Disparity Space Sub Iter 2 f Image Space Weight System DataCost Weight 20 Use GPU Smoothness Truncated 10 CPU Threads Num 4 M Generally most parameters can just use the default values The parameters highlighted with red rectangles may need to adjust for different sequences Block Resample This group of parameters indicates the strategy for large resolution image The first two parameters X and Y represent the partition number of the image width and height respectively Overlap means that two neighboring blocks should have Overlap overlapping If the image resolution is large you should set X a b a b and Y larger or set Resample small e g 0 5 especially for GPU acceleration If Resample 1 the program will downsample the sequence for depth recovery and then upsample the estimated depth maps with further depth refinement Neighbor Frames Selection For each frame how to select its neighboring frames for depth recovery is important
4. I c3 tow T rix D test act ERES T ACT Project act m Figure 3 12 save project 3 5 Export result Click File gt Export or press key ctrll le choose the exporting path file name and click save Figure 3 13 Export Data FH T Camera Track Filet txt T Hv iB Camera Track File Ok txt 3l Studio Max kt ms Export W Camera Paramters I 3D Points Option Scale the coordinates p Figure 3 13 export 3D points and camera parameters The format of Simple Camera Track File is defined as follows The number of frames Intrinsic Matrix 3 3 Rotational Matrix 3 3 Translational Vector The number of 3D Points XYZ 4 The User Interface 4 1 Overview of the GUI This is the overall GUI of ACTS Main menu bi roject window Main toolbar Main window 3D view toolba Property window EEx gi N toy test wap File Edit Actions 3DScefe Play View Window Help 9 E B x 6 p t Project Property sutol4 a Rupee Ww Autol3 Y rac La Lamera ai Anto General EEZ TT Q Autoll Q AutolO Label Camera j Autos Film Width 360 000000 Q Aut of DUE Q AutoT ilm Height Q uto Unit Q nino Resolution 960 540 Q utog CAN Auto3 Ratio 1 TTTTTS Auto2 Pixel Aspect 1 000000 Q9 Autol Q utol SZ 8 Q 10000 19999 dud dpud Focal Length 1378 98877 Q 30000 33561 c Manual Tracks R
5. Meta oe AR E er Unit Resolution 950 540 ryryryryryo c Ratio 1 TTT TTG Pixel Aspect Rotation kA radian x 0 000000 Y 0 000000 z p o00000 Translation TimeLine 207 8 Untitled YIP Image Sequences T SequenceD e Auto Tracks e Manual Tracks 2 EM ere TimeLine Parameters Graph Output racio MD i Ready NUM Figure 3 4 image sequence gt Configure the camera type known constant variable focal length as shown in Figure 3 5 Property ener al Advanced Focal Length Constraint Type Constant T P Constant Principal pYariable A 4Ta3 500000 I 263 500000 Radial distortion Doo E Figure 3 5 camera configuration 3 3 Track camera 1 Click actions quick track or press Ctrl to open the tracking dialog Figure 3 6 2 Click to open the feature tracking dialog Figure 3 6 i Track Dialo C f he 1 Track Feature he 2 Select Superior Tracks and Key Frames h 3 Solve Camera 4 Adjust Camera 5 Depth Recovery Current Process Frocess Figure 3 6 tracking dialog 3 Set the minimum track length In this example set it to 15 Click oK as shown in Figure 3 7 Feature Tracking Properties Feature Tracking Froperties Advanced Properties Option Algorithm Preview m Al Frames Zelected Frames COSIFT CONLT 9 Hybrid 1592 Features EE Iracking Range Image Presmooth Sigma stort Zo
6. including the frame interval and the maximum number Generally the default setting is ok However for a low frame rate sequence or sparse wide baseline images the frame step and the maximum frame selection count should be set smaller The number of selected frames will directly affect the running time Meanshift Segmentation the parameters of mean shift Generally the default parameters are ok Disparity Configuration Level Num the number of disparity levels used for first pass Belief Propagation optimization It directly affects the running time and memory requirement Min Dsp amp Max Dsp the disparity range of the scene If the disparity range is 0 0 01 it means that the nearest depth is 100 and the farthest depth is infinite VDR can automatically estimate the disparity range with the estimated sparse 3D points For providing a more accurate bounding box the user is allowed to manually specify the disparity range Depth Expansion Sub Levels the number of expanded disparity levels for each iteration during hierarchical BP optimization If the value is 10 each disparity is further quantized into 10 levels i d dy 2n di di 4 t 0 20 Sub Iter the hierarchical iteration number of BP optimization If the initial disparity level number is 101 with two iterations the disparity level number should be 10001 Geometric Coherence Measure measure the geometric coherence in image space or disparity
7. indicates whether camera solving has been done Note The camera solving step can only be performed when feature tracking is completed which is in turn preceded by importing the image sequence 4 6 2 The sequence property page Property Property Track bel Camera 4b E Track je Camera 4 a mane General BEY General Advanced Initial Frame 21 Label SequenceD File N wecam toy test0000 jpg Key Frames thers r 0 Moti on type Free Move 1 Camera Camer all Interlace None a T Frame rate 25 8 D 3 ames 10 i z 12 r Start Frame O 13 14 Step M 15 16 End Frame 140 17 18 19 20 21 T 26 2T 30 31 34 35 38 3T 39 4n M General All the information here is actually specified when the image sequence is imported Please see section 4 2 1 Advanced 1 Initial frame Solving the sequence from the beginning is usually not a good solution Our system can automatically select the optimal initial frame for structure and motion initialization In general user does not need to manually set it 2 Key frames Since solving all frames simultaneously is not efficient our system first estimates the structure and motion in key frames and then solve other frames The initial frame is selected from the key frames 3 All All frames of the sequence 4
8. ma 1 3 1 Automatic Tracking Our system supports to track two camera motion types i e pure rotation and free moving The whole process is very simple only requiring several clicks on the buttons Some interactive and visualization tools are also provided allowing the user to conveniently examine the tracking quality 1 3 2 Support for Camera Tracking Under Varying Focal Length ACTS can efficiently and robustly handle the long sequences with varying focal length without requiring any prior knowledge 1 3 3 User friendly User Interface The graphical user interface is provided for easy use The tracking results can be visualized in both 2D and 3D ways In addition 3D objects can be imported and manipulated to examine the tracking quality 1 3 4 About Help To make this users guide more readable the table below lists several notations that will occur in later parts of this document 1 3 5 Minimum System Requirements Microsoft Windows 2000 XP or Server 2003 Intel Pentium III 600 MHz Display resolution of 1024 768 pixels 24 bit color OpenGL compatibility 128Mb of memory 512Mb or higher recommended 100Mb of free disk space 2 Overview 2 1 Image Sequence The input should be an image or video sequence The system does not directly support a video format file The user should decompress the video clips into image sequences beforehand Most image formats are supported such as BMP JPEG PNG etc Notice
9. 6 3 The track property page Frame Range e Coordinates x 581 821106 Y 457 341217 3ll Coordinates i ausu Y 48 305258 z Error Level Resi dual Average Res General 1 Label the name of the feature point 2 Frame the frame index of current frame displayed in the main window 3 Range specifies the frames on which the trajectory of the selected feature point exists Computation result 1 2D coordinates 2D coordinates of the feature point in the image 2 3D coordinates 3D coordinates of the reconstructed feature point in the 3D space 3 Error level for valid feature tracks this is set to Bundle Adjust or Outlier Otherwise the error level is uninitialized 4 Residual the reprojection error of the selected feature point in current frame 5 Average res the average reprojection error of the selected feature track 4 6 4 The camera property page General Advanced Fila Width Film Height 480 000000 Resolution Ratio Pixel Aspect 1 000000 Focal Length 536 366394 Rotation hf radian 0 436979 I 1 141T24 Tranzlation Ak 196 931564 Y 44 880985 i 111 sm General 1 Label name of the camera 2 Film width width of the film 3 Film height height of the film Property 1 s Track BENNETT TS ener al Focal Length Constraint Type Constant AF Initial 1473 8T58TT Principal Point 479 500000 Y 26
10. E 313 UT3132 Scale sedi General 1 Label default object name with index Transform 1 Radian show the rotation parameters in degree or radian 2 Rotation the rotation of the selected virtual objects in degree of radian 3 Translation the translation of the selected virtual object 4 Scale the scale of the selected virtual object 4 7 The timeline Project window Current frame Timeline ticks NO FN O Autozz iutoz3 amp utoz4 D autozs Qm QJ AutozT D Autoz8 T iutos Qmo s lii Parameters Graph Output ee NN ae kN X mi S TimeLine Range of trajectory Selected trajectory The timeline visualizes the temporal variation of the feature points Range of trajectory of feature points this is perhaps the most important property related to feature points From the figure it is clear that some feature points can last across the whole timeline while some others may last only for several frames Currently selected trajectory this will be highlighted 4 8 The parameters graph Selection window Timeline scales Current frame Parameters Graph Eg Camera Farameters 1 20 120 180 Translation 20228371 E LELIT 118 35481 T bese E Z Axis 28 545926 9 icon i cen X X Axis W Y axis 152 2918 Z Axis N A42 07107 ia Focal TT TimeLine Parameters Graph Output Numerical scales Parameter curve Th
11. FA D Import Image Sequence lh tes t0013 j lh tes tD014 j li test0015 j lh test0016 j lk test001T j lh test0018 j ih tes t0019 j lh tes 002 j lh test nz1 j lh testO0Z2 j lh tes t0n23 j lh tes t nz4 j lh tes tDU25 j ih tes tOD6 j AH tes t D27T j ih tes tODz8 j l tes tOnz9 j leh testO030 j la tes tOD31 j h tes t0032 j h tes 0033 j l tes tO034 j Gh testO035 j Ak test0036 j h tes tOD3T j Gh test0038 j l amp k tes 00 39 j AF test0040 j GF tes tDDA1 j l tes tD042 j leh tes tD043 j leh tes OO44 j ah test0045 j ld test0046 j lh testOO47 j l tes tD048 j l amp k tes 004 j ld tes 000 j lh test0051 j A11 Image Files x PNG TIF TIFF v ELA BE ASUTTFT E TPE T Figure 3 2 choose the first image of the sequence gt click OK import 140 frames of image sequence as in Figure 3 3 Import Image Sequence Caner Frame rate amp 5 Motion type FreeMove Interlace Hans Start Frame Figure 3 3 import image sequence Figure 3 4 shows the imported sequence Untitled File Edit Actions 3D Scene Play View Window Help 22h S2RRRRBOPOGEL M Project 2 48 Untitled YIP Image Sequences Ti Sequencel Auto Tracks i Manual Tracks Wan oF Eu gir inia T x0 ACA S A Film Width tea Canera E i 3 Eel ETE sy 7 OFA 3D Primitives A E x ai MSs NE xad Film Height
12. This step can be used if the solve camera is not very good Estimate 3D Points The user can use this function to estimate the 3D positions of more feature tracks not only superior tracks Estimate 3D Points Estimate the 3D positions of the tracks Track Length gt E Always Re Estimate Eeprojection Error lt 2 Notice the operations above actually don t differ very much from each other T p sd Dialog 1 Track Feature BRasdome m 2 Select Superior Tracks and Key Frames Keyfr ame he 3 Solve Camera Lamera 4 Adjust Camera 5 Depth Recovery Current Process Click on the right to open the setting dialog for feature tracking There are various parameters in the dialog gt gt gt VV ON V Y Y Option choose to track all frames or only selected frames Tracking Range specifies the start and end frames for feature tracking Minimum Track Length minimum trajectory length of a feature track It is a very important parameter which directly affects the computation time and reconstruction quality Because structure and motion estimation with longer trajectories is more reliable and robust than with shorter trajectories our system only selects those feature trajectories longer than the specified minimum threshold Algorithm ACTS supports three feature tracking methods SIFT and KLT method is just as the standard ones The Hybrid method is a combination of SIFT and KLT m
13. displayed in the main window Frame Index whether the frame index is shown Scene Coordinate whether the coordinate axis of the 3D scene is shown 4 2 7 Window lie Standard Toolbar 5 Alt 4 Status Bar 3D View Toolbar Alt 5 v Project Alt 6 Property Alt 7 TimeLine Alt 8 v Track Graph Alt 9 v Output Alt 0 Customize Toolbars contains a sub menu that controls the visibility of each window in ACTS Status Bar determines whether the status bar is visible 4 2 8 Advanced Tools Video Stabilization e Video Stabilization the goal of video stabilization is to remove annoying shaky motion from a video sequence Video Stabilization tool is implemented based on the following paper Guofeng Zhang Wei Hua Xueying Qin Yuanlong Shao and Hujun Bao Video Stabilization Based on a 3D Perspective Camera Model The Visual Computer 25 11 997 1008 2009 B Click Video Stabilization to open the setting dialog for video stabilization There are various parameters in the dialog Wideo Stabilization Motion IvIodel Selection DIotion Filtermg fe 3D Camera IvIodel raussian Filtering Smoothness R 10 Rotation 100 Linear Optimization Zoom 100 Translation 2 View Warping Constant depth t Homozraphy IvIodel C Affine IvIodel Planar Impostor Output File Steps step 1 Motion Filtenng atep 2 Notion Compenstation Stepi View Warping amp Output Pun Steps 1 3 gt Mo
14. space System set if use GPU and the number of working CPU threads They could be set simultaneously The estimated depth maps are stored in DATA directory You can refer to the following site for the introduction of depth data format http www cad zju edu cn home gfzhang projects videodepth data By click Save Models you also can output the obj 3D models for each estimated depth map which are also saved in DATA directory and can be viewed in 3D software such as Meshlab 3D MAX etc Min Depth o Max Depth 1000 Dsp Discontinuity Threshold i3 2 d max d min OX cm 4 2 9 Help About ACTS About ACTS copyrights information about ACTS 4 3 Toolbars There are three toolbars in ACTS the main toolbar the 3D View toolbar and the playback toolbar 4 3 1 The main toolbar Standard gt S E The main toolbar contains buttons that correspond to the basic project management and playback mode Associated commands for each button are illustrated as follows 5 import image sequence d a play once a E load project auto reverse nt E save project loop 4 3 2 The 3D view toolbar Sit R k Be EM The 3D toolbar contains buttons that correspond to commands involving the 3D scene such as importing 3D models translation rotation etc pe import vof file P translate on the Z axis ri save virtual objects into vof file e translate on the viewing plane M import Wavefront
15. 1 ee T Y e oe 0 000412 2209980 G A a Pe 0 014664 auto9979 Aut o9978 Translation Z auto9992 J 20 on joo Auto9991 auto9990 autogga9 auto9988 gt TimeLine Parameters Graph Output rac ow oe i l Ready NUM I Figure 3 10 tracking result 7 Figure 3 11 shows the recovered 3D structure of the scene ee N XtoyXtest act File Edit Actions 3D Scene Play View Window Help S SNR 1 s iE EgSXxNk R 99trU QJ 0 9999 auto9999 auto9998 42102297 auto9996 auto9995 auto9994 I Qo Auto9993 A 4 e N Film Height 540 000000 uto9992 Lv Se oe osx Uni t 4uto2291 1h ad 42102220 pir ie Teme Te e a 4uto9989 A CES 1 hn Ratio WwW Auto9988 1 4 Pixel Aspect auto9987 Lx auto9985 uto9384 e 7 S Focal Length 1379 988 4uto9283 SK Resolution auto9982 42102981 auto9980 auto9979 auto9978 Rotation fe Translation kA radian D 000412 0 014864 0 013724 4ut09292 J 20 op 00 auto9991 auto9990 EET 42109289 ds auto9988 fd lt gt TimeLine Parameters Graph Output rac ow MD i 5l Ready NUM Figure 3 11 3D structure of the scene 3 4 Save project Click File save project or press ctrl choose file path input file name and click save Figure 3 12 Save Project BEATAE
16. 9 500000 Radial distortion ue ETXSES 1 Focal Length Rotation hf radian Translation amp 0 126764 4 Unit the measurement unit for the film s width and height 5 Resolution resolution of the images in the sequence 6 Ratio ratio of width to height 7 Pixel aspect defined in section 2 3 1 Advanced 1 Focal length constraint type if this is set to be user fixed then the next parameter Initial would be used throughout the solving process as the camera s focal length The other two options both consider Initial as the initial value for the camera solving step and constant means that the camera s focal length in the whole sequence remains constant while variable allows fer changing focal length 2 Initial initial value of the camera s focal length 3 Principal point the camera s optical center described in section 2 3 1 4 Radial distortion also described in section 2 3 1 Parameters 1 Focal length focal length of the camera of current image frame 2 Rotation rotation parameters of current frame s camera 3 Translation translation parameters of current frame s camera 4 Radian indicates whether radian or degree is used as the unit for rotation Note providing extra information about the camera would help the camera solving step for better tracking results 4 6 5 The 3D object property page Property Label Rotation he radian 1 023044 0 995133
17. ACTS Automatic Camera Tracking System UsER MANUAL 2 0 STATE KEY LAB or CAD amp CG ZHEJIANG UNIVERSITY TABLE OF CONTENTS 1 Introduction eene ener 4 Ld POUE o Pec CaO ostenta ubt ate sut epu ICM ct St aede 4 1 2 Featured Functiortalit8S sese raris tastes pe oap t meo eue haec me Sx nsi votre nes ptas See 4 1 3 Technical Specifications eeeeeneenenre nennen 4 1 3 1 Automatic AC da RE OD TIS 4 1 3 2 Support for Camera Tracking Under Varying Focal Length 5 1 3 3 User friendly User INterface cccccecccsscccsscceeeceeeceeeeceeeceaeceeeceeeeceeecesseceaeees 5 1 3 4 ADOUAT T 5 1 3 5 Minimum System REQuIrEMe Nts cceeceeecceecceeecceceeeeeceecceeceeeeenseceeeeeeeeeees 5 EMO Ta i REM 6 Doe Iae o GUE NO E e A ETE ceptum bea petet uu Uh Uus S 6 PP Foure W g s o ET 6 2 2 1 FI SCS arate ss OO cme E ENTE 6 2 2 2 AG MINE OE 6 2 3 Camera tracking esseistinen enhi nene 6 2 3 1 EE A E AEE AAEE A AEII ANEA A 6 2 3 2 C TAC OS a OUT 7 2 3 3 B EEEE dg E E A E E E A E A 7 2 3 4 The DEWI ONEN eera ane terete oat een ears gee aeetee utes EE MEUM 7 3 A simple example eese 8 Cs gi ei E 8 3 2 Import image SeQtellleO eee eee eee ee eo MA UEM en eee eee en ee eee 8 33 CI CIME eer EEEE ERE OEA E AEA E EE AE 11 3 4 Save 0 6 S Re ne m mm 14 2 9 EPON T E ene ee eee ere A eee ee a ee ee 14 4 The User Interface
18. For not missing your results please save the project whenever necessary 2 2 Feature tracks 2 2 1 Feature points Feature points refer to the interesting points in the image The system can automatically detect the feature points and match them among consecutive frames 2 2 2 Track lifetime The matched feature points constitute the feature tracks It corresponds to a 3D point in the scene A key characteristic of feature tracks is the track lifetime the number of frames over which a point is visible Long tracks are more useful than short tracks ACTS allows the user to specify the minimum track length so that the feature tracks short than the specified threshold will not be used for camera estimation 2 3 Camera tracking 2 3 1 The camera Each image frame has intrinsic and external camera parameters The external camera parameters contain camera rotation and translation The intrinsic parameters include Principal point the center of the lens Its default value is the center of the image Focal length the distance between the optical center and the focus plane Pixel aspect the ratio of pixel height pixel width The formula can be further expanded into width of image resolution width of film height of image resolution height of film For example if the film is 32mm 24mm and the image taken has resolution 640 480 then the ration is 640 32 480 24 1 0 For most cameras the ratio is close to 1 0 e Radia
19. Project Ctrl 5 Save Project As Export Ctrl E Import Feature Tracks Export Feature Tracks Exit Ctri X Import Sequence imports an image sequence The filenames of the images should be in a numbered format for example imgO000 bmp img001 bmp img002 bmp or imgO jpg img1 jpg img2 jpg Import Image Sequence File W hxtrackitoyitestO000 ipz m Start Frame Step End Frame In the Import Sequence dialog the user can set several parameters gt Labef default name of the imported image sequence gt File the filename of the first image frame in the sequence Click to select the image gt Motion type the user can decide the motion type of the camera It could be either Rotation Only or Free Move Camera the corresponding default camera name of the imported image sequence Interlace specifies whether the image frames are interlaced Default value is None Frame rate the frame rate of the sequence Start Frame the offset of the start frame from the specified first image frame VV VV WV Step specifies the step in which frames are imported If this is set to N then ACTS imports 1 frame from every N frames gt End Frame the offset of the last frame from the specified first image frame Load Project loads a previously saved project in the format of act ACTS Project Save Project saves the current project If it is a new project ACTS would let the user select a l
20. RR Start Octave Sigma tons Mask Enable Filter NULL Insid C Qutsid AT eR Maximum Feature Count 1000 NE Minimum Feature Distance Drm Minimum Track Length Figure 3 7 feature tracking options 4 Click process and wait Figure 3 8 If you wanna stop the solving Click stop Figure 3 8 start tracking 5 After process completes click lexit Figure 3 9 he 1 Track Feature eat aie e 2 Select Superior Tracks and Key Frames Keyframe he 3 Solve Camera Lamera 4 Adjust Camera 5 Depth Recovery Processing Track Feature Figure 3 9 tracking completes 6 Figure 3 10 shows the camera tracking result ee N XtoyXtest act File Edit Actions 3D Scene Play View Window Help 2E Sea KRES OL E gt 6 Leal W N 0 9999 auto9999 auto9998 42102297 auto9996 E X m wn n Auto9995 3 4 eit OU a f Film Width 4uto2994 Lh ER Ix Ri e INS auto9993 ERI GF 4 ET DN uto2922 Bh d af Lou ra TOR c qyex505 Unit svto9991 udi p r ias en autos990 E he E we e sutosga9 3 y er oe DIE A NP Eatio EM 4uto2288 P AER f 7 E I i e is m uet i Pixel Aspect 4uto9987 6 Pent Se auto9986 ee amp uto2985 4 A AN Parameters Film Height Resolution 5 2 Auto9984 J alias lt I Focal Length Auto9983 oe NE Seer i d ECCE NELLE ALL ee E e S 41023282 Rotation hA radian uto298
21. du etum tese hese 38 1 Introduction 1 1 Product Specification ACTS Automatic Camera Tracking system is an automatic camera tracking with dense depth recovery software which is able to effectively recover the camera parameters and dense depth maps from an input 2D image video sequence The recovered 3D data are useful for many applications such as video composition 3D modeling and animation For examining the reconstruction quality the user can insert a virtual 3D object into the scene and playback the compositing sequence to inspect whether there is a drift 3D interactive tools are also provided for conveniently manipulating the inserted 3D objects Once tracking is complete the results can be exported for use 1 2 Featured Functionalities There are three main modules contained in the system listed as follows The feature tracking module automatically extracts the feature points and track them The camera estimation module solves for the external and intrinsic camera parameters based on the tracked feature tracks The3D object testing module inserts 3D virtual objects and composite them with the video The compositing video can be used to examine the tracking quality 1 3 Technical Specifications Input a video sequence Output the camera parameters sparse 3D feature points and dense depth maps Supported export formats Y Simple tracking format txt v 3D Studio Max ms Y Maya
22. e nodes the corresponding property page in the property window will be activated and show the related information The nodes in the project tree are explained as follows E project file the working project file name Tw image sequence the imported image sequence Currently ACTS only supports one image sequence in one project 5 auto tracks sparse feature points recovered by ACTS U feature point groups the feature points are grouped into groups of 10000 w invalid feature tracks tracks with trajectory length less than the minimum track length see section 4 2 23 They are ignored in camera solving and don t have valid 3D structure 0 valid feature tracks tracks with trajectory length larger than the minimum track length They are included in camera solving and their 3D structures are reconstructed after tracking W camera the camera of the image sequence Currently ACTS only supports one camera in one project Fr 3D objects here shows the names of all inserted 3D objects in the scene Note the selected item will be displayed in bold letters right click on the items for menus Valid feature 4 5 The main window In the main window valid feature points are displayed in green and their corresponding 3D points projection in yellow Invalid feature points are displayed in red Image Current frame index 3D object 3D object Camera of current frame 3D Feature points Came
23. e parameters graph visualizes the temporal variation of the camera s parameters To visualize parameters use the mouse to selected parameters that you want to review Hold Ctrl for selecting multiple items The curve corresponding to X axis will be drawn in red Y axis in green and Z axis in yellow Parameter curves Users can have an intuitive look at how the camera s parameters change over time Note by selecting multiple parameters users can directly observe the relationships between their variations 4 9 List of hotkeys Import image sequence Ctrl I 2012 12 3
24. ethods The slider bar gives a quick tuning on the number of extracted features Image Presmooth Sigma smooth the image with Gaussian filter before feature tracking This will reduce the noise in the image Start Octave Sigma the start sigma of SIFT feature detection Enable Filter control the density and number of SIFT feature Maximum Feature Count the extracted maximum number of SIFT feature Minimum Feature Distance the minimum distance among extracted SIFT features Preview a preview of extracted SIFT features Feature Tracking Properties Feature Tracking Properties Advanced Properties Option Al gorithm Preview m All Frames _ Selected Frames C SIFT ELT Hybrid Use GPU 1592 Features Iracking Range SEA Image Fresmooth Sigma Start ieee start lletave Sigma Mask hf Enable Filter fe HILL f Inzid C Qutsid i EE Maximum Feature Count 1000 Minimum Feature Distance i Dr Minimum Track Length 20 Click camera to open the setting dialog for camera tracking Parameters include Track Camera hf Optimize Initial Frames Selection E Smoothness Constraint 0 1 C Manual Track j Full Adjustment j Cancel gt Optimize Initial Frames Selection use the initial frame selected by our system or by default i e the first frame gt Smoothness Constraint impose smoothness constraint for camera translation and focal length It is especially useful for tracking the sequences wi
25. l distortion current version assumes there is no radial distortion It should be noted that the current version of ACTS assumes the principal point is at the image center the pixel aspect is 1 0 and there is no radial distortion The focal length can be unknown and varied 2 3 2 Camera constraints Besides internal parameters users are allowed to select some prior constraints on camera motion type i e pure rotation or free moving See 4 6 2 2 3 3 Camera tracking The camera tracking step solves for the camera motion as well as the 3D positions of the sparse feature tracks 2 3 4 The 3D environment After camera estimation the user can immediately review the tracking results The provided 3D view mode allows the user to inspect the recovered 3D trajectory and 3D positions of the tracked feature points The user can insert a 3D object into the 3D scene The interactive 3D tool allows the user to freely change the position orientation and scale of the inserted object 3 A simple example 3 1 Run ACTS 3 2 Import image sequence gt Click x or press ctrl to open the dialog to import image sequence as in Figure 3 1 Import Image Sequence Label Sequen File Browse Notion type Camera Panera Interlace Hone Frame rate 5 oes yS Start Frame Step End Frame 0 Figure 3 1 dialog for importing image sequence gt click browse and select the first image of the image sequence as in Figure 3 2
26. le Adjustment adjust the scale of the scene Type the number and click Adjust Preference 3D Scale Adjustment 3D Scale Adjustment Seale Coordinate 3D Vi ew Preference SD Seale Adjustment 3D View View Frustum Hear Plans Point Size Far Plane 100000 Line Width lt Background Color Red Green E Blue Estimate Track Color estimate the color of each 3D point The 3D view will render the 3D points with estimated color Clear Track Color clear the color of each 3D point The 3D view render the 3D points with default color 4 2 3 Actions Quick Track Ctrl c Track Feature Kev Frames Solve Camera Adjust Camera Estimate 3D Points Quick Track opens the tracking dialog and checks all steps Track Feature opens the tracking dialog Only the Track Feature will be checked This action will detect and link all the feature points in the image sequence Key Frames opens the tracking dialog Only the Select Superior Tracks and Key Frames will be checked This step will link the feature points and select the superior tacks whose trajectory length is larger than the preset minimum length Solve Camera opens the tracking dialog Only the Solve Camera step will be checked This step can recover the camera parameters and sparse 3D points which must be performed after feature tracking Adjust Camera opens the tracking dialog Only the Adjustment step will be checked
27. obj model o rotate remove virtual object E scale 2 select virtual object e 9 free operations Note after activating the free operations tool holding specific hotkey and dragging the left button will operate on the selected 3D object In summary hold Alt to scale hold l to translate in the Z direction hold to translate the object in the current viewing plane hold Shift to rotate the object 4 3 3 The playback toolbar oc E o P Pi The playback toolbar corresponds to the play menu which is described in section 4 2 5 e go to first frame olay forwards e step backwards A e IF step forward D e play backwards e go to last frame browse the sequence quickly stop 4 4 The project window Project id xI ES new hybrid merge camera trackl5 EE new hybrid merge camera _trackl5 Wea En Image Sequences oD Image Sequences ll Sequencell Tug Sequenc ell CJS Auto Tracks T SA ZTBT3 a Auto Tracks TU34 2T5T3 88 Manual Tracks LE Qj 0 9999 ESI Cameras Q iuta fea Camer al Autol Bp 3D Primitives intz Ww Autos QJ Autot Ww Auton Ww Autob The overall information of the working project is shown in the project window in a tree structure The root node represents the project with four major child nodes representing the image sequence all feature points the camera and the 3D objects respectively When the user double clicks one of th
28. ocation and input a project name otherwise the program will simply save to the previously loaded project Save Project As saves the project as another file Export exports the results of camera tracking to files compatible with txt file 3DSMax or Maya format User can also scale the 3D coordinates of all feature points while exporting Export Data GFE QD D toy 4 E PAS zm r HAJEE HERE FITA T Camera Track File lk txt T Roi Camera Track File Ok txt 30 Studio Max Gk ms Maya Ok mal Export I Camera Paramters I 3D Points Option Scale the coordinates p The exported data may include The camera s parameters 3D coordinates of all feature points import Feature Tracks import the feature tracks from a text file The format is defined as follows gt The first line is the number of track count Then list the property for each track B Track length 3D valid 1 indicates the 3D position is valid 3D position X Y Z B Frame no image position frame no image position Export Feature Tracks export the feature tracks to a text file Exit exits ACTS Note To avoid losing data it is recommended that users save their projects after each tracking step 4 2 2 Edit Preference Ctrl F Estimate Track Color Clear Track Color Preference the preference of ACTS There are two options as follows gt 3D Sca
29. otation hA radian MI Cameras X 0 002488 uin y Poms of 3D Primitives TUE v 0 022409 lt lil E Translation TimeLine Duis x iux i ai Ps o 1 1 P a r UIS D UMEEMMMNM huto1344 4uto1343 huto1342 amp uto1341 v lt lil 3 ix TimeLine Parameters Graph Output raw lt amp i gt Pi Ready NUM Playback bar Timeline Status bar Main menu bar the main menu includes all commands Project window users can select project file image sequence feature points camera and 3D objects here The related information is displayed in the property window Main window allows users to view the scene in either 2D or 3D mode It shows the image frames feature points and 3D objects Detailed description can be found in section 4 5 Property window specific information relating to the currently selected object would be displayed here For detailed description please refer to section 4 6 Timeline allows users to browse the image sequence Parameters graph visualizes how the camera s parameters change over time Status bar status information of the application 4 2 The main menu bar Here is a quick look at the main menu bar In the following sections each menu and the corresponding sub menu will be introduced in details File Edit Actions 3D Scene Play View Window Advanced Tools Help 4 2 1 File Import Sequence Ctrl I Load Project Ctrl l Save
30. ra s trajectory The main window can display the scene in both 2D mode and 3D mode The 2D view mainly displays the image sequence and feature points while the 3D view presents the 3D Invalid feature point Red reconstruction result of the scene 0 0 o o o 9 Image current image Valid feature points displayed in green with their 3D points projection marked in yellow Invalid feature points displayed in red and have no 3D information The selected feature point highlighted and labeled Current frame index the frame number The 3D object imported by the user in the format of Wavefront object obj The user can access the tools in the 3D view toolbar to operate the object Trajectory of the camera the trajectory of the camera in the image sequence Camera of current frame illustration of the camera in current frame Note when no 3D tool is activated users can scale and translate the 2D view with mouse Hit to restore the position of the image in 2D view Hit to restore the 3D transform in 3D view 4 6 The property window 4 6 1 The project property page Property x Label new hybrid merge cam sequence Imported Tracks Tracked Camera Solved General 1 Label name of the project 2 File path of the project file State 1 Sequence indicates whether an image sequence has been imported 2 Tracks indicates whether feature points tracking has been done 3 Camera
31. th varying focal length Y Manual Track not available gt full Adjustment use bundle adjustment for the whole sequence Not used by default 4 2 4 3D Scene sm Import Wavefront Object Add Virtual Cube Import Wavefront Object import Wavefront obj models Add Virtual Cube insert a default virtual cube to the scene 4 2 5 Play Goto Start Goto End Step Forward D Step Backward A stop Play Forwards Play Backwards nce Loop Auta Reverse Goto Start go to the fist frame of the image sequence Goto End go to the last frame of the image sequence Step Forward go to the next frame Step Backward go to the previous frame Stop stop playing the image sequence Play Forward play the image sequence Play Backward play the image sequence reversely Once stop playing the sequence when the last frame is reached Loop playing the sequence in loops Jump to the first frame after the last frame is reached Auto Reverse when the first or last frame is reached reverse the playing order 4 2 6 View view 2D made Ctrl D View 3D made Ctrl F Image Tracks Predictions Frame Index Scene Coordinate View 2D Mode view the scene in 2D mode View 3D Mode view the scene in 3D mode Image whether images are displayed in the main window Tracks whether feature tracks are displayed in the main window Predictions whether predictive 3D projections of the feature points are
32. tion Model Selection three motion models i e 3D camera model homography model and affine model If you select 3D camera model you should first track features and solve camera For other two models you only need to track features first If you select 3D Camera model you need to set the smoothness weight Motion Filtering Gaussian Filtering is available for Homography Model and Affine Model Linear Optimization is available for 3D Camera Model gt View Warping only available for 3D Camera Model gt Output specify the file path of the stabilized sequence gt Steps there are three steps Just click Run Steps 1 3 Video Depth Recovery VDR Step 1 Depth Initialization Step 2 Bundle Optimization Step 3 Depth Level Expansion Run All Save Models The goal of VDR is to automatically recovery a set of dense depth maps from a video sequence This version can handle a high resolution image sequence Given a video sequence with very large resolution the user can run VDR tool in a down sampled sequence and then upsample the estimated depth maps for the original sequence in addition the user also can equally divide the image to a set of blocks neighboring blocks has overlapping and compute the depths for each block independently VDR is implemented based on the following two papers 1 Guofeng Zhang Jiaya Jia Tien Tsin Wong and Hujun Bao Consistent Depth Maps Recovery from a Video Sequence
Download Pdf Manuals
Related Search