Home

ACTS

1. ACtivMEDIA Color Tracking Software User Manual Copyright 2000 ActivMepia Rosotics LLC All rights reserved Under intemational copyright laws thisdocument orany portion of it may not be copied orin any way duplicated without the expressed written consent of ActivMeDIA ROBOTICS ActivMepiA Rosotcs developersand users are authorized by revocable license to develop and operate custom software for personal research and educational use only Duplication distribution reverse engineering or commercial application of ActivMebia Rosotcssoftware or hardware without the expressed written consent of ActivMepia RoBoTCs is explicitly forbidden The various names and logos for products used in this manual are often registered trademarks or trademarks of their respective companies Mention of any third party hardware orsoftware constitutes neither an endorsement nora recommendation ActivMEDIA ROBOTICS LLC 44 Concon Street Peterborough NH 03458 USA http www ActivRobots com 603 924 9100 Fax 603 924 2184 ACTS User Manual version 2 August 2000 Contents Page WEL OME DO AG TS vvsnsestaa caticssaccttsnaiccsacad tccantancnaansicannctandacacandduaanseanasasnaadenaaadaaansadeacassavaaaatediannasacacadaddanocaran 1 BASIC COMPONENTS visa coartada cdi idad da adi dci dai anda 1 USER SUPPUED COMPONEN Ti aida 1 AAIGISEND ULIOTAS A A a NA its 1 AN 1 WINIOWS IB NV a ct ee tad 1 OPTIONAL ACCESO RES aio idad 1 WHERE TO G ETHE S
2. for example ACTS sendsa variable length maximum 5248 bytes data packet over the client server TCP socket that describesthe blobsit has found in the current frame on a channel by c hannel basis Each ACTS packet consists of a 64 byte header which describesthe numberand index forthe blobsin each channel followed by a sequence of 16 byte blob data structures Packet Header The ACTS packet header is 64 bytes long fourbytesforeach of the thirty two color channels in omerone through thirty two The four bytes actually correspond to two numbers Each numberis encoded using 6 bits per byte This lets usencode numbers larger than 255 without having to wony about sending a NULL character 0 in the middle of the packet The first of these numbers number isthe starting blob numberand the second number also encoded astwo bytes isthe total number of blobs described for that channel in the packet For example the header sequence 0 2 2 0 2 O 2 0 2 4 6 O 6 O 6 1 meansthat the packet contains blob information for channels 1 5 and 7 only Channel one hastwo blobs beginning with the first blob zero indexed asblob 0 channel five has four blobs described in the packet starting with the third out of ten total blobs and channel eight has one blob starting with the seventh Blob Data Following the ACTS packet header come the blobs Each blob isdescribed in sixteen bytes Ta ble 3 including its area number of pixels x and y pixel
3. High Bright slider 16 Image View Mode 22 Info menu 13 25 Linux 4 Live Video button 22 Load Channel 24 30 Load Image 24 Load Runtime Config 24 Low Bright slider 16 Magnification 23 Manual Train 16 Mesa 5 Newsgroups pioneer users 3 saphira users 3 Number of Blobs Tracked 11 22 OpenGL 2 Output text window 16 21 25 Overlay mode 12 20 24 30 Pick tool 16 21 pioneer support 3 Quit 14 24 Raw mode 20 24 30 Remove tool 16 21 30 Reset Limits button 16 ACTS Client Server Interface Resources 3 Save Runtime Config 14 Select Current Channel 21 30 Settings menu 16 20 24 Single Frame Grab mode 30 Single Frame Grab Mode 22 Software Download site 3 Support pioneer support 3 TCP 4 8 19 Thresh Color 20 24 Thresh mode 20 24 30 Timing 25 Training Tools 15 20 24 Training Window 11 12 15 19 20 23 24 25 30 Training Window Closed Mode 23 Training Window Commands 22 Undo 12 23 View Mode 24 Visible mode 20 30 Window menu 25 X Window 5 31 Warranty amp Liabilities The ACTS software is delivered asis with no implied warranties or promises of service Developers marketers and manufacturers of ACTS shall bear no liabilities for operation and use of the software orany accompanying hardware The developers marketers or manufacturers shall not be held responsible for any injury to persons or property involving ACTS
4. current channel in white and all others in black Blobs contiguous regions of interesting pixels get highlighted bya Figure 7 Extracted blobs overlay purple square The center point of each blob isa single purple the Training Window pixel E2Z Train Thresh Retum to Overlay mode by right clicking inside of the Training Window three times or by pressing the 1 key while the mouse isinside the Training Window Ezetrainanresh Figure 8 ACTS eye Thresh view of interesting pixels Figure 9 Refine the training to track the entire obiect Notice by comparing the sample Thresh image with the onginal picture in this manual that we have not captured all of the pixels that make up the book orthe cup Go in and Add more pixels to better define the objectsasblobs aswe have done to create Figure 9 12 Concepts and Tutorial Live Continuous Step 7 View Blob Data aa Arlo dis Viewing the blobsin oursample Figure 8 notice that Magnification 3 Undo gq gt ACTS is tracking only the book and the cup even though two other smaller objects also contain Brightness 20 J interesting pixels This is due to two factors Contrast 100 Ip First ACTS only tracks your selected number of blobs two in this tutorial as you had set in Step 4 from the Channel Commands largest to the smallest objects in the image size Blobs Tracked Channel Filename chosen by area in pixels 4 3 gt A Chee Second very
5. continue where you left off in Step 9 The image of the book and the cup should be visible with the blue Thresh overlay The tracked blobs should have the same statistics as well 14 Training Tools and Tips Training Tools and Tips Training a color channel to accurately track objects overin a variety of environments is a skill that takesa bitof practice to master The tutorial does not do justice to the diffic ulties attributed to tracking live video There are a numberof factors which can greatly influence the tracking quality of a channel ACTS contains several tools including a manual training mode as well asthe EZTrain client that help you define and refine the colorsthat ACTS will use to distinguish colored objects Channels The digital elements pixels that make up a grabbed video typically are 24 bit values three times eight bits for each of the red green and blue components that make up each pixel ACTS classifies each pixel by comparing its RGB components against a lookup table stored inside of each of the thirty two color channels If a pixel s components are found there it is classified as interesting by the channel and tracked Pixels that are not indexed in the colorchannel are ignored Storing all of the possible color combinations fora 160x120 24 bit image would require nearly 17 million distinct index locations That s per channel multiply by thirty two to account for all the channels ACTS supports Besides
6. environmentsthat complicate color training let s start with a neany ideal collection of colored objectsand a clean environment found in the sample static image we provide with the software For the purposesof this demo please load the tutorial static image by pulling down the File menu of the EZTrain console to the Load Image item and select test ppm from the dialog The demonstration version of ACTS opens with a promotional Figure 5 Sample EZ Train message in the Training Window Please contact static image salesGactivmedia com to ordera licensed full featured version of ACTS Load the Tutorial training image from the EZTrain Console The Training Window starts up ata resolution of 160 horizontal by 120 vertical pixels Use the Magnification section of the EZTrain console to increase the image size by up to six times Step 3 Select Channel and Number of Blobs Tracked In the image there isa red book and a red teacup Laterin this tutorial we describe how to track these two objects First select to train Channel 1 by clicking its comesponding button on the EZ Train console And since there are only two objects that we plan to track click the arrows on the Number of Blobs Tracked on the EZ Train Console until the number is 2 Step 4 Select Objects a EZ Train Overlay ry Click the Add button on the EZTrain console to start adding colorsto yourtracking channel In the Training Window click and hold the mou
7. in the window displays the full sized brush Hotkey d ne l The Erase tool works the same as Draw except the tool removes brush sized segments from the channel Hotkey e Pick When the Pick tool is selected from the EZTrain console clicking in the Training Window causes the corresponding RGB color values forthe selected pixel to be printed in the Outputtext window of the EZTrain console Hotkey p Select Current Channel 61 8 9 16 17 24 625 32 i elslsislsl zis Several tools and dialogs of the EZTrain console enable you to set the number of parameters for each of up to thirty two color tracking channels of ACTS Channel Settings Select Current Channel This radio button section of the EZTrain console is perhaps the most important since most other features of the console refer to this setting to altera channel sdata Click on the buttonslabeled 1 8 9 16 17 24 or 25 32 to selecta different bank of eight channels Clicking on the various square buttons will set ACTS to that new channel When changed ACTS automatically switches to the working parameters of the selected channel including Number of Blobs Tracked next section colorinformation in the channel and so on These may or may not be the same as forother channels or even forthe filena me stored version of the channel if you have altered the channel information in the current session When you select a different channel the raw image in
8. small blobs of pixels lessthan 10 pixels 61 38 09 16 17 24 625 32 across will be completely ignored by ACTS as they are usually indicative of noise E AN 4 s s z e Select the Blob Stats option from the Info menu of the EZTrain console The output field at the bottom of the Blob 1 a 748 x 100 y 44 l 74 r 123 t 33 b 57 4 j iqti Blob 2 221892257 Yo31 IS 1 67 te24 b 40 conso le disp la ys the sta tistics of the curently trac ked blobs including area a centerof masscoordinates x and y and the row and columns of the bounding box surrounding the blob Figure 10 Blob statistics appearin the console outout window EA uaa a Up one directory No matching files i Home Step 8 Save the Channel Data Root Current dir Once you are satisfied with the training in a visible files particular channel save its color lookup table LUT All files to a disk based file for later use Notice the default Hidden files filename chamnell lut in the EZTrain console Directories Retyping that name does not save the data i actsconfig Rather select the Save Channel item from the File menu in the EZConsole to actually save the LUT file _0K cancel Use Save Channel As to selecta different directory orchange the name and click the OKbutton to save Fiaure 11 Savina a runtime confiauration the channel Step 9 Create a Runtime Configuration Runtime configurations in ACTS are saved collection
9. to cancel the last training operation done on a channel Each channel has itsown separate undo buffer so you may come backand reverse a change in one channel even though you may have modified several settings in another channel EE Magnification _ Y Shrink or grow the size of the Training Window by clicking the left shrink or right grow arows in the Magnification tool of the Training Window Command set in the EZ Train console The minimum magnification is 1x and the maximum is 6x Hotkeys and Brightness 20 J Brightness and Contrast Contrast 700 J The Brightness and Contrast sliders adjust the brightness and contrast of the images captured by the frame grabber These have no affect ona static image Clear Training Clear Training Remove all the color values from the current channel with a single swift click on the Clear Training button A dialog will pop up making you willfully erase the channel because frankly there is no salvation with Undo here MEA aa Sea cine Up one directory No matching files File Menu xi Home Root The File pull down menu of the main Sue ae menubar along the top of the EZTrain BATES console iswhere you load and save the les various files releva nt to ACTS channels configuration sets and the static images for alco Directories training Operate the File menu aswell asall other menus along the main menubar of the EZ Train console as you do for most other Cha
10. when these settings are altered A useful trick isto train a channel on a colorand then adjust the brightness slider until the training islost Then add the new brighter or darker pixel colors to the channel with the Add or Draw tools Iterate this process until the channel has been trained over all 17 ACTS User Manual possible lighting conditions This will help make your colortraining less affected by changes in lighting and shadows around the object Choosing Colors High saturation colors such asneon are excellent for tracking because they are distinct And try to choose colored objects that stand out from their background Forinstance tracka bright Orange object ona forest green carpet but nota bright green object againsta dark green background If the lighting conditions change or if the object moves further away from the camera the amount of light seen by the camera reflecting off the object diminishes The differences between a light colorand a darker version of the same color will become hard to see If at all possible also try and choose objectsthat are matte instead of glossy Glossy objects reflect colors from the objects around them in the worst case and will nearly always reflect the light sources In the latter case the object in question will appearto have large white spots on it making colortracking useless on those parts of the object Camera Effects What may be good forhome videos may actually interfere wi
11. OFTWARE caian adas 2 NUX FRAME GRABBERS siii aa 2 WINDOWS FRAME GRABBERS cooocccccooonccnnnononononanonononononnnannnnnnnnnnn cn rra rn nn nro rn nara o rr n nn ran n nano ren rra nr rn nrnn na nn na nn nn nnrannn nacen 2 SUPPORTING SOFIWARE E A susdsuncvssdavesvscdauvavcdacassvacavasiucdavasvecdavarduedgeasebidivaiaicdavasstativeathlseustandevsavactemiiaa 2 ADDIMONAL RESOURCES vii cd ccceciduscva vied seve aaeei Eda eraa aa aoea ia AERAN ER AEEA AEAN Sa aad EAA SEE daer did 3 ACTS SOWA Te eaa eet a A A EA Aa e EIEE td 3 PACTS ING WSO UP ni a e a ct eee dee a 3 SUD AAE A T T O E A A E TA A 3 NEW TO VERSON Lili didas 4 ACKNOWLEDGEMENTS iaa 4 INSTALLATION AND QUICK START csccccssssscccsssseeccnnsseeccnnsseecnenseeecncsseecencsseecensssecncesseeenessseeenssseenensssennensnes 5 LINUXGINSTALEA TION oia aaa dicta id a ica dais 5 WINDOWS 95 98 NT WIN32 ccoo 6 DEMO QUICK START A A tid antieb teats 6 LE ENSEDQUIEKRSTA ataca a albo 7 Maining Mode Live VISO vota 7 Training Mode Static IMAGE ee nn rr 7 Serwer Ony Mod Enirante ale Sages eae aden Seca eo den yas ed a re ed cn A dns SEO da lat 7 Sa TUP OP UO MS s 6 eases aaa sttscrageece a aoe A 8 CONCEPTS AND TUTORIAL o gt 22550 9 OWA GATS WORKS r a ebudeccnwuedvalsesesdisdduduelsdeaduetacscdoalacdeguduns edectye ageataavedvatavtagedyestuluctedscgactecttGcuseataccers 9 PERFORMANGE G ONSIDERA TIONS italia ladito iii 9 TYPICAL O PERATION SEQUENCE sssscsssssssccsssscsecesssescussseue
12. Train instead of starting with live video froma frame grabber Pathname isthe path and the filename C ACTS bin test ppm forexample C configfile Load a colorblob lookup table configuration set and run ACTS t configfile Start up ACTS servers only and use the specified configuration set S portnum Specify the client connection TCP port Default is 5001 d device Specify the frame grabbersdevice name Default is dev fg0 Linux only Win32 systems automatically choose the first available frame grabber C contrast Set the image contrast from the frame grabber Default is 100 Linux only B brightness Set the image brightness from the frame grabber Default is 20 Linux only W width Set the minimum width of the image captured by the framegrabber The default width is 160 H height Set the minimum height of the image captured by the framegrabber The default height is 120 All windows in ACTS may be closed by either pressing the ESC key while the mouse is positioned over the window or by selecting the close button from the window manager The Training Window The Training Window contains either live video from a frame grabber ora static image file as well as ACT5 generated ovena ys and displays of the selected channel colorsand extracted blobs Use it to watch ACTS work and to train its color lookup ta bles from EZTrain and Manual Training mode The middle button whe
13. a dde Ai 21 SleCECUTENECASANe limito at oe ita Ena Ea E deaava educa ny abate dawn eddie ada dene aa et 21 Numberof Blobs Tracked Ree 22 ChamelrFlename mata a a a a a 22 TRAINING WINDOW COMMANDS ioaad atenn aaa aaeain nan aaea daraa aeaaaee aia Saanaa a taa ia darai r aran nrannanns 22 Continuous Fame Grab MOE iii iia a a a a 22 Sngle frame Grab Md Erme ae a a ASA Aa Aa aeia 22 Image VieW Mode iwciek ace ns wea a a ee E dae aie aaa 22 Training Window loed MOG eG accio il le doll ned tie 23 Un O ruina naaa ada ld 23 Ma gnific ato Mainisin aAa cect rrereryrrer rere rrerer AR LITA ARA DAA E CARD AAA errr Terre 23 Brightness and Contra dd edo 23 Cll MING ti a wit oy cee Comets eaves aati a iio 23 PEE MENU AEE E E cas EE A AN A PA E E A E E E 23 Load and Save Channel AMAS A O teas A A NNA 24 load Runtime C ONNO ura da ed a 24 Kote AIG CEE AE T EA a Dc Rv I eae Aaa vs Rete 24 ELOT EE EER TTE E EEEE AENEA E LEE TENET TESE EAT ET EET AETA EA E E E EE 24 SEMNGS MENU cota rial a a ciao A a D E a a 24 VAE Aao KATETTA E T A A A E A E A ex Gu E E A A TE TEASTA 24 Brue SZE her coros roman dances 98 AS AES AA Drs raS IS EE E te cout vale EIS IAE Cousin Od TeS S S 24 BINAE oN O E E E E E E EAA E A T A dudes valsieMiediaudentanndn 24 MPA EELO TEE A I EI o ne NA 24 INFOS MIEN Ubi Sc cacucecatscutacavavucdasesauoassvatuetcnvsdudeassaceetas sotcagavauaetuave cules autivarysaucuiaguedtactesciaadaassivuatgaascadegavavatiualavadaseans 25 WMG et O ea heehe
14. cal Operation Sequence 1 Setup system and camera to view objects for tracking in the tracking environment 2 Startup ACTSin EZTrain Live Video mode simple invocation of the actsexecutable with no command line arguments Select one of thirty two channelsto train in the EZTrain console Load orcreate a new Look Up Table LUT forthat channel Use the EZTrain toolset to select colored objectsto track If possible reposition objects within environment and add remove colors from the training channel 7 Refine color selections with Manual Training mode 8 Save the channel LUT 9 Repeat steps 3 through 8 foreach different type of object fortracking 10 Prepare a runtime configuration file containing the saved LUT filenames 11 Shut down EZ Train and ACTS 12 Restart ACTS in server mode Linux only see Table 2 1 for start up options and connect blob tracking information with application clients such asa mobile robot controlled with Saphira or Ayllu au Rw Tutorial This 10 step tutorial takes first time users through a step by step training sequence we commonly use to prepare ACTS fortracking colored objects We presume that you have at least browsed the preceding chapters of this manual and have installed the software in the default directorieson your Linux or Windows95 98 NT based PC Since we ve ka rr used a common GUI operation of the Linux and Win32 File Settings versions of ACTS is virtually identical We ll mentio
15. coordinates of its center relative to the top left comer of the image and the pixel coordinates of the comers of its rectangular bounding box also relative to the top left comer of the image The blob data are specially encoded so that the data do not contain NULL 0 value bytes by adding 1 to each of the 6 bit bytes We recommend that you examine the example code we provide in the distribution for details They illustrate well how you might go about extracting an ACTS packet in yourown application 27 ACTS User Manual Table 3 Blob Data Structure Byte s Label Description each are encoded as6 bits per byte each byte 1 0 3 Area Number of contiguous pixels that make up the blob 5 X Pixel coordinates for the center of the blob 7 Y relative to the upper left comer of the image 9 Left Bounding coordinates of rectangle that encompassesthe blob 11 Right relative to the upper left comerofthe image 13 Top to remove NULL bytes 15 Bottom ACTS with Saphira SRI Intemational Inc s Saphira isa full featured develop ment environment for controlling intelligent mobile robots including ActivMEDIA RosoTics s Pioneer Mobile Robots as well as related robotic accessories Saphira comes with a demonstration client that you may modify and add features including ACTS handlers We ve done just that to create a Saphira client that makesa robot chase a colored object around the room We also provide a Saphira
16. d files Read the enclosed REAME files for more details A file named Version contains additional version and update information ACT5 looks best in 24 bit video mode However 8 bitand 16 bit video modesworkfine None of the windows in ACTS are larger than 640x480 pixels so even the most basic window configurations minimum 8 bits should work Linux Installation ACTS runs over RedHat Linux 6 x and requires the X Window system while training EZTrain The training display may be exported acrossa network so the machine running ACTS doesnot have to be the display machine Of course the performance of ACTSacrossa network depends on the speed of that network and will certainly be slower than when execution and display are on the same machine READ CAREFULLY You do not need to install FLTK or Mesa ACTS comes with the required FLTK and Mesa OpenGL libraries compiled for RedHat 6 x If you do your own installation of these graphicsand GUI software please make sure to install the Mesa ones before FLTK And after unpacking the distribution configure the makefile with configure enable shared prefix wherever Mesa was installed Otherwise follow the instructions for installation of these two library packages included with the distributions They should compile directly out of the box Finally don t forget to set the environment variable LD_LIBRARY_PATH to include the directory of the Mesa and FLIK libraries Once ACTScan find all of t
17. e command line arguments given in Table 1 from an X teminal with Linux or from the Start menu s Run item under Win32 Table 1 ACTS Start Up Options Option Argument Meaning none none Native EZTrain client mode with no channels or configuration defined Training image window will contain live frame grabber video if available or tutorial static image h none Display a summary of the command line arguments f imagefile Load an image for EZ Train instead of starting with live video from a frame grabber Pathname isthe path and the filename bin test ppm for example C configfile Load a colorblob lookup table configuration set and run ACTS t configfile Start up ACTS servers only and use the specified configuration set p portnum Specify the client connection TCP port Default is 5001 d device Specify the frame grabbersdevice name Default is dev fgo C contrast Set the contrast of the image captured by the frame grabber B brightness Set the brightness of the image captured by the frame grabber W width Set the minimum width of the image captured by the framegrabber The default width is 160 H height Set the minimum height of the image captured by the framegrabber The default height is 120 Concepts and Tutorial Concepts and Tutonal ACTS is state of the art client server software Its servers process frames of video information to identif
18. e OE a aed See eae eed Seagate atl A adic Seedachh ath faa cet es Oa 25 A O 25 WINDOW MENU gienen a sauvcuusvautcdvassusscustailbedesseiiedeadvarbcdusentiacadtabuavecsieads 25 EMM ETA T T T e is T T 25 RUNTMEC ONG lidia 25 ACTS C UENF SERVER SOFTWARE cccoocccoocccoooocoooonooncnn nc 27 CLUENT SERVER COMMUNICA TON asia 27 ACIS DATA PACKE ua dd A 27 PacketH ad errete e a rinda dae 27 A EA RT 27 ACTS WH SAPHIRA ia idas icon de eiii rado 28 RODOtA PP liC AHOM actor tar dote By Fens A Pi bp See Ae Gate nn Pees Ag cae ons Pes ha eaten Sp E Og ED hd 28 Saphira PIU IO on Gin dae Lid ais WO iG WA Gite anaes Min dine Maa a hee 28 TNE AA T AT 31 Welcome to ACTS The ActivMebia Color Tracking System ACTS is software which in combination with a color camera and frame grabber hardware in a PC lets your applications track up to 320 colored objects Developed by Paul Rybski currently with the University of Minnesota ACTS is very useful asa vision sensor for robotics and for object identification surveillance and many other machine vision applications This manual describes how to install ACTS in your PC how to train and edit each of up to thirty two discrete colorchannels with the EZTrain utility how to use ACTS with ActivMeEDIA s Mobile Robotics software asa powertul vision sensor and how to integrate ACTS with yourown applications Basic Components Y ACTS User Manual EZ Train ACTS blob tracking servers Figure 1 ACTS tracks up t
19. e the MOTORS on the real robot orelse you and it will get really frustrated Saphira Plugin Saphira comes with an interactive robotics control environment called Colbert Colbertisan interpreted language whose suite of features may be extended through plugins d yna mic a lly loadable and shared libraries that we provide to support intelligent robot accessories like ACTS and which you may also create to add featuresto the basic Saphira client The ACTS plugin for Sa phira Colbert is called sfActs Its Win32 dynamically loadable library sfActs dil and Linux shared object sfActs so come with the licensed distribution along with the source code accessory files and makefiles Visual C workspaces Forconvenience copy these libraries into the Colbert directory of your Saphira distribution typically usi local Saphira ver62 c olbert for Linux systems or CA Saphira ver62 colbert for Win32 28 ACTS Client Server Interface To use the plugin simply load it from the Colbert interaction window at the bottom of the Saphira client s main window from the Load file option in the File pulldown menu from the Saphira main window s menubar or included asa load statement in a Colbert activity Once loaded initialization part of the loading process the ACTS plugin for Saphira continuously attemptsto connect with an ACTS server on the default TCP socket 5001 When connected the plugin automatically queries for blob data ata rate of
20. erent objects and then mix and match which objects to track by reconfiguring which channels you include in a particular runtime configuration Only eight channels can be viewed ata time so switch between the various banks by clicking on the buttonsfound along the top of the window Also runtime configurations are vital for running ACTS in server only mode since there is no other way to tell ACTS what it should track Selecting the Runtime Config option enables the ACTS Configuration File Manager in which you define and save a particular run time configuration on disk A configuration consists of the number of channels that are processed foreach frame of video as well asthe saved parameters foreach channel 25 ACTS User Manual Channelscan be made active or inactive as well If they re active ACTS computes statistics foreach blob up to the Number of Blobs Tracked in that channel All channels default to active but we recommended that you make the unused channels inactive Note Saving a runtime config file will not save individual channel files If you modify the channel files you need to save them separately from the actsconfig file 26 Channels 1 8 Channels 9 16 Channels 17 24 Channels 25 32 Chan 1 ate Hhome rybski activmedia ACTS acts src Channell lut Load Save Chan 2 ee ometyestinctvmesiajaCTS cto orcichannez lut Load Save Chan 3 ict homenyoskitotvmedavACTSRcts
21. essceeeesssseusessceeueesuaeeecsaceusessueeueesaceeseeesseeeessceeueesaneneseasaass 10 TORA ica 10 Step 1 Start Up ACTS for Tha iinQ ococonnnncnnnnanccnnnnncnnnonarncn cnn cnn rare 10 Sep 2 Load the Tutorial Picture 0 ec cece cece cece cece eee nn nrrnrrnrrrrrr 11 Step 3 Select Channel and Number of Blobs Tracked oooococcnnocccnnnoccccnnnccncnnnann cnn r nan cnn rar nan r ran nrrrrnrn 11 Jep 4 ST A AA 11 Step 5 Remove ColorS And Undo o oo cnn oran nr rr 12 Step 6 Change Training Vie WS ic A id 12 Step 7 Miew BlOb Data icon cant ch outa de ck agate cou dale hak dase dae id Gedy na dee dd chan dese pan Gesu chan dd dos 13 Step 8 Save the Channel Data oonnnccconnccconnnccccnonncncnnonnnnn narra 13 Step 9 Create a Runtime Configuration ccc i i cnn non cnn 13 Step 10 Quit EZ Train and Restart in Server Mode coocccccccconncnnnncnnncconnnnnonnnnnncnonnno non rrr ran crnnnnr nor rr rar rr ran rra rra narcos 14 TRAINING TOOLS AND TIPS cocccccococcccoo once 15 CHANNEL Sas 15 AG TSARAINING TO Obie A E E A A on EAU aaa ao 15 MANUAL TRAINING MODE atada letal lali dad 16 GHING CONDMON Shen in adds 17 CHOOSING COLORS idad 18 CAMERA EFFECTS cti hs 18 RUNTIME OP MON S cuida cda id dino 19 THE TRAINING WINDOW ia 19 Training Window MOUES viaria iii 20 THEEZ TRA N CONSOLE cito tics inline ir daa 20 o NAAA 20 REMOVE e e A RANTING eo Mn e ao e ae 21 DW tt 21 A AAA 21 Pa a a a a a a A e a ona ob ari ne as ae Pe 21 CHANNEL SETING Sh
22. extension plugin that you may use with your Colbert programs And we provide the source code forall of these as well as for other ACTS Saphira supporting code so that you can build yourown Saphira clients Robot Application We ve created a Saphira client acts_robot exe to provide you with an example application in which the mobile robot followsa color literally driving around to find and then follow a colored card that you move around the room You may run the acts _robotapplication on a real robot or fake it with the simulator that comes with the Saphira distribution In eithercase the PC that runs the application must contain an active frame grabber and host both AC TSand the Saphira client To run acts robot first start up ACTS either in training mode or loaded with a pre trained configuration In either case we assume that you will train ACTS to follow some colored card preferably a 5x7 card of a wild color such as fluorescent green ororange Once ACTSis ready execute acts_robot exe The Saphira client s main control window appears containing an iconic representation of a robot at its center From the Connectmenu on the main menubar pull down to connect with your robot either Local for the simulator pioneer exe already started orone the serial ports Once the robot gets connected the application also makesa connection with the ACTS servers and will automatically begin following your colored card Don t forget to enabl
23. he libraries it will be able to run The demo and full distribution versions of ACTS for Linux come asrpm archives and they come in two flavors one for common BI8x based frame grabbersand one forthe Imagenation PXC 200 frame grabber The default location for all their components is usy local and usylib If you locate ACTS elsewhere or place its supporting libraries ata different location you ll need to redirect various symlinks and makefile pointers To install ACTS in the default location you ll need to be root Copy or download the rpm file to tmp assuming you have the proper pemnissions and then cd tmp rpm Uhv acts 1 2 i386 rpm common BT8xx rpm Uhv acts 1 2pxc i386 rpm Imagenation PXC200 only ACTS User Manual Later to completely remove ACTS from your system rpm e acts 1 2 or acts 1 2pxc for PXC200 installation You might also consideradding ACT5to yourexecutables PATH export PATH SPATH usr local acts bin bash version setenv PATH PATH usr local acts bin sh csh tcsh If you have a frame grabber installed make sure you have a dev fgO This may mean creating a symlink to your frame grabber device For example with the BT848 drivers ln sf dev bt848 dev fg0 Windows95 2000 NT Win32 The distribution ACTS EXE isa WinZip http www winzip com self extracting archive Simply double click the mouse on itsicon or otherwise execute the ACTS EXE program When prompted you
24. hogging memory initializing and iterating through this much data would overtax even the fastest processors Instead we use a scheme which involves projecting the pixel s data onto a set of three color planes Ratherthan have an RGB cube store the data three color planes a Red Green RG plane a Blue Green BG plane and a Red Blue RB plane are used instead To be classified as interesting pixels from the image must have their red and green components located in the RG plane their blue and green components located in the BG plane and their red and blue components located in the RB plane ACTS Training Tools a Add Remove Draw Erase Pick The most common and quickest way to add colorsto an ACTSchannel is with the mouse driven Add tool It sthe default active tool when you start up ACTS with EZ Train As described in the tutorial clicking and drag this tool in the Training Window to draw a rectangular overlay in the image inside which delimits the set of pixelsthat will be added to the channel However simply adding pixel values to the channel generally is not enough to robustly tracka colored object In addition to the single pixel value a set of additional pixel values which neighbor that value are also added to the colorchannel Thus each of the three color planesreceives a square of color consisting of multiple pixelsto be added to itstable Thisredundancy helps to make a much more robust colortraining by making sure that the regio
25. icitly exhorting you to save any unsaved channels so be sure to save everything before quitting ACTS also may be quit by hitting the ESC key orby closing the window through the window manager Settings Menu The Settings menu on the main menubar of the EZTrain console consists of a series of hierarchical menusand items that control various parameters of how ACTS operates on the Training Window View Mode The View Mode items of this menu select the appearance of channel data in the Training Window including Ovena y Thresh Raw and Visible See the discussions about the Training Window for details Brush Size Brush Size lets you select from one pixel to a 9x9 square of pixels forthe Draw and Erase tools Draw Color By default all of the Training Tools use a red rectangle in the Training Window to delimit their areas of influence Thiscan be bothersome when trying to train on red objects Hence with this Settings menu you may change the Draw Colorto blue orgreen for better visibility and back to red Thresh Color In Overlay mode the interesting pixels are covered bya Blue overlay Use the Thresh Color Settings menu to change the overlay colorto red orgreen for better visibility depending on the background image 24 ACTS Client Server Interface Info Menu The items found in the Info menu on EZTrain s main menubar display their results in the Output window at the bottom of the console s window Timing This g
26. ile Settings Info Window The EZ Train Console l Continuous Single Frame The EZTrain console iswhere you manage the color M um Undo 1 ini Tai Brightness 20 J The Training Tools section of the EZ Train console lets a el Sa training process you choose a mouse operated training function for the Training Window including adding and removing individual or whole regions of pixels from the currently Channel Commands active channel and sampling RGB values Blobs Tracked Channel Filename Activate a particular tool by clic king its icon on the 4 3 wi EZ Train console or by pressing a hot key while the Select Current Channel cursorisin the Training Window The mouse pointer 1 8 9 16 O17 24 25 32 changesto the tool sicon when it is over the Training Window A 4 s 7e Ad d Dutput The Add tool lets you define a rectangular region of pixelsin the Training Window by clicking and dragging the mouse Pixels within the so defined rectangle get added to the channel Hotkey a Figure 15 The EZ Train Console 20 ACTS Client Server Interface Remove Like Add the Remove tool lets you define a rectangular region of pixels with the mouse in the Training Window Removed pixels within the rectangle get removed from the channel Hotkey r om A The Draw tool lets you click Brush Size squares of pixels in the Training Window and thereby add them to the channel Holding the pointer
27. ill remove pixels starting from a clockwise and counterclockwise direction try it to see The High Bright adjuster changes how much of the lighter valued pixels closerto white are classified as interesting This limits and removes pixels from the upper right part of the plane The Low Bright adjuster changes how much of the darker valued pixels closer to black are classified as interesting This limits and removes pixels from the lower left of the plane The Fill Plane button tums every pixel in the corresponding color plane on If each plane is filled in this fashion the ACTS has classified every pixel asinteresting and the EZTrain window should be completely highlighted ACTS could be trained by filling each color plane and then adjusting the limits on the planesto subtract what is not wanted Thisisa different approach from the additive mechanism used in the EZTrain window but may be more useful in some circumstances 16 Training Tools and Tips The Clear Plane button clears every pixel in the corresponding color plane reverting it to white O Ghannel Parameters E Red Green Plane Blue Green Plane Red Blue Plane ADE Red Green Limits Blue Green Limits Red Blue Limits Red Blue Red Ml df ses pip aaf 210 bil Ma 310 pl Green Green Blue mei 115 pil ala so pw aaf 100 pl 4 High Bright High Bright High Bright a 51 dla Ml df ses pl aa sto pip Low Bright Low Bright Low B
28. in any way They shall bear no responsibilities or liabilities for any operation orapplication using ACTS or for support of any of those activities And under no circumstances will the developers marketers or manufacturers of ACTS take responsibility for support of any special or custom modification to the software ACTS User Manual v2 August 2000 44 Concord Street Peterborough NH 03458 603 924 9100 603 924 2184 fa x http www activrobots com
29. ing Mode You will find on occasion that the EZ Train toolset just isn t precise enough for stubbom objects The Manual Train control panel lets you more finely tweak various settings for the curently selected channel Choose it from the Settings menu in the EZ Train console If you change to another channel the settings in the Manual Train console automatically change too Each color plane that comprise the ACTS color channel has four associated adjusters and three buttons entitled Reset Limits All Plane and Clear Plane Modify the various limits to tim a plane s included selection of values These parameters two color levels and two brightness levels define what color pixels are allowed in the channel Initially no limits are set meaning that all parameters are set to their maximum or minimum values and all pixel colors are allowed The two colorthreshold adjusters for each plane change the amount of that particular color that is allowed in that plane For instance changing the Red adjuster in the Red Green plane will add or reduce the amount of red pixels that are allowed into that plane This reduction is shown by the white pixels of the plane being replaced by black pixels Any colors that happen to be in this plane when these para meters are changed are tumed to blackand are no longertracked To restore them either change the appropriate adjuster to reveal the coloragain or hit the Reset Limits button The color limit adjusters w
30. ities for these graphics G UI toolkits may be downloaded from their manufacturer s websites http fltk easysw com software html http www Mesa3d org download html You don t need to download FLTK or MESA Their salient libraries are included with ACTS Additional Resources Every new ACTS customer gets three additional and valuable resources v A private account fordownloading ACTS software updates and manuals v Accessto the private vision users newsgroups Y Directaccessto the ACTStechnical support team ACTS Software We maintain a 24x7 Web server where customers can obtain ACTS software and supporting materials http robots activmedia com ACTS Accessto complete distributionsand upgrades of ACTS is restricted to licensed customers To gain access enterthe usemame and password that accompanied your orginal purchase of the software For Pioneer Mobile Robot customers this ma y also be the same usemame and password that you use to gain accessto some but not necessarily all other areas of the website ACTS Newsgroup We maintain an E maiktbased newsgroup through which ACTS and VisLib owners share ideas software and questions about the software To sign up send an E mail message to ourautomated newsgroup server To vision users request activmedia com From lt your return e mail address goes here gt Subject lt choose one command gt help returns instructions lists returns list of newsgroups subsc
31. ives you a sense of ACTS performance with your system When selected Timing displays how many milliseconds it takes for ACTS to processa single frame of video It also showsthe number of frames per second fps currently being achieved Under most conditions ACTS should operate efficiently between the maximum rate for most frame grabbers 30 fps Blob Stats Blob Stats displays all of the statistics for all of the blobs being tracked in the selected channel Blob n a aaa x xxx y yyy 1 111 r rrr t ttt b bbb n blob blob 1 10 a area of blob in pixels x x index of the blob center of mass y y index of the blob center of mass left most column of the bounding box sunounding the blob r right most column of bounding box t top most column of bounding box b bottom most column of bounding box Window Menu Selecting items from the Window menu of the EZTrain main menubar enablesthe various accessory windows of the console Close windows by pressing the ESC key when the mouse cursor is inside the window including the EZ Train console or by closing them via your windows manager menu EZ Train The EZ Train item in the Window menu activates the Training Window which displays the image data that ACTS will train its channels on Runtime Config A runtime configuration isa disk file that contains the names of the channels that you want loaded en masse to start a tracking session This lets you train channels for diff
32. may select to install the software anywhere on your system The default path is CA ACTS Unless you plan to use static picturesonly the Win32 version of ACTS only works with an Imagenation PXC 200 frame grabberand requires the card s supporting softwar typically stored in CA PXC2 Once you have expanded the ACTS archive please copy these following dynamic load library dll files from the bin directory of the Imagenation frame grabber s software directory typically CA PXCA bin into ACTS bin directory typically CA ACTS bin Y Video_32 dll Y Pxc_95 dIl Y Pxc2_95 dll v pxcdd dll Info Window v Frame_32 dll z Grab Continuous Altematively copy the aforementioned Win32 frame Frame Single Frame grabber software files into your Windows System Magnification directory where they automatically will be found by the ndo 1 gt operating sytem and shared by all applications Brightness 20 I including ACTS Contrast 100 J Demo Quick Start Channel Commands Blobs Tracked Channel Filename In the demonstration version of ACTS you may not connect with a client otherthan ACTS own EZTrain 4 3 Nonetheless all of the many training features and blob Select Current Channel tracking capabilities of ACTS including itsinterface with 1 8 69 16 617 24 25 32 a camera and frame grabber are intact and on full sot a Abie it en Per sis baat Te display for your operation and study 1 2 3 9 5 6 7 8 Wi
33. n the minor differences aswe go along Continuous Having a frame grabberand camera connected is Da Single Frame good although not required to perform most of this Magnification tutorial Demo users are forewamed that they won t be Und 1 gt able connect with a client other than EZTrain but you Brightness 20 I may use a camera and frame grabberthroughout and Contrast 100 J there is still a lotto see and leam besides Step 1 Start Up ACTS for Training Channel Commands Start up ACTS in its native mode which is connected to 4 Blobs mane gt Channel Fiename a the EZTrain client with Live Video if available and with AN the default none configuration and LUT files Simply Select Current Channel see Chapter 2 for details 1 8 9 16 617 24 625 32 E AE SA BA gaia O da ejeje jeje cd usr local ACTS bin Output acts Win32 Double click acts exe icon inc ACTS bin directory oruse the Startmenu Run item s dialog type in C ACTS bin acts and click the OKbutton Figure 4 The EZTrain console 10 Concepts and Tutorial Once ACT5starts up it will open and display two orthree windows The EZTrain console and another EZTrain window containing an image A third window one opened by Windows or the original X terminal with Linux contains system messages that you may ignore Minimize it Step 2 Load the Tutorial Pic ture Rather than fuss with your frame grabberand the mynad of real objects and
34. n the mouse isin the Training Window acts like the Grab Frame button in the EZ Train console When grabbing live video itchangesthe view from a still snapshot to continuous grab Figure 14 ACTS Training Window 19 ACTS User Manual and backagain Click the right mouse button in the Training Window or pressan associated hot key to cycle between modes Overlay Thresh Raw Visible The active mode if not already apparent in the image also appearsin the window stitle Training Window Modes In Overlay mode the Training Window shows the raw pixel data asit appearsto the framegrabber orasit appearsin the loaded image file All pixels that ACTS finds interesting are colored either blue green orred depending on your Thresh Color selection from the EZ Train Settings menu default overlay color is blue Hotkey 1 Thresh mode displays all interesting pixels as white and all uninteresting pixels as black The logical blob groupings appearas purple rectangles bound the interesting regions The center of mass for each blob is shown by a single purple pixel inside of each rectangle Number of Blobs Tracked in the EZTrain console limits the number of rectangles displayed Hotkey 2 View the unencumbered image in Raw mode if overlays get distracting Hotkey 3 Visible mode is like Raw mode but displays all interesting pixels as their natural colors instead of white No rectangle forthe blobsis displayed 0 A E Hotkey 4 F
35. nnel lut GUI applications Click and hold the mouse F on the menu item na me then drag to the Loreo eea pulldown item of choice that appears La AAA AAA AAA A e Figure 16 Common File menu file selection 23 ACTS User Manual Release the mouse button to select that highlighted item of choice or move off the pulldown menu and release the mouse button to cancel the operation The various items of the File menu share a common dialog Figure 16 Load and Save Channel and Save As When selected the Load and Save Channel and Save As itemsof the File menu activate a common file selection dialog for loading or saving respectively the current channel s data file on disk The filename which you may change in the dialog s textbox initially isthe one that you may have entered in the Channel Filename textbox on the EZTrain console Load Runtime Config Use the file selection dialog activated by this item to load a runtime configuration set of channels into ACTS Load Image Load Image lets you selecta ppm file to load into the Training Window This is useful for computers that do not have frame grabbers or if you simply want to demonstrate ACTS without the headachesof hardware ACTScan load ppm files of any size aslong asthey are saved in 24 bit color and will resize them to fit the current window size Quit This isthe graceful way to quite ACTS You are prompted to venfy the quit giving you the chance without expl
36. ns added to the color channelsare more orlesscontiguous This helps compensate for variable lighting conditions and otherenvironmental noise 15 ACTS User Manual The Remove tool erases pixel values from the channel Use it just like the Add tool except its selected color contents get removed from the channel The pixels removed from the channel are not only the selected pixel but also several neighboring pixel values Thus trying to remove a few peripheral pixels from the channel like those unintended pixelsthat were selected in the tutorial example ma y actually remove pixels from seemingly unrelated colors The Draw tool lets you point and click to select Training Window regionsto add to the channel By default Draw addsa single pixel to the channel each time you click the mouse Change the size of this tool with the Brush Size settings in the EZ Train Settings menu from a single pixel up to a 27 pixel square The Erase tool like the Draw tool selects discrete spots in the Training Window image but it removesthe colors from the channel Its size also is set by the Brush Size item in the EZ Train Settings menu Finally use the Pick tool to click a single pixel in the Training Window and have its RGB values printed in the Outputtext window at the bottom of the EZTrain console Pick isa valuable diagnostic tool to see how a particular pixel s value may change with changing environments such asthrough a moving shadow Manual Train
37. nux Frame Grabbers For you RedHat Linux folks there are several Brooktree BT848 868 878 based PCI bus frame grabber cards available generally as well as through ActivMebia ROBOTICS For instance the Intel Smart Video lll frame grabber works fine with ACTS for desktop PCs The Imagenation PXC 200 isthe one we recommend and support for use with the Pioneer Mobile Robot s onboard PC Use the correct Linux driver and ACTS distribution There are special version of ACTS and Linux drivers for the Imagenation PXC200 PC104 card Linux drivers for the various BT8xx frame grabbers for Linux also are available from the ActivMeDIA RoBOTICS support website Installation and configuration instructions come with the software http robots activmedia com bt8xx Windows Frame Grabbers For Windows95 98 NT users ACTS currently requires an Imagenation PXC 200 or equivalent frame grabber and the company s supporting drivers Purchase directly from the manufacturer http www imagenation com orthrough ActivMeDia RosoTics The required frame grabbers support software for Windowscome bundled with the hardware Supporting Software Forall versions ACTS makes use of OpenGL Mesa version 3 0 graphics toolkit and the Fast Light Tool Kit FLTK version 1 0 9 from Easy Software forthe graphical user interface The required libraries come bundled with ACTSand are part of the installation process Complete versions including sourcesand support util
38. o 320 Supporting FLIK and OpenGL graphics and colored objects in up to 32 GUI software libraries independent color Gnu and MSVC version 6 x sample C channels sourcesand makefiles LNA lt User Supplied Components All distributions 150 M Hz Pentium or better PC system including mouse and min 8 bit color display 32 MB RAM 8 MB free space on hard disk drive Colorcamera NTSC or PAL SSSS Linux v Redhat Linux version 6 x v Matrox Meteor or Brooktree BT8xx frame grabber v Associated frame grabber drivers Windows 95 2000 NT Y PXC200 frame grabber from Imagenation Inc Y PXC200 driver software Optional Accessories C development tools SRI Intemational s Saphira Robotics Application Development Environment ActivMeDIA s Ayllu Multiagent Robot Application Development Environment Pan Tilt Zoom Robotic ColorCamera SSS ACTS User Manual Where to Get the Software distributions as well as free updatesand upgrades all are available from the M OBOTICS http robots activmedia com ACTS Versions and download instructions are detailed on the ACTShomepage Contact sales ActivMedia com for full distrib ution licensing details ACTS also lets you train its blob tracking channels from static image files That s fine in some situations for demonstration purposes for instance But for most other situations you ll need a frame grabberand supporting software to train and operate the ACTS servers Li
39. of course A Saphira or Ayllu client might also be controlling a mobile robot for example and thereby use the blob information to guide the machine towardsa moving colored target Your own client application might count and sort parts passing by on a conveyor belt according to their AC TS extracted color and size Performance Considerations ACTS curently can smultaneousl track up to 320 independent blobs at the frame grabbers maximum rate of 30 frames per second ten blobsin each of thirty two independent color channels We ve done this repeatedly with a variety of objectsand environments ACTsperfomance its ability to find and track blobs of course dependson a variety of factors including computing resources RAM processor speed quality frame grabber and so forth environmental varia bles such asvariance in luminance lighting and shading subtle color variations among similar objects background colors and many other real world complications The one most important factorin ACTS performance however is how well you train the system for the colored objects you wish to track There isno magic training formula To a great degree training ACTS isan art Experience is invaluable Using odd colored things helps a lot too Having fast and easy to use training tools asprovided by EZTrain makesa big big difference too We give you a number of clues on how to optimize training in the forthcoming Tutorial ACTS User Manual Typi
40. once every 100 milliseconds the standard Saphira cycle time Exa mine the sfActs source code for supported functions and data structures 29 ACTS User Manual Appendix A Hotkey and Mouse Shortcuts The following hotkeys are active in the Training Window O 00O07 gt 50RARUNE Ovenay mode Thresh mode Raw mode Visible mode Add Tool Remove Tool Draw Tool Erase Tool Pick Tool Increase magnification of the Training Window Decrease magnification of the Training Window Grab a new frame Single Frame Grab mode The following functions are invoked by the mouse while clicking in the Training Window Mouse Button 2 Grab a new frame g hotkey Mouse Button 3 Cycle between the various Training Window modes Overlay gt Thresh gt Raw Visible gt Visible Double clicking in any of the channel selectors in the Select Current Channel area will bring up a Load Channel dialog 30 Index ActivMedia 1 Add tool 16 20 30 Blob Stats 13 25 Brightness 4 8 17 19 23 Brush Size 16 21 24 Channel Filename 22 Channel Settings 21 Clear Plane 4 17 Clear Training 12 23 Configuration 25 Continuous Frame Grab Mode 22 Contrast 4 8 17 19 23 Control Window 20 Draw Color 24 Draw tool 16 21 24 30 Email pioneer support 3 pioneer users 3 saphira users 3 Erase tool 16 21 24 30 EZ Train 11 25 File menu 14 23 Fill Plane 4 16 FLTK 5 Grab Frame button 19 22
41. patible with older versions can still read channel files that were saved by older versions y New Visible EZTrain window mode in which interesting pixels display in the colorsthat are reported to ACTS while all other pixels are black On the fly contrast and brightness controls forthe frame grabber currently Linux only Arbitrary image sizes Increased flexibility for manual training of channels with the addition of Fill Plane and Clear Plane buttons SNS Acknowledgements Special thanks to Michael Trosen for his work on the Windows port and for managing the distribution and support ofthe product Installation and Quick Start Installation and Quick Start ACTS currently works only with RedHat Linux or Microsoft Win32 based PCs Each version of ACTS has its own peculiarities which we detail in the following sections In common you may train ACTS and perfom a few simple tests without using a camera and frame grabber But for most work you ll need that hardware and its supporting driver software Please installand test your frame grabberaccording to the instructions provided with the hardware and the drivers Most difficulties with ACTS derive from impropeny installed or configured hardware Also in common ACTS creates its own directory hierarchy usy local ACTS or CA ACTS containing the acts exe or acts demo exe program accessory files development resources and source files as well asa variety of support libraries an
42. re Channe ut Load Save Chan 4 cive Hhomerrybski activmedia ACTS acts src Channel4 jut Load Save Chan 5 este ometyestiectmeciajaCTS ct orciChanne ut Load Save Cren N CES crenv MN EE Figure 17 ACTS Configuration File Manager ACTS Client Server Interface ACTS Client Server Software With the licensed distributions we provide sample applications and supporting software including source code for interfacing ACTS with ActivMepia s Mobile Robotics software including Saphira and Ayllu Thischapteralso describes the data structures and interface details that you need to know to integrate ACTS with yourown C language based client applications Client Server Communication ACTS servers communicate with a client through a TCP socket either on the same machine or over a network The default socket address is 5001 You may configure ACTS to respond on a different port with the p lt portnumber gt start up option Include the W Soc ketC lientComm cxx SaphiraAC TSC lientc xx and related header file in your application code to enable ACTS client server communication Visual C users also need to include the system s wsock32 lib library in their project On inspection of the SaphiraACTSClienth file you will find commandsto open and close the ACTS server socket as well as query its status send commands and retrieve data ACTS Data Packet Upon request comm gt requestPac ket
43. ribe unsubscribe Our SmartList based vision users listserver will respond automatically After you subscribe send your E mail comments suggestions and questions intended forthe worldwide community of Pioneer users To vision users activmedia com From lt your return e mail address goes here gt Subject lt something of interest to all members gt Access to the vision users newslist is limited to subscribers so your address is safe from spam However the list currently is unmoderated so please confine your comments and inquiries to issues conceming the operation and programming of ACTS and VisLib Support Have a problem Can t find the answer in this or any of the accompanying manuals Or do you know a way that we might improve ACTS Share your thoughts and questions directly with us support activmedia com ACTS User Manual Your message goes directly to ActivMepia s technical support team There a staff member will help you orpoint you to a place where you can find help Because this isa support option nota general interest newsgroup like vision users we reserve the option to reply only to questions about problems with ACTS New to Version 1 2 Y Supports 32 channels All clients must use the up to date TCP socket communications files provided with the distribution Clients compiled with the older files will not work with the new version of ACTS Y Anew Channel Save file format Downward com
44. right ala iso yim ala so imi elf gt ol Reset Limits Reset Limits Reset Limits _FiliPlane_ Clear Plane Lighting Conditions Lighting quality is extremely important when training and tracking colored objects Properly illuminated objects are so very much easierto track Indirect illumination from all sides is probably the best lighting since point illumination sources light bulbs will cast shadows and reflect off of glossy surfaceson the objects being tracked Try not to have the target back illuminated since this will cause shadows and will probably wash out their colors Fluorescent lights are better than incandescent lights because they have a whiter color spectrum When training a channel to track an object it is very important to view the object under various conditions After training under one lighting condition view the object under another lighting conditions Add training asneeded For instance aftertraining a channel to trackan object of a particular color casta shadow over the object This should reveal particula dy when the Training Window isin Thresh mode several pixel colorson the object that the channel isnot tracking Those colorscan then be added to the channel to make the overall color training more robust ACTS for Linux allows you to change the contrast and brightness of the images captured from the frame grabber However the trained pixels in a color channel do not change their values
45. rts up in continuous fra me grab mode unless you don t have a frame grabber running in which case a default image appearsin the Training Window In continuous frame grab mode ACTS works as quickly asit can to have the frame grabbergrab a frame then processand display that image from the Training Window The quickest ACTS can grab frames is 30 persecond When the Continuous radio button is selected the Grab Frame button is grayed out Grab Continuous Single Frame Grab Mode amo gens Click the Single Fame radio button to stop continuously grabbing framesand display the last grabbed image Click the Grab Frame button to take a new snapshot Hotkey g besides switching to single frame mode also grabsa new frame for the Training Window Live A ontinuod Image View Mode ue TARE Meee If you ve loaded a static image from a file into the Training Window switch to frame grabbing mode by clicking the Live Video button Selecting this button will command ACTSto display images from the framegrabber again 22 ACTS Client Server Interface Training Window Closed Mode Close the Training Window by clicking its close window button in the uppercomer of the window in order to test the optimal timing and performance of ACTS colortracking while training In the absence of an image the Training Window Commands frames related button becomes Show EZ Train Click it to re open the Training Window Undo pole Click the Undo button
46. s of channels and operating parameters Commonly used to define the operating environment for ACTS in the service of an application client such asin server mode an ACTS runtime configuration also may be loaded for editing with the EZTrain client 13 ACTS User Manual In this tutorial you ve defined and edited one channel channell lut and set its blob tra c king number Now save that set up asa runtime configuration file Select Save Runtime Config from the File menu in the EZTrain console In the dialog choose your destination directory and perhaps change the filename then click the OK button to save the configuration Note Saving a runtime config file will not save the individual channel files These need to be saved separately in addition to the acts config file ci amas ES Select Quit from the File menu or pressthe ESC key to quit ACTS Now restart ACTS with that 7 Type the name of a program folder document or Internet saved configuration we named it actsconfig resource and Windows will open it for you and used the testppm static image for training Open JC ACTS bin acts c C ACTS bin actscontig CA we From an X terminal window in Linux Cancel Browse cd usy local acts acts f docs testppm c actsconfig Figure 12 Start up ACTS with a configuration file With Windows95 98 NT start ACTS from the and static imaae Run item in the Start menu In both cases ACTS should start up and
47. se button and drag it to create a rectangle inside the red book Change the shape of this rectangle to select as many of the pixels inside of the book aspossible Be careful not to select any pixels outside of the book Figure 6 Add tool rectangle selects training colors Release the mouse and the pixels under the red box automatically get added to the colors of the current channel The Training Window isin Overlay mode so that the blobs found by EZTrain get highlighted in the image Resize the image to make it easier to select individual pixels 11 ACTS User Manual Step 5 Remove Colors and Undo Similar to the Add tool click and operate the Remove woot ato select areasof colors you do not want included in the channel As you remove colors previously highlighted blobs also should disappear Resize the image to work more closely with the color Throughout you may also click the Undo button on the EZ Train console to reverse the last training operation that modified the channel sdata Click the Clear Training button to completely erase the channel data and start overtraining from scratch Step 6 Change Training Views Overlay mode in the Training Window can be confusing Click the right mouse button inside the Training Window or press the 2 key while the mouse pointer is inside the Training Window giving you an ACTS eye view of blobs called Thresh mode The program displays the interesting pixels colors appeaning in the
48. th ACTS Forinstance some cameras have an automatic white balancing feature in which the camera changes the color values of the image pixels relative to itsneighborsto produce a clearer picture But remember we are tracking colors not edges Since different lighting conditions have different colorsin their spectra the sun versus fluoresc ent lights auto white balancing will assuredly produce very different pixel colors forthe same object This also means that if the object that the camera istracking gets very close to the lens and thusappearsto be very large in the image the colorsof the other objects in the image will start to vary asthe camera attemptsto balance the colorvalues Tum off auto white balancing if you can 18 ACTS Client Server Interface ACTS Reference This section describes all of the controls and modes of ACTS and the EZTra in client Runtime Options ACTScomeswith a variety of startup options to address various operating environments Use the command line arguments given in Table 2 from an X terminal with Linux or from the Startmenu s Run item under Win32 Table 2 ACTS runtime options Option Argument Meaning none none Native EZTrain client mode with no channels or configuration defined Training image window will contain live frame grabber video if available or tutorial static image h none Display a summary of the command line arguments f imagefile Load an image for EZ
49. th Linux start the demo version of ACTS from an X catgut terminal window usr local acts demo bin acts demo With Windows95 98 NT open the ACTA bin directory and Figure 2 EZ Train Console Installation and Quick Start double click acts demo exe with the mouse or from the Start menu Run item s dialog type CA ACTS demo bin acts and click the OK button With all demonstration versions ACTS opens two windows The EZTrain console and an associated graphics window containing purchasing information A third window containing DOS like text also appears with the Windows versions Pulldown the EZ Train File menu and choose Load Image From the resulting dialog select your own orthe prepared bin testppm image Alte matively click the Live Video button on the EZTrain console to activate your frame grabber Licensed Quick Start The licensed versions of ACTS start up a bit differently than their demo cousins Training Mode Live Video When executed without command line arguments double clicking the acts exe icon for instance ACTS starts in EZ Train Live Video mode wherein the EZTrain console window appears along with another window containing live video frame grabs from your camera If the frame grabberis not working a promotional image will appear in the video window with an error message to that effect Training Mode Static Image Run 2 x If you don t have the frame grabber installed orcamera attac hed y0U Ma
50. the Training Window won t change but because the colortracking parameters may probably change blobs indicated in the window s overlay and thresh modes will change 21 ACTS User Manual Blobs Tracked Number of Blobs Tracked 4 3 gt Use the Number of Blobs Tracked thumb dial tool to set the number of colored objects you want to track in a the current channel Click the arrow to the left of the number to decrease the value minimum one click the right arrow to increase the value to a maximum of ten ACTS will report to the client information about a maximum of that number of the largest blob it detects from the Current image frame The default Number of Blobs Tracked is three Channel Filename Channel Filename Entera pathname into the Channel Filename textbox to give the channel data a saved file destination on disk This will be the filename ACTS uses when you simply Save the channel from the File menu Altematively entera pathname and pressthe ENTER key to load a saved channel s data into the current channel ff the file exists Click the ellipsis button to activate a file selection dialog which contents get loaded into the current channel Training Window Commands The set of controlsin the Training Window Commands section of the EZTrain console manage various parametess of the Training Window Al Continuous Continuous Frame Grab Mode Frame Single Frame The licensed version of ACTS nomally sta
51. y Type the name of a program folder document or Internet also start ACTS thata static picture of resource and W indows will open it for you yourchoice appearsin the image window associated with the EZ Train Open C ACTSbin acts f C ACTS bin test pprel y console You then operate EZ Train as ou would on a Live Video image y j Cancel Browse With Linux start ACTS from an X windows terminal and with a command f f o argument that identifies the image you Figure 3 Starting Win32 ACTS with a static image wish to load Forexample to start the licensed version of ACTS with the test ppm static image supplied with the distribution from within ust ACTS bin acts ftestppm With Windows95 98 NT systems run the licensed version of ACTS from the Start menu Run item as shown in Figure 3 Server Only Mode With Linux you may start the licensed version of ACTS in server only mode This way ACTS s servers operate most efficiently in the background forconnection with your client applications Serveronly mode requires that you already have prepared and saved your color lookup tables in an ACTS configuration file So for exa mple to start the ACTS servers in the background using the sample configuration we ve prepared for you in the distribution acts t actsconfig ACTS User Manual Startup Options ACTScomeswith a variety of startup options to address various operating environments Use th
52. y and track colored objects blobs and share that extracted information with clients The integrated ACTS EZTrain client software lets you train ACTS to find the colored objects you want the servers to track How ACTS Works ACTS needs specialized hardware in your PC a colorcamera and frame grabber to take digital snapshots frames for AC TSto process Once that video data gets captured ACTS servers take overand perform an operation known ascolorsegmentation by sorting the colorencoded digital elements of the frame pixels into two groups interesting and uninteresting Interesting pixels match the red green and blue RGB elemental color components of those stored in any one of up to thirty two different color look up tables that you create when you train the system with the EZTrain client Disinteresting pixels of course don t match any of the RGB colorsin any of the tables Once identified interesting pixels are grouped into blobs identically colored regions of adjacent pixels at least ten pixels wide Blobs smaller than ten pixels are usually noise and thereby ignored The ACTS servers characterize each blob according to its matching channel position in the video frame size and dimensions and finally communicate that information to clients such as EZ Train or the Saphira and Ayllu robotics application clients we include with ACTS It s up to the client to decide what it will do with the blob tracking information

ACTS

Contents

Download Pdf Manuals

Related Search

Related Contents