Investigation of a New High-speed Tabletop Interaction System
Contents
1. The magnifier application communications with the camtraker program under TUIO protocol As long as the magnifier is on the table surface it will be recognized as active status Once the position of the magnifier is updated it will take a snapshot of the screen acceding to the coordination received from the camtracker in real time Then the magnifier enlarges the snapshot and 25 displays the enlarged image inside its frame 4 3 2 Color Filter Application Color blindness is color vision deficiency which is the inability to perceive differences between some of the colors that others can distinguish The frequency of color blindness is fairly high among males 8 among Caucasian 5 among Asian and 4 among African 8 Generally there are two methods to avoid color confusion The first method provides an additional clue which is not relying on color information For example usually the color of a visited web link is red while the unvisited one Is blue Obviously color blind viewer can not distinguish this difference by color To solve this problem we can design the webpage as following The link status Is Showed as a prompt message when the mouse cursor Is hovering on a web link The second method is using high degree contrast color combination in the user interface However colors that are seemingly contrasting and perceptible to a normal color viewer can appear less so to a color blind viewer W2. drawing task200H2 l 1 45 overall 5 _ BBB __ loo ____ drawingtask250H2 __ Subject Subject2 Subject3 Subject4 Subject5 Subject6 Subject7 Subject8 Subject9 Subject10_ speed a l_ a4 5 2a4 a 4 a 4 3 nw consistency 4 o 41 5 4 a a 3 a 31 a influence aj 3 a4 _ 35 5 24 3 so a ____ 7 _____ wrtingtask100H2 __ Subject Subject2 Subject3 Subject4 Subject5 Subject6 Subject7 Subjects Subject9 Subject10_ speed 03 4 3 2 2 ni 2 2937 ir consistency 4j 2 2 2 9 aif 2f 1 influence 3 2 2 2 2 a mw 2 i overalt 4 11 2 2 2 i 2 i wrting task 150HZ speed consistency influence over all wrtingtask200H2 oloolo loo __ Subject Subject2 Subject3 Subject4 Subject5 Subject6 Subject7 Subjects Subject9 Subject10_ speed 83 3 a a 31 31 a a4 3 3 WIN W W LU w w Ww N N N Uw N N W W N N W W LU T L W K HHIRIN 46 eS ee ee ee ho oh a ME a a PES Subject Subject2 Subject3_ So yo 2 y 3 4349 2 3 3 a 2 A LE OOO LL __ Subject Subject2 Subject speed speed Aa laa 5H 5 consistency 3 3 041 30 04 S5 2 2 23 influence 4 3 041 30 a 5 3 83 83 3 overall 35535 s5 333 3 PASOS IESO lo i __ Rame ash tone o __ Subject Subject2 Subject3
3. A IN AIEA AA E E AN 28 LIS OIT SAA EAS AA AA EA A TETO 29 5 0 2 1 Write Drawing Application PLOT diia 29 5 3 2 2 Air Hockey Game Application PI A a ias 32 TERO EDURE nu Ga O as ANAN 32 KE rF ANAIS Tasa A AAA AA E A 33 3 DF TTI TTT 34 K AU Hockey GOING LAS ANA 3 De SER S LUDY DATA ANALY SUS a od 35 6 DISCUSSION costa NANANA NIAN RA NANA NN ANA iaa 38 6 1 DISCUSSION ON SETTING UP QUALITRACK SYSTEM eee 38 6 1 1 Static Calibration VS Dynamic Calibration 2000777 7 aa 38 Gb KA O AA Ea 38 0 2 DISCUSSION ON NEW INPUT DEVICES odiada 39 6 3 DISCUSSION ON USER STUDY EXPERIMENT cocina ride 39 Ol WIT VS DAWN TTT 40 6 3 2 Statistic Explanation on Air Hockey Game Task 40 Tx CONCEUSTON ac ae a i GW Y a GT UD CYFN YA CU aE 41 Ss FUTURE WORK ni a 42 Sal ABOUT QUATTIRACK SYSTEM aa NT AG aT Y WE NW WY YNA DF 42 D4 ABOUT DU ER STUDY Ga GR asas 42 REFERENCE NANANA NTN UA FD E ARNO 43 APFENBDIX GG AA AA 45 APPENDIX I ORIGINAL USER STUDY DATA 0d ANGIN ANNA ANG 45 APPENDIX 2 TIPS TO SET UP QUALITRACK SYSTEM SOFTWARE PARTI 49 1 Introduction In a European collaboration between t2i Lab Chalmers University of Technology CTH Sweden and ICVR research group Swiss Federal institute of Technology ETH Zurich a new high speed interactive tabletop infrared tracking system called Qualitrack was developed A smaller part of this system approximately 20 was developed at CTH BJ For a set of input devices positioned
4. Both of the two algorithms are used in Qualitrack system They can be shifted by change the calibration mode 19 The following is the formula of the Two Points Algorithm We define that the coordination of the left top corner as Itx Ity and the coordination of right bottom corner as rox rby in camera coordination system then fatiox rbx ltz screen width rahoy rby lty sereen height A ratoz lt Y Y ratioy ly Here X Y is the point coordination in display coordination system Three Points Calibration Algorithm is more complex but accurate than Two Points Calibration Algorithm We define that the point coordination in camera coordination system as Xc Yc and define it as X Y in display coordination system Then A AXc BYc L 1 Y DXc Erc 1E 2 Since we samples three points so we have the following six equations Xl AXcl BYc C A2 AXc2 BYc2 C XS3 AXcS BY c3 C Yl DXc1 EYc1 F Y2 DXc44 EYc2 F Y3 DXc3 EYc3 E After synthesize the above equations we can get the coefficients A B C D E and F E Xc1 X03 Yc2 Yc3 Xc2 X03 Fel Ye3 A X1 XD Yc2 Yc3 X2 X3 Ycl Yc3 E B Kel Xe3 NGO X3 lol Xc3 Xc2 Xc3 E C Tell Xc3X2 Xc2X3 Yc2 XcIX3 Xc3X1 Yc3 Xc2X1 Xc1X2N KE D 1 YAY e3 1Y2 3 el Yc3 E E Xc1 Xe2XY2 YS Yl YS Xc2 Xc2d K F Yc Xc2Y2 Xc2Y3 Yc2OC1Y3 Xc3YD Yc3 Xc2Y1 Xc1Y2 E Substituting these coefficients into e
5. Our Alternative hypothesis H1 was H1 There is at least one above equation doesn t exist After Analysis of Variance Anova we found that all for the three factors were non statistic significant Namely we do not have enough reason to deny the null hypothesis according to our current data And the difference observed between system tracking update rates is likely due to chance rather than the test condition This could be the result of small sized sample We can do this experiment by enlarging the data size and see If it will works better in the future 40 7 Conclusion In this project we reconstructed a high speed table top system successfully based on some existing work At the beginning of this project all the old hardware are lost or damaged Moreover we were lack of detailed system specification document Thus it took us quite a long time than we could imagine rebuilding Qualitrack system Form the procedure of setting up Qualitrack system we found the following conclusions 1 Dynamic calibration is essential in Qualitrack system 2 The accuracy of Qualitrack system is influenced by the arrangement To enrich the input devices of Qualitrack system we created two new devices the magnifier and the color filter Most of the system user thought the feature of the magnifier is quite nice and useful At the mean while we discovered that the material of an industrial product influences the users experience quite a lot A th
6. Subject4 Subject5 Subject6 Subject7 Subjects Subject9 Subject10_ speed 3 o A s 3 speed og aaa gas influence 5 3 over all 5 4 2 2 _ TE Subject4 Subject5 Subject6 TE TIE TE Subject10 speed COD IST 3 intuence O ss 3 3 3 overall 4 30 s5 830 a4 5 83 8237 4 83 ___ 47 aa sanetesk 00M _ a a __ Subject Subject2 Subject3 Subject4 Subject5 Subject Subject7 Subjects Subject9 Subject10 5o sl al consistency 4 4 _ 3 IONES 3 5 o af influence 5 3 3 3f 4l o a a it GM Y overall 5 3 3 Ll ____ gametask250H2 __ Subject Subject2 Subject3 Subject4 Subject5 Subject6 Subject7 Subject8 Subject9 Subject10_ speed 3 24 a4 30 a 3 a a 3 al consistency 3 2041 3 30 5 30 a _ a a _ a influence aj 53 3 30 sa 4 s l sa s8 a 48 Appendix 2 Tips to set up Qualitrack System Software Part 1 About SyncUnit Question 1 How can know the Olimax Sam7 p256 development board works or not Answer Run the example code demo at91sam7 p64 blink flash on Olimax Sam7 p256 development board Details please read lt lt ProReflex MCU Technical Reference gt gt Question 2 How can know the double end camera cable works or not Answer Use Docklight to test the double end serial cable Pay attention to set the baud rate within 19200 in
7. computer interaction 2 Edition pp 530 2007 17 Jodi Shephard The Use of Rapid Prototyping for Interface Design http www cc gatech edu classes cs6 51 97 winter Topics rapid proto T1 0 18 Ken Wakita Kenta Shimamura SmartColor disambiguation framework for the colorblind Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibility SESSION Assistive technologies for individuals with visual impairments ll pp158 165 2005 19 Luke Jefferson Richard Harvey Accommodating color blind computer users Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility SESSION Vision pp 40 47 2006 20 Paul Hoffman Accommodating Color Blindness Usability Interface Vol 6 No 2 pp 1 1999 21 Smith Thomas Guild John The C I E colorimetric standards and their use Transactions of the Optical Society 33 3 pp73 134 1928 44 Appendix Appendix 1 Original User Study Data drawing task 100HZz Subject1 Subject2 Subject3 Subject4 Subject5 Subject6 Subject7 Subject8 Subject9 Subject10 speed 2 3 3 3 2 2 2 2 2 3 consistency 3 3 3 3 4 2 2 3 2 2 influence 3 1 1 2 2 2 3 2 2 3 __ lo loo oo drwngtakl50He 1 __ Subject Subject2 Subject3 Subject4 Subject5 Subject6 Subject7 Subject8 Subject9 Subject10_ ___ 7 ___
8. dock light Docklight can t support a baud rate over it Details about Docklight please read lt lt Docklight V1 9 User Manual 05 2009 55 2 About Camtracker Program Question 3 How to set up RXTX Answer Unzip the rxtx 2 1 7 bins r2 Zip file which is under the root folder of the camtracker program Copy the rxtxSerial dll file into your JDK_HOME bin folder Copy the RXTXcomm jar file into your J DK HOME jre lib ext folder clipse project property gt Java Build Path gt Libaray gt Add External J ARs add RXTXcomm J ar into the camtracker project Question 4 What is the difference between TUIO cursor and TUIO object o ID Position Orientation Angle Area objet fv lv o Y cursor Y 3 About Air Hockey Game Application Question 5 How to eliminate warning C4819 in visual studio Answer Project gt Properties gt C C gt Advanced gt Disable specific warnings enter 4819 Question 6 How to eliminate the fatal error C1083 Answer Project gt Properties gt C C gt General gt Additional Include 49 Directories enter corresponding head file name Note In a VS project setting up the following issues is the basic knowledge Project Properties gt C C gt General gt Additional Include Directories add corresponding include directories Project gt Properties gt Linker gt General g
9. document We tried our best to reuse existing software components and hardware design so as to keep the original 3 system features as much as possible Object oriented thinking and software reuses thinking reduced our time cost efficiently and enhanced the code quality effectively The two project group members worked separately in a long period 6 weeks One worked in Switzerland with all the hardware and the other focused on software development in Sweden To communication efficiently and get the user experience response in time we chose prototyping as our software development method Session 4 User Study experiment This is the last session of this project Ten volunteer participants are involved in a three tasks user study experiment During the entire four sessions we used some engineering and experimental methodology These details will be explained in Chapter 3 1 3 Delimitations The user study was only carried out within a small group size for the reason of limited time This project will develop and set up Qualitrack system platform and develop the entire user study hardware and software environment so that other people can do a further user study based on our work in the future 1 4 Thesis Outline This section outlines the thesis structure A brief description of each chapter and its contents is mentioned below Chapter 1 Introduction introduces the general context in which the project is carried out and describes Quali
10. for the three tasks 5 4 1 Handwriting Task Two sample sentences are displayed in the work area They are the same sentence I love Oualitrack One is ligature The other is block s letter The participant is instructed to aim for both speed and accuracy when writing the sentences underneath the two sample sentences with the infrared pen The participant is required that if a mistake was made he or she is to ignore it and 33 continue with the sequence After finish writing the participant is asked to switch the infrared pen to the mouse mode and use it to press the OK button Then the participant s work will be saved in the hard disc driver of the host computer automatically In the new trail the system will be set at another testing tracking update rate randomly without telling the user which update rate it will work at in case that the participant will predict the system performance After the participant finishes all the four trials by press F2 the grading user interface is launched The participant s work will be load into the screen in parallel so as to the participant can grade them The participant grade the their experience on the system s reaction speed system reaction consistency and system overall performance from 1 to 5 for example if the system generate the scratch immediately after the participants wrote without any human perceptible delay then the system reaction worth 5 On the contrast if the scratch I
11. from brainstorming meeting we followed a set of procedure and obeyed some rules H First of all we determined one clear and concrete topic for every meeting instead of a dozen of general topics Secondly all participants was noticed and ensured to clear the topic and asked to do some preparation such as read relative materials before the meeting Thirdly we kept the group size around five people If the group size is too small we can t collect enough inspiration On the contrary the meet will become hard to control if there are too many participants Fourthly we cleared the three kinds of roles and their responsibility in our brainstorming meetings the moderator the sectary and the participants At the beginning of the meeting the moderator must emphasis the topic and the discipline of the meeting During the meeting the moderator made some summary in time and led the meeting to another climax The sectary s task was to record all participants the new ideas and write them on the white board so that everybody can see the points clearly We required every participant must perform active enough Fifthly it was definitely forbidden to make any judgment on people s idea during the meeting in case of influencing the participants positivity Judgments were left to after the meeting The last but not the least our brainstorming meeting was controlled within 30 to 45 minutes every time If the meeting continues t
12. the hardware he definitely has to spend quite a lot time to modify the software so as to match the hardware However at that time due to the complexity of the multifunction software it will take much more time than just modifying a pure functionality unit 12 4 Qualitrack System 4 1 Hardware 4 1 1 SynchUnit The SynchUnit hardware was developed based on the Olimex SAM7 P256 development board equipped with an Atmel AT91SAM7S256 micro controller This unit is responsible for the synchronization of the digital camera and all infrared flash input devices It also takes charge of processing the camera output signals and forwarding the devices statues data to the host computer Figure 4 1 1 1 is the Olimax Sam7 p256 development board Figure 4 1 1 1 Olimax Sam7 p256 development board In order to filter out potential noise and make the signals which access to the camera more stable a RS 232 485 converter was built This converter transforms the RS 232 signals into differential 2 chanel signals used in the RS 485 standard A three ends RS485 cable was built One end of this cable was connected to the serial port of the SynchUnit the other two ends were connected to the service port and data port of the digital camera respectively This design enables the use of longer cable without decreasing the signal quality Figure 4 1 1 2 is the photo of the RS 232 485 converter Figure 4 1 1 1 RS 232 485 converter 4 1 2 Input Devices The
13. IO client component receives the TUIO package bundle it will check 22 the arriving one is a TUIO object package bundle or a TUIO cursor package bundle Then it will check the message type If it is a set message the TUIO client component will parse this massage and assign its information into some specific variables If this set message belongs to a new session then a new TUIO object will be created Otherwise an existing TUIO object within the same session will be updated When an alive message comes the timestamp of an object within the same session will be updated If the received message is a frame message TUIO client will scan the label of the frame massage and do relevant operation The process of a TUIO cursor package bundle is quite similar with a TUIO object package bundle 4 2 3 2 Air Hockey application The AirHockey_App is a C program developed base on the OpenFrameworks OpenFrameworks is an open source C library OpenFrameworks is based on some well known libraries such as openGL openCV and so on It offers us a lot useful libraries with more convenient and appropriate call interface than the base libraries It provides a nice framework for building up a game Figure 4 2 3 2 1 shows the structure of this framework main cpp Figure 4 2 3 2 1 structure of OpenFrameworks The main cpp file is the starting point of the applications It declares the resources used in the program such as OpenFrameworks and start t
14. IT 10 024 Examensarbete 30 hp Juni 2010 Investigation of a New High speed Tabletop Interaction System Kaijie Sheng Institutionen for informationsteknologi Department of Information Technology UNIVERSITET Teknisk naturvetenskaplig fakultet UTH enheten Besoksadress Angstr mlaboratoriet Lagerhyddsvagen 1 Hus 4 Plan 0 Postadress Box 536 751 21 Uppsala Telefon 018 471 30 03 Telefax 018 471 30 00 Hemsida http www teknat uu se student Abstract Investigation of a New High speed Tabletop Interaction System Kaijie Sheng In this project we built up a high speed interactive tabletop infrared tracking system based on the existing Qualitrack system 1 2 This system can detect the position orientation and state of each infrared emitting input device in real time The output of Qualitrack is implemented according to standard TUIO protocol Any applications software understanding TUIO protocol can be docked with Qualitrack smoothly Based on the documents of the existing system we built up the whole system architecture the infrared pen the infrared air hockey paddle and corresponding application software for Qualitrack system As an in house contribution we created two new input devices namely the magnifier and the color filter to enrich the input device types of Qualitrack system In order to explore the influence of tracking update rate on the user interaction performance three user exper
15. Task A ball will be served from the gate which is at the top center of the screen The width of the gate is one thirds of the screen width The participant is asked to uses the infrared air hockey game paddle to beat the ball back to the gate If the participant succeed he or she will get 1 score When the participant gets a goal or the ball flies out of boundary a new round will start The ball will be served from a random position near the gate at random orientation and velocity There are three second break between each round so that the user can be ready The whole trail takes lasts for ten balls The participant will grade their experience whenever the task is finished a trail immediately The participant is required to grade his or her feeling on the system reaction speed system reaction consistency and system overall performance from 1 to 5 In this task the meaning of system reaction speed implies the reaction speed of the paddle image when the participant moves the infrared paddle And the system consistency here means if the image of the paddle is continues and stable then it worth 5 On the contrast if the image of the paddle twinkles obviously then the system reaction consistence is bad In the new trail the system will be set at another testing tracking update rate randomly without telling the user which update rate it will work at so that the participants will not be influence by their own prediction 3 5 User Study Data Anal
16. Thread Trigger SynchUnit and IR Cemera y C End Figure 4 2 2 1 1 flow diagram of the init method When input data stream comes SerialPortEventListener calls serialEvent method automatically to handle serial Events Figure 4 2 2 1 2 is the flow diagram of the serialEvent method san N Data Available Convert Data to MarkerArrayContainer Send MarkerArrayContainer to Deviceldentifier Figure 4 2 2 1 2 flow diagram of the serialEvent method When a data comes from SynchUnit hardware through the serial communication port the SynchUnitConnector object validate the arrived data in serialEvent method The legal data will be packaged into MarkerArrayContainer format and forward to Devicel dentifier object 17 4 2 2 2 Device Identification 1 Bit Code Algorithm Every infrared input device in Qualitrack system has its unique eight bits identification code which is called bit code The identification code is used to distinguish devices so as to support more than one device work at the same time Correspondingly a special bit code identification algorithm is used in Qualitrack system The core of this algorithm is the two rigorous bit code selection constraints which will be mentioned in the following context Conventionally the host computer software like the camtraker program here has to analyze n camera frames to identify a dev
17. a paper based one A computer based prototype is an early stage runnable version of a program which shows its most important features The prototype gives the designer a functional working model of their design so they can work with the design and identify some of its possible pros and cons before it is actually produced Another significant aim of building a prototype is to let the user have an intuitive experience with the program so that they can support the designer some valuable response The designer will modify and refine the current prototype acceding to the response from the user and create a new prototype as a result Create prototype user experience refine prototype the same loop iterates again and again Finally there will be a prototype which meets all our requirements Namely it is the final product 11 In this project due to in quite a long time one of the project students worked in Switzerland with all the hardware while the other one developed software in Sweden From the software developer s point of view the hardware developer was just the software user The programs were sent to user as a prototype whenever a functionality of software had been implemented Then the user tested the software usability with the hardware and gave the software developer response in time It is not hard to imagine what will happen if the software developer implements all the software independently without prototyping thinking When he gets
18. at or directly above the table the system is capable of simultaneously tracking each device s position state and orientation in real time The system tracking rate of 250Hz provides smooth and reliable device registration Every input device contains an integrated infrared receiver and a signal processor They are synchronized by a hardware called SyncUnit and emit synchronized infrared light pulses According to a particular bit code algorithm each device uses a unique infrared light pulse sequence enabling device identification The output of Qualitrack system is implemented under the standard TUIO protocol so that any applications software understanding TUIO protocol can be docked with it smoothly Figure 1 1 is the structure diagram of Qualitrack system And Figure 1 2 is the photo of Qualitrack system SyncUnit Qualisys Tracking Camera Fr my x j hy P a a Interaction devices IR Flash Wy Projector Interactive surface Figure 1 1 structure diagram of Qualitrack system de Figure 1 2 Qualitrack system 1 1 Related Works Only several academic institutes are engaging in developing visual based multi input devices tabletop system in the world until now Device identification is one of the core research questions for multi device tabletop system There are several device identification methods in different systems For instance DigitalDesk recognizes the input devices by shape I
19. bration accuracy enhanced substantially We support the user two alternatives in order to make the new Qualitrack system friendly enough The user can choose to do a dynamic calibration when the system starts or load the old experimental calibration setting data from the system resource file 6 1 2 System Arrangement During the procedure of setting up Qualtrack system we encountered some problems with the system arrangement which reduced the system performance of Qualitrack in some way First of all due to the size of the ProReflex MCU240 infrared camera is too big 38 242 X 97 X 133 mm and too heavy 2 4 Kg to be set vertically under the table surface we have to use a 45 degree angle mirror to reflect the input infrared light to this camera Thus the quality of the mirror became a big threshold of the system accuracy Secondly a brown color glass was used as our table top surface namely the screen In order to prevent the brown glass to be scratched by the hard tip of the infrared pan we covered an extra normal plastic glass on it However the difference refractive indexes of the two kinds of glass cause the infrared light leave its expected track and decreased the system accuracy 6 2 Discussion on New Input Devices The second goal of this project is to create a new input device A homocentric Squares shaped hardware was developed around this goal At the mean while the magnifier application program and the color filter a
20. ce created by the International Commission on Illumination CIE in 1931 Figure 4 3 2 1 is the CIE 1931 color space chromaticity diagram which is based on the human eye perception of chromaticity From the human eves point of view the percentage of red 26 green and blue in a white light is different Thus we need to add different weight value to different color channel so as to mix a gray scale which is similar with gray level in human s eyes Figure 4 3 2 1 CIE 1931 color space chromaticity diagram a International Telecommunication Union ITU defines several recommendation standards for gray scale translation The most popular ones are ITU R BT 601 and ITU R BT 709 Formula 1 is gray scale transformation formula in ITU R BT 601 In this formula R is short for red G is short for green B is short for blue and Y is the gray scale Y 0 298912 R 0 586611 G 0 114478 B 1 Usually we do not need so accurate value in programs so we accurate the coefficient to three figures behind decimal point like formula 2 Gray 0 299 R 0 587 G 0 114 B 2 The gray scale transformation formula of I TU R BT 709 is showed in formula 3 Y 0 212649 R 0 715169 G 0 072182 B 3 Figure 4 3 2 2 is the gray scale result of based on ITU R BT 601 standard mu TAr La AR Figure 4 3 2 2 gray scale result of based on ITU R BT 601 21 5 User Study Experiment A three tasks user study experiment is designed to find out h
21. d one interface which are been used in this project CommPortl dentifier class which is the central class in RXTX It controls the access of communication ports SerialPort class which defines the functionality of serial communication ports SerialPortEventListener interface which propagates serial port events In a typical serial port communication application CommPortldentifier discovers and opens an available communication port in initialization phrase Then the application can call the methods in class SerialPort to access data through this port Usually there are two ways to monitor the input data stream One is polling the other is listening In polling mode a polling thread has to be created manually In this project listening mode is used with the SerialPortEventListener interface which supports standard Java event driving mechanism These are tones of corresponding tutorials online but most of them mislead the reader about how to deploy RXTX To deploy RXTX we need to copy RXTXcomm jar to lt jdk home gt jre lib ext and Copy rxtxSerial dll to lt jdk_home gt bin 3 Methodology 3 1 Brainstorming Brainstorming is a kind of group creativity collection method created by Alex Faickney Osborn in 1938 The aim of brainstorming is to sour people open their mind so as to generate new ideas Osborn proposed that groups could double their creative output with brainstorming I In order to get a satisfied result
22. devices were built as self powered units consisting of a microcontroller an IR receiver a 3 7 Volt battery a voltage converter and one or multiple IR LEDs When the receiver detects an external infrared light the micro controller will validate it in order to ensure it respond to the synchronization signal rather than a noise signal The amount of LEDs depends on the specific application design of a device Point device such as the pen are equipped with one single LED since their position on the tabletop surface is the only valuable information for this system Line device are equipped with 3 LEDs For example the air hockey game paddle whose orientation is required in addition to its position Figure 4 1 2 1 is the photo of the input devices of Qualitrack systems Figure 4 1 1 1 input devices from left to right the magnifier the air hockey game paddle the IR pen 14 4 2 Sottware 4 2 1 SynchUnit SynchUnit is a real time embedded solution developed on Olimax Sam7 p256 development board It takes the responsibility of synchronizing the whole system and forwarding data from the infrared camera to the host computer Our project is base on some exciting work We got the bin file of the SynchUnit SAM Boot Assistant SAM BA is used to burn this bin file into the flash memory of Olimax Sam7 p256 development board Before being able to use SAM BA a recovery procedure which consists of copying the SAM BA Boot Assistant into the f
23. drawing than writing 37 6 Discussion 6 1 Discussion on Setting UP Qualitrack System The first goal of this project is to reconstruct Qualitrack System We know Qualitrack system more deeply during the procedure of rebuilding it and found several existing problems in the original system Here we discuss theses problems and our corresponding solutions in the following 6 1 1 Static Calibration VS Dynamic Calibration The original Qualitrack used static calibration solution in its implementation of camtracker program Namely the same empirical calibrate parameter values were used all the time to calibrate the input devices The system arrangement the relative distances and angles among the projector camera and table will influence the calibration accuracy of the whole system obviously So it is very hard to find a set of responsive empirical calibration parameter values Thus it is not hard to imagine that the calibration accuracy of the original Qualitrack is really bad Before this project there isn t any user graphic interface application for Qualitrack system Therefore the former developers ignored this problem since they couldn t see the result intuitively To solve this problem we applied a dynamic calibration solution Namely we did a calibration test to get the fresh calibration parameter values before the core of the camtracker program started running As a result of this dynamic calibration solution the system cali
24. e writing task There is an additional sample picture section in its in information area A sample picture is loaded by this program automatically whenever the application runs The participants can draw their picture in the work area with the infrared pen After they finish their work they can save their work by pressing the OK button as well By press F2 we can shift to the system grading user interface It is used for the participants to grade the system performance Figure 5 3 2 1 3 is the system grading graph user interface Instraction Please grade the system s reaction speed NO Picture NO Picture from 1to 5 1 is the worst and 5 is the best You may give the same grade more than once NO Picture NO Picture Figure 5 3 2 1 3 system grading graph user interface This interface can be divided into two parts as well In the left there is the instruction area In the right there is the grading area The grading instruction and attentions is showed in the instruction area The participants will be asked to grade the system four times from four different aspects Each time the grading topic will be displayed with red color and big font in order to remainder the participants what they are grading for This program loads the user work pictures into the grading area automatically The participants can grade their work by pressing the grading button under each user work picture The grade button will change its color to r
25. ed for the needs of tabletop tangible user interfaces encoding and transmitting the attributes of tangible artifacts that are found on a table surface Since TUIO is based on Open Sound Control OSC it can be seen as a Slandered implementation of OSC There are three main kinds of messages in TUIO protocol SET message ALIVE message and FSEQ message Set message describes the status of an object such as position and orientation Alive message is used to identify the objects which are working right now with a specific session ID FSEQ message is defined as the unique tag to distinguish the frame update TUIO protocol forwards the encoded data from a TUIO server such as a visual based tracker to TUIO client application which can decode TUIO package Except for working as a rule TUIO also supports a framework which defines this common protocol and APIs There are more and more third party development libraries and TUIO client implementations in various programming languages Support this protocol 2 3 RXTX One of the conventional J ava serial port communication development libraries is comm2 0 jar which is published by Sun in 1988 The other one is the open sourced RXTX which covers all the functionality of Sun s comm2 0 and can Support more operation systems There are thirteen interfaces and classes in RXTX library To access the details about these prototypes please check RXTX J avadoc Here only introduce the two classes an
26. ed immediately after being pressed At the mean while the other buttons within the same group will become disabled automatically so that the user can not grade more than 31 once for the same feature of the same user study task trail by mistake The grading result will be record into a log file in the hard disc driver automatically These data will be used for the user study analysis 5 3 2 2 Air Hockey Game Application Program The air hockey game application is a C application for the air hockey game task of the user study experiment This game is processed in ten rounds A ball is served from the goal at the beginning of each round from random positions with random orientation The user s task is to bat the ball into the goal with an infrared air hockey paddle The ball will rebound back if it hits the white borders Whenever the user gets a goal or misses the ball the current round is finished and the next round will start in 3 seconds Figure 5 3 2 2 1 shows the interface of the air hockey game application program Figure 5 3 2 2 1 user study application for the air hockey game task 3 4 Procedure The user study consists of three tasks a Handwriting b Drawing c Air Hockey Game Participants perform all three tasks over two sessions The first session is a 32 warm up session which takes five minutes In this session the users are demonstrated how to do the following task and free to do anything they want in order t
27. ements all the device identification functionality in a thread loop And hence it works during the whole tracking life cycle Figure 4 2 2 2 2 1 is the flow diagram of the 18 run method Unpack MarkerArray Container Calculate Frame Position Yes Handle Device Active Issues Update Device Status Update Device Position Figure 4 2 2 2 1 flow diagram of the run method When the Deviceldentifier object receives camera frame data it calculates the frame position If the arriving one is an identification frame the Devicel dentifier object will generate Active Device Events If the arriving frame is a status frame the Devicel dentifier object will generate Update Device Status Events Regardless of the frame type the position of the infrared devices will always be updated 4 2 2 3 Calibration Algorithm When an infrared point is captured by the camera its coordination is noted within the camera coordination system But the logic coordination system of the display device is different from the camera In order to present the position of the infrared point logically it is essential to transform point coordination between the above two coordination systems This procedure is called calibration The most common calibration algorithms are the Two Points Calibration Algorithm and the Three Points Calibration Algorithm
28. he program The testApp cpp file is the center class in this framework It contains the business logic of a specific application There are three important methods in this class The setup method is used to initialize resources for instance objects and variables It is only called once in the program The update method is called continuously by a background loop In this method the objects and variables update their status The draw method is used to draw the screen according to the latest resources status After receiving the TUIO package from the camtracker program this program decodes the TUIO packages and updates the screen according to the new data 23 Figure 4 2 3 2 2 shows the data flow of the Air Hockey Game application program amp Start a gt y initial game environment Init ball receive TUIO package Yes update paddle Yes he T rebound ball No update ball position N Yes Yes score o round a gt C end Figure 4 2 3 2 2 data flow of Pong Game application The program environments such as the goal gate position and so on are only initialized one time at the beginning of the game But the ball will be initialized in every new round When a new TUIO packaged bundle is received the status of the paddle will be updated If the ball hits the paddle or the walls it will be
29. hen color is used as the primary means to encode information people with color deficits may not perceive the information as intended 19 t is impossible to let normal color viewer and color blind viewer absolutely experience each other s feeling about the same color So it is very hard to find the best combinations which can express the same color information to the two groups at the same time Meanwhile a table top system as a platform can not and should not change the user interface design of the third party application which runs on it But from this point of view we can help the color blinder viewers works easier by designing a visual assistant device on this platform An efficient and effective approach to solve the color confusion problem is to take a snapshot of the environment a screen grab and remove the color so you are looking at it in grayscale The color filter device is designed an implanted acceding to this method The algorithm of the color filter application is quite similar with the magnifier application The only difference is the magnifier enlarges the snapshot while the color filter application converts the snapshot to gray scale image Of course we also can combine the magnifier and color filter functionalities together Then we will get a color filter magnifier In the study of the perception of color one of the first mathematically defined color spaces was the CIE 1931 XYZ color spa
30. ice Hence the real output tracking update rate would drop to the system tracking frame rate divided by n To solve this problem Constraint 1 is involved Constraint 1 only one O bit is allowed between two 1 bits Under this condition when a device enters the view of the infrared camera the camtracker program waits for eight frames to identify it Initialization phase But for every subsequent frame the device position is updated whenever its bit code is 1 for a frame Hence the update rate only drops to half the system tracking frame rate Another problem is that we suppose many devices are used in parallel and then they will all be active in the same camera frame If two devices are very close to each other it will be hard to distinguish them So Constraint 2 is that the bit code must be chosen as few as devices have a 1 bit at the same bit position After eight bits bit code there are two more status bits They can be used for example to check if a button is pressed Obviously this algorithm enhances the output of the tracking update rate effectively The only shortcoming is that there isn t any mathematics formula to classify it until now This will be an important future work definitely 2 Device Identification Implementation Deviceldentifier class identifies infrared input devices updates device position orientation status and generates TUIO events The center method of Devicel dentifier class is the run method It impl
31. imental tasks were designed and user studies were conducted The corresponding user study application software was also developed Key words tabletop system infrared tracking TUIO protocol user study Handledare Morten Fjeld Amnesgranskare Bengt Sandblad Examinator Anders jansson IT 10 024 Tryckt av Reprocentralen ITC Acknowledgement My deepest gratitude goes first and foremost to Morten my dear supervisor He gave me the opportunity to finish my master thesis in t2i lab really appreciate his constant encouragement and guidance Second must send my heartfelt gratitude to Ramon and Thomas Without their help it is almost mission impossible for Anders and me to recreate Qualitrack also must say thanks to my lab colleagues Tommaso Holger and Sandra as well They gave me a lot valuable suggestions and help And Yang Song gave me quite a lot useful suggestion on the user study data analysis in statistics Friendship never dies must show my warmest thanks to all the volunteer participants involved in the user study experiment really appreciate their positive altitude and valuable time The text of Chapter 4 1 on the paper is also appeared in the Master Thesis of Anders Rudback since we worked together on this project focused on software development and he focused on hardware development Without his dedicated work cant finish this project along absolutely He is my best Swedish friend The last thanks w
32. information Under it there is the task instruction section The user study task instructions and attentions are showed here The following is the OK button After the participants finishing their work they can shift the infrared pen into mouse mode and use it to press the OK button Then the program will save their work as a JPEG format picture into the hard disc driver in the host computer automatically There is also another way to save the user work by press the Enter key in the key board The last section is the status section After the participants save their work the program will notice them in the status section saying that their work has already been saved in the hard disc driver In the work area the participants are demonstrated with two sentences Both of them are l love Oualitrack but one is ligatures the other is written by block s letter The participants can write their sentences underneath the sample sentences with the infrared pen If the user made some big mistakes they can press F3 to undo one step of their operation or press F4 to clean the whole work area By press F1 this interface can be shift between the window mode and the full screen mode Figure 5 3 2 1 2 is the graphic user interface for the drawing task Deen work pA pens EB L Tl glir 01T ali et Fur P ee Pa pn Dez it raho Figure 5 3 2 1 2 user work interface of user study application for the drawing task 30 It is similar with the one for th
33. irst two sections of the on chip flash must be performed as follows Power down the development board Close the Jumper TST Power up the development board Wait 10s Power down the development board again Remove the J umper TST Power up the development board According to our experience it is essential to unplug the J Tag from the Olimax Sam7 p256 development board when carry out the above recovery procedure When the Olimax Sam7 p256 development board is connected to the host computer at the first time the operating system requires a configuration file to identify and initialize it The initialization file SynchUnitDriver ini is modified from a FreeRTOS demo application 4 2 2 Camtracker The Camtracker program is the core software of Qualitrack system It takes charge of starting up the tracking system both hardware and software receiving input data from the SynchUnit identifying the infrared input devices generating and forwarding TUIO packages Figure 4 2 2 1 is the structure frame chart of Camtracker program Camtraker Program be la Ident Event PAA Figure 4 2 2 1 structure frame chart of Camtracker program The Camtracker program can be divided into three main modules Controller modular controls the initialization and configuration of the whole system Device Identifier modular identifies the input devices updates the devices status and generates corresponding event Eve
34. it which generates the input data such as infrared pen In some multi tangible implementation like NUI Group s Community Earth even human finger tip can be used as input device Tracking unit which is the core of a visual based system Usually a digital camera is used to capture the physical input signal such as infrared light The input data is processed by a host computer And the output data is forwarded to the display unit Display unit which show the static background images and dynamitic output images on the table surface for instance a projector Tabletop system involves the thinking of Augmented Reality AR For quite a long time people paid too much attention and effort to Virtual Reality VR We tried to move everything in our real world into computer screen Without going into the social the physiology problems generated by VR at least nowadays computer technology is still not advanced enough to make people feel natural in most of the VR environment For example drawing with a mouse is definitely not a good experience However AR supplements the reality rather than completely replacing it 19 The idea of AR is merge the virtual environment and real world environment together organically Let s still take the above simple example People will feel much natural if he or she drawing on a digital table with an infrared pen instead of using a mouse drawing on a mouse pad 2 2 TUIO TUIO Protocol is specifically design
35. king of High Speed Movements in the Context of a Table Tennis Application VRST 2005 ACM Press 2005 8 Chia Shen Fr d ric D Vernier Clifton Forlines and Meredith Ringel DiamondSpin An Extensible Toolkit for Around the Table I nteraction systems Proceedings of the SIGCHI conference on Human factors in computing systems Volume 6 Number 1 2004 9 Ronald T Azuma A Survey of Augmented Reality Teleoperators and Virtual Environments 6 4 pp 355 385 1997 10 Martin Kaltenbrunner Ross Bencina reacTl Vision A Computer Vision Framework for Table Based Tangible Interaction Proceedings of the lst international conference on Tangible and embedded interaction session Integrating the virtual and the physical pp 69 74 2007 43 11 Osborn Alex F Applied imagination Principles and procedures of creative problem solving Third Revised Edition New York Charles Scribner s sons 1963 12 Phil Bartle Brainstorming procedures and process http www scn org cmp modules brn sto htm 2010 13 Ramage Magnus Evaluation of cooperative systems project Ph D thesis Chapter 2 1 1 1999 14 Alan Dix Janet Finlay Gregory D Abowd Russelll Beale Human Computer Interaction Third Edition pp 359 2004 15 Charles W Krueger Software Reuse ACM Computing Surveys CSUR Volume 24 pp 131 183 1992 16 Preece J Rogers Y Sharp Interaction Design beyond human
36. nfoTablel identifies devices by marker And pHotOluck distinguish input devices by color Regrettably there isn t any system can provide reliable device identification with high speed tracking update rate Most of the visual based tabletop system only can works at a low tracking refresh rate For example InfrActable system can identify seven input devices at the same time but its final update rate is only 20Hz which is too low Simulated Table Top Tennis is a system can work at 60Hz Compare with the other existing system Qualitrack can support five different input devices working in parallel with tracking update rate up to 250Hz which is rather higher than other similar systems 1 1 Research Aims and Questions This project was driven by three goals The first one is to set up the platform of Qualitrack system implement all the necessary devices and develop the corresponding application software The second goal is to develop a new input device for Qualitrack system The last goal is to design and carry out a user study experiment so as to analyze the influence of the tracking update rate on 2 the user interaction performance with Qualitrack system Around these goals the work package of this project was divided into three sections and each section can be refined into two or three subsections 1 Set up the Qualitrack system n Implement hardware and software n Integrate hardware and software 2 Develop new input de
37. nt Handler modular handles all kinds of events come form Device Identifier modular 4 2 2 1 Serial Communication SynchUnitConnector class configures the IR camera property and the tracking update rate of the whole tracking system in its I nit method After initialization it creates a thread to receive camera data from the SynchUnit and forward these data to Deviceldentifier class This class is based on RXTX to achieve serial communication functionality and implements the SerialPortEventListener interface so as to monitor input data stream There are two center methods in this class Init method and serialEvent method The Init method takes charges of the initialization issues Figure 4 2 2 1 1 is the flow diagram of the init method To Initialize and configure the communication between the SynchUnit hardware and the host computer the SynchUnitConnector object set up the serial communication ports and corresponding communication port properties such as baud rate and parity check bits in init method And then it mounts RXTX serial event listener to receive input data and starts a thread loop to forward the arriveds data to Devicel dentifier objects In the end it triggers the SynchUnit hardware and the infrared camera up 16 C sen gt Generate Portldentifier and SerialPort Set Port Properties Add SerialEventl istener CrTrenerate InputStream and OutPutSstream Start
38. o TUIO package and forwarding the TUIO packages to TUIO clients The writing and drawing task user study application was programmed by Java The Air Hockey Game task user study application was programmed by C Both the two user study applications communicate with the camtracker program with TUIO protocol 5 3 2 1 Writing Drawing Application Program This is a Java application for the writing and drawing tasks of the user study experiment Java Swing is used to build the graphic user interface There are three graphic user interfaces in this application program The first one is used for the writing task like figure 5 3 2 1 1 l System Instruction o switch to full screen press F1 o grade user work press F7 o undo press F3 o clean work area press F4 o save user work press Enter or click OK buttion o exit press ESC a Task Instraction rite the first sentence on the two lines underneath it in the same way that you would write a personal note on a piece of paper Then write the second senten on UG Mes undemeath it in black let love Qualitrack Figure 5 3 2 1 1 user work interface of user study application for the writing task 29 This user interface can be divided into two mainly parts In the left there is the information area In the right there is the work area The information area can be divided into four sections The first section is the system instruction section which shows the software help
39. o get familiar with the system and the task The second session is the experiment session Originally the system tracking update rate can be adjusted from 1Hz the ideal lower boundary although this value points is no meaning in practice to 250Hz the physic limitation of the SAM7 P256 microcontroller In our user study experiment we bound the range from 100Hz to 250Hz with the increasing step at 50HZ in every trail Namely we will test 100Hz 150Hz 200Hz and 250Hz So every participant should repeat each task four times After the participants finish the four trials in each task they are required to grade their felling from the following three different aspects system reaction speed system reaction consistency system overall performance The grade is designed from 1 to 5 for all the three aspects 1 means not satisfied at all 2 means not satisfied 3 mean not good and not bad d means satisfied 5 means very satisfied Above all a four fully within participants repeated measures design was used for each task The independent values are system tracking update rate 100Hz 150Hz 200Hz and 250Hz The dependent value is user grading from 1 to 5 Thus for each task we obtain 4 trials with Task 1 4 trials with Task 2 4 trials with Task 3 We assume that the participants use five minutes in each trail averagely then each participant will spend twenty minutes within each task and sixty minutes totally
40. oo long the participants will feel tired and can not concentrate any more 3 2 Lab Experi ment There are several existing evaluation methodologies which can be used to evaluate a computer system such as Heuristic evaluation User testing Field study Interviews amp Questionnaires and so on As one of the most commonly used evaluation approach laboratory experiment is particularly used to collect quantitative data about a single specific factor attempting to screen out other influences Since tracking update rate is the only independent variable in our user study our user study exactly conforms to the single specific factor condition It is common in experimental design where factors such as the quality of the user s performance are used as dependent variables and measured on a quantitative scale The idea is applied in our user study faithfully Our main goal in this project is to set up the Qualitrack system platform so as to support future project student to do a deeper user study with it So our user study can be seen as an early stage of the future evaluation With laboratory experiment method we can involve available users students as our participant to investigate one and only one point of this system within a short time Then we can leave the more complicated issues to the future project student and the real commercial end users Above all lab experiment is the most suitable methodology for user study part in thi
41. ould go to my beloved family for their loving considerations and great confidence in me all through these years Table of Content LL INTRODUCTION iN ricas 1 A SANA AA 2 TE RESEARCH AIMS AND QUESTIONS ss a O id CD so 2 TRE ANG NGA AA 3 LoS DELNITATION le E a Dela a A 4 kA THESIS OUTLINE cha cb a A di daa 4 2 BACKGROUND acusan is iia 6 2 1 VISUAL BASED LAB EOP SA TEM a OA 6 SS OS 7 20 5 A E EERTE EEA ONE T E AA SE GR Y HR 7 3 ME FHODOE OG Y GM A TT 9 SL BRAINS TORM OTT 9 3 N PE RANT Ee i y A Y OTE PER EPO Dd YN oes 10 3 3 SOFTWARE REUSE AND OBJECT ORIENTED 000002060904290 004 442444442404 non SSS NN 10 3 4 PROTOLY PING TTT 11 ASOUATL EE RACK SYSTE UML asada 13 S BA PA 13 c SIGN UN AA AAA AAA ANG 13 a IPU IEN Car 14 AD SDE WARREN rotatoria 15 AA 13 AD ed AN 15 AD 2 Serlal Conmmngnicdtion A O 16 ASIA Ea lA o IRTE AA oat o 18 F22 CAMPEONA OMA amen AL E 19 4 2 2 4Compose and Forward TUTO packard t 21 4 2 3 User Study Applications Writing Drawing App and Air Hockey ADD sss ee 22 42 3 1 Wriitime and Drawins Apple sh naang daan 22 42 32 A1 Hockey applie AN AA ANAN a AA AA ASO 23 4 3 NEW TABLE TOP DEVICE MAGNIFIER AND COLOR FILTER ccececececececectcccecececscectcecucteecees 25 44 MIG ONU LET APPL COLION YY E E N 23 LIN Calor TRAPP OT ON a Y a ANA ON YF eat 26 S USERSTUDY EXPERIMENT VTT 28 5 1 TERMINOLOGY IDAEETNEFION A E A AE EE E Medd E A E AA EES 28 I SS a p Le NC taa A IN 28 R AA 28 SO y 181 OTI
42. ow the tracking update rate influences user experience in Qualitrack system 3 1 Terminology Definition Tracking Update Rate The number of frames which can be captured by Qualitrack system within one second 3 2 Participants Two female and eight male volunteer participants are involved in this user stud y experiment All of them are university students 3 3 Apparatus 5 3 1 Hardware Hardware for the experiment consists of a Infrared pen one infrared LED is mounted at the tip of the IR pen It can be used to write and draw when the tip switch behind the LED is press b Infrared air hockey paddle three infrared LED are mounted in the IR paddle so as to distinguish the orientation of the paddle c Infrared camera ProReflex MCU240 is used in Qualitrack system Its capture update frequency is up to 250Hz d SynchUnit Olimex SAM7 P256 development board with AT91SAM75456 micro controller is used to trigger and synchronies all devices e Host computer u CPU Intel Core2 Duo E8400 3 0GHz u Video Card NVIDIA Geforce 9300GE 256M DDR2 video memory 28 f Projector ProjectionDign FL32 is used Its resolution is up to 1920 x 1200 g Screen Table a horizon table with a 100cm x 75 cm hole in its center A brown optical glass covers the hole as the screen 5 3 2 Software The Java program camtracker was run on the host computer It takes charge of receiving data from SynchUnit processing input data int
43. pplication program were developed for this new input device In the first version of the magnifier the three plastic infrared LEDs scratch the table surface when the user drag it rather than move it That noisy is really annoying and reduce the user experience a lot To solve this problem we attach four soft pads in the four corner of this device to make the LEDs leave the table surface about one millimeter Although the functionality and performance of this device is the same as before the users response enhance a lot just because the improvement of thus a small hardware design detail Most users thought the magnifier is really useful and natural We expect the response from areal color blind person in the future And we will try to find more proper material to build up the other input device so that the users can feel more comfortable working with our system 6 3 Discussion on User Study Experi ment The last goal of this project is to carry out a user study experiment so as to discover the influence of the system tracking update rate on user experience with Qualitrack system The winner of both drawing and writing tasks is 250Hz In wringing task Qualitrack system only got 3 6 score on over all user experience in the best condition The highest user over all experience grades was generated in drawing task but still only 4 Namely Qualitrack system still has a long distance to be 39 perfect According to the result of this user s
44. quation 1 and 2 we will get the point coordination X Y in the display coordination system 20 4 2 2 4 Compose and Forward TUIO package Since the usual display refresh rate of a screen or a projector is 60Hz while the tracking update rate of Qualitrack system is rather high than that if we forward the input data directly to the display application it will waste a lot of input data due to the low display refresh rate of the display equipment In this project we optimized Qualitrack system by using two successive input data to calculate a middle point so as to make the system more accurate Figure 4 2 2 4 1 is the middle point calculation algorithm Start gt N PrePoint exist calculate middle point PrePoint CurPoint v compose TUIO package v send TUIO package Figure 4 2 2 4 1 compose TUIO package with middle points If the previous point is not exist or the session ID of previous point is not the same with the current point it means the current point is the first point in the current session Then it will be composed into TUIO package and sent to the TUIO application directly If there is a matched previous point then the information of previous point and the current point such as position coordination speed acceleration and so on will be used to calculate a middle point between the previous point and the current point This middle point
45. rebounded Both the x and y speed of the ball will be reversed If the position of the ball is inside the goal gate it means there is a goal Then the player will get a score No mater the player gets a score or the ball flies out of boundary the current round will be finished immediately If the round counter is greater than ten rounds game is over Otherwise a new round will start 24 4 3 New Table Top Device Magnifier and Color Filter There are only two infrared devices exist in Qualitrack system by the time we took over it They are the infrared pen and the infrared handle To enrich Qualitrack system we developed two new devices for Qualitrack They are the magnifier and the color filter The magnifier is used to enlarge the image The color filter is a special visual assistant tool designed for the colorblind people Both of the two devices are based on the same hardware But they were design and implemented with two different software solutions 4 3 1 Magnifier Application The Magnifier Application is developed by java It is designed as an independent java component so that can be reused in other applications Figure 4 3 1 1 Shows its business logic Start A Yes Take Screen Snapshot Release Resources Get Device Location End Cut Screen Snapshot Enlarge Snapshot Display Snapshot Figure 4 3 1 1 magnifier application algorithm
46. ree task user study experiment was design and processed to find out the influence on the user experience by the system tracking update rate Ten volunteer participants are involved in this user study From the experiment data we get the following conclusions 1 System performance can be enhanced obviously along with increasing the tracking update rate in a table top system This conclusion is more obvious in low update rate range than the high update rate range 2 Itrequires more critical system performance in a digital pen based table top system which aiming at writing operation than drawing Moreover from this project we learned that user experience should be kept in mind through the whole process of designing implementing and evaluating a computer system Due to our technique education background we usually didn t think over this aspect enough before we taken this project Think much of user experience is the most valuable knowledge we got form this project and we will got the benefit from it our whole life 41 9 Future Work 8 1 About Qualitrack System 1 Formulate the bit code algorithm in a mathematical way 2 Redesign the arrangement of Qualitrack system so as to enhance the system s accuracy and sensitivity 3 Use more powerful electronic development board as the synchUnit in order to enhance the system tracking update rate 4 Create more input devices 5 Develop communication functionality so that to suppo
47. rt remote collaboration work 6 2 About User Study 1 Enlarge user study participant group size 2 Evolve more Input devices into the user study 3 Carry out the same user study experiment on other high speed table top system so as to compare the defenses between similar systems 4 Analysis the participants behavior from psychology point of view 42 Reference 1 R Hofer T Nescher A Kunz QualiTrack Highspeed TUI Tracking for Tabletop Applications Interact 2009 Part Il LNCS 5727 pp 332 335 2009 2 C Ganser Schwab A Steinemann R Hofer A Kunz InfrActables Supporting Collocated Group Work by Combining Pen Based and Tangible Interaction Proceedings of Tabletop 2007 2007 3 Thomas Nescher Hi speed IR tracking of tabletop interaction devices Internship report http www t2i se pub papers Thomas_ Nescher Internship pdf 2007 4 Wellner P Interacting with Paper on the DigitalDesk Communications of the ACM 36 7 1993 5 Rekimoto J Saitoh M Augmented Surfaces A Spatially Continuous Work Space for Hybrid Computing Environments Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit 1999 6 Nishimoto K Amano K Usuki M pHotOluck A Home use Table ware to Vitalize Communication at Mealtimes by Projecting Photos onto Dishes TableTop pp 9 16 2006 7 Rusdorf S Brunnett G Real Time Trac
48. s generate after a unbearable delay then the system reaction should be grade to 1 If the scratch is a smooth and continuous line then the system reaction consistency is 5 If it is a sharp poly line then the system reaction consistency is 1 The last but not the least the user is asked to give the overall grade to their experience of the system s entire performance 5 4 2 Drawing Task A simple picture is projected at the sample area of the drawing task user study application The participant is asked to draw it at the working area with the infrared pen After finish drawing the participant is asked to switch the infrared pen to the mouse mode and use it to press the OK button Then the participant s work will be saved in the hard disc driver of the host computer automatically In the new trail the system will be set at another testing tracking update rate randomly without telling the user which update rate it will work at in case that the participant will predict the system performance After the participant finishes all the four trials by press F2 the grading user interface is launched The participant s work will be load into the screen in parallel so as to the participant can grade them The participant grade the their feeling on the system reaction speed system reaction consistency and system overall performance from 1 to 5 The meaning of the three questions is the same with the writing task 34 5 4 3 Air Hockey Game
49. s project 3 3 Software Reuse and Object Oriented Software reuse is the process of creating software systems from existing software rather than building software systems from scratch 15 This concept is first proposed at the NATO software engineering conference in 1968 Nowadays the concept of software reuse is been extended from code reuse to design reuse analysis reuse document reuse and so on The basic idea of software reuse is that form the software by some functionality independent components Under this idea all the components must be designed as universal tool for similar work And Then the procedure of software development will become as simple as playing toy brick Namely to develop software the developers just need to assemble different components Above all software reuse is one of the goals in software engineering But there are several factors hinder we achieve this goal First of all it almost impossible to find a coincided software component since specific project usually 10 has special requirements The second obstructive factor is human factors Different programmer has different developing habits Sometimes it is even harder to modify other people s software to fit for our need than developing a new one by ourselves Thirdly some people hold the opinion that reuse code or component is not really creative work The last but not the least factor is there is few course and textbook mention software reuse in compu
50. t Additional Library Directories set corresponding lib directories Project gt Properties gt Linker gt Input gt Additional Dependencies set corresponding library files 50
51. ter science education Object Oriented OO is the one of the major methodologies in modern software development It is very hard to give the definition of object oriented since nowadays OO is not only be limited to Object Oriented Programming OOP but also been extended to several different lays such as Object Oriented Analysis OOA and Object Oriented Design OOD OO method provides a strong technical support for software reuse It can prevent software development from meaningless duplication solutions Thus reduces the financial and human resource cost in software development In this project we tried our best to reuse existing framework and program components so as to reduce the time cost and enhance the code quality Two popular object oriented programming language Java and C were used in this project And OpenFrameWorks architecture was applied in the air hockey application program The writing and drawing task user study program was developed based on slandered TUI O client application 3 4 Prototyping Prototyping is a limited representation of a design that allows user to interact with it and to explore its suitability In fact people use prototyping method unconsciously every day The most typical example of prototyping is sketching on a piece of paper when one illustrates his or her idea to others In software engineering we usually use computer based prototyping method so Called high fidelity prototyping instead of
52. the average grade in writing task In writing task Qualitrack system still got the best user experience at 250Hz on all the three factors But compare with the drawing task the average user grade 36 is lower down in all the three testing conditions In air hockey game task the differences between system tracking update rate were not statistically significant for system reaction speed F 36 1 9216 ns for system consistency F3 36 1 8328 ns and for over all grade F3 36 1 4019 ns It means that the differences between the four levels of system tracking update rate in user grade for each factors are due to chance So there is no meaning to discuss the result of the air hockey game task here Figure 5 5 3 shows the comparison between drawing and writing tasks 4 U i 4 5 38 F 3 6 E 3 4 F 3 d d 3 drawing writing speed 4 3 5 consistency 3 9 3 1 ces over al 4 3 6 task Figure 5 5 3 curve chart of the comparison among the task winners From the above curve chart we can see that Oualitrack system got obvious higher user experience grade on system reaction speed system consistency and over all user experience factors in drawing task than writing On the contrast we can discover that the participants gave Oualitrack system the lowest grade on all the three factors in writing task especially on system consistency feature Namely the result of the above comparison suggests that Qualitrack is good at
53. track system in general The research questions of this project are defined and delimitations are described Chapter 2 Background renders the background knowledge of the project Necessary relative concepts are defined Chapter 3 Method covers the engineering and experimental methodologies adopted to carry out this project Chapter 4 Qualitrack System provides the technical details of the Qualitrack system elaborates the design and implementation of the new magnifier and color filter devices Illustrates the design and implementation of the user study application software Chapter 5 User Study Experiment describes the user study experimental 4 design and analysis of the experimental data Chapter 6 Discussion discusses the problems in this project and explains the result of the user study experiment Chapter 7 Conclusion concludes the observations and results Chapter 8 Future work shows some topics for future research 2 Background 2 1 Visual Based Tabletop System A tabletop system supports a digital horizon surface which a group of people can work around in collaboration Table top user interfaces can preserve many of the familiar and useful properties a physical tabletop affords and allow the natural interaction Figure 2 1 1 shows an example of tabletop system Figure 2 1 1 a collaborate working around a digital table A typical visual based tabletop system usually contains three parts Input un
54. tudy experiment Qualitrack system needs to further enhance its tracking update rate At the mean while as we discussed in chapter 6 1 2 the arrangement of Qualitrack needs to be redesigned 6 3 1 Writing VS Drawing From figure 5 5 3 we can discover that Qualtrack system performed worse in writing task than drawing task intuitively There are two reasons can explain this phenomenon First of all when the participants wrote their scratch was shorter than drawing In other words the tip of the infrared pen left the table surface more frequently in writing task Whenever the tip of the infrared pen touched or left the table surface Qualitrack need eight frames to recognize this state change So the system tracking update rate is decreased eight times during this procedure Secondly due to the participants are more familiar with writing than drawing in their daily life their scratching speed is faster in writing task than drawing task As a result of the above two reasons it requires more critical system response time and scratch consistency performance when design a digital pen based table top system which aimed at writing operation 6 3 2 Statistic Explanation on Air Hockey Game Task In air hockey game task our null hypothesis HO was There is no difference between the four levels of system tracking rate in system reaction speed system consistency and user over all experience grade of which formal express as HO X1 X2 X3 X4
55. vice n Design and implement hardware and software n Merge the new device into Qualitrack system 3 Carry out user study experiment n Design user study experiment n Develop user study application software n Carry out user study and analyze data 1 2 Research Process This project was divided into 4 sessions Session 1 Literary study During this session we got familiar with Qualitrack system by reading old documents And due to none of us study exactly human computer interaction HCI one of us studies Computer Science and the other s major is Mechantronics we had to read some materials and relative papers in this research field so as to get a clear sketch of HCI and distinguish the similarities and differences between HCI system development and usual computer system development Session 2 Project design In this session we delimited the work package which is mentioned in Chapter 1 1 and design the user study experiment Brainstorming offered us quite a lot inspiration in our group meeting And we decided to use lab experiment method in our user study due to its controllability feature and our technology education background Session 3 System development Due to this system was put on the shelf for about two years all the hardware devices had lost or been damaged we have to rebuild all the hardware devices And we must develop almost all the application software by ourselves under the situation that lack of relevant specification
56. will be assembled into TUIO package and sent to the TUIO application At the same time the previous point will be refreshed by the current point 21 4 2 3 User Study Applications Writing Drawing App and Air Hockey App 4 2 3 1 Writing and Drawing Application The writing and drawing application is designed for the infrared pen This application is built based on the TUIO reference implementation which is part of the reacTI Vision framework Its graphic user interface is developed with Java Swing The most important part of this program is called work area component It is a Java Swing component extends from Java Component class and it is a TUIO client component which takes charge of receiving and parsing TUIO package After receiving the TUIO package from the camtracker program the work area component decode the TUIO package and update the screen Figure 4 2 3 1 1 is the data flow chart of the work area component A Receive TUIO package bundle Yes No No Yes Is TUIO cursor AT NR with obj package Yes No No No Is set message s frame message Yes Parse set message peat Ne a UES ane Add lt Remove No No No Yes Yes Is new session C y Add obj Remove obj Update obj Create new object Update object lt 4 Figure 4 2 3 1 1 TUIO client component data flow chart After the TU
57. ysis Appendix 1 is raw data of the user study experiment In drawing task the differences between system tracking update rate were Statistically significant for system reaction speed F 36 16 610 P lt 001 for system consistency F3 36 10 987 P lt 001 and for over all grade F3 36 11 095 P lt 001 Figure 5 5 1 shows the bar chart of the average grade in the drawing task 32 Fin KA i Sa 5 4 e J 2 5 E io 7 5 0 GHZ Ocansistency NG 3 3 7 EH cwer al 227 29 3 4 system tracking update rate Figure 5 5 1 bar chart of the average grade in drawing task In drawing task Qualitrack system got the highest user grade on system reaction speed system consistency and over all user experience at 250Hz But from the above bar chart we can discover that the user grade gap between 200Hz and 250Hz is not so obvious than the one between 150Hz and 200Hz Namely the user experience enhance a lot over 200Hz in drawing task In writing task the differences between system tracking update rate were Statistically significant for system reaction speed F3 36 7 0548 P lt 001 for system consistency Fa ae 5 122 P lt 005 and for over all grade Fa 36 9 321 P lt 001 Figure 5 5 2 shows the bar chart of the average grade in the writing task sag PI Rar eed Geansistency Boer all c 1 8 27 3 3 1 system tracking update rate Figure 5 5 2 bar chart of
Download Pdf Manuals
Related Search