1 - Cryptography and Cognitive Informatics Research Group
Contents
1. demonstrates cooperation of GDL Gesture Description Language with Kinect controller and Mirosoft Kinect SDK This document describes only this application Manual for GDL can be found elswhere 1 Gesture Description Language Studio v1 05 is an application written in C Windows Presentation Foundation technology that gives an access to RGB and depth stream of Kinect for Xbox and Kinect for Windows controller It also enables user s body segmentation body joints tracking body joints recording and gestures recognition in real time online and from recordings off line Body joints recordings are stored in text skl files that can be easily reused by other applications The gestures recognition is made with GDL classifier that is described elsewhere 1 Gesture Description Language Studio v1 05 requires third party software and libraries to be successful installed The installation process is described in next paragraph Before you decide to use the application in your project READ THE LICENSE it is available at the end of this file Long story short this application is for scientific educational purpose only It cannot be used commercially If the software is used in other software non commercial products installations or for research it must always be cited as GDL and a reference to the Tomasz Hachaj Marek R Ogiela Rule based approach to recognizing human body poses and gestures in real time Multimedia Systems February2. gesture insensitive to user rotation Psilnsensitive and sensitive to rotation Psi Waving Waving is waving with right hand the right hand joint should be more or less at the same distance to Kinect as head joint Select hold your right hand for four seconds without movement at least 20 cm in front of body Then to disable select move your hand closer to the body Drag Drag when select is satisfied you can move your hand Zoom in ZoomIn put both your hands in front of your body at least 20 cm and spread them Zoom out ZoomOut put both your hands in front of your body at least 20 cm and shrink distance between them Tomasz Hachaj CCI Research Group COMMERCIAL LICENCES SOURCES FUTHURE PLANS It is possible to obtain commercial license for Gesture Description Language Studio v1 05 and stand alone GDL classifier library for Kinect SDK In order to do so please contact me by e mail tomekhachaj o2 pl The sources of this application has been obfuscated to prevent or make more difficult reverse engineering of it However we are planning to open the source of Gesture Description Language Studio v1 05 in few months so stay in tuned The technology is continuously developed and we are testing and producing improvements of both Gesture Description Language Studio and GDL classifier itself If you are interested in our solutions stay in tuned and read our papers Gesture Description La
3. 1 Output of GDL classification results If the GDLsl file was successfully loaded in this area conclusions of the rules names that are satisfied will be displayed The conclusions will be presented one below another and it is possible that not everyone will be fitted into screen If in the particular moment no skl recording is played the GDL reasoning is performed online basing on current skeleton stream If the skl recording is played the GDL reasoning is performed offline basing on skeleton stream from recording 2 Panel that displays a skeleton If no recording is loaded in this window the current skeleton tracking data is displayed If skl recording is loaded the panel presents skeleton from recording Double click with left mouse button maximize this window or if it is already maximize returns it to original size In order to move the window click and hold left mouse button and change the position of window by dragging it 3 Slider is use to navigate the skl recording Button Record begins the capturing of current skeleton stream into new skl file The new recording replace old recording in application however it creates new file with unique name on disk that does not override any existing file Button Play plays the skl recording that is currently in memory Button Pause is not supported Gesture Description Language Studio v1 05 Button Stop stops the recording and saves the captured data to disk into folder wher
4. 2014 Volume 20 Issue 1 pp 81 99 has to be given This form of citation does not require any permission This software has been tested by many our students but IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT INDIRECT INCIDENTAL SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES etc If you have some questions about application itself GDL methodology licensing scientific and commercial cooperation etc feel free to contact my e mail address tomekhachaj o2 pl Good luck and have fun Tomasz Hachaj Tomasz Hachaj CCI Research Group REQUIREMENTS AND INSTALLATION Requirements OS Windows 7 Home Premium Service Pack 1 Processor 3 4 GHZ Intel Core 15 2320 CPU 3 00 GHz or similar System 64bit Memory RAM 4 GB Microsoft Kinect for Xbox 360 or Kinect for Windows Microsoft Net Framework 4 0 Microsoft Kinect for Windows SDK v1 7 or v1 8 To run the application copy files Gesture Description Language Studio v1 05 exe and alut tif to the same folder and run exe file Any further installation of Gesture Description Language Studio v1 05 is not required ACRONYMS LIST GDL gesture description language technology GDLsl or GDLs gesture script language a formal grammar that is used for definition of GDL rules Kinect Xbox 360 Kinect Sensor or Kinect for Windows Sensor Application Gesture Description Language Studio v1 05 Skl skeleton f
5. OSSIBILITY OF SUCH DAMAGE
6. Tomasz Hachaj Version 1 05 2014 Kinect SDK 1 7 Kinect for Xbox Windows Gesture Description Language Studio v1 05 Author Tomasz Hachaj PhD Title Gesture Description Language Studio v1 05 User s Manual and License Version 1 05 2014 Kinect for Xbox Windows CCI Research Group Affiliation The Laboratory of Cryptography and Cognitive Informatics Pedagogical University of Krakow 2 Podchorazych Ave 30 084 Krakow Poland tomekhachaj o2 pl http www youtube com ccilaboratory http www cci up krakow pl http www cci up krakow pl gdl Krak w 2014 Tomasz Hachaj CCI Research Group Table of Contents INTRODUCTION siii ide 3 REQUIREMENTS AND INSTALLATION esse sees sesse se se se ee ee ee ee es ee se ee ee ee ee ee ee ee Re Re ee ee ee 4 ACRONYMS DIST sat ee ee eeu teria Fd CT 4 INTERFACE ME O Y ads 5 A SELECT DEVICE WINDOW a ee ee ee se ee ee ee ee ee ee ee LL A ee ee ee ee ee LLA Y Y Y ee e 5 B MAIN WINDOW ee ee ee ee ee ee ee a ee ee ee ee ee ee ee ee ee ee DL FL aeaa diarea 6 C RECORDER AND PLAYER WINDOW ees ee ee ee ee ee ee se se ee ee ee ee ee ee wawa ee ee ee ee ee FF ee 8 GDECLASSTETER iii dis 10 GDESTEXAMPEES is ee n se ee adds 11 REFERENCES AND USEFULL LINKS ese ee ee ese ee ee se ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee ee 13 COPYRIGHT NOTICE A Au akon FF tt 14 Gesture Description Language Studio v1 05 INTRODUCTION This application
7. Windows website http www microsoft com en us kinectforwindows Drivers for Kinect SDK installation of those drivers are required http www microsoft com en us kinectforwindowsdev Downloads aspx The Laboratory of Cryptography and Cognitive Informatics official website http www cci up krakow pl GDL project official website http www cci up krakow pl gdl Tomasz Hachaj CCI Research Group COPYRIGHT NOTICE Copyright c 2014 Tomasz Hachaj The Laboratory of Cryptography and Cognitive Informatics tomekhachaj o2 pl http www cci up krakow pl http www cci up krakow pl gdl AII rights reserved This product includes software developed by Tomasz Hachaj for use in the Gesture Description Language Studio v1 05 application called later Software The implementation is based on scientific paper Tomasz Hachaj Marek R Ogiela Rule based approach to recognizing human body poses and gestures in real time Multimedia Systems February 2014 Volume 20 Issue 1 pp 81 99 Redistribution and use in binary forms without modification are permitted provided that the following conditions are met e Redistributions in binary form must reproduce the above copyright notice this list of conditions and the following disclaimer in the documentation and or other materials provided with the distribution e Redistributions of source code of Software are forbidden e You may not reverse engineer decompile or disassemble th
8. cci up krakow pl www cci up krakow pl gdl Seated Mode All rights reserved The Main Window of application is consisted of following components 1 View from RGB Kinect s camera If program is connected to controller this panel is an output of RGB stream In top left part of this panel an estimated frame capture speed is displayed measured in frames per seconds fps The checkbox below the RGB image enables or disables RGB stream capturing 2 View from processed IR Kinect s camera If program is connected to controller this panel is an output of depth stream Images are colored in order to simplify the perception of data The objects that near are blue further objects are green yellow and red the color pattern in look up table is in below image In top left part of this panel an estimated frame capture speed is displayed measured in frames per seconds fps The checkbox below the Depth image enables or disables Depth stream capturing 3 View from Skeleton stream If program is connected to controller this panel is an output of Skeleton stream The skeleton tracking is based on Kinect SDK algorithm and has its features and limitation In top left part of this panel an estimated frame capture speed is displayed measured in frames per seconds fps The checkbox below the Skeleton image enables or disables Skeleton stream capturing 4 GDL library control panel Button Load features and rules run window in whic
9. e Software e This Software can be used only for scientific or and educational purpose It cannot be used commercially e Neither the name of the Software nor the names of its contributors may be used to endorse or promote products derived from this Software without specific prior written permission e If the Software is used in other software products installations or for research it must always be cited as GDL and a reference to the Tomasz Hachaj Marek R Ogiela Rule based approach to recognizing human body poses and gestures in real time Multimedia Systems February 2014 Volume 20 Issue 1 pp 81 99 has to be given This form of citation does not require any permission THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS AS IS AND ANY EXPRESS OR IMPLIED WARRANTIES INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT INDIRECT INCIDENTAL SPECIAL EXEMPLARY OR CONSEQUENTIAL DAMAGES INCLUDING BUT NOT LIMITED TO PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES LOSS OF USE DATA OR PROFITS OR BUSINESS INTERRUPTION HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY WHETHER IN CONTRACT STRICT Gesture Description Language Studio v1 05 LIABILITY OR TORT INCLUDING NEGLIGENCE OR OTHERWISE ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE EVEN IF ADVISED OF THE P
10. e application exe file is localized This button is also used to stop playing of skl recording Button Skeleton file opens the window in which user can choose the skl file to load Button Reset recording removes from application memory skl recording Check box S subrules 1f it is checked in GDL output 1 all satisfied conclusions names are displayed If it is unchecked only those conclusions are displayed that contains exclamation in their names Tomasz Hachaj CCI Research Group GDL CLASSIFIER The GDL classifier s technology has been described in our paper Tomasz Hachaj Marek R Ogiela Rule based approach to recognizing human body poses and gestures in real time Multimedia Systems February 2014 Volume 20 Issue 1 pp 81 99 In GDL implementation that we are using in Gesture Description Language Studio v1 05 the following changes has been made Kinect SDK Skeleton HandLeft Hess R ea ght A d da a Y Wri ShoulderCenter WristRight X p HipLeft ipRi Z KneeLeft Coordinate system The old NITE OpenNI like body joint set BodyParts Head Neck LeftShoulder RightShoulder Torso LeftElbow LeftHand RightElbow RightHand LeftHip RightHip LeftKnee RightKnee LeftFoot RightFoot xl yl z 10 BodyParts3 D Head Neck LeftShoulder RightShoulder Torso LeftElbow LeftHand RightElbow RightHand LeftHip RightHip LeftKnee RightKnee LeftFoot RightFoot xyz Ha
11. h user chooses GDLsl file to be loaded into memory of the application If the file contains errors is not correct GDLsl file application will returns errors The error message is an exception of GDL parser Gesture Description Language Studio v1 05 instance If the application returns no error the file if correctly loaded into memory From this moment the application begins to classify user s gestures using his or hers tracking skeleton and GDL classifier If the new GDLsl file is loaded it overrides the previous file Button Show features and rules displays window in which a parsed GDLsl rules tree is presented If no GDLsl file is loaded clicking on button has no effect 5 Device control panel Combo box element contains three values RGB Depth None Choosing one option changes background of 3 in Main Window and background of 2 in Recorder and Player Window The background might be view from RGB stream processed view from Depth stream or black background Slider steers the position of Kinect sensor tilt up and down Button Apply approve the changes made with slider Przycisk Reset tilt powraca do pozycji w kt rej Kinect filmuje r wnolegle do jego podstawy Check box Seated mode enables or disables Kinect SDK seated mode Tomasz Hachaj CCI Research Group C RECORDER AND PLAYER WINDOW GDL output Record Play Pause Skeleton file Reset recording S subrules 4
12. ile format that contains captured skeleton stream data Skl files are text files but the specification of that data format is currently not open CCI The Laboratory of Cryptography and Cognitive Informatics of Pedagogical University of Krakow Gesture Description Language Studio v1 05 INTERFACE The application is consisted of several windows A SELECT DEVICE WINDOW aj Select Kinect Device Kinect no 0 None After running the application a Select Kinect Device window appears You have to pick from list one of the positions and click Connect button All Kinects that are connected to computer are listed as Kinect no 0 Kinect no 1 etc You can connect only to one device Option None means that you will work offline without a data stream attached In this case you cannot observe live video from devices however you can play skl recordings and make offline GDL recognition of skl data In order to change you selection you have to close it and rerun the application The application uses RGB and Depth stream Tomasz Hachaj CCI Research Group B MAIN WINDOW r Baj Gesture Description Language Studio v1 0 KINECT v RGB stream V Depth stream Y Skeleton stream GDL Device About Load features and rules 4 None 5 Copyright c 2013 Tomasz Hachaj Na The Laboratory of Cryptography and Show features and rules Cognitive Informatics 5 tomekhachaj o2 pl Apply Reset tilt www
13. nguage Studio v1 05 REFERENCES AND USEFULL LINKS 1 Tomasz Hachaj Marek R Ogiela Rule based approach to recognizing human body 2 3 4 5 6 7 8 9 10 poses and gestures in real time Multimedia Systems February 2014 Volume 20 Issue 1 pp 81 99 Tomasz Hachaj Marek R Ogiela Nowadays and future computer application in medicine IT CoNvergence PRActice INPRA volume 1 number 1 2013 pp 13 27 Tomasz Hachaj Marek R Ogiela Semantic Description and Recognition of Human Body Poses and Movement Sequences with Gesture Description Language Computer Applications for Bio technology Multimedia and Ubiquitous City Communications in Computer and Information Science 2012 pp 1 8 Tomasz Hachaj Marek R Ogiela Computer Karate Trainer in Tasks of Personal and Homeland Security Defense A Cuzzocrea et al Eds CD ARES 2013 Workshops LNCS 8128 pp 430 441 2013 Tomasz Hachaj Marek R Ogiela Marcin Piekarczyk Dependence of Kinect sensors number and position on gestures recognition with Gesture Description Language semantic classifier Preprints of the Federated Conference on Computer Science and Information Systems 2013 pp 575 579 Tomasz Hachaj Marek R Ogiela Marcin Piekarczyk Real Time Recognition of Selected Karate Techniques Using GDL Approach Image Processing and Communications Challenges 5 Advances in Intelligent Systems and Computing Volume 233 2014 pp 99 106 Official Kinect for
14. s been replaced by new Kinect SDK like body joint set BodyParts HipCenter Spine ShoulderCenter Head ShoulderLeft ElbowLeft WristLeft HandLeft ShoulderRight ElbowRight WristRight HandRight HipLeft KneeLeft AnkleLeft FootLeft HipRight KneeRight AnkleRight FootRight x yl z BodyParts HipCenter Spine ShoulderCenter Head ShoulderLeft ElbowLeft WristLeft HandLeft ShoulderRight ElbowRight WristRight HandRight HipLeft KneeLeft AnkleLeft FootLeft HipRight KneeRight AnkleRight FootRight xyz have to be replaced by new one In case when GDL script language data parsing is unsuccessful the application returns error message that is an exception of GDL parser instance In this message the approximate cause of the error is indicated it is not complete information about the line and position of error Gesture Description Language Studio v1 05 GDLsl EXAMPLES I added to application one example GDLsl file and one skl recording PSI Waving Select Drag Zoom in Zoom out Select and move You can load Example gdl and play Example skl for the gestures demonstration I recommend to disable S subrules option in Recorder and Player window to see only conclusions of main rules You can change and redistributed Example gdl and Example skl files without any limitation PSI is a static gesture that is demonstrated in above figure In example file I did two definitions of this
Download Pdf Manuals
Related Search