Home

Smart Home - Tampereen teknillinen yliopisto

1. An interesting workshop study Green04 set out to discover the kinds of concerns people had about the introduction of smart home technology One primary question concerned cost how much would installation of such equipment cost how much would it increase the price of an apartment or a house how much energy does it use or how much could it 32 Smart Homes save Reliability was also important for the panellists since largely negative experiences of personal computers had made them question the reliability of such technology Backup systems and manual overrides were thus also considered as necessary features Security privacy and safety were all grouped together as the third most important item followed by ease of use and flexibility Simple intuitive easy to use controls should be used and the system should be able to adapt itself to the changing lifestyle of its users Before smart home prototypes are tested with real users some means of prototyping and verifying the concept would be helpful Since AmI environments consist of multiple de vices and UIs it is no longer sufficient to test or commercialise single applications or de vices ISTAG03 Instead the environment has to be considered in its entirety consisting of interoperable architectures and interacting components A method to achieve this is Ex perience Prototyping Buchenau00 which concentrates on users interactions and experi ences rather than technic
2. ogy seem like an obstacle rather than an enabling factor One major element in this problem is compatibility if devices appliances and applications can be made to work to gether this would certainly increase the benefits and make technology more useful easier to use and install In actual fact however there are many levels of compatibility that have to be taken into consideration e Physical Defines how objects fit together for example connectors physical dimen sions how they fit into the user s hand etc e Electrical Operating voltage frequency requirements batteries and other power supplies interference tolerance electromagnetic compatibility EMC e Communication Protocols compatibility of different kinds of signals connectors and cables e Information content Context information semantics ontologies e Social Services social acceptance personal security privacy issues e Psychological Stress burden learnability ambience e Financial Price requirements operating costs In TUT the emphasis of smart home projects has been on the first three items and later expanded to encompass other items as well Since the research has been academic and mostly funded by the department financial considerations production costs marketing standpoint etc have not been considered in greater depth Definitions 2 Definitions This chapter presents various views on smart homes the kinds of interpretations
3. The Smart Home contains numerous networks each with its own purpose and design Kaila07 The primary network is the RS 232 network to which most sensors and actua tors are connected Serial port devices communicate at 19200 bps which is more than enough for the few bytes that sensor data and commands require The wired serial link is designed to be used wherever the infrastructure permits cables and where robustness and dependability is important Because RS 232 is a point to point communication method a way to connect multiple devices to a PC was required and work on a serial network hub was also needed Since the network was a master slave type it was possible to construct a hub that would allow multiple serial ports to be connected to a single port as was done in the Living Room The new serial hub houses 14 serial ports with RJ 12 connectors that can be directly connected to compatible devices in the Living Room One serial port is dedi cated for connection to the server PC and one port is reserved for wireless connections Addressing is done by assigning a unique address to every port and the device automati cally gets the address according to the port it is plugged into The devices themselves have no record of their address the server software keeps a record of all addresses and the serial hub merely ensures that messages are delivered to and from the correct port For wireless RF devices a unique ID number has been programmed into the
4. Computers are One user many ence of the user puting integrated into everywhere around computers everyday objects us Ambient Intelli Anenvironmentthat Seamless comput Support for human gence reacts to the pres ing and communica human interactions tions Pervasive Comput ing Computation and devices are all around us Decentralised com puting diversified networked devices Information is accessible any where through any device Proactive Comput Anticipates user s Constantly monitors The human is taken ble of adapting and reacting to user s actions ing actions acts proac and analyses the out of the control tively user s context and loop learns patterns Intelligent Envi Unencumbered Brings the computer Enabling natural ronments interaction with non into the background forms of interaction computational between humans and objects computers Mediated Spaces Understands and Many sensors and Changes our percep participates in multi processing required tion of computers person interaction to understand inter and the way we use actions them TUT definition A space that is capa Networked devices An adaptive relax sensors actuators and controlling sys tems ing environment which enables straightforward con trol The remaining sections of this chapter present additional terms that are closely related to smart enviro
5. Wide area networks as the name implies cover wide geographic areas larger than cities or communities WANs connect LANs and larger area networks and cross national bound aries as the Internet does for example Typically WANs use telephone lines satellite or microwave communications Local Area Networks LAN LANs cover smaller areas such as homes offices or airports Typically LANs have high data transfer rates and rely completely on internal infrastructure LANs can be connected to other LANs using secure means for example VPN LANs use Ethernet WLAN and they can be connected to WANs using cable modems DSL modems or routers Personal Area Networks PAN PANSs are defined as the communication network in the immediate vicinity of the user within a few metres including information appliances such as PDAs computers and mo bile phones PANs are used for peer to peer communication between devices or for con necting a device to the Internet WPANs Wireless PANs use wireless communications such as Bluetooth or IrDA instead of cables Body Area Networks BAN Body area networks take a step closer to the user Sensors for measuring movement bio logical quantities or similar parameters can be worn or integrated into a device or garment that the user is using Devices communicate through special short range networks usually wirelessly and data is collected into a UI device or stored and transmitted to a base station at a lat
6. Develop ment of a Location Aware Speech Control and Audio Feedback Sys tem in Proceedings of the Seventh Annual IEEE International Conference on Pervasive Computing and Communications PerCom 2009 work in progress report 2009 Galveston Texas USA References Kanter00 Kasten01 Kawai04 KNX Kolic04 Koskela03 Koskela04_1 Koskela04_2 Koskinen03 Leen02 Lehto98 LEM T Kanter Event driven personalizable mobile interactive spaces Lecture Notes in Computer Science Lecture Notes in Computer Science Springer Berlin 2000 pp 55 65 O Kasten M Langheinrich First Experiences with Bluetooth in the Smart Its Distributed Sensor Network Workshop on Ubiqitous Computing and Communication Conference on Parallel Architec tures and Compilation Techniques PACT 2001 10 pages M Kawai I Takekawa Y Wada N Fujino Middleware for ubiq uitous and seamless computing environments Fujitsu Scientific and Technical Journal Vol 40 2004 pp 35 41 KNX Association official website 2009 available at http www knx org R Kolic Ultra Wideband the Next Generation Wireless Connec tion Technology Intel Magazine February March 2004 T Koskela Smart home usability and living experience Master s Thesis Tampere University of Technology Institute of Software Systems Usability Laboratory 2003 105 pages T Koskela K Vaananen Vainio
7. TAMPEREEN TEKNILLINEN YLIOPISTO TAMPERE UNIVERSITY OF TECHNOLOGY Julkaisu 846 Publication 846 Lasse Kaila Technologies Enabling Smart Homes Tampere 2009 Tampereen teknillinen yliopisto Julkaisu 846 Tampere University of Technology Publication 846 Lasse Kaila Technologies Enabling Smart Homes Thesis for the degree of Doctor of Technology to be presented with due permission for public examination and criticism in Tietotalo Building Auditorium TB109 at Tampere University of Technology on the 12th of December 2009 at 12 noon Tampereen teknillinen yliopisto Tampere University of Technology Tampere 2009 ISBN 978 952 15 2253 6 printed ISBN 978 952 15 2272 7 PDF ISSN 1459 2045 Abstract Rapid advances in electronics integration miniaturisation and processing power have brought about a noticeable increase in computing and electronic devices that now surround us in our daily lives We use computers and processors every day even without noticing it starting from the moment we turn on the lights in the morning use the microwave make a phone call or get into our car The ever increasing number of devices also inevitably brings new kinds of functionality user interfaces functions and increased complexity which is something we have to cope with Often however there are situations where we do not understand or remember how certain actions e g installing new devices changing configuration setting up home netwo
8. furniture in a smart space J Ahola T Holm lund S Torkki eds Helsinki Finland 2003 pp 18 25 Series 60 Mobile Phone Platform 2009 available at http www series60 com T Saito I Tomoda Y Takabatake J Arni K Teramoto Home gateway architecture and its implementation in IEEE Transactions on Consumer Electronics Vol 46 Issue 4 2000 pp 1161 1166 M Satyanarayanan M A Kozuchb C J Helfrich D R O Hallar on Towards seamless mobility on pervasive hardware Pervasive and Mobile Computing Vol 1 Issue 2 July 2005 pp 157 189 ScatterWeb wireless network solutions 2009 available at http www scatterweb com J Schenker Not Very PC Time Magazine Vol 155 No 8 2000 Sentilla Corporation White Paper A New Vision for Pervasive Computing Moving Beyond Sense and Send 2007 7 pages C Shapiro H R Varian Information Rules Harvard Business Press 1999 A Soronen O Sotamaa And my microwave is a fox Reflecting domestic environments and technologies by means of self documen tation packages in Proceedings of the Fourth International Confer ence on Cultural Attitudes towards Technology and Communication 2004 pp 211 225 T E Starner Powerful Change Part 1 Batteries and Possible Alter natives for the Mobile Market IEEE Pervasive Computing Vol 2 No 4 2003 pp 86 88 U Steiner Futurelife Conclusions and private e
9. ANT Q amp A 2008 available at ht tp www thisisant com pages news q a Helsinki Virtual Village Arabianranta fi Portal 2009 available at http www arabianranta fi ATMEL Products Product card ATmega8 2009 available at http www atmel com dyn Products product_card asp part_id 2004 Home Automation the best home automation resources 2009 available at http home automation cuqr com home automation J Barton T Kindberg The Cooltown user experience HP Labo ratories Palo Alto 2001 5 pages S Bathiche A Wilson Microsoft surface 2007 available at http www microsoft com SURFACE index html P A Bernstein Middleware A Model for Distributed Services Communications of the ACM 39 pp 86 97 References Brumitt98 BTLE Buchenau00 Bush45 Case01 CEBUS Chatterjee98 Cisco03 COBA Coen98 Connolly05 Cook03 B Brumitt B Meyers D Robbins J Krumm M Czerwinski S Shafer The New EasyLiving Project at Microsoft Research DAR PA NIST Smart Spaces Workshop 1998 Gaithersburg Maryland USA 5 pages Bluetooth Special Interest Group Bluetooth Low Energy 2009 available at http www bluetooth com Bluetooth Products Low_Energy htm M Buchenau J F Suri Experience prototyping in Proceedings of the 3rd conference on Designing interactive systems processes practices methods and techniques New York USA 2000
10. Drivers RS 232 network connections Figure 5 37 Block diagram of the Home Controller and its connections Vainio06 5 4 The eHome 2001 2005 Because of the great deal of data gathered from the Living Room in the form of findings material ideas and theories it was decided to validate these in practice The eHome was a study that integrated everything that had been previously designed into a real home envi ronment Kaila08 It is difficult to gain accurate and reliable results from laboratory stud ies so a more practical approach was needed As mentioned earlier a smart home should be a relaxing environment and offer easy and intuitive ways for users to control devices One key issue was security users should not have to worry about forgetting to turn off lights or the stove after leaving home With the help of technology many functions of the home can be improved while new kinds of UIs would add flexibility and new functions The goal of the eHome project was to obtain results on using the Home Controller soft ware new UIs and hardware in daily life as well as finding out which technical solutions and services users would find most valuable The eHome would become a unique study and also one of the few long term smart home experiments involving a real living environ ment and users A 56 m rental apartment in downtown Tampere was selected to become the eHome This was equipped with devic
11. Each service broadcasts an URL for locating the service and a certain number of attributes that are related to the properties of the service SLP is agent based consisting of Service Agents that advertise the URL and the attributes of their services User Agents that perform search for services that are being requested and Directory Agents that perform cataloguing and caching tasks in the network SLP is commonly used in print ers and printing services but a primary shortcoming of SLP is the lack of remote access of the service it provides only location and contact information The rest of the implementa tion is not specified and is left to the manufacturer FRODO Sundramoorthy06 is a service discovery protocol developed at University of Twente in the Netherlands Robustness and resource awareness were key factors in its de 148 Software Architecture velopment to make the protocol suitable for a home environment FRODO includes three different classes for devices categorised according to the estimated cost and thus com plexity and resources of the device The simplest ones 3C Cent devices have restricted capabilities e g sensors 3D Dollar devices include more functionality and are able to use resources from other devices as well e g temperature controllers 300D Dollar de vices are powerful and are able to maintain device and service registries In reliability tests Sundramoorthy06 FRODO has proven to be a tolerant prot
12. Lecture Notes in Computer Science Springer Berlin Vol ume 4663 2008 pp 197 210 YIT Corporation article in Tietoviikko magazine 25 6 2008 M Youngblood D J Cook L B Holder Seamlessly engineering a smart environment in IEEE International Conference onSystems Man and Cybernetics 2005 pp 548 553 M Zakrzewski K ytt j n mittaaminen mikroaaltodopplertutkalla User measurements with a Doppler microwave radar Master of science thesis Tampere University of Technology Electrical Engi neering Department Tampere Finland 2005 M Zakrzewski A Kolinummi J Vanhala Contactless and unob trusive measurement of heart rate in home environment 28th An nual International Conference of the IEEE Engineering in Medicine and Biology Society EMBC 2006 New York City USA 2006 pp 2060 2063 205 References ZigBee P Kinney ZigBee Technology Wireless Control that Simply Works IEEE 802 15 4 Task Group 2003 available at http www zigbee org imwp idms popups pop_download asp conten tID 5162 ZWAVE Z Wave Alliance 2004 available at http www z wavealli ance org 206
13. PC 104 Consortium PC 104 and PC 104 Plus technology 2009 available at http www pc104 org References Philips98 Pieper98 Pister99 Potamitis03 ProSys07 Raento05 Randall03 Raula09 Ritala03_1 Ritala03_2 Ritala05 Philips Research Ambient Intelligence 1998 available at http www research philips com technologies projects ambintel html R Pieper From Devices to Ambient Intelligence The Transfor mation of Consumer Electronics keynote presentation at Digital Living Room Conference June 1998 K S J Pister J M Kahn B E Boser Smart Dust Wireless Net works of Millimeter Scale Sensor Nodes Highlight Article in 1999 Electronics Research Laboratory Research Summary 1999 I Potamitis K Georgila N Fakotakis G Kokkinakis An Integrat ed System for Smart Home Control of Appliances Based on Remote Speech Interaction EUROSPEECH 2003 8th European Confer ence on Speech Communication and Technology Geneva Switzer land 2003 pp 2197 2200 ProSystems Eik6 valvontakamerakaan ole en turvallinen Isn t a security camera secure anymore 2007 available at ht tp www proshop homeip net blogi cat 8 M Raento A Oulasvirta R Petit H Toivonen ContextPhone A prototyping platform for context aware mobile applications IEEE Pervasive Computing Vol 4 2005 pp 51 59 D Randall Living inside a smart h
14. Tampere Finland 2003 K M kel Sahk6staattisen muuntimen k ytt sisatilapaikannuk sessa Usage of an electrostatic transducer for indoor position ing Master of science thesis Tampere University of Technology Electrical Engineering Department Tampere Finland 2008 V M kinen lykodin laitteiden liitt minen tietoverkkoon Con necting smart home devices to a network Master of science thesis Tampere University of Technology Electrical Engineering Depart ment Tampere Finland 2003 F M yr I Koskinen eds The metamorphosis of home research into the future of proactive technologies in home environments Tampere Finland 2005 167 pages Violet Nabaztag The first smart rabbit 2003 available at http www nabaztag com en index html K Nagel J Hudson G D Abowd Predictors of availability in home life context mediated communication in Proceedings of the 2004 ACM conference on Computer supported cooperative work Chicago Illinois USA 2004 pp 497 506 199 References NesC NFC Nokia07 Nokia08_1 Nokia08_2 Norman93 Nyseth04 Ojala08 OSBA OSGi Oxygen Panzar00 Patel08 PC104 200 nesC A Programming Language for Deeply Networked Systems 2004 available at http nescc sourceforge net NFC Forum 2009 available at http www nfc forum org NOKIA Smart Home TEKES Ubicom
15. and as a consequence con trollers with more memory were also required Surface mount components allowed circuit boards to be significantly smaller in size and more inconspicuous More modular design techniques were used and after the RJ 12 connector was standardised prototyping became much faster and easier For example the controller unit for the Smart Home window blinds was designed according to these specifications using an Atmel AT90S4433 controller with 4 kB flash memory Blinds are controlled using six MOSFET transistors and the position of the blinds is read with the on board ADC using potentiometers connected to the motor turning the blinds After numerous follow ups the next step was to concentrate on the func 134 Hardware Aspects tionality of the network adaptive software and service discovery protocols Designing and implementing hardware takes some time as is the case with writing software for every ap plication With the move towards a modular standardised platform many steps can be avoided and prototyping time reduced Mote sensor modules described in more detail in the next section offered these properties and they represent a large part of the future hard ware in the Smart Home 12C bus 18 2 3 e e Temperature sensor sensor AT90S4433 Water level sensor Figure 6 12 Schematic of the flower pot controller in the Smart Home 6 11 Smart Sensor
16. way Cameras wearable locator equipment or similar technologies constantly remind peo ple that they are under scrutiny albeit only by a computer or software Locating people is also only one part of the problem tracking identifying and coping with multiple user makes the challenges greater Floor sensors usually installed under carpets floor tiles or inside the floor structure usu ally require complex installation and cabling making them too expensive or cumbersome to install Thus it makes most sense to install them during the building phase Floor sensors only work in locations where they are installed if the apartment is not fully covered there will naturally be blind spots that have to be covered with other kinds of sensors instead Floor sensors can be made using mechanical switches KoskinenO3 first prototypes in the Smart Home capacitive means using conductive tiles Valtonen09 currently imple mented in the Smart Home or other sensor film material such as EMFi EMFi used in the Living Room Passive infrared sensors detect changes in ambient infrared radiation created by movement and do not indicate whether they detect a single person or several persons or their exact location Thus they are suitable only for detecting whether someone is moving in the vi cinity or not However they are cheap and easy to install in many locations and currently they are mostly used for automatic lighting control in areas such as hallways and
17. 2009 available at http www li net fi C L Lisetti D E Rumelhart M Holler An Environment to Ac knowledge the Interface between Affect and Cognition AAAI Technical Report SS 98 02 1998 9 pages B Logan J Healey Sensors to detect the activities of daily living in Proceedings of the 28th IEEE EMBS Annual International Con ference New York City USA Aug 30 Sept 3 2006 pp 5362 5365 Echelon Corporation 2009 available at http www echelon com K Lorincz M Welsh MoteTrack A Robust Decentralized Ap proach to RF Based Location Tracking in Springer Personal and Ubiquitous Computing Special Issue on Location and Context Awareness 2006 B MacIntyre E D Mynatt Augmenting intelligent environments augmented reality as an interface to intelligent environments in AAAI 1998 Spring Symposium Series Intelligent Environments Symposium March 23 25 1998 Stanford University CA W Mark Turning pervasive computing into mediated spaces IBM Systems Journal Vol 38 Issue 4 1999 pp 677 692 K Martinez P Padhy A Riddoch R Ong J Hart Glacial Envi ronment Monitoring using Sensor Networks Workshop on Real World Wireless Sensor Networks REALWSN 05 Stockholm Sweden 2005 R M Mayrhofer An architecture for context prediction Doctoral Thesis Linz Germany 2004 173 pages MEMSnet Micro Electro Mechanical Systems exchange 2009 available at ht
18. 3 eHome User Interfaces cccccecscceeseceesececeesececeeeeeseeeeess 98 5 5 Morphome Living in metamorphosis 2003 2005 seeseeeereeees 101 5 6 LIPS Learning and Interactions in Proactive Spaces 2007 gt 103 5 6 1 Context AWareness ccccccccccceesseecescecesseeecenseeeseeseensaeeeaes 103 D02 LIPS Software oi rieo terrere a kn iii 104 vi Table of Contents 3 6 3 LIPS Hardware 25 5 re er e E E E bea a 105 N04 SELVICEDISCOVELY ipea EEE EE E EEE 106 5 6 5 World Modelling oo eee ceeceseeeeceseeeeeeseeeeeeaeeeseeseees 107 Def SUMMAT ieseana sans sendendacestdi aaa EE aa e Ene Ters 107 6 Hardware ASpECtS sccsscccssccscsccsesssccsscsccecsssscscssscsesessescssceesesscsesseseesoeses 110 6 1 NetWOrKS e f oie code totes tc eed ea a aa 111 6 1 1 Selection Criteria saosna e EE aE A e iR 113 6 1 2 ISO OSI Network Model c ccccsccccsssecessececeseeeeseeeeees 115 6 2 Wired Communication eeececceeseceesseeceeeeeceececesececesaeceseeeesnseeees 116 6 2 1 Wired Network Types sssssesessseersssesseeesreersreererensesesseses 117 6 3 Wireless Communication ss sosseeeeeseessseeesseseeteserttsetessressetessereseeesre 119 6 4 Network Functionality and Topology sseseseeseseeessererssresrseserrrrereerens 122 6 5 Home Gateways oroesi esee ierre enro Cok Enpe okopa eoni iE 124 6 6 Sensors Measurements Detection ccccccccceesssececesesssecececenseaees 126 6 6 1 Environmental
19. Architecture 7 Software Architecture While the most visible features of smart homes are appliances sensors and devices it is the underlying software architecture that actually defines the functions intelligence and use fulness of the smart home Depending on the network type as previously described cen tralised or distributed the appliances and sensors typically contain only simple software that is designed to perform a predefined function communicate and report its status to the network More complicated functions are performed by the rest of the home systems through computers or home gateways running sophisticated adaptive algorithms middle ware and server software Home control software can contain several functions practically limited only by the im agination Normally there are at least user profiles pre set modes home away at work etc timer settings and master controls If the computer running the software is sufficient ly capable it can also function as a home media server or gateway streaming video and serving files to other devices in the home Instead of a large noisy PC it is also possible to use a more discrete embedded control unit that is dedicated to home control When server software is being designed there are many design factors and options that af fect the functionality modularity and flexibility of the system In research environments this step is often skipped and the end result becomes a myriad o
20. Availability is also dependent on time again with great vari ation depending on the user There are also different levels of interruption for example ur gent items that require immediate attention or casual items that can be attended to later Naturally the level of urgency also affects the threshold of user s availability The re searchers used a pager that sounded randomly to discover what users were doing at a par ticular moment and how available they were and how willing they were to be interrupted in various scenarios The research was primarily conducted through interviews The context that a context aware system is processing must be a composition of relevant information gathered from the smart space Mark99 A structured indexed recorded in teraction is of much greater value than a simple log of all recorded activities and sensor readings It is also vital for referencing future interactions in the space by providing an in ferential representation of what has been happening earlier By encoding the current and relevant past states of the context the system is able to communicate with users regarding their current activities Where accurately detecting contexts is difficult so is learning new ones If every slight change in sensor readings would cause a new context do be detected the total number of different contexts will quickly rise to unmanageable levels The choice is either to let peo ple verbalise the contexts themselves e
21. DLNA Digital Living Network Alliance DLNA which allows devices to play media from other devices share network resources access UIs etc DLNA is derived from a sub section of the UPnP UPnP standard and the list of compliant devices grows each day with devices such as flat screen TVs Playstation 3 game console printers and mobile phones being the most popular In the near future this kind of interoperability might actu ally form a good base for a home control system allowing information and UIs to be brought to any compliant device and network properties to be set up automatically One challenging issue with home networking is the complexity of setting up and maintain ing the networked devices Current network technologies can be rather difficult for novice users to comprehend and setting up IP addresses gateways WLAN encryption and fire wall rules are understandably complex matters The ICEbox Yang08 is a home gateway that attempts to make setting up home networks easier In addition to lower level network tasks e g automatic IP addressing it also performs certain low level encryption physi cal link layer higher level file and resource sharing functions device monitoring and discovery The ICEbox itself is a small PC based information panel equipped with an in frared port a touch screen and a key lock The screen is used for controlling devices and making actions infrared is used for registering and setting up new devices
22. Mattila Evolution towards smart home environments empirical evaluation of three user interfaces in Personal and Ubiquitous Computing Springer London Volume 8 2004 pp 234 240 T Koskela K Vaananen Vainio Mattila L Lehti Home Is Where Your Phone Is Usability Evaluation of Mobile Phone UI for a Smart Home in Lecture Notes in Computer Science Vol 3160 2004 pp 74 85 K Koskinen Lattian anturointi Floor sensors Master of sci ence thesis Tampere University of Technology Electrical Engineer ing Department Tampere Finland 2003 G Leen D Heffernan Expanding automotive electronic systems in Computer Volume 35 Issue 1 Jan 2002 pp 88 93 M Lehto Tekniikkaa ik kaikki Technology for the whole life Helsinki Oy Edita Ab 1998 180 pages LEM Current measurement transducers 2009 available at http www lem com hq en content view 25 101 197 References Leppanen03 LINET Lisetti98 Logan06 LON Lorincz06 MacIntyre98 Mark99 Martinez05 Mayrhofer04 MEMS Mikkonen00 198 S Lepp nen lyk s koti ja asukkaiden tarpeet The smart home and requirements of the inhabitants in lyk s kaluste kaluste lykk ss tilassa Smart furniture furniture in a smart space J Ahola T Holmlund S Torkki eds Helsinki Finland 2003 pp 35 37 LINET Light Control Network
23. PC and a driver or a handler for the protocol When the XML de scription is updated the device is ready for use The Home Controller can manage many concurrent user interfaces and the UIs themselves are platform and programming language independent The Home Controller has been used with WWW based UIs Java applets mobile phones Java Symbian graphical PC UIs Java and a speech control UI Concurrency is an important factor since many UIs can be used at once and possible state changes should be updated to all UIs as fast as possible The Home Controller has no form of arbitration or prioritisation for UIs though if multiple commands are received in a short time they will be handled in the same order as they were received Whenever a state change in a device has been noted the Home Controller sends an update to all UIs Depending on the particular device some devices are polled at regular intervals whereas others can be queried from an UI The server uses a text based protocol to communicate with UIs keeping bandwidth and processing requirements simple This is important since some UIs use low power microcontrollers with limited processing capa bilities and relatively slow network connections A text based protocol is also program ming language independent allowing UIs to be written in any programming language The Home Controller offers several different kinds of services such as timers preset modes grouping of devices and logging servi
24. Plug and Play Quality of Service Random Access Memory Radio Frequency Radio Frequency Identification Received Signal Strength Indication Real Time Operating System Receive Service Discovery Short Message Service Serial Peripheral Interface Transmission Control Protocol Tampere University of Technology Transmit Universal Asynchronous Receiver Transmitter User Datagram Protocol User Interface Universal Plug and Play Universal Serial Bus Ultra Wideband Virtual Private Networking Virtual Reality Valtion Teknillinen Tutkimuskeskus Tech Research Centre of Finland Wide Area Network Wireless LAN x A symmetric S ymmetric Digital Subscriber Line eXtensible Markup Language xi Introduction 1 Introduction 1 1 Smart Homes Ubiquitous Computing Pervasive computing Ambient Intelligence Smart Homes Eve ryware etc predict the coming of a new age of modern living According to these predic tions our future homes will be filled with electronic devices all interconnected and around us sharing information and making our lives easier In order to better understand the con cept of the smart home we may take a brief look at the past to see how technology has im pacted our daily lives The emergence of electric home appliances in the beginning of the 19 century changed the way we perform our daily chores at home and revolutionary in novations have made our lives easier and more comfortable Once a certain l
25. Room The DVD player VCR and surround am plifier can be controlled from the same pages making infrared remotes unnecessary The tablet uses a TCP IP connection over WLAN to the server 86 TUT Smart Home Research Kotiohjuri E E S leverhojen s t 5 E z Mi KHhhhhshs Smua mm m smua m om smua m mm emes Vahvistin On ott Valkokangas aw Audion sis ntulot hde vam VER in video DYDAD in OVD Figure 5 32 Screenshots of the Smart Home tablet UI Top main page showing lighting and curtain controls temperatures and quick selection buttons Bottom A V page showing controls for the DVD player projector and amplifier 87 TUT Smart Home Research In 2002 mobile phones were becoming increasingly versatile they came with large dis plays and it was also possible to install and write sophisticated software for them A min iaturised version of the tablet UI was written for a Series 60 S60 based mobile phone a Nokia 3650 The Symbian operating system and Series 60 user interface made it possible to write a customised UI for the phone and use the phone s Bluetooth interface for com munication Ritala03_1 The phone UI is designed for use with the multi directional con troller on the phone and when a selection is made the phone communicates with the server and the corresponding action is taken The phone UI allows users to monitor temperatures cont
26. Science Volume 4008 2006 Springer Berlin 2006 pp 85 108 M Gandy T Starner J Auxier D Ashbrook The Gesture Pen dant A Self illuminating Wearable Infrared Computer Vision Sys tem for Home Automation Control and Medical Monitoring in Proceedings of the 4th IEEE International Symposium on Wearable Computers 2000 page 87 D Gann J Barlow T Venables Digital futures Making homes smarter Coventry Chartered Institute of Housing 1999 163 pages J Gerhart Home automation amp wiring McGraw Hill USA 1999 322 pages Ghanea Hercock02 R Ghanea Hercock Autonomous Computing Workshop on Green04 Greenfield06 Grand Challenges for Computing Research Panel C submissions Edinburgh UK 2002 3 pages W Green D Gyi Capturing user requirements for an integrated home environment in Proceedings of the third Nordic conference on Human computer interaction Tampere Finland 2004 pp 255 258 A Greenfield Everyware the dawning age of ubiquitous comput ing New Riders USA 2006 272 pages 193 References Gu04 Guttman99 Hansmann03 HAVi Helal05 Hightower06 Himanen03 HiperLAN HomePlug HomeRF Homeseer Homesoft04 Hong01 194 T Gu H K Pung D Q Zhang Toward an OSGi based infrastruc ture for context aware applications IEEE Pervasive Computing Vol 3 Issue 4 2004 pp 66 74 E Gu
27. Smart Home Projects 4 2 1 Telenor Fremtidshuset Future Home 2001 2004 Telenor a Norwegian telecommunications company built a complete house in the former Fornebu airport area as a flexible living laboratory Nyseth04 The Future Home com pleted in 2001 was designed to allow researchers to explore the possibilities and ideas with new technology in private homes It was also designed to accommodate several guests for demonstration purposes For this purpose the home was fitted with a LON works infrastructure and 1 wire sensor network allowing users to control home appliances heat ing etc The walls were also reconfigurable and there was ample space for equipment Several user tests were run in the Future Home involving communication technology emotional responses and media usage among family members The Future Home also ex perimented with emotional UIs window blinds controlled by stress levels of users and odour UIs a certain odour in the home would indicate a need for communication for ex ample the mother wants her children to call her Another research topic involved distrib uted families i e employing flexible rooms and communication technologies to help children feel relaxed when moving from one home to another 45 Related Work Figure 4 1 Telenor Fremtidshuset Photos Telenor archives author 4 2 2 Philips Home Lab Philips research group in Eindhoven Netherlands have built a research lab for testing Am
28. UI WWW UI 2001 2005 customisation home environment testing Morphome Proactive home applica Different design proto 2003 2005 tions interdisciplinary types new design princi design ples LIPS Proactivity adaptive soft EPIS protocol mote net 2007 2009 ware service discovery work context recognition The figure below illustrates how the focus of different smart home projects at TUT have evolved Living Room Smart Home Morphome LIPS Figure 5 50 Overview of TUT smart home projects showing overlap of research areas Compared to the other research projects presented in Chapter 4 TUT smart space research has had adopted a slightly different approach From the outset the emphasis has been on a practical approach using current standards and affordable components Whereas some other projects have concentrated on simulations and interviews all TUT implementations have involved real physical hardware This has enabled us to perform practical testing ver ification of the functionality of our designs and to identify practical usability issues Most important all designed prototypes and applications have been connected to the home in frastructure to make the system function as a single entity instead of a collection of numer 108 TUT Smart Home Research ous small independent components Interaction with the smart home system has been confined to various physical UIs and control methods
29. and in the latter nothing will happen with the new device until the server configuration is changed manually An adap tive dynamic protocol would allow the system to locate another temperature sensor nearby and use its temperature data instead and in the case of a new device it would be able to identify it and utilise its functions and facilities Network and device reconfiguration also requires no action on the part of the user making it simple to install new devices move them around the house and modify the network topology Service discovery is discussed in more depth in Chapter 7 Because LIPS is based on new hardware with stricter requirements for data transfer and service discovery it also requires a new communication protocol that would replace earlier serial protocols The protocol is presented in Chapter 7 106 TUT Smart Home Research 5 6 5 World Modelling In order to capture contexts and adapt to users actions the system requires a model of the environment One option is to build a model using sensor measurements for building a mountain model Assuming that sensor measurements have a particular of distribution for example they might be normally distributed they are used for building a two dimen sional mountain that grows in height as measurement results increase This means that sensor data for one event or context becomes more frequent it is accumulated and stacks up When this mountain is high en
30. and installing them into multiple locations is restricted only by the amount of manual labour involved For example temperature sen sors in the refrigerator on the balcony and near the kitchen stove could provide the system with contextual information on what people are doing in the home Humidity and air qual ity sensors reflect the atmosphere in the home and they can be used to indicate possible needs for ventilation or air conditioning Light level sensors can indicate the lack or need of lighting or measure the amount of ambient light Figure 6 8 Environmental sensors used in the Smart Home showing a humidity sensor left tem perature sensor middle and light sensor right 6 6 2 Location Sensors Sensors used for locating people such as passive infrared PIR pressure sensitive switch es capacitive strips Doppler radar etc are used to obtain an accurate image of where peo ple are moving inside the space Some of these are passive i e do not actively transmit anything and thus require no active components or worn equipment whereas active sen sors can require transmitter receiver pairs or similar equipment Location information can 126 Hardware Aspects be used to control lighting gather contextual information for security measures and for learning people s daily routines Unobtrusiveness is an important factor as users might feel uncomfortable seeing that their movements are being watched or monitored in some
31. and the key lock for restricting physical access to the box When a new device is introduced to the home network it is first brought physically to the ICEbox during which services and other data are exchanged over the infrared link The device then receives network settings and is set up for the current home network Access Networks Home Network Phone line DSL Figure 6 7 Block diagram of connections to a home gateway 125 Hardware Aspects 6 6 Sensors Measurements Detection Sensors are primary sources for information in a smart space indicating actions move ment and conditions both inside and outside Sensors can be either static fixed installation in the space or mobile worn or carried by the users they can be integrated into struc tures or be part of a device Sensor technology and integration has made huge advances in the last few decades and a multitude of integrated sensor modules are available today The issues to be decided now are the kinds of sensors that are useful for smart home applica tions where they should be installed and how the measurement data will be used 6 6 1 Environmental Sensors Temperature humidity and light level sensors are useful for measuring local or remote conditions that are very relevant to comfort well being and health By installing sensors in critical or interesting places it is possible to obtain even more detailed information These kinds of sensors are typically very cheap
32. annual seminar report 2007 10 pages Tulevaisuuden tietotekniikka Kuudes aisti k nnyk ss IT of the future Sixth sense in a mobile phone Tietokone Magazine Is sue 12 2008 page18 NOKIA Smart Home solutions 2008 Product in brief available at http smarthomepartnering com cms wp content uploads Nokia HomeControlCenter pdf D A Norman Things that make us smart Addison Wesley USA 290 pages A Nyseth N Khalayli J Eriksen et al The future home Telenor Research and Development 2004 82 pages T Ojala J Riekki H Kukka M Leskel Kaikki sujuu ubiikissa kaupungissa Everything runs fine in the ubiquitous city Proses sori Magazine Issue 11 2008 pp 24 26 MIT Open Source Building Alliance White paper 2003 15 pages OSGi Alliance The Dynamic Module System for Java 2009 available at http www osgi org Main HomePage Massachusetts Institute of Technology Laboratory for Computer Science Artificial Intelligence Laboratory Oxygen Project 2001 available at http oxygen lcs mit edu M Panzar Tulevaisuuden koti The home of the future Helsin ki Kustannusosakeyhti Otava 2000 285 pages S N Patel M S Reynolds G D Abowd Detecting Human Move ment by Differential Air Pressure Sensing in HVAC System Duct work An Exploration in Infrastructure Mediated Sensing in Proceedings of Pervasive 2008 Sydney Australia 2008
33. ate modify and query a group of devices send control messages and open close connec tions A new protocol was required because the standard serial protocol was limited to a maximum of 15 data bytes and its structure was also unsuitable for group messages and simultaneous transmission to multiple recipients Smart Home UIs are connected to the server using TCP IP socket connections and whenever a change is detected the server broadcasts the new status to all connected UIs A standard UI protocol message has the fol lowing structure length version command IDGroup ID value where is a start character a stop character and a delimiter character The length field defines the total length of the message maximum is 999 version indicates the UI version and command e g adjust create delete getvalue denotes the desired action IDgroup and ID number point to specific groups of devices or a single device and the val ue field is an optional parameter For example adjusting a light in the living room a dim mable light ID number 2 to brightness value 50 would be done with the following command 6 1 adjust linear_lights 2 50 Once a command has been processed an adjusted message is sent back from the UI which allows the server to update the state of the device and thus other UIs as well 7 4 3 LIPS Service Discovery and Network Protocols The LIPS project introduced new requirements fo
34. be bi di rectional it must be possible to send data both to and from a device Chatterjee98 This is different from some traditional home automation networks which only allow com mands to be sent to devices in one direction For example a timer can command the lights 111 Hardware Aspects in the building to be turned on but if someone else manually turns the lights off afterwards the network controller has no way of knowing this without this feature As far as the con troller is concerned the lights are still turned on Bi directionality is also important for ver ifying that commands have been received making reliable transmissions possible However existing devices that use only one way communication such as remote control led home A V equipment should also be considered If there is no feedback through the network it needs to be enabled another way for example visually through the user If an infrared transmission is sent to a device but there is something blocking the communica tion for example a person standing between the transmitter and the receiver the signal will not be received The only way for the system to be aware of this is to indirectly mon itor the device by measuring current consumption operating mode etc or by having the user re transmit the command Networks in a home environment can be divided into five different categories according to range application and distribution Wide Area Networks WAN
35. bient Intelligence solutions Philips The Home Lab is a fully equipped home used for testing prototypes in a realistic scenario with the latest technology The laboratory features speech recognition and user positioning along with several screens for displaying informa tion In order to study how users behave in the laboratory microphones and cameras are mounted in every room Tests were conducted with family members friends and young sters who were able to share digital media and discover new ways of interaction Philips vision is that convergence will continue in the future leaving users with fewer individual non compliant appliances A TV set will contain everything needed in the living room from a digital receiver amplifier and speakers to a media server Equipment can be con trolled in new ways using gestures and speech commands leaving the user free for other activities at the same time Some of the technology that was tested in the Home Lab is al ready available in current products 46 Related Work A Capial Peers rr NAL 424097 Figure 4 2 Philips Home Lab Photos Philips Research 4 2 3 The Adaptive Home 1999 The Adaptive Home was an experiment with adaptive technology in a home environment Mozer05 A house in Colorado USA was fitted with HVAC and lighting control numer ous environmental sensors and actuators The control system was based on neural net works and used reinforcement learning and predictio
36. boards that are used for wireless networking already include temperature and humidity sensors For context recognition there is also a need for more sophisticated and specialised sensors Location Sensor To determine the location of the user the Smart Home is being equipped with a capacitive positioning system Valtonen09 These sensors can detect a person s location with 30 cm accuracy and they will be installed in every room except the bathroom and sauna The system can be used for tracking people and also to determine whether they are seated standing or lying down Another sensor is installed in the couch near the TV which will report if users are sitting on the couch Additional sensors can also be installed in chairs around the coffee table in the living room 105 TUT Smart Home Research Table Sensor For recognising the eating context the dining room table will also be fitted with various sensors Under the wooden tabletop directly beneath the veneer surface a matrix of tem perature sensors will locate cold or hot objects on the table top Capacitive strips between the sensors can detect the presence of conducting material e g fluids An infrared sensor can detect if a person is sitting directly in front of the table With these sensors it should be possible to detect with reasonable accuracy when people are sitting at the table having lunch Living Room Sensors A few additional sensors are also needed in the living
37. but users are not explicitly aware that they are using them and subordinated the functional side of the UI is subordinated to another aspect for example aesthetic or personal Nonetheless it is beneficial Green04 Koskela04_1 to maintain traditional manual con trols for equipment and instead offer other new ways of controlling them all Practical studies have shown Randall03 Koskela03 that centralising all smart home UIs into a sin gle master UI for example a touch panel on the wall or a web based UI is not beneficial as there are already multiple different scenarios in everyday family routines A centralised control panel for example might be useful when people are nearby but there will be cases where the UI is out of reach of the user and which might prompt the user into using manual controls or doing nothing at all Physical graphical and auditory user interfaces can all be utilised each having its own pur poses and advantages In the ubiquitous computing scenario UIs can be detached from their respective devices and placed anywhere This makes it possible to group UIs and use any device from any UI However this feature also requires more complexity from the UIs since it is now possible for several people to control a device at the same time Multi user issues can be accommodated either by assigning priorities to users users with higher priority overrule others or by simply offering first come first serve type functiona
38. concentrate on solving a certain problem by using the latest technology and methods available The Adaptive Home became an adaptive space capable of learning with the assistance of neural networks whereas the Aware Home again serves as a platform for several individual research projects each focusing on im proving our lives communicating with family members etc MavHome relies on agents actuators and sensors to manage the space and create a pleasant experience for its users and the impressive number of projects at MIT include living laboratories computer labo ratories and conference rooms The Smart House at GatorTech again concentrates on an open OSGi platform and elderly care Of these projects the Adaptive Home has been a pioneer in adaptive and proactive home control systems and research on this subject is also being conducted at TUT The Aware Home presents a complex and interesting smart home infrastructure but the research con sists mostly of small independent projects and a complete home control system with ap propriate UIs seems absent Research projects with a commercial background are naturally more inclined towards pro moting new products services and technologies Telenor s future home concentrated on communication technologies whereas Philips Home Lab concentrated on media sharing and home entertainment Microsoft s Easyliving laboratories demonstrated disaggregated computing new kinds of UIs and home middleware an
39. currently doing Data for the context in formation will be collected by various sensors in the Smart Home time date and learned data in the context recognition engine The Context Messenger will also have a more im portant role functioning as a primary interaction point between the user and the context recognition engine If a new context is detected or changes in known contexts are record ed the user can be greeted with a pop up window requesting input on details of the context data This function is generally needed initially when new activities and contexts are being recorded Later as the system has gathered more data the amount of interaction required will be significantly less The Context Messenger is designed to run on a wall mounted panel display but other platforms are also under consideration 104 TUT Smart Home Research 5 6 3 LIPS Hardware In previous projects almost all hardware has been designed in house at the university but in LIPS a decision was made to move to acommercial standardised platform Initially two options were evaluated Scatterweb Scatterweb and the mote platform developed by UC Berkeley motel V Both were battery operated wireless sensor platforms with embedded sensors a transceiver some memory and a microcontroller both are presented in further detail in Chapter 6 11 Due to the popularity and maturity of its software development kit the mote was chosen as the primary platform for LIPS The
40. doing something wrong in that particular context To correct this the system learns the new settings and sets itself to perform them automatically next time the context is recognised In this way users can teach the system to perform certain functions automat ically when a context has been recognised In terms of the actions and adjustments per formed by the user a challenge lies in distinguishing between those that are relevant to the current context and those that are not For example the user might be in the habit of wash ing clothes in the washing machine in the evening and watching TV at the same time However this should not imply that whenever the washing machine is detected running in the evening that TV should also be turned on Such discrimination will become hard to per form especially when new sensors and appliances can appear in the home at anytime One option is to let the user decide which sensors and devices are relevant to a given context Context awareness can be used in conjunction with communication with other remote us ers Instant messenger applications have become very popular and users frequently share messages files and data over the Internet A similar application is being designed for use in LIPS named Context Messenger This application would be used in a similar fashion as other messenger applications but the detected context of users will be available to other users This allows people to see what others are
41. each appli cation Technologies such as DLNA can enable connection of different kinds of devices allowing them to share information and controls Other kinds of compatibility problems e g physical and electrical can be overcome by using adapters and converters but if the content or information is completely of a different kind it can be a strenuous effort to create a system that would understand this information and be able to convert it into a common format For example it would be a rather simple task to convert temperature readings from different sensors or different video signals into a common format but in the case of con textual information e g the user is eating the TV is broadcasting news knowledge about the user s tasks it would be a much more complex case Compatibility and standardisation are closely related but there are also several issues that standardisation cannot and possibly should not affect For example social and psycho logical issues vary greatly between users and user groups and could probably not be alle viated this way One further thing of note is that standards can be short lived and if the IT world is any indication there is a steady flow of new technologies on to the market to re place the existing technologies which quickly seem outdated and limited IT standards typ ically last only a few years there are exceptions for example the 3 5 floppy disk which has been an IT standard for over 20 years and
42. either body mounted sensors or computer vision for recognising different gestures With wearable sen sors such as the gesture pendant Gandy00 recognition can be achieved anywhere but at the cost of the discomfort of wearing sensor bands and battery packs Computer vision can be used only in front of cameras and is largely dependent on lighting position of the user and other visual conditions Gesture controls could also be implemented using other kinds of sensors such as capacitive floor sensor arrays If the sensor array is accurate enough users could double tap or make a foot movement in front of the door to open it 8 6 5 Augmented Reality Augmented Reality AR is a technology that combines what we can perceive from the en vironment around us using vision touch and hearing with artificial usually computer gen erated information Experiments with AR enhanced smart spaces have been made at Columbia University and Xerox Laboratories MacIntyre98 using both auditory and vis ual augmentations According to the authors humans should interact with smart spaces us ing means that are already natural for us such as speech context affect In other words humans should forget that they are interacting with a computer when they interact with a smart space With visual AR the user has a head mounted display to see both the surround ing environment and the overlaid computer generated information at the same time This enables the smart spa
43. fields Other aspects related to smart home development that might require change are public at titudes and assumptions about new technologies The status of current cutting edge tech nology does little to help in this respect since both software and hardware are released without sufficient testing or verification Many users are already accustomed to installing updates to software or flashing new firmware soon after a new product is removed from its packaging because it will often not work with the pre installed version People will hardly be motivated to buy such products if they must perform these tasks each time they buy a device for their homes Smart home devices and technology should be based on proven reliable technology and also presented to users as such New technology should be introduced into peoples homes gradually rather than suddenly or on a large scale Leppainen03 This will help users to become accustomed to the new functions and UIs and thus cope better with the devices in their daily lives In the case of retrofitting old buildings this would be an inevitable consequence since people would first choose to purchase a few devices install the basic infrastructure and acquire more devices and applications later as their needs become clearer Since smart homes would ideally be tailored for each application and scenario it is important that the requirements and wishes of the customer are identified clearly when the system is bei
44. home environment using various network and connection types Currently RS 232 us ing COM ports and 2 Ethernet both TCP IP and UDP and Bluetooth are currently be ing used Wireless connections such as proprietary RF and Bluetooth actually appear to the Home Controller as serial ports the RF link through the serial hub is connected via the serial hub and Bluetooth devices using a standard USB dongle use the standard serial port profile and are mapped as virtual COM ports on the PC 89 TUT Smart Home Research Figure 5 35 Smart Home server in the bedroom Home Controller Middleware If the server PC is the centre of the network then the heart of the Smart Home is the server software that connects to all devices and user interfaces Vainio06 The Home Controller is a Java based centralised middleware software designed to contain multiple different control interfaces From the outset the Home Controller was designed to be modular to make it easier to add more functionality while the infrastructure style approach Hong01 makes it versatile and platform independent Over the years new networks protocols and functions have been added and this modularity has proven very valuable for the Smart Home s changing research environment Currently the Home Controller uses LINET TCP IP UDP new Smart Home protocol and Smart Home UI protocols described in Chapter 7 Physical devices are modelled in the server using XML based descri
45. home need not be physically different from a traditional non smart home dif ferences will become clear in other ways To give an example of what a smart space could be like consider a space with a locationing system environmental sensors and networked home appliances Using the sensors and location information it is able to gather informa tion about the space and the actions of the users If the control system determines that an action should be taken it can perform the necessary actions according to learned patterns or weighted predictions When a person enters a smart home the space will react in some way to his her actions For example if the user is wearing an ID tag the computer will rec ognise him her greets the user and a personal profile will be loaded along with appropriate lighting levels and temperatures for the room Another scenario is when a person wakes up in the morning Instead of a buzzing alarm clock he she can wake up when the window blinds slowly open letting more light into the room If it is dark outside the computer can use the bedroom lights instead to turn up the brightness in the room This offers a more pleasant and natural wake up After a while when the user has woken up the blinds will briefly close again allowing the user to get dressed or take a shower A smart home should be able to serve its users during all stages of their lives adapting and changing itself according to people s requirements and
46. human and vice versa Tennenhouse00 Figure 3 2 below illustrates the var ious situations where it would be better for a human to relinquish control to the system The first example a is a situation that would require very precise adjustments and much concentration from the user For example when cutting metal with a milling machine a computer controlled CNC machine is capable of achieving precision far superior to that of any human being The second example b is the opposite case where events change very slowly and monitoring becomes tedious It might also be difficult to notice when something goes awry and in such a case a computerised control system would be better as it does not fall asleep or allow changes to go undetected This could be applicable to tem perature adjustment for example Example c is a case where an event occurs place ex tremely fast and reacting to it is beyond human ability for example activating antilock brakes or inflating an airbag in acar The last example d is a situation that involves sev 24 Smart Homes eral variables at the same time making it difficult for a human to comprehend and cope with these simultaneous events Too precise stressful Too quick no time to react Bounds of acceptable adjustment Too complex incomprehensible Figure 3 2 Different cases where computer involvement would be preferable These cases make it clear that there are certain situations where
47. image issues can also affect the willingness to in vest in smart home technology Additionally the cost of possible third party services be come important especially if they involve long term commitment Application developers and device manufacturers on the other hand view value gained from smart home technol ogy from a different perspective they care more about bringing new products on the mar ket or developing existing products to suit the new smart home scenarios The danger is that proprietary solutions remain on the market and slowly kill off the idea of a networked smart home Already today there are multiple competing standards both open and closed and there is no clear winner that would become a de facto choice as far as smart home networking implementations are concerned One major reason for this is the sheer amount of standards that designers have to choose from Instead open standards and developer forums would give opportunities for everyone to enter the market and develop compatible products This would also help researchers because there are often many inde pendent groups researching and developing solutions for the same problem with little knowledge of what is being done elsewhere in the world When a particular part of smart technology is being designed care must be taken to ensure that the smart home in its entirety is considered and not only small individual items As with many other applications the value and us
48. indeed contain processing power memory and some form of user interface and in a modern home the number of electronic devices is considerable However the idea of networked resource sharing computers has still not materialised with applications still isolated and unaware of the rest of the system Some gadgets such as the Nabaztag Nabaztag06 a rabbit with WLAN that checks e mails weather reports and reads news and Ambient Orb AmbOrb07 a similar device that changes its colours to reflect changes in weather new e mails etc have network connections but they do not interface with any home electron ics Moreover the task of configuring and connecting these things already requires exten sive knowledge about wireless LANs and related settings not to mention the effort of actually setting up a secure wireless home network Definitions 2 2 Ambient Intelligence The term Ambient Intelligence AmI was introduced by electronics giant Philips in the late 1990s Philips98 The AmI concept was built around telecommunications home elec tronics and computing which naturally relates to the company s product selection The idea of Aml is to create electronic environments that are sensitive to the presence of peo ple anticipate and react to their needs In order to make this possible unobtrusive hard ware seamless communication and computing and human centric user interfaces are needed Philips has complete facilities for testing and d
49. is actually desirable In such a case pervasive communication technologies can enable users to choose when and with whom they want to socialise making it easier to suit the require ments and needs of the individual For the elderly and handicapped it would also be beneficial to provide other ways of inter acting with equipment in the home Wheelchair bound people would benefit from a wire less context aware controlling device which would allow them to easily open doors and control lights Elderly people would again have reminder functions health monitoring and suitable UIs that would make their lives safer and more comfortable Health care manage ment in a smart home would also involve family members friends professional medical staff and the local community Adaptivity The ability to adapt to changes in lifestyle living habits routines and other circumstances is another major advantage of smart homes Physical adaptivity and modularity of the home is an important matter but the possibility to tailor and adjust the settings of the be haviour and functions of the home are at least as important since they dictate how the home reacts to different situations After all family size is quite likely to change over the years the abilities of users can change or users might simply wish for different experiences Cer tain details can be profiled and specified when the system or equipment is purchased but subsequent changes must also be possible
50. is almost 2010 a short update might be interesting as it would seem that many requirements can still not be met For example smart materials MEMS devices and energy harvesting are not common in 30 Smart Homes today s hardware but on the other hand integrated sensors and low power radios are wide ly available Seamless networking and compatibility has not been realised either users are still required to cope with WLAN encryption keys Bluetooth device pairing and incom patible data formats On the UI front some advances have been made for example capac itive multi touch touch screens and touch pads allow users to control devices with hand and finger gestures and this trend seems to be spreading quickly to all kinds of mobile de vices 3 2 2 User Requirements Users of different ages and from different demographics have different abilities desires and needs and these must all be considered in the smart home design process However many products today are not customisable to the needs of individual users being designed for the average user and the mass market This limits the adaptability of the product its flexibility and popularity in the long run Lehto98 People s attitudes also differ with re gard to technology adoption Some such as early adopters appreciate having the latest technology in their homes and cars while others fear or mistrust anything new and unfa miliar Tiresias08 Security privacy and safety are th
51. it can be enhanced further One example of such a network is ANT ANT which is a low power sensor network radio transceiver that houses an efficient protocol and an 8051 based microcontroller ANT is designed for sensor networks and its performance is less than excellent however it is op timised for very low current consumption ANT protocol is capable of forming simple point to point networks or more complex mesh networks and it is widely used in sports health and other personal gadgets Z Wave ZWAVE is a wireless low power communication standard developed for home automation Standardised by the Z Wave alliance it is targeted for controlling home elec tronics devices lighting and household appliances The Z Wave network is a mesh type with multi hop capabilities and a central controller that contains routing tables and data of devices included in the network Operating at 868 MHz it can achieve an operational range of about 30 metres inside a home environment Devices can be added to the network by pairing them with the network the device registers other devices nearby and their re spective signal strengths For higher bandwidths wireless LANs offer a flexible alternative to fixed Ethernet cabling However their higher complexity and large power consumption make them less desirable for low power sensors and battery operated devices WLAN access points may be re quired in several locations to guarantee sufficient coverage but there a
52. m very low 2 4 GHz WiFi 802 11g 2 4GHz 56 Mbit s lt 1W 25 600 m moderate GSM EDGE 900 1800 9 6 kbit s 2 1W several kilo moderate UMTS MHz Mbit s metres UWB UWB 480 Mbit s lt 100 mW 3 10 m cheap 802 15 3a ANT 2 4 GHz 10 kbit s 1 mW 30 m very cheap Z Wave 868 MHz 9 6 kbit s 25 mW 30 m cheap 6 4 Network Functionality and Topology The properties of smart home networks can be divided into two main categories according to their functionality and topology Topology is related to the physical way in which de vices are connected to each other and thus it has a bearing on reliability complexity cost and performance Functionality on the other hand relates to how different functions are divided among networked devices and in which way they share resources Precise classi fication is difficult because there are numerous ways to implement networks and software the line between two different categories being very fine A centralised network for example a star topology relies on a central node for all its func tions All devices function through the central node and connections to other networks and remote networks also go through this node Remote nodes can be sensors actuators net work adapters or user interfaces and the central node for example a PC This topology greatly reduces the complexity required by remote nodes communication networks and protocols while the central node must have relatively fast processors and n
53. may be a long wait before smart homes become commonplace 183 Analysis 10 Analysis This chapter discusses the current and future state of smart homes what has prevented the smart home from becoming more popular than it is and what could be done to make it a more attractive option than the traditional home Discussion of these topics is then fol lowed by some concluding remarks 10 1 Reasons Why Smart Homes Still Are Almost Nonexistent Modern apartment buildings are not greatly different from their 1970s counterparts the ba sic layout and functions having remained much the same Windows doors walls bath rooms and so on have changed little and function in the same way as they have done for decades Naturally there have been certain changes in the form of new materials and new kinds of cabling Home networks are routinely used and installed but rarely does the uti lisation of state of the art technology extend beyond such innovation The homes of IT millionaires office buildings and concept homes usually contain a great deal more tech nology mostly related to building automation communication entertainment and securi ty Smart homes in the general sense are largely nonexistent This prompts the question as to why the smart home has failed to become accepted by the wider public despite the technological advances starting thirty years ago when the first smart home made its ap pearance The primary motivation for end u
54. menting energy saving fea tures Communication Healthcare issues staying in Choosing the right communi touch with relatives social cation methods knowing aspects when people are available Adaptivity Changes in lifestyle personal Modular design adaptive soft preferences ware Interaction Feedback to the user person Programming AI alised experiences 3 4 Reliability Ideally any networked system should be stable and sufficiently reliable to run 24 hours a day but in practice there will inevitably be periods of downtime for various reasons Net works may be down blackouts may occur software can crash or a mechanical or electrical fault can disable parts of the system In a home environment uncertainty is to be expected of a smart home system that connects everywhere and controls all appliances lights and sensors in the house Koskela04_1 Indeed a widespread concern about smart homes is what would happen if the controlling computer crashes or if there is a blackout Problems with computer software instability bugs etc have understandably caused people to worry about using such technology in their homes People are also somewhat mistrustful of a sys tem that does things behind their backs Tiresias08 and instead they prefer to remain in control of decision making in their daily lives Complete control could be handed over to the system only when there is nobody at home letting the system take care of lights secu rity and temperatur
55. more exotic variations such as ges tures and emotions have not been used as these require significantly more signal process ing and are more difficult to detect accurately Since many projects have been funded by the Academy or the department most of the research has been free of commercial empha sis or interest and no constraints have been imposed by third party services or partners Each project has had its own focus the Living Room emphasized comfort and user acti vation the Smart Home compatibility and integration the eHome user tests and usability and LIPS will concentrate on adaptivity and proactivity 109 Hardware Aspects 6 Hardware Aspects The backbone of a smart home is formed by networks and devices that perform measure ments adjustments and transfer information This chapter describes the various choices that smart home designers face available hardware and a discussion about various related aspects Hardware that has been used in TUT smart home research has been previously presented in Chapter 5 Devices in a smart home include sensors actuators home appliances entertainment devic es and computers Devices can be categorised according to their processing power intel ligence designed purpose and other features The following figure illustrates a categorisation that has made in terms of the perceived visibility of one device e g how users perceive it if it is clearly visible when used or if users are unawa
56. needs Moreover in order to keep the smart environment a relaxing and pleasant environment there have to be intuitive ways of controlling devices systems and appliances in the home This can be achieved by offer ing additional ways of communication interaction and control whilst also maintaining tra ditional ways of using devices in the home This also means that manual control should always be an option and that users can choose the most suitable way of controlling the en vironment For example if users want to change lighting levels in the living room they can either use wall switches voice commands a mobile phone UI or control them from a computer screen Offering many control UIs makes devices and applications ubiquitous as there can be multiple ways and locations from where a certain device can be controlled This definition does not specify what kinds of networks should be used or what kind of central intelligence there should be nor where it should be located An early description of the fundamentals of TUT smart home research is still valid today TUTOO When talking about the future home people usually envision complex home automation broadband networks talking machines robots etc People rarely start to think about practical issues and if the technology can really be useful to us in everyday life In the TUT 13 Definitions Smart Home research project these things are viewed from the users point of view Ho
57. newer standards are not necessarily back wards compatible Furthermore in spite of international standardisation efforts coopera tion and open forums unexpected standards can still emerge and become widely accepted by the industry even if they were not originally intended to be so It is however likely that eventually any standard becomes obsolete or is replaced by an improved version This means that the smart home system of the future must be adaptive and modular and be able to accept often radical changes and new technologies The other problem caused by poorly designed devices and UIs is not correctable with modularity home networking or extra software Instead the change should take place at a significantly earlier stage i e already during the design phase Improving usability mak ing it easier to connect and use networked devices and dispelling user irritation with tech 187 Analysis nology can be achieved by introducing good user interfaces Confusing button layouts misleading labels inconsistent terminology for example could theoretically be avoided by using standardised intuitive UI models Bringing UIs together within a single location will help to promote the creation of common norms and standards making it easier to man age a myriad of devices this could of course also be counter productive if the devices are poorly designed Allowing devices to be controlled from the UI of another device can achieve the same
58. no applications because of a lack of compatible products and there are no compatible products being developed because there is no application to use with them This also makes smart home research challenging because there exist relatively few installations and complete smart homes to allow larger scale studies to be conducted Mayra05 Studies must therefore be performed in labora tories or with mock up equipment making it even more difficult to test and develop real life smart home technology Ultimately there will perhaps be no single dominating de fac to home communication standard but many different ones will be supported instead This also makes economic sense since it would allow every application to be designed around a suitable infrastructure 185 Analysis Currently in smart home development there seems to be no killer application to make the technology more appealing and while the other benefits of smart homes as described in Chapter 3 are substantial they are either unknown to the general public or not attractive enough to justify the investment required A rapidly growing elderly population clearly points to the need for innovation and development in the fields of healthcare and assisted living at home and already society is investing huge sums to meet this burgeoning growth in demand Smart home development could thus capitalise on these healthcare issues by providing savings to the social services through helping the eld
59. now with the first element indicating that the UI must be constantly in stand by mode and the second that it should be available for use immediately Furthermore tasks executed by the UI should be made using as few actions as possible for example using shortcuts or customised presets The second re quirement is that UIs should be available wherever the users happen to be thus pointing towards context aware control and mobile UIs A final issue is to enable users to overcome their distrust of new technology and applications If trust has to won time and again it will severely undermine attempts to create the safe and relaxing atmosphere that a smart home should offer This relates to the broader objective that seeks to ensure that users remain in control of events in the home whether these are automated tasks or learned behaviour 9 4 Discussion Smart Home projects at TUT have shown that the use of multiple UIs can provide intuitive flexible ways for users to interact with the environment without the need for greater effort Since the tenants in the eHome are a young student couple it is clear that more research is needed on different age groups and different family units The next step in smart home re search would be to expand the research target group to include families with children Fur thermore elderly people would benefit greatly from smart homes since the technology would enable them to stay in their homes longer without support f
60. number of users and age groups made it dif ficult to conduct serious multi user research Mobile UIs were thus not very important ei ther even if the tenants would have preferred to use the TV remote control for other purposes such as adjusting lights and blinds The remote could be pointed at a specific de vice and could be used to directly control it which would be faster and easier than doing this from a graphical UI in a similar way as in Ritala03_2 Another improvement would have involved the dimmable halogen lights in the hallway and kitchen which in the eHome could only be controlled via graphical UIs Wall pushbuttons could not be used as they were wired to control 240 Volt lights only through the LINET network resulting in problems if the Home Controller was out of operation for some reason Probably the big 180 Findings gest improvement to the eHome which was suggested by the tenants was an adaptive home control system that would actually make the home smart The home control system could make observations about the actions of the users and make decisions and adjust ments on the basis of this learned data A thorough report on the findings from the eHome project was can be found in Tiiu Ko skela s Master s thesis Koskela03 According to the author of the thesis there are two im portant requirements to be considered when smart home UIs are being designed These requirements can be expressed as right here right
61. of domestic technology since the beginning of the 20th century 3 8 1 Early Smart Home Projects In 1945 after World War II Vannevar Bush wrote his article called As we may think in which he reflected upon the post war situation Bush45 After the war there would again be freedom for researchers to continue their studies into peaceful applications However he foresaw one particular drawback there was already a plethora of research and results being published worldwide that it would be very difficult to keep abreast of all the new information In order to speed up communication and make information more widely ac cessible he envisioned the need for new technical innovations such as automatic typewrit ers innovations in photography speech recognition and a device he named the Memex The Memex the name was derived from memory extender was for storing personal in formation such as text notes images and other items of interest This device would make information easily accessible and manageable acting as an extension of the human mem ory Bush s work includes many other innovations that have already come true but there are also some that still await implementation One early report from 1962 Englebart62 concerns the increasing complexity of everyday problems and the rate at which these should be solved New ways of improving the human response to these problems are studied and analysed One solution is to increase the inte
62. of interacting with the smart home also bring new challenges for human com puter interaction Intuitive UIs allow users to communicate using gestures speech and oth er natural ways whereas artificial intelligence can use similar ways of communicating back to the user Multimodal multi user and multipurpose UIs also present new challenges for UI design as UIs have to be able to filter information extract contextual information and present different kinds of content Different kinds of signal processing such as pattern and speech recognition also become important when information is being filtered from sensor data In order to attain seamless interoperability machine machine interaction must also be intuitive and suitable for cross platform operations Requirement 5 Dependability and security In order to achieve a safe dependable and secure AmI environment testing and verification methods have to be developed Both physical and psychological issues have to be taken into consideration and the system must also be secure against deliberate misuse Self or ganising and self testing software might provide some relief on the software front and as sist in the verification phase Different kinds of user identification e g biometrics also introduce the need to secure sensitive and private data These requirements seem reasonable and important issues to be considered in the design process However these requirements were written in 2001 and as it
63. of the potted plant In order to have better control of the amount of light the plant would receive an external dimmer module was also incorporated to enable control of an external flower light Figure 5 21 Living Room sensor module with temperature and humidity sensors left improved flower pot sensor module right 77 TUT Smart Home Research Motorised Blinds Screen and Curtains Light sources inside the apartment can be used when it is dark outside but otherwise there is a need for some form of control over the amount of ambient light entering the rooms Light level management might sound simple in theory but in practice there are numerous variables that affect the way we perceive light in the home Thus it is important to have many ways of creating or removing light either naturally or artificially With a proper con trol system it is also possible to save energy because lights left on during the day or when nobody is present waste energy The burden on the user is reduced and the cost and energy consumption of the controlling system can quickly return the investment All windows in the Smart Home are equipped with motorised window blinds and curtains that can be controlled and adjusted to a desired position Blinds can be used for precisely the precise amount of ambient light entering the room while curtains can be used to block all light which can be useful when watching TV for example There are projection screens in the di
64. on a cen tral UI is not necessary or useful for example in the case of the washing machine Perhaps it would be useful to monitor the machine or select a washing programme from the remote UI but the laundry still has to be manually loaded into the machine making a remote con trolling facility seem unnecessary Ideally as every appliance and device in a smart environment is able to share its function ality to other devices a question about where the UI tasks should be handled arises Ritala05 The first option is to mange all processing in the device itself i e UI menus and task descriptions are transferred to the remote device essentially a dumb terminal that knows nothing about the device that is being controlled The second option is to have some processing and handling done in the UI device leaving only commands to be sent to the target device Intermediate solutions can also be made depending on the processing power of the target and UI devices If processing is allocated to the UI device this con serves resources expended on other devices since these no longer require as much process ing power Maintenance also becomes simpler because future updates are required only for the software in the UI device It is not always clear however in which way functionality should be shared and therefore it must be considered in terms of each individual case and scenario 8 8 Natural UIs Natural UIs are essentially specialised control UIs th
65. pp 424 433 V Bush As We May Think in The Atlantic Monthly July 1945 J Case N Spanbroek The Relocatable House in Proceedings of the Dreaming for the Future conference Helsinki Finland 2001 10 pages Electronic Industries Alliance EIA 600 Standard 1984 available at http www eia org S Chatterjee Towards rapidly deployable intelligent environ ments in papers from the 1998 AAAI spring symposium AAAI press 1998 pp 31 35 CISCO Systems Internetworking Technologies Handbook Cisco Press 4th ed 2003 1128 pages COBA Connected Open Building Automation 2009 available at http www coba group com M H Coen A prototype intelligent environment in Proceedings of the First International Workshop on Cooperative Buildings Inte grating Information Organization and Architecture Lecture Notes In Computer Science Vol 1370 1998 pp 41 52 M Connolly F O Reilly Sensor Networks and The Food Indus try Workshop on Real World Wireless Sensor Networks REAL WSN 05 Stockholm Sweden 2005 D J Cook M Youngblood E Heierman K Gopalratnam S Rao A Litvin and F Khawaja MavHome An Agent Based Smart Home in Proceedings of the IEEE International Conference on Per vasive Computing and Communications 2003 pp 521 524 191 References Cybermanor Dearle03 Dertouzos99 Dey01_1 Dey01_2 Dey05 DLNA Dommel05
66. program nodes on the fly Otherwise it would only be possible to adjust certain parameters and if the appli cation or environmental factors suddenly change a simple adjustment in the node s pro gramming would not suffice Figure 6 9 Illustration of a message hopping through nodes to reach its destination 130 Hardware Aspects Current sensor network modules are still macroscopic in size but the technology is predict ed to shrink significantly making millimetre size modules possible Smart Dust Pister99 is one possible outcome of such miniature sensor networks making it possible to create affordable flexible networks with dense sensor coverage Sensors could be deployed rap idly by dropping them from an aircraft or shot on site in a mortar shell Smart Dust sensors are essentially Micro Electro Mechanical Systems MEMS that in addition to miniature electronics also have mechanical functionality Smart Dust sensors are too small to con tain batteries but instead they can harvest their energy from the environment for example from ambient heat movement or light Possible uses for Smart Dust include forest fire de tection environmental or biological monitoring military applications e g surveillance and healthcare patient monitoring 6 8 Actuators Controllable Devices Actuators perform the actual physical tasks in the home Traditionally a home has very few or none networked actuators and most actions are perform
67. push familiar as using the TV proprietary icons buttons and nat and touch screen ural UIs interface Popularity 3 4 times per 2 4 hours per Laptop on most Used most of the day day of TV usage of the time time heavy usage The tenants in the eHome showed great concern for safety and energy consumption They wanted to ensure that all unnecessary appliances were turned off even standby mode was undesirable and devices were sometimes unplugged They were therefore pleased that they could monitor and control appliances and lights in the home either from the WWW UI or remotely from the mobile phone Light switches in the hallway were also pro grammed to turn all lights off at the press of a button and the tenants made use of this whenever they went out A feature that the tenants wanted to have but one that would have been very difficult to implement was to use the mobile phone for transferring data and files from home to work and vice versa Though this would have allowed them to seam lessly take their work with them it would also have meant leaving their home computer switched on all day something they did not wish to do Many visitors to the eHome had learned in advance that the apartment was in some way different and thus had various preconceived ideas about what to expect There were people afraid even to enter thinking that they might accidentally break something or that the in telligence in the home wo
68. research in general spans multiple disciplines this focus will naturally limit the scope of the study to hardware software and communication Practical findings results from user tests and other issues related to everyday use are also presented Usability issues aside from prac tical issues psychological perspectives and software engineering e g PC software sig nal processing algorithms are thus excluded Smart environments or smart spaces are a more general name for this kind of research but in the context of this thesis the environ ment in question is a home as a physical space Mobile computing that takes place outside the home and is not related to controlling functions in the home wearable computing e g smart clothing and related technologies also lie beyond the scope of this thesis Networks exclude body area networks BANs and wireless personal area networks WPANS pro vided they are not used for controlling or interfacing with the home networks and devices 1 4 Contribution During the years that smart home research has been conducted at the Department of Elec tronics the research contribution has been made by a large group of research assistants and scientists in the Personal Electronics Group During the first years the group consisted of the author and two colleagues while in 2001 the group had almost two dozen people work ing on projects related to smart environments As a result there have been numerous people Intr
69. result improving interoperability and adding flexibility As the findings in Chapter 9 demonstrate an optimum situation is reached when users have many interfac es to choose from This means they can choose the most suitable UI for any given situation whether it is in the living room in the backyard or away from home This also presents a possible solution for improving social and informational compatibility as this is often a personal and cultural issue which is also easily affected by other people around us Finally as a home user interface the mobile phone is becoming an ever more suitable control de vice and may eventually become the most popular smart home UI as its features and con nections improve In order to provide an answer to the research question raised in Chapter 1 for users to fully benefit from smart home technology a smart home should e Provide sufficient benefits for users e Be modular and adaptive with room for future expansion and modification e Be easy to install and maintain or include a maintenance service e Provide improved usability and controllability through appropriate UIs e Allow the user to be in control and to dictate higher level system behaviour Moreover if compatibility problems are the cause of frustration and confusion among us ers it is logical to assume that mitigating this problem would already be a large step to wards increased usability and reduced complexity 10 4 Conclusion Today i
70. room The TV set is equipped with a serial port and with a mote connected to this port it is possible to obtain various types of information from the TV e g turned on off which channel is being watched etc This information is useful for determining how the TV is being watched Additional informa tion can be retrieved from a microphone sensor which also uses a mote for measurement and communication The microphone records only sound levels and performs no addition al filtering or frequency discrimination Sound levels are used for detecting whether people are talking listening to something or relaxing in quiet Sensing in the LIPS project is designed to be unobtrusive so as not to disturb the user or be in any way conspicuous This also means that there are to be no wearable sensors or actions required of the user in order to perform a measurement or detection Floor sensors can function without the user noticing them and presence and biosignals can be recorded remotely with a Doppler radar Zakrzewski06 5 6 4 Service Discovery In order to keep track of all devices sensors actuators and their functionalities in the home a dynamic protocol that can easily adapt to changes is required A hard coded piece of software such as the one currently running in the Smart Home is unable to deal with a situation where a sensor fails or a new device is connected to the home network In the former case the sensor reading will simply be unavailable
71. services and layers but the most important features are component models and applications that can be started stopped and installed without requiring a reboot This makes it is easy for the application to download new drivers when new devices are added and remove them when they are no longer needed The original specifications were designed with service gateways in mind but lately the standard has also spread to the automotive automation and entertainment industries However the list of certified products includes only six items to date 7 2 2 COBA Connected Open Building Automation COBA COBA is a standardised operating system developed for building automation which provides a standard interface to all building management systems Development started in 2000 together with telecom IT and construction industries and the first applica tions emerged a few years later The goal was to create a worldwide open de facto standard for a building operating system which would give users easy and secure access to the functionality of a smart environment using various UIs COBA is directed mainly towards building management security and electronic real estate control and its focus has largely been on large buildings factories and offices rather than homes However the situation has changed during the last few years and now COBA is also making a move towards homes 146 Software Architecture COBA uses definitions of the building its te
72. that exist and the terminology closely related to the topic The definitions are compared to the guide lines used in smart home research at TUT In addition a comparison is made between smart homes and home automation There seems to be little universal agreement as to what is meant by the concept of a smart home This is immediately evident from the many different names used to refer to the sub ject itself smart spaces ubiquitous computing pervasive computing and ambient intelli gence to name a few There are several variables and nuances that differentiate between the different definitions A distinction can also be made between information processing e g ubiquitous computing and control of the environment e g proactive computing The space that these definitions refer to can be any building or room provided it contains the necessary infrastructure e g intelligent environments Differences between defini tions start to emerge when considering what the smart environment contains how it works and the kinds of functions it offers and the kinds of interactions being contemplated For example the term smart home can be used to refer to a traditional home where technology works unnoticed in the background or it might refer to a high tech home with broadband networks video on demand and big screens on the walls Frances Aldrich Aldrich03 presents a classification for smart homes according to differ ent levels of communication whether t
73. that uses sensors and actuators to achieve its purposes The MavHome name comes from Managing an Adaptive Versatile Home and the focus is to maximise the comfort of its users and minimise operating costs In order to achieve this the house must recognise and predict the actions of its users and adapt to their routines For this purpose MAVhome uses several prediction algorithms The MAVhome agent architecture consists of four layers a decision layer for executing actions based on information obtained from other layers an information layer that gathers generates and saves information that could be beneficial lat er acommunication layer that manages information exchange between layers and a phys ical layer which consists of physical devices in the smart space Smart spaces can provide users with automated features energy savings improved safety and security According to Youngblood05 a smart environment is able to gather infor mation from the environment and use this and its knowledge on the inhabitants to improve the user experience For testing purposes the project has two test environments at the uni versity called the MavPad and the MavHome A virtual counterpart a simulated work space is also used for verification The test environments are equipped with sensor networks powerline communication for lighting adjustable window blinds and HVAC controls Sensors include light humidity smoke temperature and motion and they ar
74. they have found their way into ordinary de vices that are used in daily life As a consequence these devices also contain fairly complex software that has been written for a specific application White goods such as washing ma chines microwave ovens etc have their own rather straightforward tasks and programs while higher end DVD players TVs and stereo sets are able to connect to home networks read USB memory keys and display pictures from digital cameras Even tiny temperature sensors fingerprint readers and light switches can contain embedded software Since the applications and requirements vary significantly it is clear that the embedded software in all these devices can also vary considerably from device to device In theory it matters lit tle what programming language has been used for writing the software what kind of CPU it runs on or how the device interfaces with the rest of the world provided it conforms to certain standards and protocols This is also a reason that systems designed at TUT use text based messages for communicating with each other e g the Smart Home UI proto col they are easy to parse even in microcontrollers they are easy for developers to debug and they are platform independent For more demanding applications there are many options available Certain real time op erating systems RTOS such as uC OS II UCOS allow software to be written to run in almost real time e g meaning that it is possible
75. timer can execute multiple tasks upon activa tion A timer can be set to repeat at any interval or it can be activated only once and then terminated User Ul s Parser interfaces Connections Message Dis patcher Timer Tasks Device Devices A Comnections XML Figure 5 36 Components of the Smart Home server Vainio06 Software Modules Running on the Server Currently the server PC runs the Home Controller core software which in turn connects to other modules running on the same PC or throughout the Smart Home network For speech control the SCARS software continuously monitors microphone inputs ready to input the received sound to commercial speech recognition software Philips Freespeech Viva A learning fuzzy software module monitors behavioural patterns using sensors and actuators in the Smart Home updating its database and performing tasks when appropriate Another PC in the Smart Home runs a fingerprint scanner software which compares fingerprints of people to a database opening the door for authorised individuals Once a positive match has been made it connects to the server opens the door and outputs a welcome message using the SCARS interface 92 TUT Smart Home Research Wall panel UI Tablet UI SCARS JI Mobile phone UI Smart Home network Vs UI E Controller Fuzzy Device Groups Timers engine Models Device lt
76. tmote sky module contains a MSP430 microcontroller 1024 kB flash memory 2 4 GHz 802 15 4 compliant radio trans ceiver USB connector and humidity temperature sensors A mote node is powered either by two AA size batteries or by a DC power source and the radio has an effective range of 125 m which is sufficient for general indoor communication Motes are usually pro grammed to run TinyOS TinyOS an operating system that has been specifically devel oped for sensor networks and other mote applications TinyOS applications are programmed using NesC NesC a dialect of the common C programming language The modularity and popularity of the TinyOS software platform makes it possible to use motes for many purposes and many kinds of application software can already be found on the Internet New devices will all be connected to the home network using motes legacy devices con nected will be either using existing networks or connecting them to the serial port on a mote One problem remains however the serial port on the mote is shared with the radio transceiver and thus care is needed to ensure that these two do not conflict Additional sen sors are also required for detecting contexts and for this purpose new sensors are also be ing installed in the Smart Home Sensors Sensors designed for LIPS are intended to complement the sensors already present in the Smart Home Some sensors are already being obtained automatically since the mote sensor
77. to perform almost any func tion in the home A portable wireless switch could be helpful for wheelchair users for example being much easier to use than reaching for wall mounted switches When location awareness is added a context aware light control could be imple mented e Touch sensitive areas using various types of technology for sensing touch weight or pressure floor sensors capacitive film pressure sensitive switches strain gauges An array of floor sensors could also function as UIs for example double tapping with the foot in front of the door could indicate a desire to communicate to the home soft ware that the door should be opened These UIs might be completely hidden inside the home structures or furniture which might make them impossible for guests to detect and use but they would also be unobtrusive for people who know where they are and how they operate e Auditory UIs that recognise sounds for example the classical light switch that recog nises the sound of hand clapping More advanced features are available through speech recognition using microphones and recognition software but problems arise when ambient sound levels rise making it difficult to filter out spoken words from background noise e Optical sensors that detect changes in light level light level sensor recognise shapes low resolution IR or video camera or motion passive IR are on the borderline between sensors and UIs These sensors provide a
78. to their power consumption and can seem as an unnecessary step for users as external transformer units require their own space around power outlets Furthermore as mobile devices mostly are battery powered they are charged only momentarily with the power supply running idle for the rest of the time it can seem to users as a waste of energy Indeed various voltage converters power supplies and battery chargers add to the energy con sumption of the device and until a more efficient way of providing power to devices is invented the only way to remedy this is by creating more efficient power supplies and bat teries Wireless power transfer could eliminate the need for power cables but it will prob ably not increase energy efficiency in any way Chapter 3 2 1 presented five requirements for future Ambient Intelligence technology ISTAGO1 with the first requirement being unobtrusive hardware This would imply small wireless sensor and network nodes that are able to perform measurements and tasks without unnecessarily disturbing the user The second a seamless communications infra structure and third dynamic and massively distributed device networks requirements re quire a flexible network infrastructure and service discovery protocols that are able to connect devices to the home network without requiring much or any input from the user The fourth requirement natural feeling human interfaces benefits from embedded sensors and unique and natura
79. used and pressing a few buttons on the phone is usually much faster than turning on the laptop PC or TV for making a simple adjustment The WWW UI was primarily used for making more complex settings or supervising conditions as it was too clumsy and slow to start up just to turn off a light or adjust window blinds The WWW UI was also rarely used for making adjustments to another room since the eHome apartment was too small for this function to be useful and both tenants usually shared the same room with light switches easy to reach Users would default to manual control when not in the same room as the laptop and as a result in these cases they made limited use of the WWW UI Table 9 1 below summarises the properties and findings for eHome UIs Table 9 1 Summary of eHome UIs Phone TV Web Manual Pros Always on Big screen Touch screen Familiar con available famil familiar control customisation trols physical iar style intuitive UI Cons Expenses Shortcomings of Slow start up Very limited dependent on infrared controls dependent on the functions availa network cover loud computer ble age 178 Findings Table 9 1 Summary of eHome UIs Phone TV Web Manual Mobility Excellent Limited range Inside the apart None for remote con ment limited trol a few metres battery life for laptop Ease of use Easy if phone is Easy the same Medium uses Very easy
80. without the need for expertise in programming or hardware engineering Adaptivity is also important for the comfort of the users by let ting them modify the home into a pleasant living environment Interaction In addition since a smart home contains several kinds of user interfaces and is capable of many kinds of feedback there are numerous ways of interacting with users This interaction can be used in many ways to enhance the experience and activate the user for example the system can alert the user to items requiring attention or to potentially dangerous situa tions It can be used to relay interesting information e g display or read aloud news head lines or e mails or even to establish a casual dialogue between the user and the system All of this naturally depends on the complexity and capabilities of the artificial intelligence in the software components of the smart home Table 3 3 summarises main benefits from smart home technology and also lists related challenges 35 Smart Homes Table 3 3 Summary of smart home benefits Driving Force Main Challenges Ease of use Improved usability remote con Zero configuration protocols trol UIs natural interaction automatically reconfigurating UIs Safety Added safety for users remote Privacy dependability of the control and monitoring system Energy savings Cost savings environmental Increasing efficiency imple impact
81. 1 Example of how middleware can be implemented Bernstein96 In a home environment the middleware would essentially be a software package running on a home server A dedicated home gateway or a similar device could also function as a home server but recently the future of such poorly expandable and upgradeable machines has become uncertain Game consoles home theatre PCs and set top boxes are constantly gaining popularity and new features and they may soon deliver similar functionality at an affordable price However a dedicated control unit would be preferable as it would be more reliable and tailored for this specific application compared to a PC with random soft ware modules running at the same time but problems arise when updates or changes to the program have to be made Interfacing with a home control box would probably require access through another terminal or similar complicated tasks When devices are added services or preferences are modified so that changes to the server probably would have to be made either by the user or automatically by the system A home server with appropriate middleware could offer services such as timers logging functions preset modes distributed UIs and group controls The middleware also makes controlling the home more user friendly allowing users to make higher level decisions in stead of worrying about network standards software packages and driver versions Valtchev02 The middleware can also b
82. 28 3 2 2 User Require Ment eneren e a an 31 3 3 Benefits That a Smart Home Can Offer 0 0 ceeeeeseceseecereeeneeeeneees 33 BA Reliability cosciensa ee E N E En Eik 36 3 5 Mobility and Mobile Computing ssssesesseseesessreesssrerrsrssrsssrsrrsreeeess 38 3 6 Adaptive Systems Learning essseeeessessesessrrrrsrtersrrerrrrererrereserseseet 39 3 7 Business Perspectiye noeneen ne eee ieee ee aat 39 3 8 Smart Home History and Future oo eee cesses ceeeeeeceeeeeeeeeenes 40 Table of Contents 3 8 1 Early Smart Home Projects 0 0 eee ceseeeeceeeeeeeeeeeees 41 3 8 2 A Brief History of Domestic Technology s es 42 3 83 FUTE cann ace ores tes ek eee es ee ce eee 44 4 Related Work swsccsssccsccccccscatescuscevescccdeatesdusstecsecccussessecdeitesuscutesseccsedesudeccetsesce 45 A LiOverview 220 00 tiie nein ie nee tc nce a 45 4 2 Other Smart Home Projects ce eeeeesessecenceceeeeeseceeeeeseeeaeeceeeeenees 45 4 2 1 Telenor Fremtidshuset Future Home cccceeeeceeeees 45 4 2 2 Philips Home Lab ou eee eee eee ceseeseceeceeeceeeeeeseeseaeeseens 46 4 2 3 The Adaptive Home 0 cee eceseececseecsecnsesaeeeeeseens 47 4 2 4 Georgia Tech Aware Home o0 ececeeeeeseecseeeeeecneeeseens 48 4 2 5 Microsoft Easyliving 0 eee ceceseeesceeeeeeeeeeeeeeeneeseeeaee 48 4220s Ma VHO E ee n a e E a E esis a 49 4 2 7 HomeSoft LONIX smart home concept cece 50 4 2 8 Duke Smart Home ccecceeesceesscecesseceseeeeee
83. 5 Home Gateways A home gateway or residential gateway is a device that connects various home networks together and or to the Internet while offering diagnostics remote control facilities and user interfaces Valtchev02 Since home networks can consist of many different kinds of networks and have incompatible physical connectors and protocols a gateway can be used for connecting all these together Home gateways are often based on standard PC hardware and thus possess sufficient capabilities to run software that enables them to do other things as well For example it can offer users a way to control the home from a remote location and a third party can perform diagnostics and maintenance tasks on the home from their offices Home gateways can be divided into two groups according to their function physical gate ways and service gateways A service gateway connects functional software blocks serv ices together and through them provides different kinds of services to devices in the home network such as service discovery content sharing and remote management A popular platform for implementing service gateways is OSGi OSGi A physical gateway is a sim pler device as it is more concerned about physical connections routing and firewalling A physical gateway thus connects many kinds of networks together streams media and con nects the home to the Internet In the early 2000s as the Internet boom was gathering momentum there were severa
84. Aldrich s classification of smart homes Aldrich03 presented in Chapter 2 TUT smart home projects can be classified to fit the second third and fourth category e Homes with intelligent communicating objects intelligent appliances and objects are able to communicate with each other and exchange information The Living Room e Connected homes homes with both internal and external networks making them and their services accessible from inside and outside The eHome e Learning homes homes that record and adapt to behavioural patterns of their users control the devices accordingly and predict the user s future actions The Smart Home Table 9 2 summarises the research goals and contribution of TUT smart home projects and it also presents short comments related to each project 182 Findings Table 9 2 Summary of requirements and findings from TUT research projects structure Creating arelaxing environment User activation Centralised control trollable devices A relaxing and pleasant environ ment First prototypes Project Research Goals Contribution Findings Living Room e Testing space for e A smarthome lab Centralised UI smart home infra oratory with con inadequate Other ways of con trol required Implemented net works were too primitive and device interopera bility Expanding research to a com plete apartment laboratory Home Controller middleware EPIS service dis covery pro
85. As these often com puter based powerful ICT systems penetrate our lives their effective and dependable con figuration and integration becomes ever more challenging Tiresias08 In the worst case every new device has its own user interface operational logic and func tionality that users have to learn Trying to manage all devices making them work together and attempting to transfer data or information between them can prove difficult or even impossible All this creates additional frustration stress and problems the very opposite of what such technology is supposed to deliver Koskela03 Indeed the motivation behind the design of smart homes is to reduce stress and improve the usability and living experi ence of the people living there Weiser96 making their lives easier Smart homes should also avoid automation and instead assist users in controlling their environment Intille02 making the user a supervisor not a controller The emergence of smart home technology however is not entirely without problems One problem is that new kinds of technologies and new applications are introduced to the mar ket with no consideration of their usefulness in real world scenarios and everyday life Thus many technological innovations are available but they have not been designed to work in the various scenarios where they could be used or together with other available technologies This lack of a common standard for connecting different home applianc
86. B European Installation Bus is a European network communications protocol for building automation Nowadays it is known as the KNX and administered by the Konnex Association KNX KNX is compliant with EN 50090 the European Standard for Home and Building Electronic Systems The EIB KNX standard aims to connect all electrical functions of a building into a single functional network HVAC lighting and security are all controllable with a single system The standard includes three levels of configurations that devices can have A E or S mode ranging from plug and play zero configuration to fully customisable hardware Physical interfaces for KNX include twisted pair cabling powerline networking radio wireless and Ethernet Networked devices can range from small 8 bit microcontrollers to PCs using various operating systems Other kinds of communication interfaces can also be used for example HAVi HAVi which is a based on IEEE 1394 Firewire HAVi allows home audio video devices to be connected using embedded middleware making it possible to share content and offer serv ices to applications One practical limitation however is caused by the maximum cable length of 4 5 metres Proprietary networks also exist for example LINET described in Chapter 5 which is primarily used for controlling lights and electrics and LonWorks LON which is mostly used in larger office buildings and industrial applications The following table summar
87. Biosensors Sensors for monitoring the space s inhabitants present the biggest challenge They should be as unobtrusive as possible without interfering in users lives or requiring extra atten tion Wearable biosensors e g electrocardiogram or blood pressure have not been con sidered in this research but other ways of measuring and monitoring people have been investigated Sensors mentioned in the previous paragraph can also be used for this pur pose Heart rate for example can be monitored using Doppler radar Zakrzewski05 with capacitive or pressure sensitive sensor films installed under the floor or in the bed EMFi Body temperature can be sensed with infrared cameras weight with pressure sensitive sen sors and height with capacitive sensors Valtonen09 or by visual means e g using video cameras Movement and activity can also be detected with accelerometers and angular ve locity sensors installed in various locations such as UIs furniture and small portable de vices These can indicate whether or not a device has been used or the kinds of movements the user has performed Biometric measurements could give useful information to the home system on the inhabitants state of mind alertness and activities sleeping tired stressed etc Also when the biometric data is sent to a health centre patients could be monitored at home without having to stay in hospital This would facilitate earlier recovery and help reduce the
88. Duke Edwards01 Emery03 192 Cybermanor Your premiere home technology design and integra tion partner 2009 available at http www cybermanor com A Dearle GNC Kirby R Morrison A McCarthy K Mullen Y Yang RCH Connor P Welen A Wilson Architectural support for global smart spaces in Lecture Notes in Computer Science 2574 proceedings 4th International Conference on Mobile Data Manage ment MDM 2003 Melbourne Australia 2003 pp 153 164 M L Dertouzos The Oxygen project the future of computing Scientific American Vol 281 Issue 2 1999 pp 52 55 A K Dey Understanding and using context in Personal and Ubiquitous Computing archive Springer Verlag Vol 5 Issue 1 2001 pp 4 7 A Dey P Ljungstrand A Schmidt Distributed and disappearing user interfaces in ubiquitous computing in extended abstracts of the Conference on Human Factors in Computing Systems CHI 01 Seattle USA 2001 pp 487 488 A K Dey J Mankoff Designing mediation for context aware ap plications in ACM Transactions on Computer Human Interaction TOCHI Vol 12 Issue 1 2005 pp 53 80 Digital Living Network Alliance available at http www dlna org home H P Dommel R Wagner A Doran R Edwards A Middleware Framework for the Adaptive Home in Third International Confer ence on Smart homes and Health Telematics ICOST 2005 Sher brooke Qu bec Canada 2005
89. Duke Smart Home 2009 available at http www smart home duke edu K Edwards R E Grinter At Home with Ubiquitous Computing Seven Challenges in Ubiquitous Computing G D Abowd B Bru mitt S A N Shafer Eds Berlin 2001 Heidelberg Springer Ver lag pp 256 272 V K Emery1 P J Edwards J A Jacko K P Moloney L Barnard T Kongnakorn et al Toward achieving universal usability for old er adults through multimodal feedback in Proceedings of the 2003 conference on Universal usability Vancouver Canada 2003 pp 46 53 References EMF Englebart62 EPoSS08 Forman94 Futurelife Galushka06 Gandy00 Gann99 Gerhart99 Emfit LTD Emfit Ferro Electret Film 2009 available at http www emfit com sensors sensors_products emfit film D Englebart Augmenting Human Intellect A Conceptual Frame work AFOSR 3233 October 1962 139 pages RFID working group of The European Technology Platform on Smart Systems Integration EPoSS Internet of Things in 2020 Roadmap for the Future 2008 27 pages G H Forman J Zahorjan The Challenges of Mobile Computing ACM Computer Vol 27 Issue 4 1994 pp 38 47 Futurelife project presentation 3 years future house and in future 2003 available at http www futurelife ch 19 pages M Galushka D Patterson N Rooney Temporal Data Mining for Smart Homes Lecture Notes in Computer
90. Engineering in North Carolina USA Duke It has been designed from the outset to function as a test environment where students can experiment and improve different technologies that the smart home contains Students are also expect ed to create their own designs and also provide input on current installations The project also has industrial partners seeking to strengthen the market for integrated smart home sys tems The building contains a general area for relaxation a computer lab media room and student bedrooms The key element in the design is energy efficiency and many of the building s systems are based on this philosophy Rainwater is collected for use in bath rooms solar power is used for heating water and the building uses heat pumps for heating and cooling Removable panels on the walls allow students to install and test new designs Figure 4 8 Duke Smart Home Photos Duke University 4 2 9 HP Cooltown 2001 In 2001 Hewlett Packard laboratories at Palo Alto developed a laboratory named Cool town for ubiquitous computing HP describes it as A vision of mobility connectivity com 51 Related Work munity and transformation based on open standards and user needs Barton01 One of the main themes in Cooltown is Internet connectivity everyone everything and every space is connected to the Internet and through which many kinds of devices can interact HP believes that the Internet or more specifically the World W
91. F 84 TUT Smart Home Research power of the module is about 10 dBm which permits a range of 20 m in a normal living environment and the maximum data rate of the module is 20 kbits s Figure 5 29 Smart Home RF module Mikkonen06 Bluetooth devices are connected in a similar way using a proprietary Bluetooth module The module houses a Samsung BTMZ5012A0 single chip transceiver and an interface for initiating a pairing sequence or re connection The module has to be paired with the Blue tooth dongle in the server after which it opens a virtual serial port connection between the two Thus the Bluetooth connection also appears as a standard serial port to the server The Bluetooth module consists of a Samsung class 1 integrated transceiver module RJ 12 con nector buttons for pairing and connecting and indicator LEDs Figure 5 30 Smart Home Bluetooth module Myllymdki03 An alternative to wired RS 232 communications is Ethernet and a module for converting the RS 12 connector specification used in the Smart Home to standard Ethernet UDP com munication was also made Makinen03 This made it possible to communicate over larger distances and other networks 85 TUT Smart Home Research Figure 5 31 Smart Home Ethernet module with RJ 45 and RJ 12 connectors Mdkinen03 5 3 2 Smart Home User Interfaces Smart Home user interfaces were designed to be totally different from the prototype UI in the Living Room A touch sc
92. Kosteusprosentti 36 m Kello 10 52 11 Figure 5 8 Main screen of the Living Room UL To the left are quick buttons for accessing lighting air quality home theatre etc The main page shows temperatures and device information f KotiOhjain Ohje m J rjestelm t Monitorointi Valaistus Kotiteatteri Kukat PE Iman tila Ovi i Turvalaitteet Figure 5 9 Lighting control screen Individual spots can be selected from the image and brightness adjusted from the slider 67 TUT Smart Home Research KotiOhjain iol xi Ohje m J rjestelm t Pikan pp imet __Videotykki Monitorointi m Kotiteatteri DYD A K Vey Video moodi C Video vo CO AN On zr gi Valaistus Kukat Tietokone moodi gt gt Pad laguna vaikko Y Valitse Kauko ohjaimet liman tila ce pe m Turvalaitteet Sulje laitteet Videotykki DYD Video Vahyistin Figure 5 10 Home theatre controls with mode selection buttons 5 2 3 Controllable Devices in the Living Room Flower Pot The first device constructed for the Living Room was a flower pot monitor Here it was decided not to implement automation since an aspect of a pleasant environment is active participation The user would take care of the plants though there was a backup system to activate reminders should the
93. LIPS presented in Chapter 5 Another TUT research project the Morphome is also presented in this thesis because it is closely related to other projects However the author has not been involved in this particular project Introduction The aim of this thesis is to contribute to smart home research by presenting the following items e Practical experiences from three smart home projects e Review of related work and research e Findings and thoughts on real world implementation of smart home hardware soft ware and user interfaces e Advantages that smart homes can offer challenges that still remain e Guidelines for creating a home network and control system 1 5 Motivation As mentioned above modern life involves the use of a wide range of electronic devices and appliances One major problem is that each device has its own operational logic and user interface and the way one device works might be very different from another similar unit from another manufacturer With the ever increasing number of devices being intro duced into our daily lives it is hardly surprising that users are overwhelmed by the need to remember how to operate them all Furthermore the number of features and functions that a single device is able to offer is constantly increasing as well All this might lead to confusion frustration and cause situations where users only employ the basic features of a device and ignore the rest Green04 For example a user o
94. Landay An infrastructure approach to context aware computing Human Computer Interaction Vol 16 Issue 2 2001 pp 287 303 References Hossain08 Hyv nen03 H kkinen00 IBM98 inHaus Intille02 IPv6 IrDA ISTAGO1 ISTAGO3 ISTAG06 Jeong07 M M Hossain V R Prybutok Consumer Acceptance of RFID Technology An Exploratory Study in IEEE Transactions on Engi neering Management Vol 55 Issue 2 2008 pp 316 328 J Hyv nen Speech control system for intelligent environment Master of science thesis Tampere University of Technology Elec trical Engineering Department Tampere Finland 2003 T H kkinen M Viskari J Vanhala HereUR A Personal Loca tion aware Information Manager for Wearable Computers in Pro ceedings of the OZCHI 2000 Conference Interfacing Reality in the New Millenium Sydney Australia 2000 pp 22 23 IBM Pervasive Computing 1998 available at http www 304 ibm com jct03004c businesscenter venturedevelopment us en featurearticle gcl_xmlid 10406 nav_id transopp inHaus Innovationszentrum der Fraunhofer Gesellschaft 2009 available at http www inhaus zentrum de site_en S S Intille Designing a home of the future in IEEE Pervasive Computing April June 2002 pp 80 86 IPv6 The Next Generation Internet 2009 available at http www ipv6 org Infrared Data Association 2009 available at http w
95. Life is probably the first smart home experiment with real inhabitants Parents Ursi and Dani Steiner receive regular visitors to the house and interested parties can also visit the project website for further information and webcam feeds The house contains hardware and applications from several companies who have also pro vided most of the funding The house uses the EIB bus for intra house communications and controls together with a Fitbox service provider portal that provides home automa tion controls and remote control facilities The home also contains controllable lights shutters windows doors and a front door with access control employing keys fingerprints or wireless tokens Control is facilitated through a wireless SIMpad tablet speech recog nition and web based UIs Energy savings are also important and the house utilises fresh air ventilation by opening and closing windows in suitable weather conditions Fresh food and goods are delivered to a Skybox essentially a refrigerator from a third party provider The Skybox contains sensors for registering its contents and users are able to order sup plies from a wall mounted panel as needed The lawn is mowed by a robot and an auto matic irrigation system waters the plants The Steiner family enjoyed the remote control facilities and the possibility of adjusting various parameters in the home However they also observed that a few single appliances do add to the comfort but t
96. Myllym ki J Vanhala Living Room in Proceedings of Dreaming for Future Future Home Conference University of Art and Design UIAH Helsinki Finland 2001 page 12 L Kaila Expanding Smart Clothing with Smart Environments in Proceedings of the Doctoral Colloquium session IEEE International Symposium on Wearable Computers ISWC 2005 Osaka Japan 2005 pp 5 7 L Kaila A M Vainio J Vanhala Connecting the Smart Home in Proceedings of IASTED International Conference on Networks and Communication Systems NCS 2005 Krabi Thailand April 2005 6 pages L Kaila L Lehti T H kkinen P Myllym ki V M kinen J Van hala The BluePost a smart car heating system in Proceedings of the 25th International Conference on Discributed Computing Sys tems Workshops ICDCS 2005 Columbus Ohio USA 2005 L Kaila J Mikkonen K Palovuori J Vanhala InfoCube A 3D Handheld user Interface Poster in Ambience 2005 Tampere Fin land 2005 L Kaila J Mikkonen A M Vainio J Vanhala Open Architecture for Practical Implementation of Smart Homes in Proceedings of Telecommunications Networks and Systems TNS 2007 Lisbon Portugal 2007 3 pages L Kaila J Mikkonen A M Vainio J Vanhala The eHome a Practical Smart Home Experiment Pervasive 2008 workshop Per vasive Computing Home Sydney Australia 2008 L Kaila J Hyv nen M Ritala V M kinen J Vanhala
97. Nodes Since many features and requirements of networked nodes in a smart home are quite sim ilar it makes sense to create a generic network node that can be used in multiple applica tions These kinds of modules are available from many manufacturers and they typically contain a low power microcontroller wireless radio interface some I O connectors flash memory power regulation a few built in sensors and A D converter These modules can be used as sensor platforms utilising built in sensors network interfaces to other equip ment actuator controllers or network relays Ideally a node would be millimetre sized and thus small and cheap enough to be used in vast quantities and distributed over a specific area This smart dust as mentioned before could be used as a deployable sensor field embedded into structures etc Practically however these modules are not size limited as the batteries that they use are large anyway The rest of this chapter presents a few com mercial sensor modules and their primary properties 135 Hardware Aspects Berkeley mote The mote platform briefly introduced in Section 5 6 3 moteIV was developed by Uni versity of California Berkeley and has since been very popular for prototyping sensor net works around the world The mote family consists of several different models MICA telosA telosB Imote etc that vary in size performance and abilities Motes are common ly used in conjunction wi
98. Sensors essssssssssesseeeeessrsssserseressereeseseesee 126 6 6 2 Location SeMsors o cien e e e a a E 126 6 6 3 BIoS nSorS moe e e e AEE hee ee 128 6 6 4 Sensor Data Processing sseseseeseeesereessererrsreressserrreererers 129 6 7 Sensor Networks erein eE Era EEEE KERRE i 130 6 8 Actuators Controllable Devices ccccccesscceeesesssececesestseeeeeeeseaees 131 6 9 Power COnSUMPTtiON oo eee ce cess ceeeeeeeeaeeseecaeesaecaecaeeseeeeeaeeeaes 132 6 10 A Typical Smart Device oe cseetecneceneceeenseeeeeeeeeeeeeaeeaees 133 6 11 Smart Sensor Nodes serii erra eer EEr ES 135 612 DISCUSSION ea E dansk E a mens 140 6 13 SUMMAtY ceiseniieinoes s ernisee veeti iei EE E EEE EEE EEEE E iE 142 7 Software Architecture sccsccssccscscsscesccssssscesscssesssseseesscssscssscsssseesoeses 143 7 1 Centralised vs Distributed Intelligence ole eee eeeeereeeees 143 TQ MAddeWate ceana ets ciated eee ees 144 7 2 1 OSGi Open Services Gateway Initiative 146 7 2 2 COBA Connected Open Building Automation 146 TB Service DISCOVELY sites ieee cage Wa aera ee as MA Ses 147 7 4 Early TUT Communication Protocols eeceeeeeeseeeeeneeeees 149 7 4 1 Living Room Serial Protocol MOB Bus eee 150 7 4 2 Smart Home Serial and UI Protocols ccceeseeeereeeees 151 7 4 3 LIPS Service Discovery and Network Protocols 152 7 5 Embedded Software ccccccccsssssssecessnsecessssessscec
99. al functionality For example instead of focussing on a lamp as a physical item and its functionality the designer would concentrate on the light as an expe rience and how it affects the space surrounding it Experience Prototyping is valuable for communicating ideas to an audience understanding existing user experiences and context as well as exploring and evaluating design ideas It simulates important aspects related to relationships between people objects and environments By allowing other people to di rectly engage in testing it gives everyone a shared point of view Experience Prototyping enables researchers to understand user experiences and their contexts while new designs are being evaluated and testing can be done either in a laboratory environment or in more realistic surroundings New tools such as 3D modelling allow easy visualisation of pro totypes and environments and provide the possibility of quickly modifying the environ ment and performing user trials inside the virtual environment 3 3 Benefits That a Smart Home Can Offer The motivation behind the creation of smart homes is to improve the quality of living make life less stressful and enable people to spend more time with their families From general safety ease of use customisability to remote user interfaces smart homes have much to offer Ease of Use Probably the greatest advantage smart homes offer is the ability to flexibly control the home New UIs communica
100. all lights on off dining area lights bedroom lights kitchen lights and living room lights dimmable All controls are touch sensitive buttons operate by touch and the slider by touching and sliding a finger on the scale If the panel is out of use for approximately ten seconds it turns itself off A capacitive touch sen sor performs touch recognition and a small speaker beeps when adjustments are made The panel is connected to the home network using Ethernet and serves as an alternative to wall mounted pushbuttons An alternative design was also created being similar in func tion but the button layout was redesigned to display the floor plan of the apartment instead By touching the appropriate room the light controls for that room can be controlled and a slider for the dimmable living room lights is also present A further updated version of the panel Ulis currently being designed with more intuitive graphics and functions The ulti mate goal is to make the electronics thin enough so that they can be embedded in a thin film or inside the actual building material plastic wood textiles etc Figure 5 34 Smart Home wall panel UI prototypes one embedded into the end of a cabinet left and a newer version located next to the front door Raula09 Home Server The centre node of the Smart Home network is the server PC This is basically a standard miniature sized PC running Windows 2000 and is connected to all devices and sensors in the
101. also offer control of electrical equipment but it also needs adapters and transceiver modules for control of home electrics One advantage is that a light can be directly controlled without needing relays or extra installations if screw in socket modules see Fig 5 48 are used In apartment buildings a practical limitation arises from the fact that it is rarely possible to control the heating and ventilation of a single apartment In large buildings such facilities are centrally controlled and users have only indirect or minimal control of the temperature in their home by opening windows and closing doors etc Only air conditioning or limited temperature controls are usually available This lack of control presents problems for con trolling the home as HVAC controls would be very useful for regulating conditions inside the home to implement the energy saving features that smart homes could offer On the other hand in apartment buildings a control network would also make it possible to cen trally monitor security energy consumption and environmental conditions using net worked devices and sensors In the case of a detached house the situation is simpler the HVAC system can be modified for advanced functions and flexible programming giving users greater control of all functions that the infrastructure supports 3 2 Design Challenges The challenges for the smart home as a platform for compatibility and integration of fun damental component
102. and depending on what kinds of sensors are available the system can gather enormous amounts of data For this reason it is important to choose the right kinds of sensors for an application otherwise the amount of sensor data can become overwhelming or the nature of the data can be irrelevant for the application in question For example context aware applications would benefit greatly from sensors that monitor the users and their activities whereas an application concentrat ing on optimising energy consumption would mainly require current consumption and en vironmental sensors A smart home would not be very useful if there was nothing for users to control and use Actuators home appliances and mobile devices provide users with means of physically af fecting the environment and controlling their homes Legacy home appliances and other non controllable devices can cause problems as it might not be possible to monitor or con trol them in any way creating a black hole in the home infrastructure where no informa tion is available Power supply and power consumption are other issues which are a concern not only for economical reasons but also for practical reasons cables transform ers and batteries are an unwanted sight in homes and it is not uncommon to see a great number of entangled power cables behind television stands and computers Most devices cannot use mains power directly and thus require a transformer or power converter which adds
103. andable and accessible by a third party It is divided into several application layers the physical layer covers all physical devices in the home sensor platform converts physical devices into software services service OS Gi framework that maintains active services knowledge reasoning engine context man agement detects and registers contexts and application associates behaviour with contexts layers The Smart House building is located in Gainesville Florida and contains a multitude of smart devices For example the smart floor is for user tracking smart plugs with RFID tags sense which appliance is connected to the power outlet and the microwave oven de tects what food is being cooked The sensors contain a small amount of code that they transmit to the server thus registering themselves in the network and providing informa 55 Related Work tion on themselves The server deals with context awareness and sensor data abstraction and performs adjustments according to decisions from the reasoning service Figure 4 12 GatorTech Smart House Photos University of Florida 4 2 15 FutureLife 2000 FutureLife is a one family house located in Cham Switzerland Futurelife It was built in 2000 to demonstrate what smart home technology has to offer and provide an insight into future living The house contains innovative applications the latest technology and sys tems as well as a resident family of four people Future
104. ant inventions had now been made and there was nothing more to invent The laws of physics were becoming apparent when it came to speeds of cars aircraft and communication The telegraph was as fast as electric communication can be faster cars could not move people around any faster on the congested highways and personal flying vehicles did not emerge either 42 Smart Homes Later in the 1960s and 1970s everything changed again There were new visions about ro bots artificial intelligence and electronic brains and the future seemed bright and interest ing The role of the traditional housewife changed for good and women were now making careers outside the home New appliances such as food processors electric razors and hair dryers sewing machines etc were flooding the markets Thermostats central heating and limited degrees of home automation also emerged In 1969 the ARPANET a project be tween the US department of defence and Massachusetts Institute of Technology became operational It was the world s first packed switching network which later evolved into the Internet A network was needed because computers were very expensive at the time and researchers across the country needed a way to exchange information and data In the 1980s and 1990s colour television made its breakthrough and video recorders were soon to follow Microwave ovens tumble dryers and cordless phones became common in every household Home entertainment wa
105. apartment EM Fi EMFi is a sensor film material that converts mechanical pressure into proportional electrical energy and vice versa Mechanically it is constructed of thin air isolated polymer layers acting as charged plates of a plate capacitor Due to the relatively high cost of the sensor film material it was not possible to cover the entire floor area with sensors and as a result sensor film was installed at locations where people would be most likely to walk when moving from one room to another Though the integrated EMFi film amplifier gives both analogue and digital outputs the latter was used to obtain present not present type information from the sensors One problem with EMFi film was however that it only re sponded to state changes The effect of this was that when a person stepped onto a sensor pad the sensor gave a reading but when the person remained standing on the pad it ceased to register anything more Only once the person had stepped off could another reading be obtained This made the sensor useful for movement but unsuitable for measuring static conditions we Balcony Kitchen Dining area Living room eile Figure 5 15 EMFi sensor board and control unit top approximate placement of EMFi sensors bottom PC 72 TUT Smart Home Research Odour sensor A sensor measuring air quality odour levels air pressu
106. are in making it easier to control lighting at room level Apart from those in the living room all lights in the Smart Home are controlled through the LINET network with pushbuttons in the wall controlling corresponding light groups However as an exception halogen lights in the living room are controlled using the dimmer module from the Living Room project Light groups are as follows Sauna bathroom kitchen kitchen desk dining room living room hallway balcony and bedroom 80 TUT Smart Home Research Speech Control and Audio Feedback System Speech control allows users in the Smart Home to give voice commands to the system and thus removes the need of buttons screens and other physical UIs With PCs having enough processing power to recognise words and sentences and the availability of speech recog nition software this presented an interesting option SCARS Speech Control and Audio Response System is an experimental user interface which uses speech as input and com puter generated audio as output Hyvonen03 Kaila09 A portable wireless microphone is used to give speech commands to the server and the system can provide audio feedback to the user from ceiling mounted speakers The microphone is also equipped with a loca tor so the system knows where the person is using the microphone making it possible to direct audio and audio feedback directly to the user Another advantage is context aware ness issuing commands in the l
107. as conducted between 1999 and 2009 at the Department of Electronics The focus is thus on hardware design and imple mentation communications user interface prototypes and home networks each of these topics is presented in a separate respective chapter In order to obtain results from authentic life situations these systems have all been built and implemented in real environments and later tested and evaluated by various test persons The findings of the study are presented and discussed with some conclusions on the future of the smart home Preface This thesis presents research that has been carried out from 1999 to 2009 in the Personal Electronics Group at the Department of Electronics at Tampere University of Technology Tampere Finland The work has been funded primarily by the department with certain projects also being funded privately or by the Academy of Finland Partial funding also came from the Graduate School in User Centred Information Technology UCIT The impetus for smart home research at the department came from Prof Jukka Vanhala whom I would also like to thank for the inspiring research themes and his guidance over the years A debt of gratitude is also due to present and past members of the Personal Elec tronics Research Group PEG who encouraged and assisted me in this research Our friendship also extends outside the office environment Smart home research at TUT has been a collaborative effort and none of it would hav
108. as furnished with a comfortable couch and chairs a dining table was placed near the windows and a small kitchen with a bar table was built in another corner Lighting was accomplished with small halogen spots which provided the room with warm ambient light Before the laminate floor was installed spe cial EMFi sensors were installed on the floor by VTT EMFi These were to be used to monitor movements inside the apartment The artificial balcony presented an interesting design challenge as it was basically a narrow corridor separated from the living room In order to create a pleasant view from the living room windows a large poster with a view of downtown New York was put on the balcony wall A handrail was also built and two high power artificial daylight lamps were installed in the ceiling To further enhance the illusion of a city apartment an ambient sound system was also installed Speakers emitted random city sounds and background noise With this setup it was possible to create artifi cial daylight and the atmosphere of a big city apartment 63 TUT Smart Home Research Figure 5 4 Living Room dining area and balcony As the Living Room was intended to be a testing space with numerous controllable elec tronics a central user interface was needed For this purpose a miniature PC was embedded into a small coffee table and equipped with a touch screen and batteries This UI was also intended to function as a multi purpose remot
109. at Georgia Tech for multidisciplinary stud ies 4 2 5 Microsoft Easyliving 1998 Easy Living is another project where prototypes and architecture are being tested ina smart environment Brumitt98 Microsoft s emphasis is directed to data processing modelling and combining sensor data modalities The laboratory uses computerised vision with the 48 Related Work help of multiple cameras to detect and identify people in the laboratory and computer agents and models to control the space Cameras are also used to track devices while de vices are tracked with tags or through RF networks using signal strength indicators Con trollable devices in the laboratory are computers A V equipment and lighting which are controllable from graphical UIs on screens in tablets or PDAs or using gestures and voice commands The Easy Living Geometric Model maps devices and their relationship with the physical world and a middleware platform InConcert is responsible for message pass ing and device addressing The Easyliving laboratory is an example of disaggregated com puting in which tasks are performed by multiple computing devices such as PCs PDAs mobile phones Figure 4 5 The Easy Living laboratory Photo Brian Meyers Microsoft Research 4 2 6 MavHome 2001 The MavHome is a smart home project at University of Texas at Arlington Cook03 The multidisciplinary research project attempts to view the smart home as an intelligent agent
110. at have long established roles for each device The concept of natural user interfaces involves items that are designed to fit a pur pose as described earlier in the beginning of this chapter For example door handles can have different forms and shapes but in general people are aware of how they are used no matter where in the world they travel In addition to the physical aspect natural UIs also include forms of interaction that are natural to people such as gestures speech and touch VTTO3 Whatever the case there is no need to change the natural way in which we use certain de vices if we add electronics and connectivity to a certain device it merely adds to the func tionality without altering the traditional way of using it Vanhala02 For example a smart flowerpot capable of measuring water level humidity and the temperature of the flower can alert the user if the plant needs attention But the pot will not water the flower auto matically this must be done in the conventional way with the aid of a watering can Mo torised window blinds can be controlled through a graphical UI by using sliders and position controls but it is still also possible to adjust them by hand by turning a knob Thus preserving natural UIs is recommended wherever possible as it allows people to use the device in a familiar manner Enabling manual physical control for all devices is also essential to ensure that there is always a way to control devices even
111. atives for locating people were passive infrared sensors PIR which detect the movement of people Such a sensor is in use in the living room area and detects if people are seated around the living room table For accurate user tracking however they are unsuitable and too inaccurate Experiments have recently been conducted with capacitive sensing by utilising the metal lic sheets inside the Smart Home floor tiles By using a wire as a receiver and tiles as trans mitters it is possible to detect changes in capacitance when people are walking on the tiles Valtonen09 A single tile is 60 x 60 cm in size and if the metal sheet beneath it is cut into 82 TUT Smart Home Research even smaller pieces greater accuracy can be obtained Problems are caused by differences in footwear is the person wearing shoes or not and carpets both degrading the sensing capabilities However the floor tile approach would yield a reasonable accuracy and could be used throughout the apartment without the need for much additional infrastructure OO Figure 5 26 Smart Home PIR sensor left floor tiles in the Smart Home right 5 3 1 Smart Home Network Infrastructure Figure 5 26 below shows different networks and their connections in the Smart Home lab oratory Bluetooth Dongle Bluetooth Bluetooth z Uls device Serial device i Figure 5 27 Network infrastructure of the Smart Home 83 TUT Smart Home Research
112. bject explains the differences between these and presents the ideas that have been used in research at Tampere University of Technology TUT Chapter 3 discusses various aspects of creating smart homes the kinds of obstacles one might encounter on the way what kinds of benefits smart homes could offer users as well as a brief history of the development of domestic technology In Chapter 4 there is an overview of current state of the art technology and related research projects are presented Chapter 5 presents smart home research projects from a nine year period at Tampere Uni versity of Technology Chapters 6 and 7 present different hardware and software solutions such as networks sensors middleware etc that are commonly used in smart home appli cations Chapter 8 presents different types of user interfaces and interaction methods with smart homes while Chapter 9 summarises the findings from smart home projects at TUT This chapter also considers the various problems and challenges encountered during the research and summarises the findings and lessons learned during this nine year period In Chapter 10 there is further discussion along with an analysis of the current and possible future state of smart home technology and finally some concluding remarks 1 3 Scope of the Thesis This thesis concentrates mainly on the technical aspects of smart homes as this is as it is also the standpoint from which research has been conducted Since smart home
113. ble today and some devices are com pletely without a communication interface From a technical point of view a smart home does not require state of the art technology or futuristic gadgets the technology for creat ing smart homes has been available since the 1970s and technically there are no obstacles to prevent smart homes being constructed already today Tiresias08 However there are several factors that have prevented smart homes from being widely available Though sev eral commercial and non commercial showcases technology demonstrators and proto types have been built Nyseth04 HPO1 the smart home remains a distant dream In order to find out what kinds of opportunities barriers and future prospects there could be for smart homes a research group of which the author was a member was formed at the Department of Electronics Research started in 1999 by converting former laboratory space into a smart home laboratory The author s first project was to implement smart de vices for this laboratory something that later grew into participation in the implementation of three smart spaces Introduction 1 2 Structure of the Thesis This thesis discusses the idea of smart home technology in general how the idea of a smart home was conceived and how it has been interpreted around the world This chapter sets out the scope of the thesis and defines the research question Chapter 2 starts by introduc ing various interpretations of the su
114. c networks has been made IST requirements for AmI technologies ISTAG03 also concern software for example re garding self configuration and on the fly programming The former is important for adap tive environments and is essentially service discovery on a larger scale enabing devices to adapt themselves to the environment For example sensors can change parameters accord ing to how quickly they have to react and a motorised door would keep the door open a longer time if the home is occupied by elderly people On the fly programming is a way of configuring devices without requiring them to be removed from their operating environ ment This method is however a larger task than simple reconfiguration as it practically reprograms the entire device allowing it to be modified to conform to a new communica tion protocol for example 7 7 Summary A crucial part of the smart home infrastructure is the middleware which is responsible for connections translations and user interfaces Properly designed modular middleware ar chitecture allows the environment to be modified at any time accepts the addition or re moval of devices and is adequately fault tolerant A suitable middleware reduces the need for compatible protocols but does not render the system completely independent of com munication protocols as these still lay the grounds for all communication in the smart home Together with artificial intelligence and adaptive control software
115. cally Assisted Rehabilitation TAR 2009 Berlin Germany 2009 A M Vainio Ohjausj rjestelm lykk lle kotiymp rist lle Control system for a smart home environment Master of science thesis Tampere University of Technology Department of Informa tion Technology Tampere Finland 2006 D Valtchev I Frankov Service gateway architecture for a smart home IEEE Communications Magazine Vol 40 Issue 4 2002 pp 126 132 M Valtonen M Proactive and Context Sensitive Home Control System Master of Science Thesis Tampere University of Technol ogy Tampere Finland 2006 M Valtonen A M Vainio J Vanhala Continuous time fuzzy con trol and learning methods ISCIT 2007 International Symposium on Communications and Information Technologies Sydney Australia 2007 pp 346 351 M Valtonen J M entausta J Vanhala TileTrack Capacitive Hu man Tracking Using Floor Tiles in Proceedings of the Seventh An nual IEEE International Conference on Pervasive Computing and Communications PerCom 09 Galveston Texas USA 2009 J Vanhala A Flood of Intelligence the Living Room Project in ERCIM News Number 47 European Research Consortium for In formatics and Mathematics ERCIM 2001 pp 14 15 J Vanhala Emerging Man Machine Interfaces in Proceedings of the Fourth International Conference on Machine Automation IC MA 02 Tampere University of Technology Tamper
116. ce to present contextual information on a specific user and this in formation can be seen only by the individual in question Worn information displays can further be augmented by wall mounted and other displays in the space allowing informa tion to be split into private and public sections Auditory augmentation is achieved by using a set of speakers microphones and worn in frared badges People can listen to audio messages or they can record auditory footprints for others to hear Audio messages can be tied to the physical location of the user the us er s context or other relevant information There are problems however with such a setup since information presented from speakers is not private and it is also impossible to direct information to a single user without using precise locationing and user recognition systems or headphones The Digital Desk Wellner93 is a physical desktop environment with a pair of projectors that are used for projecting images onto the desktop or onto documents placed on top of the desk Cameras detect the motion and track the actions of the user either fingertip or 168 User Interfaces pen and also read paper documents that are placed on the desk With the Digital Desk us ers are able to drag and drop images or paper documents by moving them digitally to other areas on the desk These will then be combined digitally to form a new document Other applications include a drawing program cal
117. ces It also contains mechanisms to create ar tificial intelligence in the home environment The intelligence is created in objects called tasks and these are adjustable from user interfaces and in turn they command the device objects There are many kinds of tasks but they all have a unique identification number This number is used for example to find the appropriate task when one component is sending commands to another Tasks have access to device objects and thus they are able to manipulate physical devices Apart from these common properties the tasks can be very different the simplest kinds of tasks are used for sending values to device objects so that user interfaces can adjust physical devices Other tasks only monitor device objects and notify user interfaces as soon as their state changes Tasks can send commands to other tasks and there are several different ways to group tasks When commands are sent to a 91 TUT Smart Home Research group the tasks belonging to that group are commanded in tum usually after some processing Some tasks are activated when certain conditions are met so that they can com mand other tasks in turn New forms of intelligence can be added by designing new tasks and new types of devices can be brought into the system by implementing the appropriate device interface The timers that the Home Controller offers are available for every user interface There can be an arbitrary number of timers and each
118. changing from the traditional supplier developer model involving manufacturers of electric household goods and construction companies into the current model which also accommodates brown goods companies i e home entertainment and IT industry In addition to these certain types of service providers offering various sup port and maintenance will play an important role in the future ISTAGO3 Classification into different market segments and specialities is becoming difficult because the distinc tion between hardware and software is no longer clear making it more difficult to exploit and promote products In some cases the growing ICT industry has brought large benefits 39 Smart Homes and economic growth as in the case of South Korea Current plans include a complete ubiquitous city the u City Jeong07 that will link traditional industries together with the ICT industry converging technologies to form a ubiquitous network of information serv ices and content Similar plans exist in Oulu Finland where sensor networks new kinds of information access and urban interactions are being integrated into the city environment Ojala08 From a business point of view there are many factors that affect the value of smart home technology compared to traditional technologies VTT03 The user would be interested in energy savings additional value from home systems and user interfaces safety and the resale value of the property Status and
119. chnical systems and proposed user roles to create a COBA compliant conceptual model of the building COBA definitions include open interfaces to all systems related to building management as well as modelling of the physical building different spaces devices systems and users Using control groups and information points the system can obtain information and control building automation lighting control consumption management access control and security systems Current COBA compliant hardware includes security cameras access control systems fire alarms and water heat regulators IV VERKOSTO Ga e e e 2 e Ga d Ga Ga Ga e a e e oe ow Ga a oc Figure 7 3 Screenshot of a COBA heating system control UI Photo Lonix 7 3 Service Discovery The point of a service discovery feature is to provide clients with a method for locating services that they require and devices to advertise the features that they are able to provide Sundramoorthy03 In other words it allows providers to advertise themselves and serv ices to find resources that they need For example a user interface wanting a temperature reading from the living room could initiate a search for temperature sensors in that area select an appropriate device and request a temperature measurement from it This kind of implementation also makes the network more robust and fault tolerant since it is possible to discover non functional devices and perfor
120. conjunction with the LIPS project Some networks could also benefit from being redesigned the serial hub for exam ple can only communicate with one device at a time and can become flooded if too many messages are sent in succession Major problems will result if the hub locks up all traffic to the Smart Home devices connected to it The communication protocol also needs im proved algorithms and greater privacy and error correction methods At present it does not have enough fault tolerance and the wireless traffic is basically unencrypted 176 Findings The Home Controller has been very useful in bridging together different network types An intelligent centralised gateway makes it relatively easy to combine different devices and networks even when they are totally incompatible Even with the limitations men tioned above using different network technologies and connecting new devices has been made easier because of the upgradeable server software and the versatility of the PC plat form This is a major advance on the tabletop UI used in the Living Room which lacked modular design and appropriate software interfaces Visitors often ask how much such a smart home system would cost and there was also considerable interest in the tablet UI and its home theatre controls Naturally it is difficult to answer questions on the cost of such a prototype system but it is clear that people are very interested in such a system 9 3 Findings From
121. culator and remote interaction with an other user also using a Digital Desk Another example of an augmented UI is using standard camera phones as computer aug mented user interfaces Rohs05 The display of the phone when set to function as the camera viewfinder normally displays the image obtained directly from the camera In ad dition the software running in the phone is able to detect when the camera is pointed at a smart object or a controllable device Identification is achieved by machine readable tags that the software detects from the image information When the object is centred on the screen a user interface for the particular device would appear on the screen for the user to use Thus by turning on the camera and by moving the phone around in the smart space users would be able to find and activate controllable devices easily 8 6 6 Calm Technology The term Calm Technology was coined by Mark Weiser Weiser96 to describe the proc ess of designing the kind of technology that does not overwhelm the user with a surfeit of information or command attention in annoying ways Calm Technology attempts to pro vide information on a human scale and also remain unobtrusive Calm Technology moves between the centre and periphery of the user s attention depending on the nature and ur gency of the information Humans are able to follow several events simultaneously but there are limits to the number than can demand our full attention By movi
122. d HomeSoft with its home automa tion package attempted to make it easier for customers to customise and build the home of their dreams Cooltown allows users to exchange URLs and context bridging the physical world and the virtual world of the World Wide Web The inHaus conglomerate in Germa 58 Related Work ny integrates products and services from various companies into their innovation centre with their first project concentrating on working at home Technology demonstrators and showcases are useful for increasing public awareness and for allowing users to have a hands on experience with technology but often these sce narios are built around marketing new technologies instead of trying to solve a certain real life problem Thus it can take a considerable time before this technology becomes com monplace and compatibility and upgradability problems can further inflate this problem There are not that many research projects that have involved long term user tests many laboratories have been inhabited temporarily either by research staff or volunteers but these tests have only lasted from a few days to months The Duke Smart Home is also an other living laboratory but in this case there inhabitants are students and the laboratory their permanent home The Orange at Home project experimented with different groups of people both with and without technical backgrounds in order to survey the usefulness and usability of new tech
123. d activities taking place around them Mayra05 If consistency is lost it can lead to users becoming frustrated and confused as the application or device is not doing what the users are used to expect Table 3 2 below shows examples of what kinds of requirements different age groups might have what benefits they could have from smart homes and what kind of technology would be required to fulfil these requirements Table 3 2 Requirements and benefits for different age groups Requirements Benefits Technology Infants Safety monitoring Infants cannot access Surveillance access certain dangerous control controllable functions electrics Children Safety entertainment Safe play at home As above children can use some functions of the smart home Youngsters Entertainment com Customisable con Distributed UIs home munication efficiency trols distributed data networking mobile remote access systems Adults Usability communica Remote access easy As above energy con tion efficiency secu management of the sumption monitoring rity entertainment home cost savings sensor networks Pensioners Communication Improved usability and Sensor and communi accessibility management cost sav cation networks ings remote access The elderly Safety accessibility Ability to stay at home Sensor networks bio without further assis sensors communica tance tion networks remote access
124. d and when should the system be silent In a study by Kristine Nagel a case of context mediated communication the knowledge of user availability is made known to trusted parties before any communication or interruption of the user is attempted The goal was to find out if it is possible to accurately predict the availability of users in a home environment According to the author a home environment is especially difficult in terms of knowledge of user availability Availability is dictated by three principal factors co presence are there other users present and is there currently any interaction between 19 Definitions them location where are users located and activity what are the users currently do ing For example a user in the kitchen might be cooking and occasionally available for interaction but if she is washing dishes her hands are wet and interaction has to be limited to verbal or visual means The greatest variation however is caused by individual habits and preferences Different people have different specifications for when and where they are available and these preferences can vary enormously from person to person One user might consider the bedroom a place for undisturbed rest whereas another user might regard it as a place for relaxation where the user is always available Parenthood can also drasti cally change opinions when full attention is turned towards the baby and everything else is of secondary importance
125. d and wireless e Communication protocols suitable for smart homes e Modular home control middleware e Multiple user interfaces for smart homes e Learning and adaptive control software e User test results from both laboratory and long term tests However in spite of all this research there are a great number of questions that still remain unanswered and research directions that would be interesting to explore For example the commercial aspect has been practically overlooked devices and applications have not been commercially taken up nor have they been designed to be mass produced in any fash ion The only exception has been the Blue Post Kaila05_3 which was designed directly for a commercial purpose From a researcher s point of view it can be difficult to envision the requirements and future of the commercial market which is why a commercial partner could have brought more insight into smart home research In addition to hardware and home appliances there is also the third party service aspect which could provide a valu able connection to services located outside the home Another regrettable issue was the premature ending of the eHome project it would have been interesting to continue the ex periment with different user groups and with an upgraded smart home system This would have enabled researchers to gather data from a larger group of users as well as further de velop the user interfaces and control software Using Frances
126. d methods of natural interaction using multimodal technologies Computational intelligence adapts to human behaviour and together with contextual awareness it can present relevant information for its users depending on the current conditions The same report emphasises that AmI is a research field that incorporates expertise from technological societal and economic fields According to the report it is also important to stress that in AmI the intelligence is provided through interaction and participation ad dressing the real needs and desires of the user This is different from the traditional view of artificial intelligence as an inferior abstract entity that people cannot really communi cate with When the intelligence resides in the environment networked devices and con tent it can provoke negative reactions at first because it is very unfamiliar invisible and unobtrusive The goal of the group is thus to promote the benefits and possibilities of AmI technology Components AMBIENT INTELLIGENCE Smart materials Media management amp handling MEMS tech amp sensor tech Natural interaction a Embedded Systems Computational intelligence A E Ubiquitous Communications Contextual awareness hd 1 O device technology Emotional computing Po Adaptive software Platform design Software amp Service Architectures Design Engineering and Integration Experience prototypin
127. daptive Systems Learning Smart homes are usually tailored for specific applications or to fulfil the requirements of the inhabitants Edwards01 Modifying the system at a later stage for example if the house is sold or if there are changes in lifestyle or situation can prove difficult and costly An adaptive home monitors the actions and reactions of its users and keeps track of what they have done in each situation Eventually the system becomes trained to the routines and behaviour of its users and can take over some functionality of the home In an ideal case the users will no longer need to worry about manual control and device management Usually adaptivity fits into the middleware software see Chapter 7 that controls the home placing it between the home UIs home network infrastructure and devices Dommel05 Using various algorithms for example pattern learning or by correlating in teractions between users and devices the adaptive software builds a database to support itself when actions or changes are detected This information is then used for determining whether something new has been learned or if it was something that had previously been learned More complex systems such as the Adaptive Home Mozer05 see Chapter 4 utilise neural networks for self programming and artificial intelligence 3 7 Business Perspective The emergence of smart homes offers new business opportunities for companies in many industries The market is
128. dditional data and possible parameters e g adjust lights in the living room or data The entire frame is composed of the following bytes address header command data parameters etc Every device connected to the serial hub either by wire or through the RF network has its own address which translates into the physical port on the hub the device is plugged into The communication speed is 19200 bps For example commanding the lights to be turned on in the living room would require the following command 0x01 0xA3 0x01 0x02 OxFF address frame adjust lights living room maximum brightness 151 Software Architecture Valid addresses for wired devices are 1 to 15 and 20 to 29 for wireless devices Specifica tions also require devices to reply to a command within two seconds otherwise the session will time out The new serial protocol performed more reliably than its synchronous predecessor though there was still no collision detection or error correction in use Collisions are theoretically possible even if all traffic is still being initiated from the network master There is also a limit of how many bytes the serial hub can store in its memory limiting the maximum number of bytes that can be transferred The new serial protocol was used in the eHome and it is still in use in the Smart Home Smart Home UI Protocol User interfaces in the Smart Home use a separate textual protocol which allows UIs to cre
129. dequate information for detecting whether a person is present mobile or if the surroundings have changes This infor 171 User Interfaces mation could be used as an input method thus communicating a request for a certain action Computer vision is the most advanced alternative Using one or more video cameras and video processing software or hardware it can be used for recognising patterns individuals or motion e Mobile UIs that include mobile phones remote controls etc These include numerous everyday items like garage door openers infrared remote controls touch screen tab lets and other kinds of portable hardware e Wearable computing can also introduce new UIs for smart homes using UI function ality on the garments themselves For example a jacket with integrated textile buttons can turn into a home remote control when the user enters the building Furthermore possible UI elements on the wearable item such as wristbands watches and bracelets can be integrated in the home control system e Tangible UIs are objects that can be manipulated in a physical way to perform tasks They can represent a real physical object e g a remote control or a ball or be com pletely different from the object they are control Tangible UIs can provide users with multiple ways of controlling equipment for example through touch movement or visual means e Augmented UIs as previously described combine what we can perceive in the real w
130. e 49 Related Work networked with a proprietary sensor network Various learning algorithms models and agent software are tested and simulated and these are later verified in practice using the test environments If the system works as planned it will attempt to minimise the users interaction with home devices In other words a functional smart home reduces the amount of adjustments and actions that users have to perform 4 2 7 HomeSoft LONIX smart home concept 2003 HomeSoft was a Finnish software company that specialised in smart home control Homesoft04 In collaboration with LONIX a supplier for LON based home automation it supplied smart home technology to one family houses HomeSoft went bankrupt in early 2006 and the future of the concept remains unclear The idea behind the consortium was to offer customisable home automation packages which would be delivered on a turnkey basis The customer therefore would no longer need to consult numerous separate con tractors for heating plumbing electricity etc and try to make them work together In stead the HomeSoft package would be negotiated only once since the contractors would be working under the consortium The concept was intended to deliver comfort safety and energy savings that small house builders would find useful The system included pre set states which could be changed from a wall panel switch at home daytime at home night away away for a longer per
131. e Digital Versatile Disc Electrocardiography Electronically Erasable Programmable Read Only Memory Electromagnetic Compatibility Efficient Protocol for Intelligent Spaces General Packet Radio Service Global Positioning System Global System for Mobile Communications Human Computer Interaction High Definition TV HyperText Markup Language Home Theatre PC Home Phoneline Networking Alliance Heating Ventilating and Air Conditioning Input Output Inter IC bus Integrated Circuit Information and Communication Technologies Institute of Electrical and Electronics Engineers Internet Protocol Infrared Infrared Data Association Industrial Scientific and Medical Information Society Technologies Advisory Group Information Technology Internet Protocol International Organization for Standardization Local Area Network List of Abbreviations LIPS LON MCU MEMS MOSFET NFC OS OSI PAN PC PCB PDA PIR PLC PnP QoS RAM RF RFID RSSI RTOS RX SD SMS SPI TCP TUT TX UART UDP UI UPnP USB UWB VPN VR VTT WAN WLAN xDSL XML Learning and Interactions in Proactive Spaces Local Operating Network Microcontroller Unit Micro Electro Mechanical Systems Metal Oxide Semiconductor Field Effect Transistor Near Field Communication Operating System Open Systems Interconnection Personal Area Network Personal Computer Printed Circuit Board Personal Digital Assistant Passive Infraeed Powerline Communication
132. e Finland 2002 pp 47 50 H Ailisto A Kotila E Str mmer Ubicom applications and tech nologies VTT Research Notes 2201 2003 58 pages References Vuorela06 Wang07 Weiser9 1 Weiser96 Wellner93 Wiio78 X10 Yang08 YIT Youngblood05 Zakrzewski05 Zakrzewski06 T Vuorela K Surakka J Vanhala H M Jarvinen Instruction level energy measurement of general purpose microprocessor a case study AT90S8515 and ATMEGA8515 in proceedings of the 6th International Conference on Machine Automation ICMA 2006 2006 Sein joki Finland 8 pages J Wang A Novel Magnetic Communication System for Wireless Transmission Operating at 14 9MHz IEEE Radio and Wireless Symposium 2007 pp 59 62 M Weiser The computer for the twenty first century Scientific American September 1991 pp 94 10 M Weiser J S Brown 1996 Designing calm technology avail able at http www ubiq com hypertext weiser calmtech calmtech htm P Wellner Interacting with paper on the DigitalDesk in Commu nications of the ACM Vol 36 Issue 7 1993 pp 87 96 O A Wiio Wiion lait ja v h n muidenkin Wiio s laws and some others as well Weilin G s Espoo Finland 1978 X10 Industry Standard 2009 available at ftp ftp x10 com pub manuals technicalnote pdf J Yang W K Edwards ICEbox Toward Easy to Use Home Net working
133. e TV viewing is a more challenging context as there are many situations when the TV is turned on but no one may be watching Casual viewing where the TV is on in the background watching news in the daytime or watching a movie in the dark are all different scenarios and require separate modes for lighting ambient sound levels etc Detecting when the user is sleeping is easier by using bed sound and light level sensors it is fairly straightforward to deter mine if a person is sleeping in the bed or not 103 TUT Smart Home Research These contexts are clearly insufficient for general use and a major challenge arises when all the possible contexts in peoples daily lives are taken into consideration as discussed in Chapter 2 Accurately detecting a context and predicting the subsequent context is cru cial for proactive computing and thus of great importance in the LIPS project 5 6 2 LIPS Software LIPS middleware is being built around the core software used in the Smart Home Previous experiments with fuzzy logic and adaptive home control software Valtonen07 have al ready provided a foundation for building further functionality The control software is con stantly monitoring the user s actions and recording every event If the user makes an adjustment when a context has been recognised itis recorded and taken into consideration If the user has to perform the same adjustment multiple times it is an indication that the system is
134. e basement through the suspended ceiling making installations tidy and unobtrusive Figure 5 39 eHome control room in the basement top electric distribution board showing relays and the LINET master box bottom Installing electrics devices and other hardware took a few months after which the tenants were able to move in The total cost of the installations was about 50 000 though a sig nificant amount of this was invested in the adaptability of the structure which would not be required in normal living conditions The eHome was in use for three years after which it was dismantled and concluded in late 2005 Some of the hardware items were reused in the Smart Home while others were kept for debugging and testing purposes 95 TUT Smart Home Research Figure 5 40 Bedroom and kitchen in the eHome apartment User Interfaces n g SS pons ah E oe se 2 E 0 Mis eHome network Figure 5 41 Floor plan of the eHome apartment showing devices yellow dots network connections blue lines and network components green boxes 96 TUT Smart Home Research 5 4 1 eHome Networks For the most part the eHome contained the same networks as the Smart Home Serial ca bles were used for most devices sensors actuators and LINET master and RF transceiv ers were not required Ethernet was used for connecting server and UIs together and LINET provided the control method for lights and mains
135. e would largely benefit from devices that are able to iden tify their own location Hightower06 This information is especially important when de 127 Hardware Aspects vices can move around the space be relocated or removed However obtaining this information can be difficult as very few devices contain technology for this purpose Most current positioning hardware is built around GPS receivers but these are not usable inside buildings or in dense urban surroundings One alternative is to create a radio network that can estimate the location of each node by sending out signals to nearby nodes and calcu lating the relative distance from these using Received Signal Strength Indication RSSI Lorincz06 This would yield an approximate relative location which might be sufficient for most applications Another alternative is to search for nearby devices whose location is known and attempt to establish a position relative to these Table 6 4 Comparison between different location technologies Sensor Price Accuracy Processing Other requirement PIR very cheap 10m Tow detects only move ment Video camera moderate centime high depends on ambient tres metres conditions Radar expensive low very high high low very high Floor sensors cheap cm moderate expensive to install GPS moderate 10m low does not work indoors RSSI free 10 m low depends heavily on ambient conditions 6 6 3
136. e been possible without the originality and enthusiasm of the researchers I am also grateful to the reviewers of my thesis Prof Albrecht Schmidt and Prof Petri Pulli for their valuable comments on the manuscript Dr Harri Raittinen and Dr Jaana H nnik inen for their assistance and Alan Thompson for proofreading the manuscript Thanks also go to everyone who sent me pictures for this publication to the Drunken Old Farts Internet community and Christopher Franke for his inspiring music Finally I would like to thank my family and relatives for their enduring support especially my wife Pilawan for being that special one for me Tampere December 12 2009 Lasse Kaila iii Table of Contents Table of Contents ADSULACE si scscs secs sennscused sdacassessteseset sdedascceosessssscesessndccsesissevessssssscsesdesesecnsdesdeccsasese i Preface scssccscsscsscssccccsscsscsssnssessssssesnseesessssesssesesssssssesssessssessenssesseessesseeees iii Table of Contents ssccscssssssccscssscsssssssssssssssssessessesssecssscsssenssssnsensssnseessoess v List of Abbreviations s ssssisssssssissssesossorssosssssssosesssssssssessssssorsessessssssistososossssse ssss x 1 Introduction scsscssssssscccscsssssssssscsesecsssssscssessessssseessecsseesessnesseseneseessons 1 LV Smart HOMES 3328 css vd as See enee ha has aaa OE lees csn te ee ee 1 1 2 Sttucture of the THESIS rra eo ieeen Mite i aba 2 1 3 lt Scopesof the Thes
137. e control for example A worst case scenario would be a completely computer controlled environment with all UIs and functionality tied to graphical and electronic interfaces In this case if a crash or network problem occurs all functionality is likely to be lost and users would even be una ble to turn on lights or open doors Even minor faults can be difficult for users to diagnose because several faults can cause similar problems Finding out which device has a flat bat tery which one has lost its configuration data or which one has completely locked up can be confusing for users 36 Smart Homes Is server up and running Check network connection Is manual control working Tech support required Is there power available Reset device restart server Reset serial HUB reconnect Restore power to device Does not work Check functionality Figure 3 3 Flow diagram of troubleshooting a device in the Smart Home Ideally nothing should happen if the central system or network goes down the home should function as before only without computer control and graphical UIs Home appli ances can still be used from their own UIs and other controllable devices can function with traditional manual controls The Smart Home at TUT has been designed according to this philosophy In the case of an electric blackout very few devices will function in a mod
138. e control for A V equipment Figure 5 5 Coffee table and embedded computer right picture Batteries are located under the low er surface Initial plans for network infrastructure were based on simple low cost solutions Eventu ally serial RS 232 cabling was selected because it was widely available in off the shelf de vices and microcontrollers An infrared link was being constructed for wireless applications 5 2 1 Living Room Network Infrastructure The first network in the Living Room was designed around the tabletop PC A serial cable link was used for lighting controls there being no need for wireless because the lights were 64 TUT Smart Home Research mostly already embedded in the living room infrastructure For more remote locations an infrared network was designed This was constructed using a powerful master IR transmit ter which by means of a large array of IR photodiodes was able to transmit and receive from longer distances than standard IrDA transceivers Remote devices were equipped with transceivers slaves and adhered to a synchronous serial protocol as described in Chapter 7 Serial HUB Living Room PC IR Master Light controller Tabletop PC Figure 5 6 Living Room network infrastructure Tests with the IR network showed that the transmission power of the IR array was more than sufficient but the smaller slave devices with less powerful transmitters had difficul ty in t
139. e major concerns of users People are also concerned about reliability long term durability and the time it might take for a long term investment such as a smart home to repay itself The willingness to invest in smart home technology can also vary greatly according to the demographic classification of users For example there is the so called second youth or third age which involves people who have recently retired and are experiencing new interests and more free time In many cases these are consumers with spending power and free time to devote to new hobbies activities and services The most important user requirements for smart homes are e Customisability the system can be modified to suit the needs of its users e Improved usability improved ways of interacting with the smart home system e Safety increased safety for the users less things to worry about e Privacy sensitive information and data must be kept safe e Consistency UIs and systems should work in similar logical ways A smart home should offer something for all either in the form of adaptivity or modularity as the greatest advantages can be achieved by customising the system to its users For youngsters different kinds of entertainment and communication are important and for par ents additional safety can be achieved with security and monitoring facilities For example unused electrical outlets can be turned off in the children s playroom the kitchen sto
140. e responsible for creating user interfaces and depending on the UI this can be a fairly demanding task Dynamic UIs that reconfigure themselves accord ing to what devices and services currently are available would require a middleware with extensive knowledge of where devices are and what features they offer Simpler static UIs only require up to date information about the states of each device Figure 7 2 shows the connections of a home server with middleware different kinds of networks and devices and how they connect to middleware components 145 Software Architecture Service provider Remote users Home server w Middleware Wide Area Network Service Service User Interfaces Interfaces Home Network Device drivers y Figure 7 2 Connections of a server running middleware software 7 2 1 OSGi Open Services Gateway Initiative The OSGi Alliance OSGi was formed in 1999 to promote open standards for a middle ware Java framework The OSGi platform was designed to make it easier to design main tain and deploy applications in networked computer environments The OSGi alliance consists of developers and technology innovators and the platform has been a topic of many smart environment middleware research projects For example Tao Gu et al used OSGi to create an environment for rapid smart home application prototyping Gu04 The OSGi specifications include APIs
141. e thus consisted of the following parts Header 0xA 4 bits P Data 8 bits P Data 8 bits P Data 8 bits P P denotes a parity bit which in this case was agreed to be even The recipient of this data was required to respond within ms by returning a special acknowledgement byte Ox7E This byte was also used to test the communication link and attached devices In this case the master device sent 0x7E and the recipient replied by returning the same byte Elec trically data transfer looks like this CLK 1 bit 2 bit 3 bit 4 bit 5 bit 6 bit 7 bit 8 bit parity pausel bit 2 bit 3 bit 4 bit DATA L__J L__ EI L__ Figure 7 4 MOB BUS communication From Fig 7 4 top CLK bottom DATA In idle status both CLK and DATA are set to 1 data transfer is initiated from the master by pulling CLK low for 1 ms after which the next clock cycle contains valid data The slave can initiate data transfer by pulling DATA low and waiting for the master to react 150 Software Architecture For example for a temperature monitor the data sent for the measure temperature com mand 0x01 would have been OxAI P 0x01 P fixed header OA plus 1 data byte followed by one command byte To which the recipient temperature monitor would initially reply with Ox7E P acknowledgement After this the monitor would pe
142. each device are described in the service advertisement message 155 Software Architecture Below is a sample XML file in this case it is for a sensor node containing temperature humidity and light level sensors lt xml version 1 0 gt lt DOCTYPE device SYSTEM properties dtd gt lt Example definition file for a mote sensor device implementing gt lt temperarure humidity photosynthetically active radiation and visible gt lt spectrum including infrared gt lt Heartbeat interval is 30 minutes in active state and 60 minutes gt lt in sleep state gt lt This application doesn t require short sleep intervals preamble gt lt 500 ms in normal state and 10000 ms in sleep state gt lt device name MoteDevice type mote poll_period 30 sleep_poll_period 60 preamble 30000 sleep_preamble 30000 conn Tiny set ON gt lt DEVICE ATTRIBUTES gt lt Attribute Temperature INDEX 2 gt lt 16 bit unsigned integer fixed point 6b 8b gt lt Read only attribute gt lt opt func temperature value 0x0 valuetype uint minval 0 maxval 0x3FFF rules read gt lt descr gt Temperature lt descr gt lt opt gt lt Attribute Relative humidity INDEX 3 gt lt 16 bit unsigned integer used 12b gt lt Read only attribute gt lt opt func temperature value 0x0 valuetype uint rules read gt
143. easier by providing contrib utors the required building blocks Universal Plug and Play and similar protocols on the other hand can assist in inter device connectivity In the case of a single server running middleware software functionality is centralised making it easier to debug but also makes the system vulnerable to errors and failures Distributed middleware components increase fault tolerance but it also makes it more difficult to locate errors and faults as there might not be any clear indication of where the fault has occurred Challenges with smart home software are clearly related to artificial intelligence and adap tivity as these are the most complex parts of the system Creating a set of rules and behav ioural patterns constantly monitoring actions in the environment and correctly anticipating future actions are complicated tasks for any piece of software Perhaps equally important are service discovery protocols which can significantly ease the intialisation and set up process as a lot of steps that previously have involved user ac tion can now be performed by the system Furthermore it will bring additional benefits in the long run as the system is being modified and updated and as mobile devices join and leave the home network The first communication protocols designed at TUT have been 158 Software Architecture extremely simple and lightweight but with the introduction of the EPIS protocol a big step towards dynami
144. ed into a tabletop computer A few years later in 2002 the department moved into a new building making it possible to design a new larger test environment Together with Pasi Myllym ki new designs were made now with modularity and expandability in mind Markus Ritala and Antti Matti Vainio designed a new modular Home Controller software package allowing the new laboratory to be more flexible and easy to develop for future needs Building the new testing space also required much work with old designs being re moved from the Living Room re installed into the Smart Home and new features being implemented New sensors for temperature and humidity were made as well as controlling electronics for window blinds lights and curtains Controlling systems for 240 V mains electricity were supplied by a third party as well as furniture and interior design Later in 2002 work expanded into another test apartment the eHome in downtown Tampere re quiring further installations and preparation for user tests During this project the author was also in charge of the TUT research team As the three year project progressed there were occasions when a visit to the apartment was necessary in order to carry out repairs or make modifications to the system Since the conclusion of the eHome project the author has mainly been responsible for the upkeep and development of the Smart Home labora tory at TUT as well as participating in new research projects such as UUTE and
145. ed manually A garage door opener or ventilation fan might exist but there is no control network for these In smart homes it is possible to install more controllable motors servos and valves and connect these to the home network Thus it becomes possible to control and monitor these from any UI in the home Features that could prove useful to inhabitants include motorised blinds dimmable lights motorised windows and doors controllable air conditioning and with the help of these it is possible to impact environmental conditions and physically affect the space Home appliances and other electrical devices are more difficult or even impossible to control remotely depending on the implementation and the functions of the particular device Certain functions can be achieved by simply switching the power supply on or off but for many modern electronic devices this might have complications due to resetting clocks or losing settings Kitchen appliances rarely have any means of remote control or communication and thus controlling them is also a challenge On the other hand there is not a great need for this since people have to be present for cooking activities in any case For safety reasons it is more practical to measure current consumption temperatures and user presence and in the event of a problem or accident the gas or electricity could be cut off Home entertainment equipment usually has infrared remote controls or utilise manufactur ers ow
146. efulness of a product may not be realised until it is integrated and tested in a proper environment However this often gives rise to a chicken and egg problem since a particular application or appliance might have no use until there are services providing content or context for it to function Thus collaboration between different parties and developers is of crucial importance 3 8 Smart Home History and Future Projects related to smart environments and smart homes have been conducted since the 60s but collaboration between man and computers has a much longer history The emer gence of new exciting technologies paved the way for innovations for both everyday chores and entertainment whilst stimulating the mind Science fiction books and TV series envisioned of robots talking computers large video screens and other gadgets that seemed quite far fetched at the time However it did not take long for the Smart Home concept to emerge something that happened in the 1980s with certain enabling technology already available a decade earlier Light switches that operated by clapping the hands computer networks adjustable thermostat heating etc were already a step closer to the reality of the 40 Smart Homes computerised home One notable milestone was the formation of a Smart House interest group at the National Association of Home Builders Aldrich03 The following section briefly presents the history of the smart home and the advance
147. eless USB and other short range applications Inductive data transfer is another way of communicating wirelessly using low power and over relatively short distances Whereas RF communication is accomplished with radio waves that propagate through free space inductive communication is based on two mag netically coupled coils tuned to resonate at a specific frequency that communicate in the near field The receiving coil can sense variations in the magnetic field caused by the trans mitting coil and data can thus be transmitted Wang07 Compared to traditional RF com munication inductive links are much more secure as the coils have to be closely matched and the signal is propagated only in the near field around the transmitter and not widely distributed like RF Furthermore the required energy is lower and there are no problems 121 Hardware Aspects with interference or overcrowded frequencies A limiting factor however is the rather short practical range usually limited to less than one metre as the fall in magnetic field strength is inversely proportional to the distance cubed Inductive links are often used in body area networks Tieranta05 RFID and wireless energy transfer Table 6 3 Comparison between wireless network types Network Frequency Max data rate Rover Range Cost Consumption Bluetooth 2 4 GHz I Mbit s gt 100 mW 10 100 m Tow ZigBee 868 MHz 250 kbit s gt 50 mW 200
148. emonstrating AmI concepts in the form of the Home Lab which opened in Eindhoven in 2002 The Home Lab contains large displays speech recognition interfaces and the latest home entertainment technology It also contains cameras and equipment for usability studies for experimenting with new technology According to Pieper98 Ambient Intelligence is e Embedded the environment contains invisible networked devices e Personalised the system knows your identity and it can be personalised e Adaptive the system can change its behaviour according to your actions e Anticipatory the system anticipates the user s needs Ambient Intelligence is not limited only to home environments it is applicable to cars and wearable personal electronics as well Another definition written by the EU Information Society Technologies Advisory Group ISTAG Aml is based on three key technologies Ahola01 ISTAGO1 ubiquitous com puting ubiquitous communication and intelligent user interfaces Ubiquitous computing as previously mentioned concerns the way processing power is being integrated into eve ryday objects such as clothing furniture and structures Ubiquitous communication is the media that enables communication between objects using wireless ad hoc networks In telligent UIs present the interface for users through which they can control and interact with the environment The emphasis of Aml is on user friendliness enhanced service sup port user
149. emporarily stopped making more complex adjustments Such faults can be very difficult for users to debug and their cause may be difficult to determine Further more smart home technology cannot be repaired or diagnosed like other household prob lems making it especially difficult for users to cope with sudden faults More serious faults however might have rendered the entire control system inoperable forcing users to default to the manual controls Initially there were major problems with the ADSL connection to the apartment but the problem was later identified as a faulty modem The lack of an In ternet connection made the phone UI inoperable and also cut off remote connections to the server This did not affect connections within the home so the Home Controller and local UIs continued to function normally Whatever the fault or the case the logfile that the Home Controller software created was of very useful in debugging problems and for mon itoring user activity The logfile contained details of every adjustment event and fault as well as a timestamp It thus provided valuable information on user activity and UI func tionality Since the eHome apartment was rather small there were situations in which the possible gains from a smart home control system were not fully realised For example it was not very difficult to check if the apartment lights were still on or to walk to the light switches and adjust the lighting The relatively limited
150. empowerment and support for interactions between humans In other words people will be surrounded by intuitive intelligent interfaces in every object and environ ment Implementing Aml is not straightforward adding computational power and commu nication links to an object does not make it intelligent A learning adaptive intelligence is also required to make everything work as a whole The IST group report ISTAG03 also lists the requirements for ambient intelligence and divides these into two categories components for ambience and components for intelli gence Components for ambience include new smart materials such as electronic paper sensor film materials that emit light and that can store data etc New Micro Electro Me chanical Systems MEMS can be used to construct tiny actuators and other kinds of new sensors can also be used to bridge the gap between human and machine senses On the hardware side self configuration on the fly programming and self repair allows for more reliable systems and rapid development whereas ubiquitous networks make it easier to ac cess any device in the space using either active or passive communication For this pur 10 Definitions pose the software in the devices also has to be adaptive and self managing in order to manage and adapt to changes in the environment Components for intelligence include components for handling and managing media for analysis and presentation of captured content an
151. ent of a serious problem 3 5 Mobility and Mobile Computing As people move around in their homes they are able to access information anywhere But instead of having people going to a specific location to access information a more flexible solution would be to make information available everywhere This is where mobile com puting has a role to play Typically mobile computing is considered as a situation where mobile users use mobile devices for example PDAs laptops or cellular phones Forman94 However as noted in the introduction mobile computing that takes place out side the home lies outside the scope of this thesis Mobile computing can also mean that information itself is mobile and in search of an ideal place to store and display itself Certain information might be suited for display on a cer tain UI or it might be better suited for storage at a location where it will more likely be needed A smart environment can recognise users and analyse their activities while at the same time fetching data that is relevant and interesting to the user In such a scenario users are not required to carry any information on their person since everything is stored in the network This however requires the infrastructure to understand and utilise the services and the capabilities of all devices in the network Devices must also have the capacity to inform the network of their capabilities and the services they offer Furthermore the re quirements fo
152. er stage 112 Hardware Aspects Intrabody Network IBN Intrabody communications take place between devices implanted inside the human body or swallowed measuring various parameters such as blood viscosity glucose levels and ECG With IBNs care has to be taken when transferring data as safety margins should not be exceeded when transmission takes place this close within the body Implants can use RFID for communication or alternatively connect to the BAN WAN Figure 6 3 Illustration of different network categories from intrabody networks to wide area net works 6 1 1 Selection Criteria When networks are compared and considered for smart home applications there are a number of factors to be considered The following list presents a few of the more important ones Range With wired networks there are practical limits on the length of cables and wire less networks naturally also have a limited usable range Range is closely linked to power usage and thus power savings can be achieved by sacrificing range Bandwidth It is always good to have spare bandwidth as it makes it possible to transfer large amounts of data without overloading the network However in order to save cost and energy itis usually wise to choose the most appropriate network technology for a partic ular application Energy consumption In wired applications energy consumption is usually not a crucial design factor but in wireless cases thi
153. er users the optimum services and UIs anywhere and anytime Data that has been input at some point must be always available from any access point and similar content should be accessible even from different kinds of devices Apart from obvious compatibility with computers and devices seamless computing requires free roaming ac cess between networks resource sharing among computers and protocols that allow serv ice discovery and ad hoc networking A seamless transition is one that involves a potentially disruptive state change such as when moving from a desktop computer with one OS to another with another OS and different versions of other software yet hardly distracts the user Satyanarayanan05 For example when a user with 3G mobile phone moves from an area with 3G reception to an area where only traditional GSM networks are available the phone will automatically switch from 3G to GSM networks without the user noticing anything However seamless computing is facing constant difficulties due to the tremendous pace at which new technologies networks protocols and devices are introduced to the market Standards do exist but they quickly become obsolete when newer improved versions emerge Autonomous computing Ghanea Hercock02 could be one possible solution to the problem by introducing evolution and the capability for growth into computer systems and software 18 Definitions Mahadev Satyanarayanan et al Satyanarayanan05 propo
154. erly and infirm to remain longer in their own homes and thus avoiding the need for nursing care Present smart home applications are mostly being developed by companies in the electronic medical and tel ecommunications industry and thus they do not necessarily cooperate on a large scale with the construction industry On closer inspection there are in fact a surprising number of firms and organisations with an interest in smart home development Himanen03 Con struction companies build according to established production models prioritising effi ciency cost and workforce utilisation Furniture and interior designers must consider aesthetic and practical issues of living while also following current trends in modern de sign White and brown goods are supplied by the electrical and electronics industry the former attempting to save people time by assisting with everyday chores and work while the latter seeks new ways to spend leisure time In addition sociologists and social work ers whose working places are other people s homes are involved with the welfare mental health and comfort of their clients Telecommunication companies are responsible for bringing multimedia video on demand and broadband networks into the home enhancing connectivity and communication All these various interest groups make are examples of how smart home development is really a huge multidisciplinary endeavour requiring sci entific and practical input from numerous
155. ern house and a backup battery for the server will be of no use if there is no power for lights motors or home appliances The term graceful degradation was coined by Adam Green field Greenfield06 to describe a situation in which a fault in a part of the system it should only be apparent to the user as the loss of some minor part of functionality instead of a complete breakdown For example if a porch light bulb is broken it would still be possible to have some light outside by turning on the light in front of the garage instead of the sys tem only attempting to turn on the broken light and doing nothing else Most humans have a fear of losing control and thus users should always have the option of manual control whenever they wish This is also desirable from a practical point of view as it is possible that the nearest UI is out of reach or unavailable Aldrich03 It would also be beneficial if functions timers and automation could be easily disabled by a single keypress and then quickly re activated if necessary there are always situations in which even the most sophisticated neural networks algorithms or adaptive systems fail to function properly For example a scenario involving TV watching can be carried out in so many ways that there can easily be a situation when the context is recognised incorrectly The TV can be on in the background users can watch a movie and opt for a darkened living 37 Smart Homes room for bet
156. ers and controllers although they are being rapidly replaced by USB ports However RS 232 is not a network it only transfers data between two devices point to point communica tion A proper network with common cables can be implemented using the RS 485 stand ard which allows for megabit speed networking over a distance of hundreds of metres Legacy cabling in the home can also be used for networking Powerline communication using mains cables makes it possible to transfer data inside the building without requiring any other infrastructure X10 X10 LonWorks LON and HomePlug HomePlug are ex amples of such technologies The simplest version the X10 is able to transfer only simple on off commands whereas the more advanced standards are designed to replace Ethernet cables Problems with powerline communication include causing interference and being susceptible to it Phone cabling has been adopted much more successfully making it pos sible to deliver Internet communications to end users From the first slow analogue mo dems that were able to transfer up to a few kilobytes per second faster and more reliable DSL modems quickly took over offering transfer rates up to tens of megabytes per second Phonelines can also be used inside buildings for example using homePNA connecting apartments to acommon DSL or leased line Standardised CAT 5 Ethernet cables Cisco03 are becoming a de facto standard in mod ern homes as it is possible to u
157. es creates compatibility and interconnectivity problems Furthermore everyday realities and human factors are rarely considered in the design process and thus render the technology incapable of adapting to changing circumstances and varying user requirements Lepp amp nen03 Current versions of smart homes have also been criticised for being solu tions to problems that do not exist Mayra05 giving rise to the claim of technology push rather than consumer demand Even the term smart home has been criticised on the grounds that it implies the user is stupid The term hints that the users are incapable of per forming the most mundane tasks in their own homes Leppanen03 and are reduced to the role of sitting on a sofa watching TV Suggested alternative terms have been media 99 ec home adaptive home or helpful home There is however a good example in the form of the modern car of an application where embedded electronics software sensors and actuators all work together Leen02 A mod ern car is a fully integrated smart environment that monitors conditions both outside and inside the vehicle and attempts to make the ride as comfortable and as safe as possible for the driver and passengers A variety of networks connect all sensors and controls together 23 Smart Homes and a computer controls actuators offers UIs and adapts to the driver s driving habits Lower speed networks manage simpler tasks such as ope
158. es for example an address class has message types for address changing registration and acknowledgement 4 bit Message Class 4 bit Message Type 16 bit Address 16 bit Device ID EPIS message classes are e Heartbeat Network control messages device power state control e Address Login messages addressing change release request e Attribute Read Write device attributes e Error Reserved for error logging 154 Software Architecture e Service Discovery Allows the server or device to perform service queries throughout the network e g find all lights in the living room Service Advertisement Each device in the LIPS network contains an XML file that contains a description of its properties and services The file contains device parameters that describe the physical de vice its power saving features and service parameters that are related to what the device is able to perform Device parameters include the following e Device name A descriptive name of the device e Device type Type of the device either a custom type or one out of the standardised device types e g light or temperature sensor e Poll period Time interval in minutes with which the device will re announce its presence in the network e Sleep poll period As above but when device is in sleep mode e Listening period Describes how often the device listens to the network e Sleep listening period As above but when in sleep mode e Connecti
159. es and technology similar to the Smart Home with a home net work server user interfaces sensors and controllable electrics The next important speci fication concerned the tenants and their profiles They should be relatively young willing 93 TUT Smart Home Research to experiment with innovation allow random visits to their apartment for evaluation in terviews and technical support and most importantly they should not have a technical background The apartment would be rented to them for a period of three years After a few interviews a couple in their mid twenties was selected a male biologist and a female culture historian The tenants were to provide researchers valuable feedback about the eHome and its functions as well as ideas for further development A usability researcher was present whenever a new UI was introduced or when regular feedback was collected to record valuable first time usage comments and experiments The eHome contained several different user interfaces including WWW pages programmable wall switches mobile phones and a television UI It also features control of electric sockets lights adjustable window blinds and different kinds of sensors The emphasis of the eHome project was on usability and practicality since it was not the aim to create an automatic system which would control everything without interaction from the users The devices and user inter faces are designed to make everyday life easier and le
160. es include sufficient Bluetooth function ality and with proper software they can be made to function as user interfaces and controllers for the home network Bluetooth as used in the Smart Home has sufficient range for indoor usage and compared to text messages for example is free of charge to the user An example of using Bluetooth in a smart environment is the BluePost a smart remote controlled car heating system Kaila05_3 Recently a low power version of Blue tooth Bluetooth low energy was developed making it possible to include Bluetooth into even smaller devices than before BTLE Another similar technology is ZigBee ZigBee based on the IEEE 802 15 4 standard ZigBee is also a mesh radio network technology designed for low speed and low power applications Zigbee has been specially designed for automation sensor networks and mobile applications Recently several manufacturers have launched low power single chip RF transceiver modules and similar solutions onto the market These offer a viable alternative to cabling and the data rate typically in the range of 20 200 kbit s is sufficient for sensors and ac tuators A single chip solution makes it possible to create miniaturised battery operated sensor modules a small antenna and a few passive components are all that is required for a matchbox size transceiver The effective range is enough to cover the an apartment or a large room and with proper alignment and larger antennas
161. essnseseseeecsnseesensees 157 7 6 Discussion vrai A EE E eee sess ERRE teins 158 TT Summary aeee roeas er ENEE E EREA iE 159 SU Ser In tera ces c sacs cick sesssecussuanscasvontconnsswaptevssenuss saasesscsnssssuse seaseedenssvsscccetess 160 Vii Table of Contents SU Feedback as cisesdu ck vcides E iE caves OEE E EEE E E EAE 162 8 2 Context A wate UIS seiri e a e Ea AAE E PEE EES ES 162 8 3 Physical UIS wi cc c sessccaseivsssecensevsseseenssscceseevasstecseevyecusectnessesvevne cveeseness 164 8A Graphical US scceccsesaeveesscveswerd eene ra ee aa rn e ea aria 165 SS Auditory UDe ra e a E EE ne alone R RRE 165 8 6 UIs in Other Smart Home Projects seseeseeeseeeeeereersrrerrrrsreersreee 166 8 0 l InfoC be in E 166 8 6 2 Microsoft Surface ssssssssesseseeesrrereresrerresesssreresrererreeees 167 8 6 3 Speech control sismi csnsstsii ienei doses rao Eeri 167 8 6 4 Gesture Control ieroci iieii aR 168 8 6 5 Augmented Reality ssesrssessrrsseserseereseese rers tvetsoererseivseses 168 8 6 6 Calm Technology sseessssesssssiesssresrsrrserresesreseeesreeeresrees 169 8 7 Control UI vs Centralised UL cee ce ceseeeeceeceeeeeeeeeeseeeeaeeeaees 169 8 3 Nat ral UIS izes horore a teea e atao ae E bea haste EEEE Ao ASES 170 8 9 Different Types of Smart Home UIs ssessseesseeeereerrererrersreersreeee 171 SLO SumMary oii iin a a pice ieee eee 172 0 FUNGUS cesssiccsdseccassosdceaesbeestudscestcsstenesssuctsnasspens
162. etermi munications nation Frame Data Link Physical addressing data format Bit Physical Media signal transmission 115 Hardware Aspects The physical layer is responsible for transmitting data through a physical communication medium whatever it might be The physical layer is very close to the hardware for exam ple it knows how many volts a logical 1 has to be and what modulation is used The data link layer defines the format of the network data checksum addresses etc and divides data received from the physical layer into frames It is also responsible for flow control and checking for errors during the transmission The network layer is responsible for wrapping data received from the data link layer into packets and relaying them to the appropriate recipient It also has to manage routing i e choose the best path shortest or the one with the smallest delay for example to the des tination The transport layer receives data from the session layer divides it into blocks and ensures these arrive at their destination in the correct order It also adjusts the transmission speed so that the recipient is not overwhelmed with data The session layer creates a session between two participants It controls opening maintain ing and closing the session and also resuming it after an interruption The presentation layer is not concerned with data transfer anymore its primary task is re sponsib
163. etworks in or der to process all information without additional delays A drawback however is that a centralised network is totally reliant on the functionality and reliability of the central node If this node fails the network can stop functioning All smart home networks created at TUT have currently been of this type mainly due to the expandability of PCs and ease of expansion Another choice is to not have a central node which means that devices can be connected to each other in any way Wired networks can be connected as a ring through hubs or di 122 Hardware Aspects rectly between devices in the case of a wireless network each node can be capable of com municating with its neighbour nodes In the former case there is always a path from one node to any other node in the network but in a wireless network there can be devices that are out of range of a particular node In practice this requires the network to support multi hop networking in which messages travel through other nodes hopping from one node to another until they reach their destination Nodes are required to create a map of nearby nodes and even if a single node is not directly connected to a node it wants to communicate with itis still possible to relay a message through other nodes Further complexity is added by the possibility of ad hoc networking i e a spontaneous network that is formed by wire less devices in a place where there is no existing network infras
164. evel of home appliance adoption had been reached the time came for the next wave in home electronics which included various kinds of home entertainment The focus shifted from work to lei sure and the technology was adopted rather quickly bringing us televisions radios and tape recorders In the late 1980s the emergence of the personal computer saw another wave of technology and it also quickly found its way into our homes Today we are accustomed to using such devices in our daily lives without giving it a second thought Technological development clearly does not end here and the increasing number of devices user inter faces and functions present new challenges in terms of compatibility management and us age Personal computers wireless networks home gateways Internet connections and device convergence have all opened up new possibilities and we can now perform computing in various locations and forms A smart home contains many kinds of devices sensors and user interfaces and in order for these to communicate with each other share information and processing power they need to be connected to each other Thus seamless networking and device interconnectivity is a fundamental requirement for a smart home as Mark Weiser Weiser9 1 observed in his views on Ubiquitous Computing Unfortunately this re quirement for compliancy and compatibility still presents a difficult obstacle as there are so many competing standards and technologies availa
165. f a modern mobile phone might be unmotivated to read instruction manuals and use only basic functions such as making phone calls sending SMS and adding phonebook entries Other functions might be hidden under a myriad of menus and submenus require initialisation or some form of setting up The user might thus be unaware of the other functions the phone offers such as calendar calculator alarm unit converter etc A similar problem can occur in a home en vironment if a power outage resets the alarm clock or the clock on the microwave oven most users have to resort to the user s manual in order to set the time When a new TV set is purchased considerable time must be spent on connecting all the devices to it setting up the system and tuning the TV channels Such problems can be frustrating and overwhelm ing making the prospect of a future home filled with even more electronics even more daunting MS07 A smart home should be able to overcome this problem a smart home containing sensors actuators and user interfaces must be capable of interacting smoothly with both devices and users However connecting all possible home devices and appliances is not enough practical issues such as interoperability and compatibility must also be taken into consid eration When using modern electronic gadgets that process information there is often a need to transfer this information to another device or computer Problems arise if the de vices or the infor
166. f separate software that run on a standard PC Instead a modular expandable core would provide an easier upgrade path and room for future improvements as well as connectivity to other software compo nents either running locally on the same machine or in a remote location The choice of programming language can also affect the flexibility of the software system for example using a platform independent programming language such as Java would allow the soft ware to be run on several kinds of platforms This chapter presents design issues with smart home control software various kinds of middleware implementations and embedded software used in smart home devices Proto cols used in smart home projects at TUT are also presented 7 1 Centralised vs Distributed Intelligence The smart home infrastructure can either rely on acentral master server that manages all connections devices artificial intelligence and resources or these functions can be scat tered all around the network as the ubiquitous computing definition suggests As ex plained in the previous chapter a centralised version has obvious weaknesses that relate to the reliability of a single computer software and network stability but on the other hand it also reduces the workload on all other the network nodes Networks and protocols and also software can be simpler since all relevant information is stored on the central node Nodes may communicate directly with each other bu
167. festyle relaxing enjoying things together as a family etc and without becoming a burden to the users As its name implies the Aware Home is designed to be aware of the people inside it where they are and what they are doing A tracking floor sensor array can monitor the movements of users inside the home and trackable tagged objects make themselves easier to find An other emphasis of Aware Home research is on aging residents and improving their quality of life by allowing them to remain in their homes and live independently Research projects focus on solving the problem of using everyday technologies and on connecting family members together In order to solve these problems new UIs and communication methods have been introduced Family members can keep contact with elderly relatives using a variety of gadgets children have new kinds of interactive toys and there are also various forms of home entertainment Some personal items have RF tags to make them easy to find when misplaced Figure 4 4 The Aware Home Photos Aware Home Research Initiative The Aware Home has two identical living spaces each one consisting of two bedrooms two bathrooms an office kitchen dining room and one living room The basement houses a control room and a shared home entertainment area The purpose is to have people living in one half of the building while the other half is used for testing and prototyping The Aware Home is used by both staff and students
168. ficult for a generic apart ment to suit all these needs The apartment of the future might be modular and flexible with shared spaces that inhabitants are able to use In order to further facilitate flexibility rooms can have multiple functions for example the living room could easily be changed into a bedroom for the night 44 Related Work 4 Related Work 4 1 Overview Smart home projects laboratories and industrial showcases exist all over the world and while some might be more advanced than others they share many similarities Generally smart home projects contain some form of network infrastructure often with wireless net works sensors and actuators Graphical or multimodal UIs are used for controlling the home while some kind of middleware software manages decision making and automated tasks using artificial intelligence agents or pre set rules Signal processing with the help of cameras or location sensors is used for locating and identifying people Depending on the research group the aim of the project can be to prove the functionality of particular technological innovations to gather usability information and practical testing results or to demonstrate the latest commercial technology This chapter reviews research projects that have produced practical data and real world re sults i e they are similar to the smart home projects at TUT their goals and findings and relates them to TUT smart home research 4 2 Other
169. flexibility and adaptability when thinking about controlling A V equipment access control monitoring energy consumption and managing security The user interface UI of every device could be made to look similar and only relevant functions could be enabled depending on the current user and context The DVD player VCR set top box would have similar controls and traditional infrared remotes would no longer be required Likewise instead of simply turning off a light somewhere in the home the switches could be programmed to perform any function in the home For ex ample one button could be a master all off switch which would turn off all lights in the home when the users are going out Certain features could be disabled to suit the users that access the system for example children could not turn on the kitchen stove and remote access could enable users to monitor the home from a remote location with a mobile phone Uls need not only be physical or graphical there are obviously other ways of interacting with the home for example by using gestures speech or other natural forms of commu nication When considering the smart home as a physical space there was one further reason that inspired us to start experimenting with smart homes Despite the number of smart home research projects in the world very few had actually implemented a complete apartment let alone in 1999 In many cases there had only been laboratory tests simulations and ne
170. formation and services to the user The real challenge lies in select ing which sensors to use and what kind of information is important for detecting contexts Experiments have been made to detect normal everyday scenarios such as dining watch ing TV and bathing using hundreds of different kinds of sensors Logan06 and results show that it is difficult to reliably detect a certain scenario Even a small variation in sensor data might cause the detection algorithm to think that another scenario is unfolding or falsely detect another scenario Context ambiguity is also a potential problem among context aware applications that as sume the sensed and interpreted contexts to be the same Dey05 Ambiguity is caused by problems in interpreting sensor data and converting it into context information for exam ple if a speech recognition software fails to catch an important phrase or a positioning sys tem reports incorrect coordinates According to the author handling ambiguous contexts requires user involvement thus helping the system to correct false detections and assist it with learning contexts In order to perform this UIs need to be able to provide redundant mediation techniques allowing the user to choose the least obtrusive and most suitable UI for interaction and flexible input methods related to both time and space Regarding user involvement there is also the matter of availability and discretion Nagel04 when can the user be contacte
171. g INTEGRATION Figure 2 2 Building blocks and technology requirements for AmI ISTAGO3 2 3 Pervasive Computing Pervasive Computing is generally considered to be a synonym for Ubiquitous Computing though the term was introduced by IBM in 1998 IBM98 The term Pervasive itself re fers to something that is everywhere all around us in this case computing As information technology has become cheaper and faster every year the amount of processing that devic es are able to perform has made it possible to create smart devices that can make our lives 11 Definitions easier and more productive Information technology has thus moved from computers into everyday devices including home entertainment personal digital devices and mobile phones Pervasive Computing makes it possible for users to conveniently access informa tion using new networks and communication technologies bringing new possibilities and improving the quality of our lives Relevant information is available whenever and wher ever the user needs it The Pervasive Computing concept also includes third party services support and other businesses such as content providers IBM has a Pervasive Computing Laboratory in Austin Texas for demonstrating new pervasive technologies artificial intel ligence voice recognition and wireless computing According to Uwe Hansmann Hansmann03 Pervasive Computing can be divided into four fundamental paradigms Dece
172. g I am eating or just let the computer assign ID numbers to them in a database Initially manually naming each context will require sig nificant effort from users but eventually the number of new contexts and thus also the number of required user interventions will decrease Context awareness allows the home control system to aid users in matters of safety for ex ample if a fire breaks out Mayrhofer04 Accidents and hazards might be avoided if the system could detect dangerous situations and take preventative action Dangerous situa tions could be fire medical emergencies or less serious matters such as a network overload or disconnection Other advantages concern configuration and preparation if the network structure or device configuration changes there could be another way of retaining func tionality without affecting performance Furthermore if the system predicts that the user is about to return home it could boot up the computer or preload some information so that everything is ready Alarms calendar entries and automatic timer settings would be done in a similar fashion With the help of auditory UIs the interaction between the user and the system could assist in planning daily chores 20 Definitions 2 11 Home Automation vs Smart Homes Smart home technology has often incorrectly been considered a synonym for home au tomation thus giving the impression that the home is an automated entity that performs al
173. g consumption and the ability to effectively control appliances and HVAC related devices in order to help users to reduce energy consumption YIT a large Finnish construction company has installed water usage meters in each apartment of a block to study the effect this has on water consumption YIT The results showed a reduction in water consump tion of one third When people were able to monitor their water usage continuously it was easier for them to try different water saving techniques and directly see the effect on con sumption Another study Homesoft04 has shown that during the entire lifetime of the house 20 of the cost of living is composed of construction costs whereas the remaining 80 is for maintenance heating etc Thus though a smart home would cost more to build in the long term it could achieve greater savings A smart home could save energy by turning off unnecessary lights and appliances by reacting quicker to variations in am bient temperature and by using daylight to collect as much energy as possible by adjusting drapes and window shutters Further energy savings can be achieved by allowing the home control system to lower temperatures by a few degrees and turn off unnecessary ventilation and lighting when people are not at home In office buildings the savings could be even greater since they are generally only occupied eight hours per day Communication Communication with family members also becomes easier peop
174. gets dark outside electric outlets that can be switched on with remote control etc The basic technology is no longer expensive and there are many manufacturers to choose from and people can even perform the installation themselves Automation Home automation systems benefit largely from simple inexpensive networks such as X10 X10 since these are also easy to retrofit into older buildings In other cases sensors and switches usually require direct wiring to a controller unit and audio video signals are transferred using traditional digital or analogue cables possibly even wirelessly Costs start to rise when touch screen wall panels remotely controllable thermostats and multi room audio systems are installed Cybermanor Homeseer 22 Smart Homes 3 Smart Homes With electronic components becoming cheaper faster smaller and more efficient there has been a huge increase in the amount of electronic devices that play a part in our daily lives Digital convergence presents us with new opportunities and accelerates the speed of the information and communication technology ICT revolution ISTAG06 Today our homes already contain many kinds of electronic devices that we use every day usually without even noticing that there is a computer or an electronic system inside it performing measurements calculations and adjustments Thus ordinary everyday objects can contain functions and potential that are currently just beginning to emerge
175. h forecasts proved false and with interactive digital TV services being practically nonexistent and later completely discontinued in Finland these products failed to appear on the market The eHome was thus also left without a home gateway leaving the project without a TV interface For this purpose a home theatre PC HTPC was used instead mimicking a home gateway with Internet connections and media playing ca pabilities The HTPC was connected to the TV set and had a DVD drive as well as an in frared receiver allowing it to be controlled with a standard remote control Software created a graphical UI on the screen and processed remote commands which were forward ed to the Home Controller server The TV UI had similar controls as other UIs the empha sis being on entertainment and commands related to TV viewing lights blinds preset modes Figure 5 47 eHome TV UI showing lighting controls 5 5 Morphome Living in metamorphosis 2003 2005 The Morphome project Living in Metamorphosis Control and Awareness in a Proactive Home Environment was a joint effort amongst TUT University of Tampere and the Uni versity of Art and Design Finding was from The Finnish Academy Research Programme 101 TUT Smart Home Research on Proactive Computing Since the author was not directly involved in this project it is mentioned only briefly here Morphome M yr 05 was started to discover how proactive home applications should be des
176. h still not perfect there are programmes that can al ready process and recognise a a variety of commands efficiently Using speech as a form of interaction allows users to focus their attention elsewhere since there is no need to con centrate on reading text on a screen pressing buttons etc Coen98 Moreover the increas ing complexity of graphical user interfaces and the number of available functions in a smart environment call for more intuitive ways of interfacing with the home system This is especially important when considering mobile devices and other applications where UIs are usually restricted in size small screens and tiny buttons can reduce usability drastical ly A speech control interface would usefully allow hands free operation in numerous com mon everyday situations such as when coming home from the grocery store carrying large shopping bags or when the physical controls are out of reach Text to speech synthesizers can create audio feedback and read aloud e mails news items and other important notifications Audio feedback can be directed to the room where the user is present and the system can be made to read any text string that the server produces This way it is possible to construct a system that would listen to commands from users process them and provide audio feedback back to users The practical implementation would be more difficult because microphones must be used either requiring users to carry them around or to i
177. h the eHome control sys tem affected the users there were interesting changes in their daily routines and lifestyle Timer options group controls and remote control became increasingly popular during the course of the project and tenants soon realised that they had grown pleasantly accustomed to the system For example in the evening when the lights were programmed to turn off at 11 pm the tenants reacted by registering that it was getting dark and time for bed In the morning the timer was set to waken them by opening the blinds slowly turning on the bed room lights After a few minutes the blinds closed again allowing the tenants to go and take a shower The tenants soon found this way of waking up much more pleasant than a noisy alarm clock The eHome system ran continuously for three years and the infrastructure worked reason ably well throughout the experiment Apart from a few blown fuses caused by a broken light dimmer and a broken computer power supply the system required almost no atten tion from tenants or researchers Devices were added during the course of the project and some were modified for example the TV UI but otherwise there were no major changes The eHome software however encountered a few problems As is common with proto type software there was downtime caused by updates bugs and network failures Depend ing on the problem it might have meant that some of the UIs were unavailable in which case the tenants t
178. he home infrastructure but was a stand alone device programmed to respond to certain IR tag ID numbers At a later stage the infrared detector in the handle was changed to a capacitive sensor that reacted when a person was touching the handle This enabled faster and more reliable sensing and it removed the need for the person entering to touch the handle in a specific way i e putting the hand completely around the handle in order to block the IR receiver transmitter pair Figure 5 13 Door opening unit with control for the electrical lock 70 TUT Smart Home Research Halogen Lights Lighting in the Living Room was implemented using ceiling mounted halogen lights All lights were controlled by the computer with a special controller card which also enabled stepless dimming Each light could also be controlled separately making it possible to ad just brightness in different parts of the room The controller also allowed lights to be grouped according to their physical location as well as enabling manual control from tra ditional wall mounted switches The dimming feature and control of individual lights en abled users to set different moods and also control the lighting of the space in a precise manner Lights could also be controlled using various sensor data for example according to movement data from the floor sensors The lights were controlled primarily from the tabletop PC using a graphical UI but manual light controls were als
179. he home is capable of learning and if it is aware of its occupants and installed devices The classification contains five groups e Homes that contain intelligent objects standalone appliances that contain some kind of intelligence e Homes with intelligent communicating objects intelligent appliances and objects are able to communicate with each other and exchange information e Connected homes homes with both internal and external networks making them and their services accessible from inside and outside e Learning homes homes that record and adapt to behavioural patterns of their users control the devices accordingly and predict the user s future actions e Attentive homes homes that constantly monitor their users and use this information for anticipating their needs The Smart Homes Association on the other hand defines smart home technology as the integration of technology and services through home networking for better quality of liv ing Tiresias08 2 1 Ubiquitous Computing Over a decade ago views and theories began to take shape of computers and computing capacity that is available everywhere anytime and via any device and also about how com Definitions puters would disappear into the background Weiser91 Mark Weiser s theory was that evolution would go from one computer many users mainframe era into one computer one user personal computer era and in the future to one user many computers ubiqui
180. he major point is to really have an effective use and added value is the network of different appliances applications and services underlining the impor tance of the connected nature of the system Steiner01 The family was also reasonably technically proficient since they were able to make pre set modes and create various mac ros that they used on a daily basis suchas a movie mode for the living room and ventilation 56 Related Work schemes for bedrooms Sensors in the Futurelife home also contained a form of learning mechanism allowing users to teach them various kinds of desired behaviour Figure 4 13 Futurelife house interior Photo Futurelife AG 4 2 16 inHaus 2001 The inHaus innovation centre of the Fraunhofer Gesellschaft located in Duisburg Germa ny is a cooperation platform for new technology and innovations inHaus The goal of the centre is to pool technologies from different manufacturers and integrate them into the inHaus centre Other themes include reducing energy consumption increasing the attrac tiveness of smart home technology and enhancing the convenience for senior citizens The facility opened in 2001 with the completion of the first building inHaus The centre forms a 250 m residential space consisting of a living area home office garage and gar den In the basement there is a mechanical workshop which serves as a location for hobbies and a place where researchers can develop software and tes
181. he system performed as a whole As mentioned in Chapter 5 data was collected from the tenants by a usability researcher from TUT as a joint project called Smart Home Usability and Living Experi ence but activity logs error logs and spontaneous user reports were also collected on a regular basis The couple living in the eHome apartment were a female of 25 years and a male of 26 and they moved into the apartment in late 2002 To the couple the home was a place where they spent most of their time a place for relaxation and being together They also enjoyed cook ing and much of their time was spent in the kitchen At the beginning of the eHome project the tenants were introduced to the UIs and functionality of the home and they were able to start using it immediately Initially they showed great interest in the TV UI as it seemed a logical and familiar way of controlling the home However they found the mobile phone less interesting as it seemed an unusual place from where to control the home and because 177 Findings they were unfamiliar with the particular model The WWW UI had a clear role from the outset as the main console from where to input the timer settings preset modes and so on Eventually however the situation was to change considerably the mobile phone quickly became easily the most popular UI in the eHome This was due to problems experienced with the TV UI that became apparent as it was being used on a daily basis and
182. http www tinyos net The RNIB Digital Accessibility Team 2008 Smart home environ ment available at http w ww tiresias org cost219ter inclusive_future inclusive_future_ch3 htm P Tolmie J Pycock T Diggins A MacLean A Karsenty Unre markable computing in Proceedings of the SIGCHI conference on Human factors in computing systems Minneapolis Minnesota USA 2002 pp 399 406 Tulevaisuuden lyk s koti Smart home of the future Tampere University of Technology Department of Electronics internal memo 2000 TUT Institute of Digital and Computer Systems TUT Wireless Sen sor Network TUTWSN Wireless Sensor Network 2009 availa ble at http www tkt cs tut fi research daci ra_tutwsn_overview html E Tuulari Methods and technologies for experimenting with ubiq uitous computing Doctorate thesis VTT Espoo Finland 2005 203 References UCOS UPnP UUTE Vainio06 Valtchev02 Valtonen06 Valtonen07 Valtonen09 Vanhala0 1 Vanhala02 VTTO3 204 uC OS II kernel overview 2005 available at http www micri um com products rtos kernel rtos html Universal Plug and Play forum 2009 available at http www up np org T Peta koski Hult J Merilahti Juho T Kanto Hannula H Kailan to M Zakrzewski New Technology Supporting Rehabilitation at Home in proceedings of the 2nd European Conference on Techni
183. huge progress during the author s ten years of involvement in the field However there is still a long way to go before most of us can sit back and relax in our smart homes However interesting times lie ahead and the smart home is assured of a place in this future 189 References References Abowd02 Abowd96 Ahola01 AIRE Aldrich03 AmbOrb07 ANT Arabianranta Atmel Automation Barton01 Bathiche07 Bernstein96 190 G Abowd A Bobick I Essa E Mynatt W Rogers The Aware Home Developing Technologies for Successful Aging in Proceed ings of AAAI Workshop and Automation as a Care Giver Alberta Canada 2002 G Abowd C Atkeson A Feinstein C Hmelo R Kooper S Long N Sawhney M Tani Teach and Learning a Multimedia Author ing The Classroom 2000 project in Proceedings of the ACM Mul timedia 96 Conference 1996 pp 187 198 J Ahola Ambient Intelligence in ERCIM News Number 47 Eu ropean Research Consortium for Informatics and Mathematics ER CIM 2001 page 8 MIT Agent based Intelligent Reactive Environments AIRE 2003 available at http aire csail mit edu F Aldrich Smart homes past present and future in Inside the Smart Home R Harper ed Springer Verlag pp 17 36 2003 Ambient Devices Ambient Orb 2008 available at http ambient devices com cat orb orborder html Dynastream Innovations Inc
184. hus reside in any location Because of this mobile UIs can be used in many different environments to control various items For example in a sports stadium users can access information on the 160 User Interfaces game players or statistics and in a shop they can access prices shopping lists and so on When an occupant returns home mobile devices can synchronise data that they have col lected during the day and access updated information from the home server In the past resources of mobile devices have been rather limited but with advances in battery and electronics technology this problem is rapidly diminishing In general the interaction between computers and users in smart homes requires new kinds of user interfaces as the activities are typically non computational Interfaces for offices computer environments and information processing etc are generally unsuitable as they are designed for another purpose Coen98 Thus the human computer interaction should become less explicit and more closely connected to the environment and situation in which it is being performed The ultimate goal would be to have UIs disappear into the back ground becoming visible only as needed Dey01_2 According to the authors there are different levels of invisibility Truly invisible UI is completely unnoticeable to the user and integrated into the environment interaction with the user is implicit transparent UI is not physically invisible
185. ibition which was held in con junction with the MindTrek 2001 conference in Tampere Finland Most hardware and fur niture from the Living Room was transported to the exhibition area and a new living space was constructed The theme of the exhibition was future technologies to which the Living Room was very well suited The exhibition contained a home theatre floor sensor demo which was built into a game EMFi demonstrator a flower pot monitor demo a smart wine bottle rack and an InfoCube demo Visitors showed keen interest in all the various technologies particularly the flower pot monitor demo with its capacity to monitor the well being of the plant and adjust lights and curtains to achieve maximum growth The ex hibition lasted two days after which the equipment was transferred back to the Living Room 174 Findings Figure 9 1 The e home stand at MindTrek 2001 exhibition showing the living area in the middle and the InfoCube demo to the right Activities around the Living Room project provided valuable educational experiences for our group of young researchers and experimentations with different kinds of hardware networks and UIs offered many ideas for further development and improvement The re search group now came to view ubiquitous computing environments and user interfaces seem as the key factors in smart home design The Living Room laboratory was in use until late 2002 when an opportunity arose to move to a new te
186. ic or the number of nodes a network can sustain Reliability Wireless networks are inherently less reliable than wired counterparts but nonetheless reliability and the ability to cope with errors noisy environments and other points of failure are very important in a smart environment Error correction the possibil ity to resend corrupted data and collision detection are examples of methods to cope with such problems Complexity To the user networks should be as invisible as possible there should be no addressing installation or setting up required of the user Standardisation Possibly the largest problem with smart environments is the myriad of different standards that exist especially for networks However since it can be difficult or impractical to use only one type of network interoperability has to be considered carefully in the design process Latency Here latency is the delay that data undergoes from the time it is sent from one device to the time it is received by the target device If latency becomes too great it can adversely affect the functionality and response time of a smart home system as messages arrive too late to react accurately to events in the home High latencies are also easily per ceived by users and they can cause frustration or be misinterpreted Privacy Concerns regarding privacy and security are understandable and of major im portance to users of smart homes since the amount of data that is being ga
187. ide Web will be the context for future ubiquitous networks due to its openness and prevalence The infrastructure con sists of beacons for example infrared ones that contain contextual information about events location etc Beacons transmit specific URLs to mobile devices which then obtain a web link where they can download relevant information For example when a user in the vicinity of a conference room where a meeting is about to start he can download informa tion about the meeting onto his PDA and decide whether or not to join in This integration of the physical world and the virtual world of the web forms a sensor enhanced ubiquitous environment as shown below PDAs phones Context amp URL Context Places printers Exchange ID resolution URL Sensing ID sensing Figure 4 9 Component relationships URL sources and contexts in Cooltown BartonO1 Cooltown laboratories have been built around the world and they typically contain a fu ture office setting a digital home large displays and mobile terminals for information ac cess Figure 4 10 HP Cooltown Photos Anton Diaz 52 Related Work 4 2 10 Orange at Home 2001 2003 Orange at Home was a smart home project started in 2001 by Orange a UK mobile net work operator The project was a smart house in Hertfordshire which was equipped with the latest technology The Orange house featured health monitoring of its inhabitants a self diagnosing heating s
188. if the central com puter or network is inoperable or if direct manual control is desired for some reason Even if the device is controlled from a graphical UI it is often possible to recreate the natural 170 User Interfaces way of controlling it on the screen for example by using a virtual representation of the controllable device 8 9 Different Types of Smart Home UIs The types of UIs that smart homes can contain are almost limitless but some of the most popular can be categorised as follows e Various types of graphical UIs GUIs using different kinds of displays and monitors Graphical UIs can present a lot of information at the same time they are easily recon figurable and they can be tailored to suit a specific application or user group e Touch screens mounted on the wall wall panels tablet PCs and mobile phones These are essentially a subgroup of GUIs but they introduce a more physical element into control by offering physical like controls and interaction For example a light dimmer control such as a graphical model of a dial on a traditional mouse operated GUI requires precise mouse movement whereas on a touch screen could be in the form of a slider that users can simply drag into a desired position e Mechanical switches buttons levers etc These can be fixed wall mounted for example or mobile wireless portable or attached to another mobile device Switches that are computer readable can be programmed
189. igned and since this was an interdisciplinary project a non technical approach was adopted The project approached domestic life as human centred experience rather than random interaction with home appliances Another goal was to create new design methods for proactive technology Soronen04 In order to verify and test this theory Morphome also involved testing in two pilot homes and other shorter term tests with other families during the course of the project Users were given a camera a diary and a notebook that they could use to document their experiences with the new technology Prototypes constructed during the project include pillows that detect each others presence emitting different sounds depending on the pillow they are next to lamps that reacted to sound levels and ambient light and were able to slowly change colours and an X10 based home network that adjusted lights in the home according to movement sounds and pre set timer settings Figure 5 48 X10 modules used in the Morphome project Findings of the project showed that multidisciplinary studies can provide results that can not be gained from specialised studies through prototype engineering and user interviews In addition acceptability of new smart technology was surprisingly low among the testers but this was considerably helped by the softness of the prototypes used in the tests e g Morphome devices were not gray boring boxes but instead friendly looking gadgets tha
190. ility for encryption decryption and as the name implies the presentation of data semantics format character sets etc The uppermost layer is the application layer which is the layer that is visible to the user This provides different kinds of protocols for applications for example e mail FTP tel net Another network related term is Quality of Service QoS QoS is not directly an indicator of achieved service quality but rather a guarantee of performance for applications Cisco03 QoS contains control mechanisms for reserving resources in a network for ex ample required bit rates maximum error rates or maximum delays It can be used to pri oritise different kinds of data according to their requirements ensuring important data receives high priority especially in congested networks For example delay sensitive ap plications such as voice over IP VoIP benefit from low latencies and guaranteed data rates 6 2 Wired Communication The first wired communication media were telegraph and telephone lines and even if the traditional switched telephone network still exists across the globe other more modern net works are slowly taking over Generally wired networks are robust dependable fast and cost efficient but even if the hardware itself is inexpensive compared to wireless alterna tives installation work can involve high cost Wired networks are highly resistant to inter ference and they offer a broad range of communicat
191. in the eHome but a Fujitsu Siemens Lifebook laptop with a touch screen was acquired as a primary terminal From the laptop users could navigate to the Home Controller page and make adjustments from there The main page showed pre set modes timer and light adjustment options The timer settings page contained settings for different kinds of tim ers for example users could set all lights to turn off at midnight each working day Timers could also be temporarily disabled if necessary A drawing of the floor plan was shown on the light adjustment page allowing users to click on a desired room and adjust the light levels from a slider on the right hand side of the page Individual lights could also be turned on or off by clicking on the icon on the floor plan The primary purpose for the web UI was to allow users to make more complex settings such as setting modes or timers 99 TUT Smart Home Research Watte hucre Pores ja shidh verhot hakaman suertoon WR ORDe Shem bas bio bsere valtsemala siasa ja panamala Pasta parece vae Q pamaos WE 1 Viikonloppuisin iso 16 19 nias z 4 F Obtusce ZS venct EY SA _ ramae T nuhe on WP 1 Joka pava ito 1200 Ena x xz ae Sea ey pee Figure 5 45 Screenshots of the eHome WWW UI Shown here is the start page top left lighting con trol top right window blind control bottom left timer settings bottom right eHome Mobile Phone UI The mobile phone UI was designed to ru
192. ind smart spaces is to reduce stress and improve the usa bility and living experience of the users Weiser96 Smart networked homes can also offer other advantages when devices are connected to each other either directly or through a server they are able to access and use information that would otherwise be unavailable They could share time information location data processing power etc with each other through the network When sensor data from all around the home is collected and combined the system and the user can gain a much clearer picture of the situation in the home For example a temperature sensor in the living room could only measure the temperature in that particular location and if the temperature starts to drop a home automation system would simply turn the heating up A networked smart home on the other hand could utilise other sensors in the room and register that the window has been opened because of bad air quality inside the room On subsequent occa sions the system will already know the reasons and proactively take the necessary action Thus information which in isolation may seem insignificant can actually be valuable when considered and compared with other measurements and information from the home Another often overlooked aspect of smart homes is the addition of intuitive and logical user interfaces that would lessen the burden of managing the home For example a graph ical home user interface would offer great
193. ing has been conducted all over the world as presented in the previous chapter Mostly however these have been theoretical studies in a mock up or laboratory environment And though this kind of research has been popular at universi ties and companies there is still a vast amount of empirical study left to do To meet this need for starting constructive and empirical research on the subject we set out to implement physical devices and spaces instead of theoretical tests simulations and in terviews Smart home research at TUT started when a simple low cost network for con necting different devices together was designed and implemented Later different kinds of sensors actuators and other devices were built and all interconnected to form the basis of a smart home The first test environments were located at Tampere University of Technol ogy and were later expanded to include an apartment with tenants for real life studies This chapter presents four different smart home implementations at TUT their initial goals design and implementation Each space served a slightly different purpose enabling research to be conducted from many points of view Fig 5 1 presents the timeline starting from 1999 showing important milestones such as completion of my Master s thesis in 2001 on the way Timeline 1999 2009 Smart Home research UUTE project started starts control in the Smart Smart Living Room Master s Thesis Smart Home amp eHo
194. ing the computer more to the background and using more natural forms of inter action between users and computers Blair MacIntyre proposes that interaction should not be through menus windows and other mouse driven interfaces but instead through ges tures speech context and affect MacIntyre98 People should be unaware they are inter acting with a computer Another definition describes Intelligent Environments as spaces 15 Definitions in which computation is seamlessly used to enhance ordinary activity Coen98 As the name implies Intelligent Environments encompass all kinds of spaces such as offices meeting rooms hospitals etc Intelligent Environments are both embedded and multimo dal which means that they use cameras microphones and other sensors for sensing real world phenomena and computer software to interpret and understand them This approach does not require the same kind of hardware and sensors as Ubiquitous Computing instead the focus is on unencumbered interaction with non computational objects Thus far seven different definitions have been presented and it would seem that they all look at things from a slightly different perspective each having their own presumptions goals and enabling technologies Ubiquitous Computing and Pervasive Computing both rely heavily on ad hoc networking resource sharing and new kinds of human computer in teraction Ambient Intelligence is also using Ubiquitous Computing as enab
195. inks Measurements control and communication are done with embedded microcontrollers in the devices Com mon features for these devices are small size low power consumption low price and wi reless connectivity A test laboratory has been built at the institute where different kinds of smart devices have been installed for usability and practical testing The TUT definition combines parts from previous definitions as there are networked de vices and sensors contextual reasoning and improved usability and security There are however notable differences For example processing power is not shared among devices as such as most processing can be done either centrally or independently in devices Us ers actions are monitored but interactions between users or with computers are not ana lysed or monitored in a specific way in order to extract their contents 2 5 Proactive Computing Proactive Computing monitors the context of the users and depending on the captured context context recognition is described in more detail in chapter 2 10 and learned pat terns the system can take the appropriate action M yr 05 The idea behind proactivity is that the system can anticipate user behaviour and prepare for the next actions in advance without the user noticing it Thus the lag will be minimal and use of available resources optimised One goal of proactive computing is to take the human out of the control loop Tennenhouse00 Interacti
196. iod of time party Each mode switches the home into a specific mode by adjusting lights drapes doors HVAC controls and security systems to a certain state Additionally the system could alert users of fire or water damage by sending an SMS as well as turning off water electricity and ventilation when necessary The sys tem was designed to be controlled from wall panels and switches and also by means of re mote controls and mobile phones or through an Internet connection Olosundemittaus 2 Yovillennys L Pimasa S Hamaras SD Valoisaa Turva sd Summeri 2 Sireni 3 LED Kuinaus Murtoturva s Sisatitan valvonnan ohitus lt 9 Ovien murtonatytin Sisatitan valvonta Vesiturva SB vodan mg venttiin Kodin eri titanteet Kotona J Juha ro O Poissa gt Pitkalin poissa Porusvalaistus Verhot yids Valot pols Varhot alas Figure 4 6 Homesoft control UI 50 Related Work The system was based on LON based modules controlling heating water lighting and electrics using various sensors to detect changes in the environment The software ran on a COBA compliant COBA platform HomeSoft Control Lighting Ventilation Security Electrics a c Alarms Figure 4 7 Example of building automation system integration Homesoft04 4 2 8 Duke Smart Home 2007 The Duke Smart Home is a large living laboratory which also serves as a student dor mitory at Duke s Pratt School of
197. ion is issued A well designed UI will clearly indicate that an action has been taken a button can give a satisfying click or a software beep when it is pressed a graph ical UI might change the colour of a button icon when it has been pressed or a light is turned on as a result Studies have shown that it is effective to combine different kinds of feedback to improve functionality and usability Emery03 People with special needs such as the elderly blind or deaf could also greatly benefit from different kinds of feed back enabling them to use computers and home equipment with greater ease Main categories of UI related feedback are visual auditory and haptic feedback Visual feedback comprises different kinds of lights and indicators that catch the user s attention by turning on or blinking for example Tangible mechanical UIs like switches buttons and levers also indicate their state to users by visual means light switches remain in the position that they were set to a software button shows whether it is on or off with a label etc Physical elements also offer direct confirmation that an action has been taken i e lights turn on door opens but this is not always evident to users depending on ambient noise light and similar conditions In graphical UIs similar approaches can be taken with on screen buttons changing colour or shape when activated with icons indicating status or simply with a text string Auditory feedback ra
198. ion speeds and in some cases they also supply power through the same cable Wired networks can be connected to other types of 116 Hardware Aspects wired networks using adapters and converters to take advantage of the superior qualities of another network or the same physical cable can be used to house multiple different sig nals Thus it is possible to use traditional electrical or phone cables for data transmission with appropriate transceivers for example making powerline communication and DSL modems possible The disadvantage of these solutions is increased complexity and cost since they require special adapters and converters In apartment buildings it is also difficult to filter out the signals emitted from each apartment and neglecting to do so would create interference security issues and other unwanted effects Wired networks are durable but are also still prone to corrosion and physical damage from excavators drills and mechanical stress Ohmic losses and other unwanted effects also dic tate maximum lengths of cables which for 100 Mbit Ethernet means a maximum cable length of 100 metres for example Wired networks in a home can be divided into four cat egories electric cabling phone cabling audio video cables and data cables 6 2 1 Wired Network Types Traditionally wired communication in embedded electronic systems is done using proven technology such as by using RS 232 serial cables Serial ports are common in comput
199. is sirere a E hii eee ae ee 2 14 Contr Dutton esa seeeesevercustests eere ee yeaa ye ah ste 2 VES MOU Vat OM e a E R dea cones A E Aa Eaa Eaa 4 1 6 Research Question deee a A ee 6 2 Defimitions cccsscsscssssssssscvorsesssesssnsessccsssssscssenssesseossscsssscssensssssenssonssesees 8 2 1 Ubiquitous Computing o eee eee cee eeeeseeeecneesaecaecneeseseeeeaeeeaes 8 2 2 Ambient Intelligence oo eee eesceeeeeeeceeeseecaeesaecaecsaseseeeeeaeeeaes 10 2 3 Pervasive COMpPUtING oo see ceeec eee ea ea aeie asais 11 2 4 Our Definition of a Smart Space 0 eee ese cseeseecsecneesecsseeeeeaes 12 2 5 Proactive Computing oo eee ee cece cee eeeeeeeeseecaeesaecaecsaeeeseneeaeeeaes 14 2 6 Mediated Spaces orreee orense sairo sodan scostseciesnesscbasneseedasscnsccbeses 15 2 7 Intelligent Environments sssssssessssesessesssseserrrsreeresreresrnresesrnressseees 15 2 8 Internet of THINE S Aeon a a E E eae SE 17 2 9 Seamless COMPUUNG nnonsnenndsnnro ii ien a E E iS 18 2 10 Context AWareness ascosi genri ienie En ae Ea ea A E EE EE 19 2 11 Home Automation vs Smart Homes ssesssessesesesseseesesesseresessesee 21 3 Smart HOMES ccccccsscsssssssssssssssssssesesssscsssesesesssesesssceseseessesssesssessesseeees 23 3 1 Building Smart Environments 0 0 eee eseceseceeceseeeeceeeeeeeeeeeeee 25 3 2 Desie Challenges erered rre ire e ee KEE R 27 3 2 1 Networks and Software sesseseseeeeerssrsssresrssrsssrssrseeseeesees
200. is only responding to a wake up message Device Registration Procedure When a new device is introduced into the network it sends out a heartbeat message When the server notices a message from an unregistered device it initiates a login procedure by sending a registration request to the new node The node responds with an acknowledge ment and is given a new unique network address by the server After this the device trans mits an XML file containing the specifics services and functions to the server The coordinates and location of the device are also updated either from the server to the device or vice versa 153 Software Architecture Server Node 1 Heartbeat Request registration Registration acknowledged Change address Address acknowledged Request advertisement Return advertisement Get address coordinates 9 Return address coordinates Figure 7 5 EPIS device registration procedure EPIS Network Protocol EPIS network communication is encapsulated inside TinyOS headers Thus the low level communication is handled by the operating system using a header system similar to the protocols in the Smart Home The message header consists of a message class and type 4 bits each a 16 bit address source or destination and a 16 bit message ID There are a total of five message classes Heartbeat Address Attribute Error and Service Discovery Each class has a number of message typ
201. ises the properties of different wired networks Table 6 2 Comparison between wired network types Network Max data rate Topology Max cable length Ethernet 1000 I Gbit s point to point 100 m BASE T tree HAVi 400 Mbit s point to point 4 5 m 100 m tree EIB 1200 bit s powerline 9 6 several depends on media kbit s twisted pair CEBUS 8 kbit s several depends on media RS 232 115 kbit s point to point 15m RS 485 100 kbit s 35 Mbit s multipoint 1 2 km 118 Hardware Aspects Table 6 2 Comparison between wired network types Network Max data rate Topology Max cable length X10 powerline 20 bit s star 100 m LonWorks 5 4 kbit s powerline 78 several depends on media kbit s twisted pair 6 3 Wireless Communication Wireless communication is a good alternative if cabling is too expensive or complicated to install Usage of wireless networks is more flexible and cost effective when cables are not required and mobility is required Most wireless communication uses radio frequency communication which can be divided into three groups depending on the frequencies that they use narrowband kHz range spread spectrum wideband MHz range or ultra wide band GHz range Energy output Narrowband Spread Spectrum Ultra Wideband Frequency Figure 6 4 Spectrum graphs of different wireless communication technologies The many existing techno
202. it can make the smart home a pleasant functional environment for the users 159 User Interfaces 8 User Interfaces In an ideal case a smart home would not require any special interactions or UIs as the home would learn to anticipate what users want to do and react to changes and situations by constantly monitoring the users actions Mozer99 In practice however some kinds of UIs are required for making more complex adjustments or remote control UIs are after all the primary means of communication and interaction between users and the home con trol system These also present the most challenging design problems as there are different requirements for different users and devices Smart homes present endless new possibili ties as far as UIs are concerned Dey01_2 and in order to maintain usability and practi cality great care has to be taken when choosing and designing these UIs Generally speaking user interfaces should have the following properties e Intuitive logical and easy to understand e Consistent they work as expected every time e Configurable users can if they want to change the way the UI works e Adaptive UI changes according to the context or user e Simple not too complicated easy to use These quite similar to general requirements described in Sections 3 2 1 and 3 2 2 which underlines once again the importance of well designed UIs Smart home UIs can be divid ed into three main categories nat
203. it soon lost its appeal to the tenants The PC in which it was running gave off a slight noise when turned on which the users found disturbing so they turned it off whenever it was not need ed This led to one particular problem with using the TV UI To start the computer and load the software took considerable time and led to decreased use of the TV UI Other difficul ties were caused by the infrared remote receiver which required the remote control to be pointed precisely at it in order for commands to be received Although this problem was solved later it did not fully restore the confidence of the users in the TV UI As the PC controlling the TV UI also functioned as a DVD player it did receive some use and con trolling the lights to watch a movie worked as expected However the tenants preferred not to open the home control UI during a movie or when watching a favourite TV program The reasons for the surprising popularity of the mobile phone UI were its instant availabil ity the phone is always on an intuitive UI the eHome control applet was made using No kia style menus that were similar to the standard menus in the phone and universal access it could be used wherever there was cellular network coverage Furthermore the phone was easy to carry it was physically smaller than the TV remote control and it could be used to turn off all lights and equipment when desired It did not have to be pointed in any particular direction when
204. it would be beneficial to have a control system managing certain items in the home This view is also one shared by the Tiresias research group TiresiasO8 who propose that The goal of equipping the home environment with technology isn t just to automate all the tasks that are carried out by the residents The objective in design is to provide tools and services that empower and enable people themselves to address their social rational and emotional needs Equality autonomy and control are the goals of empowering design This chapter presents a brief history of smart homes and provides a glimpse of what the future might bring with it There is also discussion of smart homes as physical spaces de sign issues and the advantages and disadvantages of smart home technology 3 1 Building Smart Environments The space in which a smart environment is set up naturally plays an important role in the design process In the case of a home a major obstacle is the way in which people regard their homes the home is usually considered a very personal environment almost sacred where any intrusion or modification might be considered unacceptable Koskela04_1 In practice however the home must be the installation space for all equipment so that there is little choice as to where all the infrastructure will be located With unobtrusive installa tions and ubiquitous computing the equipment itself can be made as invisible as possible to the user making i
205. ive tags Passive tags work from a range of a few centimetres whereas active tags can work over a distance of tens of metres Near Field Communication NFC is a technology based on RFID with the largest differ ence being the ability of a device to function both as a tag and as a reader NFC NFC is currently used in some prototype mobile phones enabling NFC tags to be read by touch with a mobile phone Recent developments in wireless technologies have focused on Ultra Wideband UWB which is defined as a technology that uses a bandwidth of at least 500 MHz or at least 20 of the centre frequency Kolic04 Traditional radio networks have transmitted data as long messages over a narrow channel but UWB uses extremely short wide band pulses These pulses typically last nanoseconds and the bandwidth can be several GHz as shown in fig 6 4 Thus the power of the transmission is rather large but it is spread across such a wide spectrum that it does not disturb traditional RF channels UWB radios offer very high data rates with good privacy and compatibility unfortunately a wideband transmitter or several transmitters in close proximity can cause disturbance in nearby equipment UWB is also very resistant to multipath distortion a situation where the signal bounces around and takes many different paths to the receiver and stealthy UWB transmissions can appear as background radiation High data rate UWB is predicted for use in wireless monitors wir
206. ive types of UIs for people to use and are as such the most popular controls for me chanical equipment Buttons switches and levers are familiar to users and they provide a physical and recognisable action when they are used However even traditional physical UIs can be enhanced by other means For example new kinds of functions can be added to traditional UIs like switches and pushbuttons in smart homes wall switches can be pro grammed to control a specific light source or groups of lights One switch for example can turn off all lights in the home making it less likely the user will leave a light on after leaving the house In addition to controlling lights the buttons can be configured for nu merous other tasks or almost any function in the home Thus we still maintain the tradi tional physical UIs while we are able to change their functions to something that would have been very costly or difficult to implement without smart home hardware The major advantage of physical UIs is that their shapes can already indicate the kinds of actions they can perform and their physical existence means that unlike graphical or other virtual representations they cannot accidentally disappear or be closed Mayra05 To some degree a physical device can represent a virtual counterpart for example a digital photo frame but in some cases physical size can become a limiting factor making inter action too cumbersome or impractical 164 User I
207. iving room would control devices in the living room For example if the user walked into the living room and commanded lights on the system would turn on the living room lights Positioning is achieved with infrared transmitters placed in the ceiling all around the apartment and a receiver set in the microphone unit The Smart Home laboratory is divided into six separate areas rooms each having its own speaker location ID and set of controllable devices The microphone has a push button that the user presses and holds when issuing a com mand A beep is heard from a nearby speaker after which the system is ready to accept a voice command After the command has been received the system can take appropriate ac tion and play a confirmation message from a speaker near the user Using a dedicated talk button eliminates the need for constant listening and makes it easier for the computer to capture the command Speech recognition is performed on the Smart Home server which runs speech recognition software A text to speech converter is also used for giving audio feedback to the speakers the software also makes it possible to send any text string to be read aloud to the speakers In order to minimise faulty recognition and speed up recogni tion time the software is set to recognise only a certain set of commands This allows the microphone to be used by others and not only the person who has recorded the original commands to the software An e
208. k to learn to use the UI without additional instructions however its Finnish language text provided daunting for foreigners Other UIs for controlling light ing pushbuttons touch panel are good for their primary purpose and quick to access when needed Speech control and a mobile phone UI again proved to be handier when moving around the apartment and mobility is required The Smart Home has proven that is possible to connect different kinds of incompatible networks devices and user interfaces together to form a unified home infrastructure and the value of such a system will continue to be proven in the future as more functionality and features are added However when the network infrastructure and possible applica tions were designed there were several practical issues that limited the implementation of a pervasive home network where everything would be connected especially with regard to home appliances Connecting existing home appliances to the home network varies from easy coffee maker motorised door blinds lights to difficult stove microwave ov en A V equipment Devices that originally have no network interface or remote control functionality either must be taken apart and physically modified costly difficult or even impossible or externally monitored with some kinds of sensors easier but less reliable In the latter case for example the temperature of a refrigerator could be monitored with an additional sensor or the sta
209. l lectual effectiveness of humans for example by augmenting it with computers This was well before the emergence of the personal computer and other personal electronics Much later came the Intelligent Room at MIT 1998 Coen98 an early laboratory space a former meeting room for bridging computational and everyday activities The Intelli gent Room was not intended to be a ubiquitous computing space full of computers as it was designed with minimal hardware modifications and also enabled interaction with non computational objects This was achieved by the use of wearable infrared badges pressure sensors in furniture computer vision and speech and gesture recognition systems Cameras are used for tracking people identification and UI control whereas the speech recognition system is used for controlling the system and receiving auditory feedback The authors concluded that it was difficult to obtain reliable output from the different kinds of sensors in the Intelligent Room using contemporary technology especially the cameras and com puter vision system The ambiguity inherent in human gestures also presents problems when trying to determine a user s intentions The Classroom 2000 project 1998 Abowd96 set out to enhance the teaching and learn ing experience of university students The goal of the project was to discover how ubiqui tous computing technologies can be used to improve education For this purpose electronic white boa
210. l home gateways being launched by large electronics manufacturers These were typically shoebox sized embedded computers running a customised Linux operating system com munication to the outside world was via DSL cable modem or dial up networking Digital television was also being introduced throughout Europe and interactive services such as web TV were services that were to be offered through a digital set top box This box could also function as a home gateway according to some forecasts as it already contained nec essary hardware and connections From Finnish companies the most notable product an nouncements came from Nokia Nokia Home Gateway and Patria Patria Ailonet However these products vanished completely only to reappear recently for example the reincarnation of the Nokia Home Gateway in an improved and different form Nokia08_2 Figure 6 6 Patria Ailon home gateway Photo Patria 124 Hardware Aspects The current view of home gateways has changed slightly and they now becoming a con vergence point for media networks and communications instead Saito00 Households contain digital cameras wireless devices such as laptops and PDAs HDTVs MP3 players and high speed Internet connections that all can work together with the help of a media gateway A PC can act as a gateway but it would require installation of many kinds of soft ware A new wave of home entertainment electronics has brought with it compliance with
211. l kinds of services for its occupants Home automation can however be integrated into a smart home and be a part of the system The primary role of home automation is to make our life easier by controlling mundane functions like heating air conditioning and lighting and allowing users control to set the parameters and threshold values Additionally home automation can manage and control the home when the occupants are away by taking over certain routine and tedious tasks that are easy to automate Home automation systems can be rather simple in function but re programming or reconfiguring such a system might require the services of trained support personnel Some systems only include basic threshold or timer settings with more ad vanced options requiring reprogramming or additional control units The simplest form of home automation would be a heat regulator that would keep the temperature in a room within specified limits Home Automation is generally considered to consist of applications with certain pre set operations some with programmable options and the most sophisticated including robots and complex machines Gann99 Typical home automation applications include lighting control security systems and home surveillance home entertainment irrigation and tem perature control Home Automation Smart Homes Building Automation HVAC Entertainment systems Proactive adaptive software Timers Multimodal Uls Security systems En
212. l remote control was needed Since homes al ready contain several devices that require infrared remote controls there are often a plethora of bulky remotes Additionally each remote has a different layout buttons and functions making the task of turning on all equipment and setting them up rather tedious To combat this problem an infrared network node was constructed The node named IRe mote was connected to the serial network received commands from the master PC and 73 TUT Smart Home Research transmitted them to A V devices over an infrared link This made it possible to control the DVD player video projector etc without having to use legacy handheld remote controls The Remote node was pre programmed to incorporate all the remote functions for each individual device In addition to the Remote another control method for A V equipment was tested CASIO had produced a wrist watch model CMD30B 1A with an infrared transmitter and built in control buttons Essentially this was a wearable programmable universal remote con trol The watch was able to command individual devices directly using programmable but tons on both sides When buttons on the watch were being operated the watch would send an infrared signal to the IRemote unit which in turn would perform the appropriate tasks with the A V equipment Figure 5 17 IRemote unit and CASIO IR wrist watch 5 3 The Smart Home 2002 gt Two years after the Living Ro
213. l user interfaces The final requirement dependability and security focuses more on the reliability of the system and the security of the users Reliability is dependent on both hardware and software whereas security is more related to the imple mentation and features of the system as a whole 141 Hardware Aspects Further IST requirements for AmI ISTAG03 such as new materials would mainly ben efit UIs and sensors as they would enable new sensor materials and innovative UIs to be constructed MEMS on the other hand would make it possible to design microscopic ac tuators and devices that will open up new worlds in biotechnology high frequency com munications and microsensors Tiny robots implantable technology and extremely small sensors are examples of such applications 6 13 Summary As is evident from this chapter there are numerous choices available for hardware plat forms networks and protocols for smart home designs and choosing the most suitable one can seem a major task Most smart home devices however do not require any specialised hardware and are thus not limited or bound to any specific platform or architecture It is the software and interfaces that are more important factors when compatibility and stand ardisation are being considered In the future new materials and technologies can drasti cally change the way we use and interact with devices bringing the future smart home one step closer 142 Software
214. le can leave notes to each other using home UIs for example on a display in the kitchen or near the front door Com munication to remote locations outside the home for example to relatives in another city is also possible using home UIs cameras and broadband networks A study made by IST ISTAG03 shows that today s urban populations are becoming increasingly isolated from friends families and neighbours The quality of social bonds is however relevant to the well being of people and thus of great importance By building environments that support the formation and maintenance of social networks it is possible to reinforce these impor tant bonds The Arabianranta residential area in Helsinki Finland contains homes for around 10 000 people and it is built around a virtual community with many on line services and a common Internet portal Arabianranta This Virtual Village is an experiment in volving third party services discussion forums on line calendars bulletin boards and eS ervices Each apartment is equipped with a fiber optic high speed Internet connection and wireless networks are widely available throughout the residential area The goal of the Vir tual Village is to experiment with community networking and the effects of pervasive con 34 Smart Homes nectivity and the impact this has on the inhabitants and their lives The purpose is to discover if networks make people more social or isolated and how much connectivity
215. levision channels on each side and can be seen in Figure 9 1 When users want to change the television channel they simply pick up the cube and rotate it in their hands The cube on the screen will rotate accordingly and users can see the contents of the different televi sion channels When they find the channel that they want to watch they simply squeeze the cube on the appropriate side and the selected channel will pop up and become fully vis ible Other applications for the cube include controlling similar 3D user interfaces and navigat ing through menus The cube could also be used in computer games and since it is wireless and battery powered there are no cables to restrict its range and movement Figure 8 3 InfoCube prototype 166 User Interfaces 8 6 2 Microsoft Surface Microsoft Bathiche07 has been developing a large tabletop surface which acts both as a display and an interactive touch sensitive surface The tabletop screen allows multiple us ers to view graphical elements and interact with them by touching the table surface This technology would fit well into smart homes as the entire family could browse the home functions and make necessary adjustments together The Surface table is still being devel oped but similar technology is already available albeit in smaller sizes touch screens and LCD monitors tablet computers and computer vision applications Figure 8 4 Microsoft Surface Photo courtesy of Micr
216. ling technolo gy but emphasis is on user recognition and user centricity That is AmI creates the media for person to person and person to computer interactions Proactive Computing on the other hand aims to relieve users of lower level functions and lets the computing system handle context recognition and anticipation of the user s actions This approach requires a significant amount of signal processing and a complex software architecture something that is even more important in the case of Mediated Spaces Mediated Spaces require a large network of sensors as the system has to be able to understand the interactions that are taking place in the space Gathering sensor data is rather straightforward but analysing and understanding its meaning is quite a challenge for software developers Intelligent Envi ronments leave the ubiquitous hardware and software in the background and focuses on usability natural interactions and straightforward control of the environment The TUT definition relies both on ubiquitous hardware and software but it is the seamless coopera tion of hardware and software that enables a flexible adaptive smart home system Table 2 1 summarises the main themes and perspectives of the different definitions 16 Definitions Table 2 1 Main themes and differences between the definitions Main themes From the com puter s perspective From the user s perspective Ubiquitous Com Processing power
217. lity One important UI related issue is latency if it takes a noticeable time for the issued com mand to be executed itis easily perceived as annoying or even as a malfunction A latency of a few hundred milliseconds is already discernable Koskela03 whilst a second or more already makes the user question if the command has been received at all This can lead to people issuing a repeat command which eventually leads to multiple commands being ex ecuted possibly cancelling each other out In smart home scenarios this is evident in the case of lighting control for example because the results are immediately visible Smart home UIs can also be adaptive i e they can display information and controls that are relevant to the user at a particular moment Tiresias08 For example during daytime 161 User Interfaces a control panel might display controls for adjusting blinds or opening windows whereas in the night it would instead display controls for adjusting lights This chapter discusses important elements of smart home user interfaces various imple mentations and other ways of interacting with the environment The approach taken is mainly technical since usability and related issues lie outside the scope of this thesis 8 1 Feedback User interfaces also need to provide suitable feedback so that for users will know if an ac tion has been successful or not It can be rather confusing for the user if nothing happens when an act
218. logies provide several options to choose from possible issues being mostly current consumption Wireless LANs short range Bluetooth and reliabil ity short range RF links Another problem is the limited amount of frequencies available creating overlapping frequency bands Even when using a single wireless network type the maximum number of active network nodes is usually limited Problems can already be seen between WLANs and Bluetooth links which disturb each other quite heavily as measurements conducted in the Smart Home have shown Myllym4ki03 Since wireless signals also propagate outside the intended space there are also potential security risks Without adequate security measures and encryption signals can be intercepted by other parties or they can simply interfere with another network nearby considerably reducing the usable range Thus there is poorer reliability with wireless networks than with wired alternatives 119 Hardware Aspects With wireless networks it can also be difficult to debug if problems occur a connection can work perfectly one moment but become unreliable at another Electromagnetic condi tions change constantly so it is impossible to predict the future reliability of a wireless link To users this would appear only as a general malfunction in the smart home system Bluetooth can easily be made to replace wired serial links since the protocol directly sup ports serial port emulation Modern mobile phon
219. lows for grouping of its network nodes In this way it is possible to individually turn on or off a light or mains socket or to create a controllable group that responds to the press of a button In the Smart Home a push button near the front door is configured to turn on off all lights in the apartment making it easy to ensure that all lights are turned off when leaving the apartment The LINET network comprises of one network master unit and multiple slave nodes and it uses a time division multiplexed pro tocol and twisted pair cabling supporting up to 255 network nodes Electric power to the nodes is also supplied by the same wires making cabling relatively simple even in older buildings The LINET master uses an Ethernet connection to connect to the Smart Home network enabling control of the network from another user interface as well The status of lights and relays can also be read which is a major advantage when displaying items on a graphical UI Each controllable relay dimmer and light switch on the wall contains a LI NET node that can be reprogrammed to perform any function that the protocol supports Figure 5 23 LINET master unit top left light switches top right and network connections in the Living Room bottom 79 TUT Smart Home Research Motorised Door and Fingerprint Scanner Experiments with the motorised lock and ID tag reader in the Living Room revealed that alternative methods for implementing access control
220. lt descr gt Relative humidity lt descr gt lt opt gt lt Attribute Visible light INDEX 4 gt lt photosynthetically active range 300 700 nm gt lt 16 bit unsigned integer used 12b gt lt Read only attribute gt lt opt func visible_light_level value 0x0000 valuetype uint rules read gt lt descr gt Visible light level photosynthesis active range 1 amp lt dark 150 bright office light 1350 gt bright sunset lt descr gt lt opt gt lt Attribute Solar light INDEX 5 gt lt Full visible spectrum ir radioation 300 1100 nm gt lt 16 bit unsigned integer used 12b gt lt Read only attribute gt lt opt func solar_light_level value 0x0000 valuetype uint rules read gt lt descr gt Solar light level Visible light spectrum ir radiation 0 dark 210 bright office light 1500 amp gt bright sunset lt descr gt lt opt gt 156 Software Architecture lt Attribute Sensor report interval INDEX 6 gt lt 16 bit unsigned integer gt lt Read only attribute gt lt opt func sensor_report_interval value 0x0001 valuetype uint rules both gt lt descr gt Sensor report interval in minutes 0 disables reporting lt descr gt lt opt gt lt device gt 7 5 Embedded Software As processors microcontrollers and different kinds of programmable chips have become cheaper and more powerful during the years
221. ly age and ubiquitous computing will become part of childhood and school life all up to adulthood Increasing connectivity enables us to keep in contact even over longer distances and the communication between people can change radically The boundary between humans and computers can move ever closer towards the human body for example with implantable and wearable technology with the boundary between computers and the surrounding world will shift towards the en vironment pervasive and distributed computing The computer we know today will be very different in 2020 On the negative side the ever growing amount of interconnected devices makes it increas ingly difficult for users to understand the workings of a system and thus also how to react to problems The lack of understanding might lead to less confidence in the case of new technology something that can be observed already today How the growing dependence on technology will affect our basic skills and lifestyles re mains to be seen Will humans become too dependent on computers or can artificial intel ligence become so integrated in our lives that we no longer require the same amount of thinking and action that we need today Norman93 The home as a physical place can also face changes in the future The ongoing drift of peo ple from the countryside to the city continues to create a need for affordable urban housing and since there are many kinds of families and lifestyles it is dif
222. m a search for replacements or best possible alternatives In the previous example if the temperature sensor in the living room was not functional it would be possible to search for any devices that are able to measure temper ature and use their readings instead A network using service discovery is also useful for 147 Software Architecture sharing resources as data and processing power can be distributed throughout the network to other devices increasing performance and redundancy Network traffic can also be re duced by relocating data in the places where it is most likely to be needed In a smart home scenario service discovery is essential as the network usually is a myriad of heterogeneous devices that can appear and disappear at any time The ability to commu nicate directly with another device remotely control it or replace it with another similar device can also be very useful There are several service discovery protocols available though they are not yet standard for smart home systems due to incompatibility and re source requirements Listed below are a few common service discovery protocols UPnP developed by Microsoft UPnP uses multicast technologies over an IP network to broadcast service information and XML documents providing service descriptions The goal of UPnP is to provide zero configuration networking and remote control of devices and it has thus found its way into home entertainment devices and network hardwa
223. mation is not compatible or if the one device lacks functionality that the other one has To a certain extent traditional unintelligent electronic devices can be com patible with other devices and it has been easy to take this into account in the design proc ess Television sets for example can accept the video signal from almost any brand or type of DVD player data signal compatibility provided the physical connector fits and an en ergy saving light bulb can be used instead of a traditional one physical electrical compat ibility A smart device can also be relatively easy to design because its construction and Introduction functions can be designed to suit a specific application However compatibility problems arise when these devices are used together with other smart objects Communication be tween devices becomes impossible because this facility has not been taken into consider ation when the devices were designed The global market does not wait for common interconnectivity standards to emerge instead manufacturers continue to use their own communication standards making devices that are introduced to the market incompatible with each other from the outset This is a fact that cannot be overlooked and one that must be considered in the design of smart environments From the consumer s point of view the situation is understandably difficult and frustrating This is all the more alarming when it is remembered that the idea beh
224. me automation and things that happen behind the user s back can start to become annoying in the long run they cause stress and make the user passive Our principle is instead to activate the user i e make the user do everyday chores and take care of the home For example instead of an automatic plant monitor that would water the plant regularly the user will be reminded to do so if the plant requires it If nobody is at home the system can either set a reminder for the user or switch to an automatic mode Because the system is programmable some events can be automated For example lights can be lit when the user walks into the kitchen This can also be combined with user iden tification and a personalised profile can be created for each user Also one further requi rement is to avoid adding user interfaces displays buttons blinking lights to devices themselves but instead keep them as they have been before This is according to the Natu ral User Interface principles Controlling and using devices would instead be done cent rally for example by using a web based user interface on a PC For example a coffee maker would not need any buttons or additional user interface components the electro nics will be hidden inside the device and it could be controlled wirelessly from a control UI In order to implement a smart home devices have to be able to communicate with each ot her wirelessly This has been implemented with infrared and radio l
225. me project ended Adaptive amp learning Home LIPS project started eHome iviorpnome oO 2 a N mM q wn wo N op Q g Q Q Q Q Q Q a O O e O O O ad N N N N N N N N Figure 5 1 Timeline of smart home research at TUT from the author s point of view 61 TUT Smart Home Research 5 2 Smart Living Room 1999 2002 When smart home technology research started in 1999 there was a clear need for an envi ronment where the infrastructure user interfaces and devices could be tested and installed The goal of the Living Room project Vanhala01 was to create a relaxing home environ ment with controllable devices sensors and a user interface from where the home could be monitored and controlled Another key theme was user activation i e the system should notify the user of tasks that needed attention instead of automatically taking care of them Mikkonen00 By bringing user interfaces together to an intuitive common control UI bet ter usability and easier control will be achieved Figure 5 2 The Living Room laboratory showing the dining area kitchen and living room In 1999 when the first smart home project was being started the specifications for the first smart space were as follows TUTO0 From the beginning the emphasis of our research has been on creating relaxing stress free environments where the users don t have to worry about forgetting to turn off the lights or coffee maker the indoor plants dying or
226. measurements that are unobtrusive to the user No external de 29 Smart Homes vices that require strapping on or wearing are required all necessary electronics will be integrated into garments and other devices or locations around the user Requirement 2 A seamless mobile fixed communications infrastructure The complex heterogeneous network structure in a smart home needs to function seam lessly and reliably no matter what kinds of hardware it is connected to Wired and wireless communications need to be completely integrated and dynamically managed so that de vices do not require any configuration when they are moved from one network type or lo cation to another Plug and play or similar zero configuration protocols take care of integrating a new device into the network relieving the user of the tedious set up and con figuration tasks Requirement 3 Dynamic and massively distributed device networks The AmI network scenario is a mix of static and mobile devices in massive numbers Ac cording to the AmI requirement they must all be able to access and process data from any where in the network which requires a centralised or distributed database where all data is stored This brings the focus on data and storage management that extends beyond mid dleware and system software New standards and protocols are needed in order to create an adaptive embedded intelligence Requirement 4 Natural feeling human interfaces New ways
227. mponents and that adapts to new situations and anticipates behaviour will be the main challenge for the fu ture Chapters 6 and 7 discuss smart home networks and software in greater detail The IST advisory group has created a list of five technology requirements for Ambient In telligence and present some possible future development scenarios for the year 2010 ISTAGO1 Requirement 1 Very unobtrusive hardware The increasing pace of miniaturisation and integration will allow electronics to be pack aged into even smaller sizes in the future paving the way for nanotechnology and Micro Electro Mechanical Systems MEMS At the same time the tasks and processing that the units are able to perform increase and the integration of sensors actuators and interface systems will result in the creation of smart materials One key requirement for future AmI hardware is the ability to harvest energy from the surroundings thus minimising the need for external power supplies Very low power wireless radio links also help to keep energy requirements low When AmI hardware is designed the process should be focused on hu man factors and usability to enable the creation of a coherent AmI landscape instead of having only a group of networked computers New materials and display technologies make it possible to create seamless interfaces with new ways for users to interact with their environment This way it is also possible to create sensors and perform
228. n numbers 132 Hardware Aspects Atmel Atmega8L 4 MHz Supply voltage 3 0 V e Active state 5 mA e Idle state 2 mA e Power down mode watchdog timer disabled 2 uA A popular sensor module the telosB introduced later in this chapter states the following power saving modes TelosB mote MSP430 MCU powered by 2 x 1 5V AA batteries e MCU on Radio receiving 21 8 23 mA e MCU on Radio transmitting 19 5 21 mA e MCU on Radio off 1800 2400 uA e MCU idle Radio off 54 5 1200 uA e MCU standby 5 1 21 0 uA In wireless devices power consumption is often a compromise between wake up time maximum range for the radio network and response time if the processor is in very deep sleep it will take a relatively long time for it to wake up and become active again If the radio is turned off it obviously cannot listen to any traffic and therefore the device cannot send or receive anything Further savings can be achieved by designing and optimising the schematic power supply and with proper component selection Reducing the clock fre quency to aminimum helps in reducing the power consumption of the processor and soft ware optimisations can also help Vuorela06 For ultimate energy efficiency energy harvesting as explained earlier in this chapter can theoretically make the device energy independent or at least requiring little external energy 6 10 A Typical Smart Device Devices in the Smart Home can be divided into th
229. n on a Nokia 6310 mobile phone It was possible to load Java applets into the phone memory and a simple Home Controller UI was written to allow users to interface with the Home Controller even from remote locations The phone would use a GPRS connection which connected to the eHome server through a se cure tunnel via the university The phone UI featured basic adjustments for lights blinds pre set modes and it also had a special function to check the status of the coffee maker and turn it off if necessary The primary function of the phone UI was to facilitate remote con trol of the home and to reassure by allowing them to check whether they forgot to turn off an appliance when leaving the home Koskela04_2 Figure 5 46 eHome mobile phone UI showing lighting controls 100 TUT Smart Home Research eHome TV UI In 2001 digital set top boxes and home gateways were introduced to the market with the belief they would become the centres of home control and entertainment Multimedia gate ways with audio video controls broadband network connections and video on demand promised instant access to TV channels movies and Internet sites Set top boxes were to receive high definition digital broadcasts and provide interactive services to viewers This technology would also have made it possible to integrate smart home controls in the same unit allowing users to control their home using remote controls and their TV set However later in 2003 suc
230. n proprietary systems links and for these reasons it can be difficult to integrate them into the home network Infrared remote controls only work in one direction and com mand sets usually vary a great deal from one manufacturer to another Some devices have standard serial RS 232C ports and they accept a wide variety of commands making it much easier to control them through a home network However the integration of audio visual equipment could deliver additional functions and benefits to inhabitants for exam ple by providing feedback information or entertainment related to the user s contexts Displays and TVs can function as graphical UIs or information panels and audio systems can be used for delivering audio feedback to users Further information can be gathered if the system can know what TV channel is being watched and how high the volume level is 131 Hardware Aspects Motor Network node ae A Fre ae Curtain Beebe Switch Server User Interface Figure 6 10 Components and connections of a typical smart home actuator in this case a curtain controller 6 9 Power Consumption Power consumption is often a property that is not directly visible to the user especially in the case of devices that use mains power supply However as the number of devices grows and price of energy increases consumption begins to become important Since one of the objectives of smart home technology is to lowe
231. n techniques to create a learning home control system In other words in contrast to automated homes that are programmed by the users to perform certain tasks the Adaptive Home programs itself The Adaptive Home was designed to adapt to users everyday lifestyle patterns thus helping them save energy and enjoy increased comfort In time as the system has collected sufficient infor mation the home can anticipate the users needs and act proactively The system could op timise the house heating according to times of occupancy turn off unnecessary equipment and adjust lights and temperatures to suitable levels when users came home This design also made the house less dependent on user interfaces as users can continue using light switches and other manual controls to train the home Figure 4 3 The Adaptive Home Photos Michael C Mozer 47 Related Work 4 2 4 Georgia Tech Aware Home 2000 The Aware Home is a complete house built by Georgia Institute of Technology in Atlanta in 2000 to research emerging technologies and services Abowd02 The home has been used for testing experimenting and evaluating state of the art technologies related to fu ture living Projects in the Aware Home are aimed at simplifying management of the home assisting in everyday activities and entertaining family members during their leisure time According to the researchers technology should be introduced into the home without ma jor disruption of li
232. nd projection displays cameras for detecting and tracking users mi crophones and an array of speakers that can be used for directing sound towards users E21 is used for developing camera vision applications demonstrations and meetings The aire project also covers intelligent workspaces both from a physical and technological standpoint It provides a dynamically reconfigurable test environment that adapts to new configurations using both hardware and software The goal is to discover what activity a person is engaged in and whether the workspace could offer relevant assistance and max imise the work contribution 54 Related Work 4 2 13 MIT Oxygen 1999 The Oxygen project of the Massachusetts Institute of Technology Dertouzos99 started from the assumption that cheap electronics make powerful affordable and small devices available everywhere These devices exist all around us in the same way as oxygen But instead of making people serve computers computers must be made to serve people so that computation becomes human centred Oxygen had three goals to introduce new technol ogies to increase human productivity and to help people to achieve more while doing less Interaction with computers is to be achieved through natural means such as speech and gestures freeing users from traditional computer UIs and input devices The backbone of Oxygen is a wired wireless self configurating network called Net 21 through which a va
233. ng events to the periphery of our attention we are still able to follow them without too much loss of detail or being overburdened by a plethora of data This enables our attention to focus on more crucial issues when needed Not all technology can or should be calm but for many everyday applications in the near future Calm Technology may enrich and enhance human computer interaction 8 7 Control UI vs Centralised UI A control UI is designed to control functions and the state of a device Typically it is lo cated on the device itself for example buttons dials and displays on the front panel of a washing machine or a menu structure in a mobile phone In a ubiquitous computing sce nario control interfaces can be physically separated from devices and brought together to another place to form a versatile universal UI This would allow almost any device to be controlled from any UI allowing for certain UI restrictions For example a washing ma chine could be controlled from a mobile phone but the phone could not necessarily be con trollable from the washing machine due to the limitations of its UI Both could however be controlled from a graphical UI on a PC 169 User Interfaces A centralised UI would be a location from where all devices and appliances could be con trolled remotely Examples of such UIs are the tablet UI in the Smart Home and the WWW UI in the eHome There are cases where removing the control UI or replicating it
234. ng planned This helps to en sure that the smart home will meet the particular needs of the customer instead of being a generic pre specified package It is forecast that future living and homes will be modular R nk 03 Lepp nen03 with user configurable options when the home is purchased in 186 Analysis the same way that automobiles are today This allows buyers to select the rooms materials and floor plan they prefer whether it is a traditional cosy home with a garden a functional urban dwelling for modern living or a luxurious villa full of modern conveniences Simi larly smart home technology can be modular and customisable to make it suitable for a variety of scenarios and situations This also allows the construction of concept homes which are larger units designed around a common theme Over the passage of time it would be possible to make changes and adapt the home to suit a new lifestyle 10 3 Final Thoughts As stated in the introduction a major obstacle to smart home development is incompati bility and the problems of complicated UIs making it difficult to integrate technology and human factors The results and experience gained from projects described in this thesis provide a possible solution to this problem The solution can be divided into two parts Compatibility problems can be alleviated by bridging current networks and protocols to gether with middleware and by selecting the most appropriate technology for
235. nges from the buzzing sounds of equipment operating to beeps indi cator sounds and related status sounds More sophisticated audio feedback uses spoken or synthesised vocal messages to read messages aloud Auditory feedback is often used as a supplementary method combined with haptic or visual feedback Emery03 Haptic feedback utilises physical means of communication Switches and other physical UIs usually have tactile properties and when used they clearly click or physically com municate action in an obvious way Haptic feedback is also achieved by vibration as is the case with mobile phones and computer games force feedback 8 2 Context Aware UIs Context awareness can be a valuable asset in smart home UIs If an UI knows who is using it where and under what kinds of circumstances there are numerous advantages that such information can provide Things that a context aware UI can enable include changing the functions available and presenting functions that are predicted to be useful to the user Fur 162 User Interfaces thermore the UI layout can be customised for each user and appropriate feedback can be directed at the user depending on the location and situation For example the speech recognition UI in the Smart Home described in Chapter 5 locates the user and is able to direct audio feedback and use commands related to the room the user is in Another experimental UI Ritala03_2 was designed to replace standard inf
236. nical development new kinds of household machines and automation In the 1970s the fo cus shifted towards computers and the emerging information technology Currently in the 21 century the focus is on the enrichment of daily life new experiences and more user friendly development This development has made us increasingly dependent on technology while at the same time our lives and routines have become accustomed and shaped by it This raises two questions 1 How should new technology be designed so that users can benefit from it If users have to cope with an increasing number of electronic devices it can cause problems stress and frustration towards new technology As our homes are already filled with electronic gadg ets the way devices are used managed and installed becomes a very important issue 2 How can complexity be reduced without any loss of functionality in the process If a smart home consists of multiple networks devices and user interfaces how is it possible to create a functional usable system without sacrificing performance functions or resources When new technology is designed it is easy to focus on the technological innovations such as new processors faster networks new software features etc Other issues like usability durability and integrability are considered of secondary importance and are easily over looked When thinking about smart homes this is clearly a disadvantage making technol Introduction
237. nics or devices made the Liv ing Room an ideal testing space which is still deeply missed among researchers A large projection screen and good home theatre equipment also made the living room area a cosy place to watch movies User activation was encouraged by means of versatile monitoring and controlling facilities through the touch screen UI The UI seemed to be well designed as it was easy to use without any greater learning effort However there remained one large UI related problem the tabletop PC was still a rather fixed fitting and because it was the only Ul in the Living Room it was sometimes in the wrong place or turned off when users wanted to make adjustments or check settings A more distributed and mobile way of controlling the home would have been more useful During the Living Room project re sources were devoted primarily to creating new hardware developing different communi cations networks and building applications for the testing space Thus additional user interfaces and artificial intelligence were regarded as secondary considerations at the time As many things were new to the newly formed research team many valuable lessons were learned during design of networks protocols and devices The project members gained valuable learning experience through activities such as programming microcontrollers making robust electronics design and complying with common standards In 2001 the research team participated in the Futuuri exh
238. ning living room and bedroom that can be used for entertainment or meetings In the bedroom these can be used to show restful images to create a relaxing environment The screen in the living room is intended to be the primary screen for the home theatre and a large video projector is mounted on the ceiling for this purpose The main screen is also equipped with a controllable motor All of these devices are based on commercially available products but the supplied stand ard control units have been replaced with custom controllers that are connectable to the Smart Home network The window blinds can thus be controlled to any position and the rooms can be completely darkened if needed The projection screen in the living room can be set to lower at the when the projector is turned on Figure 5 22 Control unit for motorised blinds 78 TUT Smart Home Research LINET Network The requirement to control all the lights and electric sockets in the apartment caused a need for a dependable control network As it was considered difficult and possibly hazardous to custom design the control of mains voltages it was decided to acquire a commercially available product LINET LINET is a proven product that suited our requirements and since it was also possible to interface the LINET control unit with our Smart Home infra structure it was seen as a logical choice LINET enables switching and dimming controls for lights it controls relays and al
239. ning doors adjusting mirrors etc whereas networks for more latency critical applications connect to functions such engine parameter adjustment and stability control Car electronics have indeed become ubiqui tous with drivers unaware of their existence until they are activated or applied For exam ple the anti lock braking system ABS adjusts the braking power so that the tires keep turning and the Electronic Stability Control ESC automatically applies braking power to the wheels to prevent the vehicle skidding Volvo s 80 automobile contains 18 electronic control units linked by two CANbus networks to form a complex distributed system RED BUS 250 KBPS CANBUS GREEN BUS 125 KBPS CANBUS ABS ANTILOCK BRAKING SYSTEM AUM AUDIO MODULE ECM ENGINE CONTROL MANAGEMENT CCM CLIMATE CONTROL MODULE ETM ELECTRONIC TRANSMISSION CONTROL DDM PDM DRIVER DOOR MODULE PASSENGER DOOR MODULE SAS STEERING ANGLE SENSOR PHM PHONE MODULE TCM THROTTLE CONTROL MANAGEMENT PSM POWER SEAT MODULE GATEWAY CEM CENTRAL ELECTRONIC MODULE REM REAR ELECTRONICS MODULE RTI ROAD TRAFFIC INFORMATION SRS SAFETY RESTRAINT SYSTEM SWM STEERING WHEEL MODULE UEM UPPER ELECTRONIC MODULE Figure 3 1 Example of the complexity of the electronics in a modern car Photo Volvo In car electronics raise another issue regarding computer control automation and action reaction there are cases where a computerised system can perform significantly better and faster than a
240. nments and compares smart homes to home automation 2 8 Internet of Things Internet of Things IoT is another concept that is based on the assumption that in the near future every device will have a unique identifier e g network address EPoSS08 Having multiple networked devices around us creates a ubiquitous computing environment that is able to exchange data context information and processing capability Whereas the current situation around the Internet is that unique IP addresses are starting to run out the newer version of the IP protocol IPv6 IPv6 holds 3 4x10 8 addresses which should be suffi cient for the foreseeable future Thus a future IoT can be significantly more heterogenous than the current Internet due to the vast number of different devices that are connected The Internet today contains devices that are somewhat standardised and similar mostly person 17 Definitions al computers servers and network gear However if every toaster toothbrush and armchair is added the nature and requirements of the network will be quite different The IoT sce nario also contains a few dilemmas ubiquitous intelligence cannot be accomplished unless communication protocols and interfaces are standardised and made compatible with each other Additionally there is a fundamental design issue regarding the nature of the inter connected device network whether to design it as a centralised type using a central node or completely di
241. nologies The FutureLife house in Switzerland is an interesting case where the inhabitants are also partially the developers of the project giving them un restricted control over the home control system Short term laboratory tests are more common but it takes a longer time for users to start feeling at home and to feel comfortable and familiarise themselves with the new environ ment As there are only a few research projects that have conducted long term user tests it makes the research contribution from the eHome project significant The Duke Smart Home seems a very interesting project and the students are probably highly motivated to enhance their dormatories Technology students however hardly represent average users and similarly to the users in the FutureLife house they have skills and advantages that can not be expected from the average resident 4 4 Summary As this chapter has shown there have been numerous similar projects active during the past ten years Even if they share much in common each project has its own area of focus and interest Smart home projects at TUT have also had their own special goals and prop erties which from the beginning have been low cost practicality and technical innovation Networks have been implemented using existing technology and standards while almost all devices have been implemented and designed in house All designs have been built and implemented in a real physical environment Usability and
242. ntegrate them into another device furniture or even structures in the building itself Background noise is also a problem causing problems for speech recogni tion software The process of separating commands from other sounds is another major challenge Nevertheless a speech control UI integrated into a mobile phone for example would give users another way of interacting with their home electronics 165 User Interfaces 8 6 UIs in Other Smart Home Projects 8 6 1 InfoCube The InfoCube developed as a joint project between TUT and the University of Lapland is a cubic wireless device that can be used as a 3D controller for a 3D user interface Kaila05_4 The goal of the InfoCube project was to create a natural way of interfacing with a 3D user interface It would be suitable for usage in a smart environment or in virtual environments The 15 x 15 x 15 cm wooden cube is aware of its angular movement and velocity which are measured to control a visual 3D user interface depicting a virtual cube on the screen The model is used to create visual and haptic feedback to the user by means of which decisions can be make from visual information and the position of the cube Ro tating the actual cube will also cause an identical movement of the virtual cube on the screen resulting in a very intuitive interactive interface A sample application consisted of an image projected on a wall screen showing a rotating cube with pictures of different te
243. nterfaces 8 4 Graphical UIs Graphical interfaces have become commonplace since the 1990s with the emergence of graphical windowing operating systems and applications Graphical UIs are easily reada ble and can display a great deal of information that other kinds of UIs cannot easily present Another advantage is that the UI can be easily reconfigured to represent any in formation in any form making it possible to personalise UIs for different users or user groups For example a wall mounted graphical home control panel could restrict access to certain features for children and for the elderly it could display larger buttons and text with bigger fonts Naturally a graphical UI is only good if it is well designed poor usa bility and readability can make it even harder to use than more traditional UIs Graphical UIs can nowadays be implemented in a wide variety of devices PCs internet tablets PDAs wall panels mobile phones and even home appliances are just some examples and the list of devices continues to grow The physical size of the screens can be an obvious practical limitation to their implementation with smaller screens needing back up from larger ones to display complex data or enhanced visualisations 8 5 Auditory UIs Another natural way for humans to interact is by speaking and auditory UIs are an inter esting way of interfacing with the smart home system Speech recognition software has made great progress and even thoug
244. ntion system 55 3 Home appliances that shut down automatically 48 4 Remote alarm system 41 5 Home surveillance system 36 6 Automatic climate control 35 7 Backup power system 33 8 Anti burglar system 32 9 Telecommuting distance learning facilities 23 10 Energy consumption monitoring 22 39 Controlling the home through the Internet 1 Fear of burglars and property theft is the foremost issue whereas personal security was considered less important in this context The next preference is technology related to building automation HVAC and energy management Home entertainment communica tions and multimedia appear relatively lower down on the list which is interesting because this field is where the most rapid developments are being made 3 2 1 Networks and Software The requirements for a smart home network are rather demanding it has to be reconfig urable self organising dependable secure and consume minimal energy New algorithms for inter device collaboration and communication are needed and the coordination be tween the artificial intelligence and reasoning software is also difficult to implement The smart home platform should be easily scalable and it should be able to migrate tasks inside 28 Smart Homes itself and perform load balancing of the network Resource management for example lo cating all temperature sensors inside a room and gathering their measurements is a very important feature especially f
245. ntralisation The shift from centralised mainframe computers to distributed com puting PDAs mobile computing devices and PCs Information is accessible everywhere from any device data and processing can be shared throughout the network Diversification Numerous alternatives are available for each application and it is pos sible to tailor a suitable software hardware platform using available components Each de vice has its own purpose and even if some properties might overlap between devices there will still be different user scenarios for all of them Presenting information and managing a myriad of different devices poses major challenges for UI design and data management Connectivity A device can use multiple networks to connect to the Internet or other de vices and it can seamlessly change over to another network if the previous network be comes unavailable or a more suitable network becomes available Gateways and converter modules can be used to connect an incompatible device to another network type but ulti mately it is applications and their interoperability that determine how compatible systems are with each other Simplicity Increased functionality new networks and applications also tend to make de vices more complicated Pervasive computing should be the opposite requiring minimal effort and thus avoiding becoming a system with hierarchical menus and multiple menus Great care must therefore be taken when designing UI
246. o implemented The light controller was constructed using a microcontroller and low resistance MOS FETs The dimming function operated by switching the MOSFETs on off with a high fre quency pulse width modulation using 12 V power from a massive toroid transformer Figure 5 14 Halogen lights and controlling unit Sofa Sensor The entertainment area of the Living Room was equipped with a two seater sofa and a re cliner People were often seated on the sofa when watching a movie and so that placing sensors in the sofa would provide information about them The primary application for the sofa sensors was context detection regarding the home entertainment equipment Thus for example if a movie was playing and the viewers got up from the sofa to get a drink from the refrigerator the movie would pause automatically Sensing was achieved by means of stretch slip sensors installed inside the sofa cushions and a controller unit that sent on off data to the computer The sofa was designed for two people and it was possible to detect which seat was occupied 71 TUT Smart Home Research Floor Sensors Locating users in the Living Room was a primary requirement for the infrastructure as it allowed the system to determine the location and activities of the users This information could be used for lighting control access control and for contextual applications Sensing was achieved with special EMFi sensors installed in specific parts of the
247. ocessors sensors and current consumption Table 6 5 Comparison between sensor nodes Sensor node Radio Current con CPU Integrated sensors sumption tmote 802 15 4 5 1 uA sleep MSP430 Temperature light 19 5mA 8MHz humidity active Scatternode Chipcon 250 pA MSP430 Temperature vibra 1020 standby tion movement light 868 MHz 250 mA peak Smart its Bluetooth lt 3 7mA ATmegal03L N A available as exter 868 MHz sleep 67 mA PIC 16F876 nal modules peak Soapbox 868 MHz lt 6mA unknown Acceleration light magnetism proximity Sensinode 802 15 4 unknown Chipcon Acceleration light CC2431 SunSPOT 802 15 4 36 pA sleep ARM920T Temperature light acceleration 6 12 Discussion The hardware presented in this chapter allows smart home designers to choose the most suitable components for their homes Wireless networks allow freedom of placement and movement while wired alternatives are more reliable robust and faster The inclusion of 140 Hardware Aspects a home gateway of some sort makes network selection easier as gateways typically con tain multiple network interfaces and are able to cope with many kinds of protocols The gateway functionality does not have to be contained to a specific physical device it can be distributed between home appliances or included in another kind of device In order to sense the environment a smart home system needs sensors
248. ocol comparing well with the previously mentioned standards The table below summarises properties of different service discovery protocols Table 7 1 Comparison between service discovery protocols Sundramoorthy03 Feature UPnP Jini Bluetooth SLP EPIS SDP Architecture Peer to peer Client server Peer to peer Client server Client server Peer to peer Peer to peer Catalogue ser No Lookup ser No Directory Directory vice vice agent agent Leasing con Yes Yes No Yes Yes cept Remote con Yes Yes No No Yes trol Scope of Device type Service type Service type Service type Service type search Service type Service ID Attributes Attributes Attributes Attributes String Device ID Robust No No No No No Resource No No No No Yes awareness Workload del No No No No No egation 7 4 Early TUT Communication Protocols Three generations of smart device communication protocols have been in use at TUT each being improved on its predecessor Protocols have been designed for fast parsing light weight and simple implementation Later as requirements rose and complexity demanded more functions protocols have also evolved together with the smart home system Primary requirements for these protocols have been the ability to receive and send com mands consisting of a few bytes of data send back an acknowledgement and contain a sim ple addressing feature 149 Software Archi
249. oduction working on all the various designs that TUT smart home projects involved and in some cases it is difficult to identify individual developers and responsible persons The author has been designing smart spaces and devices since 1999 from the Living Room test envi ronment and the first sensors and actuators for it until later moving on to the Smart Home laboratory and the eHome all of which are presented in Chapter 5 During the course of these projects various contributions by the author have been published in Kaila01 Kaila05_1 Kaila05_2 KailaO7 Kaila08 Kaila09 The first implementation for the Living Room test environment was a flowerpot sensor module its infrared communication protocol and parameters This work any many subse quent projects was done in collaboration with colleagues Petri Keski Opas and Jussi Mik konen As this work progressed more devices were created and the Living Room space was built again with the help of the same people plus a few newly recruited research as sistants Building the Living Room involved much of practical work designing the layout electrics installation furnishing and supervision of contractors More devices such as lighting control floor sensors controllable door etc were built after the laboratory was completed while Jussi Mikkonen concentrated on the wireless network and protocols Markus Ritala designed the first graphical user interface and control software which was embedd
250. odules With separate device and network addresses it is possible to ac cess individual sensors or modules in the node Each node also stores a set of coordinates x y z and a space variable essentially an identifier for the room that the device is located in e g bedroom kitchen At certain intervals wireless nodes poll the network listening for transmissions in order to conserve battery power Devices send out regular heartbeats that the server monitors If a device fails to send a heartbeat for a certain amount of time its address is released and the device is removed from the network As the central node gets its electricity from the ac mains it does not have to conserve power in the same way as remote nodes and it is thus continuously listening to the network for any activity This also allows it to respond to mes sages with minimal latencies EPIS specifies three different power states for wireless de vices The Active state consumes the most power but as heartbeat and polling intervals are kept short it also allows the device to react and send data as fast as possible In the Sleep state these intervals are significantly lengthened and response times become much higher as a result The device is still active but computational power consumption is mi nimised The Standby state maximises power savings while maintaining the same tim ings as in sleep mode all other functions are essentially shut down The device
251. om project started the Department of Electronics moved to a new building in the university Plans included a dedicated 69 m laboratory space for smart home research 74 TUT Smart Home Research Figure 5 18 The Smart Home laboratory There was now an opportunity to conduct smart home research in a complete apartment with kitchen bedroom living room bathroom and sauna Furthermore it was now possible to design the apartment from scratch with prototyping and testing in mind The laboratory space was named the Smart Home and furnished to resemble a typical modern apart ment However many changes were made to the structure and design to allow easier mod ification and installation of new systems and appliances in the future A suspended ceiling houses bright controllable halogen lights electric sockets network sockets and generous room for equipment while the raised floor contains electric cables and additional space for floor sensors Rooms are separated by large movable shelves to allow changes to the room layout The shelves also have hollow ducts and space for cabling concealing cables and making later installations easy The biggest modifications were made to the electrical net work which eventually contained several kilometres of cables This was due to our re quirement that every light and electrical socket should be directly controllable with relays The main electric distribution board houses all the necessary relays and
252. om the user s shoul ders A software capable of learning would adapt to the users needs and if a wrong action is taken it is possible for users to take counter measures and thus teach the system what the desired action would have been Adaptivity is also required for UIs and in decision mak ing since the layout and type of home devices can continuously change depending on what people carry with them Context awareness will further assist the system with making de cisions and processing data as it will provide the system with more information about what users are doing 7 2 Middleware Middleware is the media that connects devices networks software and services together in a smart environment Middleware is defined as connectivity software that allows soft ware components and services to interact across a network Bernstein96 Middleware is required in order to have distributed computing services and heterogeneous platforms working together and it allows applications to be located throughout the network increas ing reliability Figure 7 1 below illustrates how middleware is situated between applica tions and other platforms 144 Software Architecture Application Application Application Programming Interfaces APIs Middleware distributed system services a ed Platform Interface Platform Interface Platform Platform e Operating e Operating system system e Hardware e Hardware Figure 7
253. ome A case study in Inside the Smart Home R Harper ed Springer Verlag pp 227 246 2003 H Raula Touch controlled lighting control panels Master of sci ence thesis Tampere University of Technology Electrical Engineer ing Department Tampere Finland 2009 M Ritala Bluetooth Enabled Mobile Phone as a User Interface De vice for a Smart Environment Master of science thesis Tampere University of Technology Electrical Engineering Department Tam pere Finland 2003 M Ritala T Tieranta J Vanhala Context Aware User Interface System for Smart Home Control in Proceedings of the Home Ori ented Informatics and Telematics 2003 HOIT2003 2003 Califor nia USA M Ritala J Vanhala Task partitioning in smart environments Pervasive Mobile Interaction Devices PERMID 2005 Munich Germany 2005 pp 39 42 201 References Rohs05 R nk 03 S60 Saito00 Satyanarayanan05 Scatterweb Schenker00 Sentilla07 Shapiro99 Soronen04 Starner03 Steiner01 Sundramoorthy03 202 M Rohs Visual Code Widgets for Marker Based Interaction in Proceedings of the Fifth International Workshop on Smart Applianc es and Wearable Computing IWSAWC 2005 pp 506 513 K R nk Asumisen uudet el m ntapakonseptit New lifestyle concepts in housing in lyk s kaluste kaluste lykk ss tilassa Smart furniture
254. on Connection type e g mote for devices that use a mote for connecting e Description Optional description of the device Service parameters include the following e Func Service function description in one word e g lighting level e Value Initial default value e WValuetype Data type of the value short signed unsigned text etc e Rules Accessibility rules for values read write both none e Minval Minimum legal value e Maxval Maximum legal value Device Attributes Each device can have up to 18 attributes which can be stored into the memory on the de vice The first two of these stored attributes are the x y z coordinates of the device and the space variable both of which are assigned by the service discovery protocol The remain ing attributes can be freely assigned and they can vary significantly between devices For example a light level sensor could store an exact floating point number indicating light levels in a certain space or it could simply store a dark light boolean value in its memory An attribute can either be a 32 bit value or an 8 bit ASCII string up to 96 characters in length Different numerical formats long short unsigned etc are accepted as well as boolean true false values Each attribute can be defined to have minimum and maximum values current value value type and privileges write read write read An attribute can also be a free word description of its properties Attributes for
255. or more critical services such as alarms and environmental controls Load balancing and fault tolerance become important when there are power out ages malfunctions or network problems or part of the network is not accessible As mentioned earlier compatibility or the lack of it is another major obstacle to the seam less integration of devices and networks Even with adapters and converters some func tionality might be lost or altered possibly leading to other problems There might never be a universal general smart home protocol for all devices and this must be considered when smart homes are being implemented In practice this means that a smart home system has to be able to support a large variety of standards and there also has to be room for future expansion Software for the smart home has to be written differently to current standalone applica tions Smart home software architecture is comprised of multiple embedded software com ponents that interact with each other and the amount of different modules agents and databases can be large Thus the way in which software is currently written using parti tioned abstraction layers with fixed interfaces connecting them is no longer a viable op tion Software systems are also usually vertically integrated with fixed linkages between components making it difficult to adapt to new interfaces and performing horizontal inte gration ISTAGO3 As a result writing software that is open to new co
256. orange house was that users as in the eHome had a single preferred UI for each scenario In other words they also found it useful to have multiple Uls each designed for its own application 4 2 11 MIT House_n 2003 Massachusetts Institute of Technology conducts research on home design and products and services related to future living As such the House_n project is directed towards re search into new technologies for all ages proactive healthcare biometric monitoring ac tivities of daily living privacy and new construction materials and solutions A research facility the PlaceLab MIT03 has been constructed at MIT to facilitate experiments re lated to the project The PlaceLab is a 93 m single bedroom condominium filled with sensors that can be used for studying people their actions and reactions in the laboratory The laboratory is periodically inhabited by test users who are remotely monitored by re searchers in order to provide data for further analysis The PlaceLab contains several different sensors embedded in various locations in the lab For example cabinets in the lab contain sensor modules for measuring environmental quantities infrared transmitters and microphones Various sensors are installed in fittings such as cupboards drawers windows and furniture to monitor their usage Video cameras and biometric sensors capture data about the users and their movements almost all the lab oratory surfaces can be used to displa
257. orld with artificially generated information This augmented information can be vis ual auditory or haptic in nature and using various user interfaces it is possible to overlay this information with information that we gather using our senses The prob lems with augmented UIs are that they usually require users to wear certain equip ment such as head mounted displays or haptic force feedback devices e Affective Computing Lisetti98 is computing that relates to human emotions using sensors to detect and analyse facial expressions speech and gestures While the com puter can be made to read certain human emotions the reverse process poses much greater challenges 8 10 Summary User interfaces are ideally the only parts of the smart home visible to the users and thus their importance is crucial Fortunately there are many choices and ways UIs can be im plemented making it possible to choose the UI best suited for a particular lifestyle and sit uation Furthermore it would seem that users will benefit from having several UIs to choose from making it possible for users to pick the UI that best suits the situation In cas es where computerised UIs are unavailable or if users merely do not want to use them manual controls have to be enabled With properly designed UIs it will be easier to control various appliances in the home add new ones and manage difficult situations For users this will instil a sense of security and comfort while al
258. ort the concept of a future smart home is rather unclear and people tend to use mental impressions to envision certain 27 Smart Homes smart home applications and ideas In the study a group of people were interviewed about the features they would like to see in a future home and what they thought about smart homes in general They were also asked about their willingness to invest in certain tech nology and services related to smart homes Factors affecting the level of investment in clude how exciting the product or service is how useful and how mature it is considered to be and how innovative it seems Also willingness to invest in installations on new build ings was considerably higher than in existing buildings According to the study the most desirable feature related to home security Fire alarms wa ter damage control home appliances that automatically switch themselves off home mon itoring systems and remote alarms were among the five most popular features The table below shows what percentage of the 1800 respondents were very interested when pre sented with various smart home applications It is noteworthy that only one per cent showed interest in using the Internet to control their home Table 3 1 The most desirable smart home applications according to the TATU re search report of 2004 TATU04 Smart Home Application 1 Fire extinguishing system 55 2 Water damage preve
259. osoft Corporation 8 6 3 Speech control Speech is a natural form of interaction for people and speech recognition has been used in many related applications offices computer control for blind people alternative input method for PCs etc for anumber of years Spoken commands make it possible to interact more naturally with home equipment for example by asking for instructions or by issuing specific commands relevant to the user s context However smart home applications still suffer from significant challenges in the form of ambient noise music unwanted speakers differences in voices between different users and the vast amount of possible voice com mands The noise is similar to the sounds that the speech control system actually wants to listen to making it even harder to eliminate However using technology similar to that 167 User Interfaces used in mobile phone hands free sets it is possible to filter out unwanted sounds and proc ess the voice that gives the commands Potamitis03 8 6 4 Gesture Control The human body can be used for making gestures that can be recognised with computer vision using cameras and video processing tools Gestures include pointing Jojic00 forms of sign language or facial expressions For example waving a hand could initiate a sequence to turn on the television lifting both hands in the air could mean that the user wants to load an exercise program etc Gesture control typically relies on
260. other kinds of everyday things Also a high degree of automation was not desirable since things happening behind the back of 62 TUT Smart Home Research the user were considered awkward and confusing Instead things that required the user s attention were brought up by the home computer and appropriate action could be taken The space itself was designed to be relaxing without a TV set flashing lights or beeping computers For this purpose the Smart Living Room test apartment was built in early 2000 A former virtual reality laboratory was renovated and remodelled into a 40 m apartment with a liv ing room vestibule and kitchen A suspended ceiling allowed easy and unobtrusive instal lation of equipment and the space was large enough to have separate rooms and areas for testing Because the laboratory room was located in the middle of the building it had no windows In order to make the apartment feel more realistic and homely it was decided to build an artificial balcony in one corner of the living room The balcony can be seen in the figure below in the upper right hand corner AN Balcony Kitchen Dining area Living room Hall C eee Figure 5 3 Layout of the Living Room According to our philosophy the Living Room was designed to be a relaxing and a com fortable living space Kaila01 The living room w
261. ough at a certain point it will be registered as a con text as illustrated in Fig 5 49 Limit for context A E Sensor A m Sensor B E Sensor C E Sensor D Figure 5 49 LIPS mountain model showing data from four different sensors There is also a clear need for a physical model of the space mapping rooms and locations onto a coordinate system This model is also used for positioning and fixing coordinates for devices according to the rooms they are in 5 7 Summary Smart home research at TUT has been conducted for ten years and during this period three different smart spaces have been constructed and two additional smart home projects com pleted The result of these projects has been valuable knowledge of software and hardware design for smart home applications physical testing spaces and home control middleware Research in the Smart Home will continue in conjunction with the LIPS project and de velopment and maintenance of the apartment will also proceed The table below presents a short summary of TUT smart home projects 107 TUT Smart Home Research Table 5 1 Summary of TUT smart home projects Project Main Themes What s new Year Living Room Relaxing activating infor Home network wireless 1999 2002 mative technologies Smart Home Different kinds of UIs Home UIs fuzzy control 2002 gt more control options system Home Controller middleware eHome Multiple UIs User studies Mobile
262. parately Useful data can also be collected from other appliances and devices in the home Many al ready come with internal temperature and other similar sensors for their own purposes and if this information could be transmitted and shared with the home network it could also provide valuable information Even a small piece of information such as on off data can be very useful when contextual information is being gathered This notion is also shared by Robert Metcalfe one of the people behind the development of the Ethernet network standard who wrote a law that states that the value of a telecommunications network is proportional to the square of the number of connected users of the system Shapiro99 The type of data received from the sensor can also vary greatly depending on the sensor type Simpler forms of data can be on off information from a switch or a pure numerical value that is usually obtained from an A D conversion A brightness value of 223 or a floor sensor capacitance reading of 42441 do not mean anything as such without a reference scale or threshold value Analogue values are difficult to convert to digital discrete infor mation for example if a temperature threshold is set to 23 C then 22 9 would be con sidered cold and 23 1 warm even if a person could not detect this difference in practice For such analogue quantities a fuzzy representation ValtonenO7 is much more suitable as itis closer to the way humans percei
263. plementing a complete home network with various user interfaces and conducting long term tests Existing and proven network standards were used as widely as possible and adopted in the Smart Home Kaila05_2 76 TUT Smart Home Research Bathroom os Rigi o 0 2A ANAL NAA IAP SPOOO LANL A TTA EA Wooo MAVEN VV VV VV VV ES Soe oon i TEH pE AO THEN Bedroom EPL fell rx Ja i Ta P E A T kOn Figure 5 20 The Smart Home EET showing rooms and furniture Sensors temperature humidity Small deployable temperature humidity sensor modules were installed around the Smart Home making this the first test with sensor networks in the Smart Home These sensors can help the system by analysing conditions both inside and outside the home The sensor module contains separate probes for temperature and humidity measurements both con nected to the module by a short cable The module can use either wired or wireless network connections In the Smart Home modules are installed in the living room sauna flower pot refrigerator and on the outside Other devices can also contain temperature sensors and their measurement data can be used to augment readings from standard sensors An im proved version of the flower pot monitor was also constructed The new version featured the former water level sensor and additional humidity temperature and light sensors for even more precise monitoring
264. ptions The descrip tion contains a brief outline of the device and its functions a list of commands and values that the device uses and understands information on the protocol and network type that the device uses device address where applicable polling interval and possible scaling or other data format options The following is an example an XML description of a temper ature sensor which gives its output as two s complement in C 90 TUT Smart Home Research lt device name OutTemperature protocol rf connection com addr 04 gt lt function id 31 name OutTemperature idgroup sensors get 01 pol lint 60000 unit C gt lt description short Outdoor temperature gt Temperature outside the building lt description gt lt return format 0lvvxx gt lt valuerange lowerlimit 128 upperlimit 127 scaletype 2compl gt lt function gt lt device gt The Home Controller software creates generic Java objects based on these descriptions When each object is aware of the protocol and the network it is using for communicating with the physical device the networks and protocols that the devices actually use become irrelevant to the server An XML parser module reads this information and creates corre sponding objects for the server software concealing the physical networks behind an ab stract interface If a new network or protocol is introduced all that is required is a physical connection to the server
265. queues at hospitals UUTE 128 Hardware Aspects 6 6 4 Sensor Data Processing Most sensors transmit data at regular intervals and require very little bandwidth transmit ting a few bytes at a time A sensor places little demand on the network itself the most important criteria are dependability and robustness Some sensors however transmit data continuously and require large amounts of processing before sending the data to the rest of the system A floor sensor unit for example would have to constantly monitor sensor modules compare readings to previous measurements and attempt to detect movement across the floor If the sensor module itself lacks the required processing power it has to send all measurement data to another processing unit which requires considerably more bandwidth and lower latency than a simple sensor network would have Using video cam eras as sensors also presents challenges for signal processing as image data requires a lot of processing power and efficient algorithms to be useful in smart home applications Cur rently microcontrollers smaller processors and digital signal processors are already able to perform complex processing on the sensor board itself reducing the load on the server However there are often situations where changes or adjustments in the processing algo rithms are required and in these cases it is usually much easier to update all software on the server instead of upgrading each sensor se
266. r a mobile node are rather high as it must be able to store and process data independently if it is disconnected or out of range of the network infrastructure Sentilla07 According to Nokia s vision Nokia08_1 mobile phones will play a major role in mobile computing in the near future Phones can communicate with other nearby phones share information and extract sensor data that they continuously gather Accelerometer data is 38 Smart Homes used for detecting movement and gestures microphones for detecting voices and sounds and various radio receivers for analysing weather conditions etc Information gathered from a large group of users could be used to detect and predict traffic jams or even epidem ics Another approach to mobile computing is to make the entire home mobile Case01 Re locating houses has been commonplace for example in Queensland Australia where mining towns and associated businesses were often relocated as mines became exhausted Physically dragging houses by tractor could be replaced by common house foundations that can accept a variety of structures making it easier for families to relocate and housing types to be changed rapidly according to the residents wishes Houses could be built on stilts allowing them to be easily removed and transported elsewhere Plans to utilise older buildings in a similar way are also under way saving them from demolition and also re ducing restructuring costs 3 6 A
267. r communications and for this reason existing implementations were discarded and a new protocol designed The ability to flex ibly add devices discover services and to implement better fault tolerance were primary design guidelines 152 Software Architecture EPIS Service Discovery Protocol EPIS an Efficient Protocol for Intelligent Spaces is a service discovery protocol designed for the LIPS project EPIS is a simple low power protocol offering service discovery data exchange and power saving schemes developed with low power sensors and simple wire less networks in mind EPIS uses a standard PC for running the main server software keeping registry of known devices and continuously listening to the wireless network Thus EPIS is also confined to a star topology which on the one hand leaves all responsi bility to the central node and also a possible single point of failure but on the other hand makes wireless nodes simpler as they only have to know the network address of the central node Remote nodes are mote devices as described in Chapter 6 Addressing is imple mented using TinyOS 16 bit addresses with OxFFFF reserved for broadcast messages Ev ery wireless node has an individual network address that it responds to and in addition to this address one node can have one or several device addresses This is required because a node depending on the hardware configuration can contain several sensors or connect to other external m
268. r concern especially if a home is filled with sensors and cameras that can gather rather personal information on the occupants Many of today s appliances and office devices contain network functionality rendering them as vulnerable to attack as any other networked equipment Printers routers game 184 Analysis consoles WLAN gadgets etc all have web servers or remote administration services run ning and these embedded software modules are seldom upgradeable or fully secure ProSys07 When such equipment is permanently connected to the Internet there are seri ous security and privacy risks often unknown to users Most people are intimidated by the spectre of a Big Brother in the home and therefore it is important that infrastructures are secure and totally isolated from the outside world Another important factor is the work and resources that have to be committed when retro fitting smart home technology into old buildings The majority of smart home installations are likely to be in older buildings involving additional costs for the user Modern homes currently have Internet access PCs or similar terminal equipment mobile communication devices and certain external services that connect to the home cable television online services etc There are however numerous features and devices that are still lacking in the form of ad hoc short range wireless communication i e sensor networks indoor po sitioning adaptive UIs and
269. r home and to address this issue a method was needed to sense if a device was turned on or off For this purpose a sensor module was designed that was capable of meas uring the current drawn by a connected appliance The device was essentially an extension cord with a sensor unit in the middle The sensor used a LEM LEM current sensor which could measure the amount of current being drawn from the phase conductor The sensor was connected to the serial hub using standard serial cables Figure 5 43 eHome current monitor 5 4 3 eHome User Interfaces Whereas the network and infrastructure in the eHome was similar to those of the Smart Home the UIs were designed to be very different The focus was not on home automation but rather on offering users various options for interfacing with home equipment While static traditional UIs were retained light switches window blinds etc new ways of con trolling home equipment were created Each UI was designed for a specific purpose and scenario with emphasis still firmly on usability and practicality UIs were designed in co operation with usability experts to create intuitive graphical UIs that users would appreci ate 98 TUT Smart Home Research Figure 5 44 Collage of eHome UIs showing the WWW UI running on a laptop top TV UI bottom left and mobile phone UI bottom right eHome WWW UI The primary UI for the eHome was a web page The web page could be accessed from any PC
270. r power consumption it would be benefi cial if smart devices themselves were also designed with maximum power savings in mind Batteries provide users with better mobility but they also impose practical limits for oper ating times One problem is the race between battery software and electronics manufac turers if a faster processor is developed it is able to run software faster which leads to software becoming more complicated and demanding with more functions and capabili ties Faster processors consume more power and the advances that are made in battery technology are quickly overtaken by these growing power consumption figures Starner03 Increasing power consumption also causes problems as electronics miniatur isation advances the smaller the physical size of a chip becomes the harder it is to provide sufficient cooling for it Conducting heat away from a physically small area is already a major problem with today s desktop processors limiting maximum operating frequencies Methods to avoid overheating include lowering the operating voltage of the processor and utilising parallel processor cores Largest energy savings can be achieved by using power saving and sleep states that con trollers and ICs offer If the software permits the controller can enter a low power state whenever possible For example the Atmega8L microcontroller s which is used in many devices in the Smart Home datasheet Atmel states these current consumptio
271. r to monitor the con dition of the building and take preventative action against damage Energy can also be saved if these conditions can be accurately predicted or quickly reacted to New materials such as self monitoring concrete and self healing structures can asssist in maintenance tasks whereas thin foldable displays and wallpaper that can function as a display all can change the way in which people interact with the smart home With innovations like these the distinction between user interface and infrastructure can almost disappear Problems arise when new devices and networks are installed in old buildings that lack any infrastructural support Thick solid walls or metallic structures may prevent or hinder the use of radio communications and even if wireless networks are used devices still need a power supply which usually means a transformer and power cables or at least batteries However it is possible to utilise existing cabling in the building using appropriate com munication equipment Phone lines and power lines are already being used for networks Gerhart99 and they can significantly reduce the need for additional cabling In most cas es these only facilitate communication not control since motors actuators and relays are still needed for controlling the home Adding new wiring or making changes to the elec trical network however can be very challenging and expensive Another interesting problem arises when considering wha
272. ransmitting their responses to the server Furthermore transmission from the master was strong enough to bounce off walls and penetrate thin transparent material but the re sponse from the slave was easily lost if there was no direct line of sight to the master The next logical step was to move to RF transceivers which at the time were becoming increas ingly popular due to lower prices and highly integrated modules The synchronous serial link was also discarded in favour of standard RS 232 transmission either over RF or cable The RF link acted like as a serial cable so that changing to a physical cable or a RF card was possible without modification to the devices themselves The RF link had a range suf ficient to cover the apartment and the matchbox size transceiver was easier to install than the cumbersome IR links The RF link is described below in greater detail 65 TUT Smart Home Research Figure 5 7 IR master array used for wireless communication left IR slave device right With standard RS 232 communication becoming the used standard in the Living Room a problem arose with the limited availability of serial ports on the server In order to connect several devices into one serial port on the PC it was decided to construct a serial hub A four port hub was considered sufficient and allowed four devices to share the same serial port to the PC Since the network was designed as a master slave type there was no danger of colli
273. rared re mote controls and light switches becoming a portable universal UI for the home The re mote control has only a four way rocker switch and a small display and the functions available on the screen change according to the device or function that is being controlled In order to detect which device is being controlled the remote has an infrared receiver Small infrared tags are placed near controllable devices and various other locations in the home the remote is able to use SCARS infrared positioning and when the remote control is pointed at such a tag it recognises the device and displays the appropriate information to the user Commands are transmitted from the remote to the home server over an RF link With the ability to sense which device the user wishes to control it is possible to present only the relevant controls This makes it easier for the user to select appropriate actions instead of having to use separate remote controls for each device or browse through a com plex menu structure on a graphical UI Figure 8 1 Context aware remote controller prototype RitalaO3_2 ContextPhone Raento05 is an application for mobile phones using the popular Series 60 S60 platform ContextPhone attempts to discover the context of the user by using infor mation that it can gather using sensors in the phone cellular network notes in the calendar etc This information is also shared with other people and passed onto other applications
274. rds tablet pads with a pen UI and web pages were used Technology was used to support students before during and after the lecture 41 Smart Homes Other Smart Home projects are presented in Chapter 4 3 8 2 A Brief History of Domestic Technology During the 20th century there has been a tremendous increase in domestic technology Chores that previously had to be done by hand could now be assigned to machines and ap pliances and thus more spare time was available and the way people used the kitchen changed for ever Most of these advances can be credited to electricity and later during the century to information technology After the 1940s the focus became very technology ori ented mostly on different kinds of machines automation and technology In the 1970s in formation technology took over with computers and networks emerging It was not until the 21st century that the users living experiences became central issues and more user friendly development started The technological development during the 20th century can be roughly divided into five periods Panzar00 Aldrich03 The period from the early 1900s to the 1920s were pioneering times for modern home ap pliances and new electric appliances The home was still very traditional and old fash ioned running water and mains electricity were far from being universally available Development was driven by the shortage of a domestic workforce and industrialisation which was
275. rdware platforms have been created using both PIC and Atmel microcontrollers Blue tooth and 868 MHz radios wired RS 232 and I2C communications Smart Its are able to collect data from their environment and share this with each other creating a collective awareness Figure 6 15 Smart Its module board Photo ETH Ziirich 137 Hardware Aspects VTT Soapbox The SoapBox Sensing Operating and Activating Peripheral Box is another small plat form for sensor networks developed by VTT Tuulari05 As the name implies the hotel soap bar sized module consists of two modules stacked on top of each other an antenna battery and other circuitry Sensors in the SoapBox module include acceleration light temperature proximity IR transmitter receiver and a magnetic sensor that also acts as an electric compass The 868 MHz radio connects to a central server and the operational range varies between two and 80 metres depending on the power level used SoapBox sensors have been used as movement and attitude detectors in handheld user interfaces for exam ple enabling navigation through the UI by tilting turning and moving the screen Figure 6 16 VTT Soapbox module Photo VTT Sensinode Sensinode is a company currently concentrating on creating small IP stacks for embedded devices but they have also created network node modules Sensinode nano and micro The nanostack IP stack is written in C and designed to be usable in low power mic
276. re The DLNA Digital Living Network Alliance DLNA alliance also uses UPnP to share media through the home network for example in the case of amplifiers TVs and game consoles where it is already in use Due to XML parsing and heavy use of multicast UPnP con sumes resources rather heavily and thus is not a viable choice for smaller low power de vices Another alternative protocol Jini Jini was designed to transfer computing away from the computer disk drive approach to a networked computer style where applications and re sources are available through the network in an invisible fashion Jini also functions in an IP network and it is based on three parts the client a server and a lookup service Jini is Java based and thus requires a virtual machine to run which also makes it an undesirable option for small low power devices Bluetooth also contains a protocol for discovering devices and their services over the wire less link However the range limitation of 10 metres Bluetooth Class 1 the inability to remotely access another device relatively high current consumption and the peer to peer style architecture also makes Bluetooth a less desirable alternative for devices with limited resources Bluetooth discovery can also be turned off from the client device making it im possible for a server to locate it and discover its services Service Location Protocol SLP is a protocol developed for use in local area networks Guttman99
277. re nitrous oxide and carbon dioxide content was installed to monitor the well being of the occupants Commercially available sensor modules made it possible to construct an integrated sensor unit capable of measur ing above mentioned quantities The quality of the air inside the home can affect users in various ways both physically and mentally Ventilation in rooms is often inadequate and wastes energy and knowing when and where ventilation is needed can considerably im prove the situation In larger cities with variable or poor air quality measurements could also be taken outside to determine if it is safe to open the windows or turn on the ventila tion The sensor unit used in the Living Room was a module that contained numerous types of sensors each with their own measurement method The air quality sensor was sensitive to abstract characteristics such as cigarette smoke coffee cosmetics and cooking Although the odour sensor did not provide a definitive reading on the air quality in the Living Room it could still be used for detecting a need for ventilation Indirectly combined with read ings from the integrated carbon dioxide and nitrous oxide sensors it also indicated when the room was occupied by several people or if certain activities were taking place Figure 5 16 Odour sensor unit TRemote Home entertainment and A V devices rely on optical infrared transmission for remote con trol and in order to control them a universa
278. re of the technology they are using and the capabilities that the device possesses e g the amount of processing it is able to perform and its level of intelligence Low level of intelligence Temperature Washing sensor machine HVAC TV controller Set top box Embedded Perceivable Digital camera Broadband router Mobile Home phone gateway PC High level of intelligence Figure 6 1 Classification of typical home devices according to how intelligent and perceivable they are For example a TV set is clearly visible to the user but a digital set top box is less apparent as users are still watching TV even if the broadcast is being received by the set top box A modern digital camera is capable of processing large amounts of image data and it also actively analyses the image content recognising faces movement etc Simple sensors are inconspicuous and numerous while their other capabilities remain very limited Network related devices such as broadband routers or firewalls actively monitor traffic filter it by 110 Hardware Aspects content and source destination addresses and decide on appropriate tasks A home gateway is a more complex device being capable of interconnecting home networks bridging pro tocols and network standards and connecting the home to the Interne
279. re only a limited number of radio channels to choose from Overlapping frequencies cause WLAN access points to interfere with each other effectively reducing their usable range There have also 120 Hardware Aspects been other competing wireless standards however many of them have fallen into disuse This is the case with HomeRF HomeRF a wireless home networking standard prepared by the HomeRF Working Group and HiperLAN HiperLAN High Performance Radio LAN Wireless optical transmission is also still largely in use however mostly in the form of uni directional infrared remote controls Infrared can also be used to create a home network but range and physical obstacles can severely limit its applications IrDA IrDA still available in laptop computers and mobile phones offers relatively high data rates up to 4 Mbit s However since its range is limited to less than one metre it is not a viable alter native for smart home applications Other short range wireless technologies include RFID Hossain08 which is mostly used in smart cards and price tags in shops for anti theft measures RFID tags can be passive or active depending on whether or not they have a power supply Passive tags lack a power source and receive the energy required for operation from the reader through induction In the case of an active tag a power source such as a battery is required providing it with sufficient range to cover greater distances than pass
280. ree basic categories sensors actuators and input output devices Sensors perform measurements actuators perform physical tasks and input output devices are the interface between the user and the home control sys tem Common properties of these are that they are aware of their own state they can send and accept commands from the rest of the system and they have a specific identity or net work address Thus they are able to receive commands perform a task e g measure ad just and report back to the network master In order to do this a smart device needs a microcontroller for processing input and output connectors for controlling and communi cating with external equipment sensors motors LEDs a power source and a communi cation link to the home network The Department of Electronics has excellent facilities for electronics board design and prototyping and thus it has been possible to implement new devices and modify existing ones Most devices have a standard connector that allows them to connect to each network RS 232 wireless RF Bluetooth This standardised RJ 12 connector houses serial receive transmit data lines common ground and 5 V power supply The device is designed to pro 133 Hardware Aspects vide power 5 V for an external wireless data module RF Bluetooth or a sensor board through the same RJ 12 connector When the physical connector is standardised there is no need for reconfiguration if the network t
281. reen was still considered to be the most intuitive method of controlling the main graphical UI but additional ways of interacting with the home were also deemed necessary If all functionality is invested in a single UI there might be situa tions where the UT is in the wrong place or users simply fail to pick it up and make adjust ments For this purpose we designed mobile UIs one for a mobile phone and one for wireless speech control The primary Smart Home UI is a touch screen equipped tablet PC Being a tablet PC it is easily carried around the home to wherever needed The tablet runs a graphical Java pro gram the Home Controller which allows the users to control and monitor equipment in the home The main page gives an overview of important temperatures shows which lights are turned on and allows adjustment of lights blinds curtains and the front door Another page is dedicated to home entertainment such as Watch a DVD and listen to music macros that allow users to input a preset mode at the press of a single button For example by pressing the Watch a DVD button the DVD player amplifier and projector will turn on and be set in appropriate modes window blinds and curtains will close living room lights will dim and the projector screen will lower Other preset modes allow users to turn off the equipment and resume normal daily routines etc The same page also includes con trols for A V equipment as in the Living
282. reliability have also been im portant criteria and for this purpose experiments with several different kinds of user inter faces have been conducted while at the same time traditional functionality has been retained in the form of manual controls Later as the research progressed adaptivity and proactivity became important themes with learning fuzzy logic based home control sys tems For the most part telecom related technologies and applications e g broadband net works video on demand multimedia services etc have been ignored due to their heavy dependence on external services and the telecom industry Healthcare on the other hand has been researched in related projects but has not played a significant role in smart home 59 Related Work research as such Results from biosensors and other healthcare related sensors can how ever be easily integrated into the smart home infrastructure The Department of Electronics also conducts research into smart clothing and since this branch is not unrelated to smart environments there are potential benefits to be gained from collaboration between both fields Kaila05_ 1 Smart home research projects at TUT are presented in more detail in the next chapter 60 TUT Smart Home Research 5 TUT Smart Home Research 5 1 Research Projects and Environments at TUT For many years different kinds of research concentrating on intelligent homes ubiquitous computing and home network
283. rform a temperature measurement and send a reply back to the master This reply would consist of a header the measurement command plus an additional data byte consisting of the actual measurement result OxA2 P 0x01 P OxYY P where YY is the measurement result 7 4 2 Smart Home Serial and UI Protocols The Smart Home introduced more devices and new UIs which partially meant that the old protocol and infrastructure were no longer able to cope with the increasing amount of func tionality A new serial protocol was written and partially tested in the Living Room later to be followed by a separate protocol for UIs Smart Home Serial Protocol The complexity and performance of the MOB bus protocol became apparent as more de vices were created When it was decided to move away from the infrared links there was less need for the MOB protocol and work started on a new version As RS 232 interfaces were available in most microcontrollers and devices it was a logical step to adopt this standard instead The addition of addressing was important as there could be numerous devices connected to a master using a serial hub or wireless link The data structure using Datapak remained basically the same consisting of three specific bytes device address header and a specific command The header informs the receiving device how many bytes of data are to be expected in the data frame After these three bytes it is also possible to send a
284. rks are performed This can cause frustration tech nophobia or reluctance to use the advanced functions of electronic devices In a home en vironment filled with automation appliances home entertainment PDAs laptops mobile phones and computers there is already now a plethora of devices we must learn to use and maintain in our daily routines Despite convergence and the integration of functions it is likely that the number and complexity of such devices will continue to increase For the future home a few years from now this increase in electronic devices raises certain issues how will these devices work together how can they all be interconnected to enable more intuitive control of the future home and what possibilities would such a home offer its occupants The smart home is a concept that promises improved ways to control func tions in the home flexible networks that can connect to any device adaptive control facil ities and peace of mind for its occupants This is the goal but how can it be achieved using present technology The key elements to achieve this are intuitive flexible user interfaces and device compatibility Three different smart home research environments were constructed to investigate these issues the Living Room the Smart Home and the eHome This thesis discusses the design implementation development and findings of these projects with brief mention of related smart home projects and technologies The research w
285. rocon troller applications Sensinodes use 2 4 GHz IEEE 802 15 4 system on chip CC2430 radi os for communication and they can be organised in different network topologies 138 Hardware Aspects Figure 6 17 Sensinode evaluation kit Photo Sensinode Sun SPOT Sun laboratories have created a Java programmable sensor module the Sun SPOT Sun Small Programmable Object Technology Sunspot Compared to previously mentioned nodes the SPOT is more powerful and complex as it requires more processing power and memory to run Java software The core of a SPOT is a 32 bit ARM920T processor running at 180MHz together with RAM and Flash memory An auxiliary Atmega 88 microcon troller is reserved for further functions Communication with other nodes is performed with an 802 15 4 compliant RF transceiver and a USB connector The internal recharge able battery is sufficient for a running time of 3 7 hours of continuous usage further ex tension of battery life can naturally be achieved by reducing the active time of the processor and using various sleep and power saving states The SPOT board includes sen sors for measuring acceleration temperature and light It is designed to be a flexible de velopment platform that is capable of hosting many kinds of application modules 139 Hardware Aspects Figure 6 18 Sun SPOT module Photo SUN Microsystems Table 6 5 shows a brief comparison between sensor nodes listing their radios pr
286. rol lights blinds curtains and the front door NOKIA Figure 5 33 Smart Home mobile phone UI Wall Panel UI While the concept of the disappearing computer Schenker00 did not mean its actual physical disappearance the concept was nevertheless quickly directed towards user inter faces Dey01_2 This prompted speculation as to whether walls panels and surfaces of equipment could be kept aesthetically clean and their UIs made invisible only to appear when needed A disappearing UI would normally pose no problem for users but if it was needed for any reason or if there was something that required the attention of the users it would readily become available Experiments with LED lights and thin veneer were very promising as they proved that it was possible to create a wood like surface thin enough for light to pass through This gave rise to the idea about creating a transparent display with a wooden surface The goal was to devise a user interface which would normally be invisible until a person walked by or waved a hand in front when the panel UI would come to life and be visible A prototype with four pushbuttons and a light dimmer control was designed using thin veneer and chip 88 TUT Smart Home Research board and embedded in one end of a cabinet in the Smart Home The buttons consist of LED patterns which light up and the light dimmer is a slider which allows users to control brightness in discrete steps Controls include
287. rom the community or relatives The same technology can be adapted to each of these user groups although user interfaces probably would require changes so they would be more suitable for the target group Another issue often overlooked is the influence of national cultures on the accept ability of smart home solutions For example in Asia the definition of a home probably is the same as in Europe but its function may be largely different home values and priorities in certain cultures might diverge from those considered in the present work The home can be an environment where many families and generations live together where people con stantly come and go and where walls do not necessarily mark the borders of what is con sidered a home In this respect the benefits of an adaptive learning home control system would also prove important TUT smart home research set out to discover how smart spaces can be built and designed and what options there are for users to control smart environments From the outset the contribution has been a practical approach to research instead of simulating and speculat ing the research group has constructed devices built test environments and conducted tests Research contribution from TUT research projects has been summarised in the list below 181 Findings e Functional prototypes that can be used tested and modified e A total of three functional test environments e Home network infrastructure both wire
288. running in the phone Sensors used by the software include microphone and camera for 163 User Interfaces context sharing Bluetooth used for detecting other nearby phones and network cell in formation for obtaining a rough location estimate Further information is received from applications and settings from the phone itself active profiles open applications recent activities etc In a context aware contact list in ContextPhone users can see the context of their contact and decide whether or not the contact can be reached For example a person in a meeting would have a silent profile activated be surrounded by other people and be near a meeting room WLAN hotspot ContextPhone also allows users to exchange pres ence information using text and multimedia messages or through short range wireless communication ContextPhone software is available from the author s website 5 Presence info lt for Kari Laa kari Laasonen am Current Kumpula HKI Kumpula HKI 0 20 ocation or the last 0 30 pe x Current Profile Meeting yRaento Mika WK W Speaker off Hermanni HKI 1 00 1 B Vibrator on Renaud Petit gz People close by Exactum 5 20 32 1 1 friend s 6 other person s E d Last phone use 0 02 ago Options Back a Close Figure 8 2 Screenshots of Context Phone Photo Mika Raento 8 3 Physical UIs Physical interfaces such as buttons levers and other natural user interfaces are the most intuit
289. s The tag was battery powered and transmitted an ID code every few sec onds A pin allowed users to wear the tag like a badge A tag reader mounted near the front door was able to read the infrared signature transmitted by the users tags If the ID received matched the list of known users the tag reader would unlock the door Another application for the tags was a wine bottle rack that identified the users accessing it Unauthorised use of the rack would register an alarm on the server displaying the person s identity or in the case of an unknown person the time and date of the theft The system was originally designed to work in the opposite way with a person carrying the tag reader and tags being placed onto objects of importance around the home H kkinen00 However since the reader was physically considerably larger than the small sized tags it was considered easier to make the tags mobile and place readers in ap propriate locations in the Living Room 69 TUT Smart Home Research Figure 5 12 Infrared tag left tag integrated into a brooch right Front Door The front door was equipped with an electrically controllable lock An infrared transmitter receiver pair was installed in the door handle to sense when a person was clasping it in order to open the door An infrared reader mounted on the outside wall read the persons IR tag and if the person was recognised the lock would open The door monitor was not connected to t
290. s and functions for pervasive tech nology From the previous design goals it is clear that there is a need to have an embedded network infrastructure and compatible devices in order to create a smart home network Pervasive devices should be optimised for specialised tasks and not for general use thus they can be designed to be easier to use and maintain 2 4 Our Definition of a Smart Space The Personal Electronics research group regards a smart space in our case the environ ment is always a home as a physical space consisting of networked devices and sensors that are able to monitor each other and the conditions inside the space The system is able 12 Definitions to detect the presence of people and react accordingly by making autonomous decisions and affecting the space in a physical way This definition is similar to the definition of T Kanter Kanter00 However merely adding and connecting sensors and actuators in a space is not enough A smart space should also be able to perform complex reasoning and adapt itself to the users routines and habits Thus even if a space consists of several very complex and advanced independent systems it cannot be considered a smart space if these systems are not interconnected in any way As aresult home automation HVAC control pre programmed systems and other standalone applications lie outside our definition as they do not offer all the functions expected of a smart system A smart
291. s becomes paramount Energy consumption is often a compromise between performance range features response time and mobility Robust 113 Hardware Aspects ness encryption and complexity often dictate the power consumption of a wireless trans ceiver chip for example Cost Smart home technology i e the hardware itself is rarely a factor For example de vices created at TUT use widely available components and an individual device typically costs less than ten Euros Certain sensor modules complex integrated circuits etc can still seriously increase the cost of a device as will the amount of time spent on its design Size Size matters especially with mobile devices but also when installing or retrofitting fixed devices The physical size of a network connector or a battery often limits the prac tical minimum size Cost might also be a factor as smaller components can be more expen sive and difficult to install Scalability Networks are by nature designed to be scalable but in many cases there are practical limits on how many devices or nodes can be connected at the same time Limits can also be imposed by power consumption a network can sustain a certain number of nodes before maximum power output is reached or delays network cables can be of cer tain length or messages can hop through a certain amount of nodes before delays become too long Finally processing power or memory restraints can dictate the amount of traff
292. s involve the diversity of the networked devices and the robustness of the system as a whole ISTAGO3 Since a smart home must accommodate almost every device connected to the home network special care must be taken during the design of the smart home platform Heterogeneity of devices compatibility and security are crucial is sues in the process Privacy and security are also important issues since devices can access private or otherwise delicate information regarding users and their surroundings A smart home can collect large amounts of data related to user s activities personal preferences and personal data that could be misused to endanger the privacy of the occupants Itis also important to note that wireless networks can also be accessed from outside the home and that mobile devices can be exposed to unsecured foreign networks where they might be compromised Marketing smart homes also presents certain problems of its own Most people have only a vague idea what to expect or demand from future living so it is reasonable to suppose that there is a good deal of uncertainty as to what a smart home could actually be like For many users even the scope and capacity of today s technology can seem like science fic tion In 2004 the research group for technology and everyday life TATU at TUT conduct ed research into how people envisioned future living smart homes and what they would like to see in the near future TATU04 According to the rep
293. s revolutionised with the Compact Disc game consoles and cable television Where the PC had originally been given a role in archiving communication and accounting it later proved to be more popular for gaming entertain ment and Internet usage When different kinds of smart home projects started the computer already had an integral role in the home a role that continues to evolve As in the past whenever new electrical home appliances were marketed to housewives the primary marketing argument was that these new appliances leave users with more time to do something else Later on as time consuming home entertainment devices came on the market the time saved doing chores faster could now be spent relaxing with a vast choice of entertainment A community of smart devices Intentional devices Proactive devices Communicating devices Programmable devices Remote controlled devices Sensors surveillance Figure 3 4 The evolution of a smart home Panzar00 43 Smart Homes 3 8 3 Future Discussion about the future of homes human computer interaction is a very interesting topic Microsoft s scientists have given much thought to how we interact with computers and information technology in the year 2020 MS07 The report suggests that the way we feel about technology in our lives will probably change considerably in the future Young er generations will be accustomed to using IT from an ear
294. se an architecture called ISR Internet Suspend Resume which works in a similar fashion as the suspend resume action in a laptop computer This would allow users to work on a workstation at one location sus pend their work and walk to another location where they can resume their work without interruption The work would simply have transferred to another location and the user does not have to carry any hardware Many users are already familiar with the suspend resume action of a laptop and since ISR would work in a similar fashion it would not re quire users to familiarise themselves with new practices 2 10 Context Awareness The concept of context is vital to humans in everyday communication with other humans but in the case of digital computer communications almost all of this information is lost Dey01_1 Body language ambient conditions people around us sounds odours etc are all information that humans process instantly without noticing In contrast a computer would need lots of sensors and processing power in order to do achieve this Contextual information could however add significant value to human computer interaction increas ing its richness and providing better services In smart environments it is possible to utilise sensors around the space making it theoretically possible to detect contexts in the environ ment Therefore a context aware system can follow the actions in a space detect contexts and provide relevant in
295. se them for several purposes The same RJ 45 connector is installed in every room and from the switchboard it is possible to re route the connectors to become phone audio or antenna jacks or Ethernet ports Thus re routing and modifying the layout is possible at a later stage making it easier to move the TV set or computer into another room for example If an appliance requires an Ethernet connection it is simple to connect this socket to the home router in the switchboard Current copper Ethernet net works reach speeds up to 10 Gbit s providing enough bandwidth for domestic communi cation applications Optical counterparts are used mainly for high speed trunk networks 117 Hardware Aspects and high bandwidth multimedia but they can also have use in other demanding applica tions CEBUS Consumer Electronic Bus is a set of standards for household networking devel oped to succeed the older X10 standard and to create a standardised way for devices using infrared communication CEBUS CEBUS can be used together with several kinds of transmission media including coaxial wires power lines twisted pair cabling and wireless RF or infrared All media are capable of transfer data at a rate of 8 kbit s and in addition to this control channel there can be extra faster data channels for transferring audio or vid eo CEBUS is not confined to a specific network topology or server instead control is dis tributed among routers and devices EI
296. sent or measured and thus require networks with higher bandwidth Furthermore video surveillance and computer vision applications have similar require ments as even compressed video generates large amounts of data Thus it would seem ap propriate to select the optimal network for every application taking into account the requirements price performance and compatibility of each network type 6 1 2 ISO OSI Network Model OSI Seven Layer Model The International Standard Organization s Open System Interconnect ISO OSI model is widely used for describing layered network communications and network protocols Tannenbaum02 The OSI model describes how data is moved between the independent layers and divided into tasks that the seven layers perform The purpose of a layer is to pro vide services to a layer above it and to receive services from the layer below it Layers are transparent so that the transport layer of a device is able to communicate directly with the transport layer of another device The layers of the OSI models are presented in the fol lowing table Table 6 1 OSI model layers and their functions Data Unit Layer Function End to end com Data Application Provides services to applications munications Data Presentation Encryption data presentation Data Session Session management Segment Transport Connections reliability Inter device com Packet Network Logical addressing path d
297. sers to invest in new technology is the balance between necessity and appeal and also cost TATU04 New technologies may be intriguing but if the application itself is considered unnecessary or too expensive consumers will hold back The cost of the latest smart home technology remains high and the prospect of in stalling potentially incompatible and marginally useful hardware into a home makes it an unattractive investment Another hindrance to the wider acceptance of the smart home is the push by the technology industries to persuade people to buy their ever more sophisti cated products Aldrich03 Too little emphasis is placed on usability and user friendli ness making technology seem foreign and unfamiliar Instead the focus should be on user friendliness and usability as well as usefulness and compatibility Developers need to remember that smart home technology should not greatly alter the atmosphere of the home and nor should it detract from home comfort or attempt to replace the essential val ues of home life Ultimately the general public have very little understanding of smart homes and it seems very little interest in ownership A study by Nokia Nokia07 bears out the above observations on the adaptation of smart home technology The study attributes this public reluctance to the absence of any com mon protocol and a lack of compatibility among devices and networks combined with high prices Security and privacy also give cause fo
298. sions or lost packets because the master always initiates communication 5 2 2 Living Room User Interface The primary UI for the Living Room was the tabletop PC mentioned above The PC was constructed from PC 104 compliant PC104 modules i e a biscuit PC embedded in side a round table and equipped with a touch screen and batteries for mobility This way the PC was basically a mobile UI that could be moved to other parts of the room if needed Batteries provided an operating time of about two hours The main graphical UI of the Liv ing Room included a master status page separate lighting control home A V equipment and house plant monitoring pages The UI allowed users to monitor conditions in the home and also make adjustments and control equipment In addition there were pre set modes for watching TV or listening to music Controls for A V equipment allowed users to dis card traditional infrared remote controls and control all equipment from a single location 66 TUT Smart Home Research KotiOhjain F Ohje m J rjestelm t Valaistus Kotiteatteri Kukat liman tila ys _Kotteates_ ue monia _Luvalsiteet_ Turvalaitteet m Turvajarestelmat r Ulkoilma Palovaroittimet Kunnossa L mp tila Murtoh lyttimet Ep kunnossa 9 C Kosteusanturit Kunnossa Kosteusprosentti 54 r Sis ilma L mp tila 24 C
299. sneceseseeesenees 51 42 9HP COOMO WI een ene enr a E hc 51 4 2 10 Orange at Home sssssesssseiseessseeessrestsresrrrrsenrssreresesreseses 53 AD A MITE House al cise etn died oe Bit seit sowed 53 4 2 12 MIT alte aeae e aE E REA A RE 54 4 213 MIT ORY SOD c 2 c556s ccsecescsesbcossedsevsss sven iniiis nivei 55 4 2 14 GatorTech Smart House cccceccceesseceesteeceseeeesseeeenees 55 ARQ ADS RUT CALE oss escectes Sette a e a ae tesco 56 42 AOA Haus verorren ete eet ere E E 57 3 DISCUSSION serae e E eee E E eke ek 58 gAs AN A E A TE TE GORA 59 5 TUT Smart Home Research cccssscsssscssssccsssccssscescsssscssssessssssssssseeees 61 5 1 Research Projects and Environments at TUT 0 0 eee 61 5 2 Smart Living Room 1999 2002 ooo eeeeescesecseeeeseceeeeeseeeseeceeeeenees 62 5 2 1 Living Room Network Infrastructure 0 ee eee 64 5 2 2 Living Room User Interface wo eee cee ce eeeeeeeeeees 66 5 2 3 Controllable Devices in the Living Room ee 68 5 3 The Smart Home 2002 gt cccccccesesccesseecsececesseceseeecesneeeneneeesenees 74 5 3 1 Smart Home Network Infrastructure cccceesseeeeees 83 5 3 2 Smart Home User Interfaces ccccececeecsceceseceeesteeeeeeees 86 5 4 The eHome 2001 2005 eeccecceceesecessceceesececeseecesneeecesaeceseaeeeseeeens 93 5 4 1 eHome Networks ccccccccesesesssseecsseecesseeecseseeecseeeenseeeaes 97 5 4 2 eH Ome DEVICES Hreini siceaees teehee enn 97 5 4
300. so lowering the threshold for human computer interaction As sensor technology and artificial intelligence progresses new ways of multimodal interaction with the home environment will also emerge paving the way for 172 User Interfaces even more natural user interfaces Capturing and understanding context is another chal lenge that requires effort from both software engineering and sensor technology and when fully realised it will further enhance the smart home user experience Uls used in TUT smart home projects include graphical physical auditory and mobile whereas more complex UIs and those that would require heavy data processing have not been implemented As the amount of sensors and devices in the home increases it also cre ates new possibilities for creating exciting UIs that can give users new ways of interacting with the smart home system 173 Findings 9 Findings This chapter presents findings and observations on the smart home projects at TUT These are based on several years of practical experience working with the projects as well as con tinuous evaluation of their development Findings on three smart home projects are pre sented 9 1 Findings From the Living Room One key design goal of the Living Room was to create a relaxing comfortable living space and based on the opinions of those who used the space this design goal was achieved Soft ambient lighting comfortable furniture and no visible electro
301. sockets The heart of the eHome network was a server PC located in the basement The PC managed connections to other Uls and running a web server in the home intranet and it also ran a modified version of the Home Controller middleware software In addition the PC had an ADSL connection to the university so that a remote connection could be made to the server if updates or adjust ments were needed The Internet connection was also shared by the tenants for purposes such as regular web surfing Fig 5 41 below shows the infrastructure of the eHome dif ferent network types UIs and hardware Buttons lights Devices Serial Protocol RS 232 RS 232 COM1 COM2 Serial HUB main PC ekoti ele tut fi i WLAN access point GPRS Home theater PC Figure 5 42 Network infrastructure of the eHome 5 4 2 eHome Devices The eHome contained devices in common with the Smart Home including motorised win dow blinds flower pot monitor temperature and humidity sensors and halogen dimmers Devices were first tested in the Smart Home and later transferred to the eHome Since a duplicate network and devices existed it was easier to test different upgrades and changes before implementing them into the eHome 97 TUT Smart Home Research Current Monitor The eHome tenants reported their concern about forgetting to turn off equipment when they left thei
302. ss stressful the functions offered by the eHome being mostly informative helpful and relaxing Figure 5 38 The eHome apartment building The test apartment is on the ground floor lower left The eHome project started in 2001 in cooperation with private companies in the construc tion Pohjola Tampere rental apartments telecommunication Elisa Nokia Sonera and electronics industry Electrolux Pikosystems as well as usability researchers TUT soft ware engineering The apartment was finished in late 2002 and the tenants moved in later that year The apartment consisted of a living room bedroom kitchen and a bathroom sau na and was modified to conform to the project s requirements Like the Smart Home the eHome had a suspended ceiling customised electrics and extra space for cables and equip ment Empty cable ducts made it easier to install equipment and modify networks if nec 94 TUT Smart Home Research essary and the suspended ceiling conveniently functioned as installation space for equipment A separate control room was built under the apartment in the basement of the building which allowed the servers and network equipment to be installed out of reach of the users It also made it possible to visit the eHome without bothering the tenants A LI NET network node was installed in each room and the master control unit was wired to the server in the basement Lights sensors and controllable devices were also wired to th
303. st amount of information and processing power is available Devices that connect to Net 21 include Enviro21s which are static computational devices e g wall mounted panel computers and mobile Handy21s smaller sized PDA style applianc es Different kinds of automation systems that manage repetitive automated tasks in the environment other systems enabling collaboration and various kinds of sensors are also connected to Oxygen networks This allows users to access and search information in nu merous ways for example they can listen to a phone conversation they have had earlier at work while preparing a meal at home 4 2 14 GatorTech Smart House 2004 The GatorTech Smart House is a project run by the Mobile and Pervasive Computing Lab oratory at the University of Florida Helal05 The Smart House is a programmable perva sive space designed to assist the elderly and disabled in their daily lives to make them more comfortable and secure The goal of the project is to create assisting environments that are able to sense themselves and their residents and create connections between the physical space intervention services and remote monitoring The Smart House runs generic middleware built around the OSGi platform OSGi that stores service definitions for all hardware sensors and actuators inside the home in es sence turning the Smart Home into both as a software library and runtime environment The middleware is designed to be easily exp
304. sting environment The Living Room was dismantled and the hardware moved to the new location 9 2 Findings From the Smart Home The Smart Home laboratory was completed in late 2002 and has been in use ever since Like the Living Room the Smart Home apartment is used primarily for demonstrations testing and also as a social space for department employees During busier periods there are weekly visits to the Smart Home by student groups schoolchildren exchange students and corporate visitors Itis in use daily and provides us with practical information on func tionality and usability in everyday life Students on the Modern user interface electronics post graduate course have also participated in usability research and electronics design and have contributed new points of view to our knowledge They have also been helpful in test ing various UIs and writing reports Overall it has been proven useful to have several different UIs from which users can con trol and monitor the home The situation changed very much from the one UI scenario in the Living Room since now there were numerous choices The most frequently used UI is 175 Findings the tablet UI which replaces bulky remote controls and offers users a unified way of using the home entertainment system It displays the status of the home clearly and allows for easy and quick control of lighting and home appliances Visitors to the Smart Home have also found it easy and quic
305. stributed with intelligent independent network nodes As wireless devices become increasingly numerous their management must be taken into consideration Changing batteries can no longer be an option and other ways of powering devices must be found Current RFID technologies are able to supply power wirelessly over short dis tances but there is more promise in energy harvesting technologies where devices extract energy from their environment This is accomplished for example by collecting solar power kinetic energy or energy from ambient heat Thus in theory the device can operate without maintenance or any external electric power supplies or wires IoT can be defined as Things having identities and virtual personalities operating in smart spaces using intelligent interfaces to connect and communicate within social envi ronmental and user contexts EPoSS08 IoT is predicted to become reality in 20 years as miniaturisation progresses power consumption of devices decreases and devices be come more autonomous IoT does not only comprise devices inside a home but is every where around us outside in the car in the office and at home 2 9 Seamless Computing When a ubiquitous computing environment contains a multitude of services computers and UIs it allows users to access information anywhere in any form The concept of seam less computing Kawai04 implies that computers and networks work together without dif ficulty and can off
306. suitable sensor matrices Preconceptions and image factors combined with general mistrust of new technology can also be major obstacles to the wider public acceptance of smart homes Leppinen03 The smart home is sometimes perceived as an environment in which humans become passive and indolent their daily lives controlled and dominated by the devices that surround them Other people reject smart homes on the grounds that they will be made to look and feel stupid by allowing technology to take over their daily activities and chores Perhaps the very term Smart Home and its connotations need to be reviewed so they convey a more exciting and user empowering image This could be accomplished through smart home in terest groups and active promotion by the industry 10 2 Future Directions As with most technological innovations there is the inevitable chicken and egg problem in the early stages of development Initially the cost of new technology can be very high because of low demand and high development investment with the result that very few people can actually afford the technology Furthermore when a new standard or feature emerges it needs a lot of support services and applications or it will quickly disappear from the market People will not buy a smart home system that supports only a few sensors and devices and companies will not develop devices for a smart home system which nobody uses Thus we are faced with a situation where there are
307. switches and thus it became significantly larger and more complex than usual This would however allow flexible control of all electrics in the apartment turn off devices as needed and form logical groups 75 TUT Smart Home Research Figure 5 19 Bedroom Bathroom Sauna and kitchen in the Smart Home The network infrastructure was imported from the Living Room with some features being redesigned for the Smart Home but both RS 232 and RF networks were retained as the primary network types The only devices that were directly used from the Living Room were the IRemote and halogen dimmer The tabletop PC was replaced with a new central computer a shoebox sized Shuttle XPC which was installed in the bedroom The Living Room was devised for conducting further studies on user interfaces home net works and other smart home technology One key research item was network and device interoperability how could incompatible devices from different manufacturers be inter connected and used through the same home network Thus the goal of this research evolved into proving that with appropriate system architecture it is possible to connect de vices from different manufacturers to a common infrastructure Creating a home network either requires the re design of all devices or the creation of network converters and the Smart Home laboratory experiments with both ideas An affordable or cheap solution to the home networking problem was sought by im
308. t 6 1 Networks The vision of ubiquitous computing relies on a network infrastructure to facilitate commu nication between user interfaces sensors and surrounding devices The kind of network actually used does not really matter provided it is compatible with all devices There might never be a single universal network that would become a de facto standard for domestic use and instead it is more likely that the home network will continue to consist of multiple network types AldrichO3 Home networks are used for inter device communication communication with user interfaces and communication with the home server or gateway Networks that go out of the home are used for connecting to the Internet phone lines cable TV etc The following figure illustrates different communication networks related to homes Satellite GSM UMTS radio Home networks Ethernet Bluetooth RF IR HAVi HomePNA Zigbee xDSL cable telephone powerline Figure 6 2 Network connections inside and outside the home The network is crucial for transporting messages and data from one place to another but also for making it possible to query devices in the network This way devices and user in terfaces can learn the status of each other and update information when necessary For ex ample a user interface always needs the most recent information so it can display the status of all connected devices properly To make this possible the networks used must
309. t Introduction worked media homes but cases where a complete functional infrastructure along with UIs and controlling software was implemented were very rare Since the Personal Electronics Group has always emphasised prototyping and practicality this seemed a logical course of research The initial goals of the research were the creation of a relaxing smart space with interconnected devices and different kinds of user interfaces Later on plans were made to conduct long term real life studies with actual inhabitants in a normal everyday scenario TUT smart home research used the following research methods e Design and implementation of smart home hardware software and user interfaces e Installation of functional prototypes into the test environment e Implementation of test environments with a control system in order to obtain experi ences from using various user interfaces controlling the home and interfacing with devices e Conducting experience prototyping Buchenau00 user and technical tests in test environments e User tests conducted by a professional researcher in the eHome e Periodic iteration and improvement of the test environments 1 6 Research Question As consumption researcher Mika Pantzar has observed the technical development of do mestic technology can be divided into three phases Panzar00 The first of these roughly from the beginning of the 20 century until the 24 World War basically focused on tech
310. t made the home more homely Technology that contributes to domestic comfort and well being is more likely to be adopted than complicated technological gimmicks 102 TUT Smart Home Research 5 6 LIPS Learning and Interactions in Proactive Spaces 2007 gt LIPS is a project funded by the Academy of Finland that started in early 2007 Since the project is ongoing this chapter describes the current situation and future plans The project is based on knowledge from previous smart home projects the Smart Home and adaptive home control systems Valtonen06 The LIPS project is aimed at researching adaptive technologies environment modelling and user studies A proactive adaptive context aware home requires little or no input and configuration from its occupants This would lessen the burden on the users make it simpler to add or remove devices from the home and make daily life easier by adapting to users routines The LIPS project research plan sets out the following objectives e To study methods for modelling of the environment and algorithms to adapt the model and design a control system for a smart home test environment e To study interactions and context awareness in a home environment and to create unobtrusive and user friendly sensing methods and user interfaces for controlling and especially training the home environment e To set up a test environment for validating the results with functional and user tests The projec
311. t due to the simplistic nature of the setup the usefulness might be rather limited they might require assistance from the central node 143 Software Architecture The important element however is the artificial intelligence itself how it reacts to new situations what kind of functionality it offers and how expandable it is Artificial intelli gence in this case is a piece of software that monitors the situation in the smart home gath ers measurement and activity data and makes decisions based on the knowledge that it has at that particular moment The most basic form of home control software would be simple home automation which at most would have some user configurable settings timers group controls and preset modes However this would not fulfil the requirements for a smart home so more functionality is required A more advanced version would allow users to program more complex settings and possibly incorporate some kind of if then else type logic For example lights could be turned off in the night if nobody has been detected in the living room during the last 30 minutes This kind of programming would already al low reasonable flexibility and many smart home functions but it takes time if users have to manually program all possible variables into the computer There would also be many cases where this kind of reasoning would fail and produce unwanted results Adaptive home software would remove the programming burden fr
312. t feel like a normal part of the home This is not such a challenging task as it may at first appear since people gradually grow accustomed to electronic devices 25 Smart Homes When people no longer notice these devices or realise that they are actually using them the devices become ubiquitous and thus invisible to the user Normally newly built homes contain little or no smart home technology at all even if in stallation would simple to incorporate during construction of the building During the de sign phase it would be relatively easy to make changes to the infrastructure home networks and electrics but the customer is rarely consulted at this stage The later changes are made the more they will cost to implement TiresiasO08 and typically this happens af ter the customer has already moved in Inevitably home networks need cabling ducts equipment needs installation space and sensors require both Additional mains sockets and network connectors can also be very useful when more equipment is added The structures in the building itself can contain embedded sensors and other means for monitoring con ditions and events both outside and inside For example humidity and temperature sensors can alert users of imminent humidity damage and external temperature and light level measurements can notify the system of hot weather and that extra ventilation or air condi tioning will soon be required With such sensors it can become easie
313. t happens when people move from one smart home to another Already today there are differences between countries and cultures regarding the items that people consider to be fixtures and items which they take with them when they move house The situation becomes much more complicated when thinking about smart home technology There can be major investment in the net work infrastructure sensors and other fixed installations and if users take these with them spending considerable time on dismantling the system the home becomes a shell with no functional infrastructure even the electrical network can be dysfunctional On the other hand if every apartment in the entire block is fitted with a similar or common infrastruc ture there is little doubt of ownership To enable control of electrical appliances and lights in a home itis feasible to wire the elec trical cables using a star topology making it possible to control individual wall sockets and 26 Smart Homes lights The electrical switchboard can then be fitted with relays and a networked control ling apparatus such as LINET LINET This makes it possible to control lights and sock ets not only from wall switches but through a network and from multiple locations All lights can be controlled from a single location making it possible to group lights and create master on off controls for turning on or off all lights in the home Traditional electric con trol networks such as X10 X10
314. t hardware Technology allows work to be transferred from the workshop to the home office upstairs making it possible to work and live under the same roof The inHaus1 centre is used for demonstrations and technology development and for studying how different systems interact and how they are accepted by users Itis planned that selected residents will live in the laboratory for periods from one to three months after which user data and findings will be recorded To date inHaus1 has resulted in a number of commercial products that are being marketed by in Haus GmbH Completed in November 2008 inHaus2 features the latest technology in energy saving in telligent construction and building materials The 3500 m building is designed to be ver satile and flexible so that it can be modified for numerous purposes Planned applications include healthcare hotel rooms and offices with research scheduled until at least 2011 with a budget of 27 million euros 57 Related Work Figure 4 14 inHaus1 facility Photo inHaus Zentrum 4 3 Discussion Previous sections present only a handful of all smart home research projects that are being conducted around the world but they do represent the diversity and different areas of in terest that have emerged over the years These projects can be categorised into three sub categories university projects corporate projects and projects that have involved long term user testing University projects all
315. t is clear that earlier forecasts for a Ubiquitous Computing revolution have been over optimistic while we are surrounded by a vast array of computational devices these are still rather independent with only limited capacity to share information or resources among each other However computers are becoming more inconspicuous so that nowa days most of us are often oblivious of the role of embedded computers and other electron ics in our daily routines Computing has become so commonplace and familiar that we no longer notice it and if would be aware of the number of devices that include a processor we would certainly be surprised Predicting the future is closely related to the subject of smart homes and a popular ques tion asked by the media guests and students is when will such homes become common place Ten years ago we would probably have suggested ten years but unfortunately this reply is still true today It would seem that Osmo A Wiio s law stating that the near future is overestimated and the far future underestimated still applies Wiio78 despite the mar keting fanfare surrounding technological innovations Another popular quote regarding in 188 Analysis novations and the future is by Petri Kotro CEO of Valkeus Interactive He observes that engineers are almighty and it is possible to build almost everything but it is much harder to invent something that is truly usable and useful Smart home research has made
316. t will use the Smart Home laboratory for user testing and for setting up the re quired infrastructure A prototype network is to be constructed and devices from the Smart Home will be integrated using new network protocols 5 6 1 Context Awareness LIPS is being built around context recognition and context awareness The prototype in frastructure will be tested and verified with a set of pre defined contexts to determine the reliability of the recognition algorithm and the kinds of sensors that are required to accu rately detect these contexts In this case contexts are restricted to contexts of the user not those of the space In order to conduct tests and limit development time all contexts are designed for a single user scenario When there are several users inside the space it be comes all the more difficult to decide how to proceed This is also due to the lack of a re liable unobtrusive identification system that could be used for identifying and locating people inside the home The primary focus is thus on the requirements of context recogni tion and creating the necessary infrastructure Three different contexts were defined eating watching TV sleeping Eating is determined as an action that takes place in the dining room The context itself would be recognised by sensors detecting situations such as presence in the dining room the presence of food or drinks either hot or cold on the table or the user sitting on a chair at the tabl
317. tecture 7 4 1 Living Room Serial Protocol MOB Bus The first communication protocol used in the Living Room was designed to be light weight and it was targeted mainly at the infrared network but it was also used in wired network nodes The interface was a synchronous serial bus with separate clock and data lines The maximum clock signal rate was 50 kHz The data consisted of a header data pak byte followed by up to 15 data bytes The header consisted of a fixed start field 4 bits and a header which contained the number of data bytes that followed The following data byte would typically consist of a specific command to the recipient e g measure tem perature adjust lights and possibly other parameters e g measure temperature in the kitchen adjust living room lights The network was of master slave type with the master device being responsible for creating the clock signal and initiating data transfer After transferring 8 data bytes and one parity byte the slave device checks for possible parity er rors If none are found the next byte is transferred In case the slave device wants to transfer data it has to request this through the data bus The wired version had no addressing as it was considered a direct peer to peer connection but the infrared network did use addressing This was invisible to the devices themselves however with the infrared master and receiver managing this functionality The frame structur
318. ter viewing the TV can be on for playing console games etc In the case of an incorrect context the lights can be turned off in a bad situation or telephones can be inad vertently muted The concept of smart home upkeep also arises in the context of the reliability and service ability of modern electronic systems Reliability is an issue in terms of the overall reliabil ity of the smart home and its dependence on the reliability of its components The image of the weakest link determining the overall strength of the system also holds true for a smart home system Furthermore a smart home system should not be so intrinsically com plicated that it would require expertise to set up and use though inevitably there will be a need for such skill particularly during initial setup Later when the system is in use users should be able to add new devices and functionality by themselves There will probably be a need for third party service to update maintain and monitor the home system if the cus tomer prefers This kind of eJanitor service would remotely monitor the state of the home system provide users with information such as electricity consumption advice on energy saving statistics etc and user support This would allow users to ignore the under lying technology and concentrate instead on using the smart home system However an obvious drawback of such a service would be the costs involved and any delays in obtain ing the service in the ev
319. th the TinyOS operating system allowing similar development environments regardless of hardware configuration The technical implementation is dif ferent between types of motes for example CPUs used are Atmel s AtMega series or Tex as Instruments MSP430 series sensors accelerometers light temperature and humidity etc and RF transceiver 802 15 4 compliant 868 MHz or 433 MHz proprietary radios Figure 6 13 tmote SKY also known as mote IV module Scatterweb Scatterweb Scatterweb is another solution for sensor networks and a scatternode is the equivalent of the mote platform It also uses the MSP430 microcontroller an 868 MHz ra dio an A D converter and 512 kB EEPROM memory The mainboard can be augmented with separate sensor and power supply modules if needed Multiple scatternodes can form a multi hop scatterweb network that is ideal for sensor networks or similar applications Scatterweb is intended for use in sensor applications energy management and structural health monitoring The Scatterweb software also includes drivers for connecting nodes to other kinds of networks e g Zigbee KNX 136 Hardware Aspects Figure 6 14 Scatternode Smart Its The Smart Its project Kasten01 is part of the European Disappearing Computer initiative and it aims to create small scale embedded devices that can be embedded into everyday objects enabling new kinds of sensing computation and communication Five prototype ha
320. the eHome The eHome project was a unique study that set out to discover how a smart home would function in everyday life and what kind of functionality it should and could offer its users This practical study produced the kind of results that could not otherwise be gained from theoretical laboratory tests and it also taught the research team much about the practical issues around smart homes It seems that the most desirable functions of a smart home are indeed the various user interfaces that allow greater flexible control of the home without increasing the amount of technology and complexity that lead to concerns over loss of con trol The home should also contain learning and adaptive functions which would gradually allow users to transfer certain functions of the home to the home controller This paragraph summarises the major findings of the eHome project with the exception of private or sen sitive information and the omission of unpublished data By far the most valuable experiences from the eHome were those gained from the reports and experiences of the tenants during their three year occupation they provided the re search group with a great deal of material to study and process When the project was con cluded and the eHome dismantled in 2005 user feedback was collected and analysed Koskela04_1 It was interesting to see how living in the eHome had changed the tenants daily routines how visitors to the eHome experienced it and how t
321. thered is very large and of a sensitive nature Tapping into wired networks requires physical access but wireless networks leak outside the home and thus have to be adequately protected against outside intrusion Strong security measures such as biometric identification should also be considered for remote access to home controls 114 Hardware Aspects Bi directionality This is not a problem with most networks but there are still some leg acy or wireless networks that work only in one direction This is a problem for smart home applications since commands cannot be verified and the status of the device cannot be en quired Bi directionality is also required for updating the software of devices through the network Networks that are designed for industrial and larger scale applications for example LON CAN bus etc might be too expensive and unnecessarily complex for domestic use For tunately simple devices such as temperature sensors and light controllers only send a few bytes at a time and have no need for high speed megabit range networks Simple low cost networks can be used keeping installation costs and energy consumption low User inter faces and more complicated devices on the other hand benefit from low latency higher speed networks Latency is critical especially with user interfaces as delays can be annoy ing and confusing in terms of usability and reliability Some devices produce data that has to be continuously
322. tion interfaces and seamless interconnectivity make it possi ble to use multiple locations for controlling a single device and group controls make it possible to control of several devices at once Customisable menus further enhance usabil ity and make it possible to create personalised UIs for each user group controls and save personal preferences Multimodal UIs allow interaction between the user and the system to be more natural while also providing a variety of feedback from the system for the user When new devices are introduced they are made easier to use because they can be control led from familiar UIs Zero configuration capabilities allow them to be easily connected to the home network with little effort from the user 33 Smart Homes Safety People sometimes leave home having forgotten to turn off an appliance and then must re turn home or call another family member to rectify the situation Such situations not only increase energy consumption but can be hazardous if they involve the risk of fire or other danger A smart home with remote control facilities makes it easy to control lighting and monitor home appliances from a remote location Similarly it would be possible to turn off electric outlets in the nursery and disable control of possibly dangerous home applianc es e g the kitchen stove when children are present Energy Savings Major topics of current interest in domestic living concern energy saving and monitorin
323. to create tasks that complete within a specific time frame creation of multiple tasks and implementing task scheduling Such op erating systems naturally consume more resources than a customised software written for a specific purpose However when more tasks are added and if multiple similar applica tions are implemented it makes sense to move to a common expandable platform upon which new components can easily be implemented The alternatives are other non real time operating systems such as the TinyOS mentioned earlier Fig 7 6 shows the flow of a typical Smart Home device software After initialisation the software loops and waits for data to be received After reception the software checks the received data performs a measurement or adjustment and sends a status report and ac knowledgement to the server 157 Software Architecture Initialisation Data received Send status report Receive amp process data Status request Action requested Perform Send back measurement acknowledgment adjustment Figure 7 6 Functionality of the software of a simple smart home device 7 6 Discussion Requirements for proper smart home software seem immense it has to be adaptive mod ular stable and intelligent Thus a great deal of care has to be taken in the design and test ing process in order to ensure a reliable and usable home control platform Standardised middleware platforms such as OSGi make the design process
324. to result in the emergence of middle class citizens From 1920 1940 mass produced standardised home appliances were becoming common place and the 1933 Chicago world s fair introduced Mankind to new stunning visions of new home appliances and all possibilities that they had to offer Electricity was becoming commonplace although it was still primarily used for lighting The development was seen as a process whereby science contributes to new inventions that industry brings to the mar ket The user s job was simply to start to use these inventions and to adapt to their require ments The new home appliances actually created more chores than earlier since these were now easier and faster to do This resulted in clothes being washed more frequently and the house being cleaned more often After the war women had grown accustomed to work in industry and now formed an im portant part of the workforce Media propaganda tried to persuade women to stay at home again by running traditional housewife campaigns and promoting family ideals The mod ern kitchen started to evolve with design focusing on accommodating all the new home appliances on the market Refrigerators washing machines vacuum cleaners and electric stoves were all making their breakthrough The late 1950s were however plagued by pes simism and changed attitudes The flood of new inventions like jet airplanes cars televi sion and nuclear energy made people believe that all signific
325. tocol for smart home unreliable hardware and soft ware Smart Home e Network and e The Smart Home e Distributed UIs were useful Home middleware is essential More user testing is required Smart Home e New UIs eHome e Getting results e eHome UIs e Project was very from real life e Usability test educational usage reports e Adaptive control e Conducting user e Three years of software was studies usage data requested e Multiple UIs e Experiences used e System stability e Security for users to improve the and reliability could be improved 9 5 Summary Based on these experiences it can be concluded that connecting home appliances and de vices together can achieve new levels of control and communication Combining different kinds of networks and protocols can be laborious and may give rise to new problems but the final outcome will be worth wile when the full potential of the networked space is re alised The result will be innovative and easy ways to control smart spaces homes and de vices A centralised gateway that binds various network technologies together will enable the building of effective and flexible intelligent environments even if the devices and net works themselves are incompatible While awaiting the advent of such a universal connec tion method we need to adapt existing solutions and use them to their full potential However since there are no compatible devices on the market it
326. tous computing According to Weiser computers were to be available everywhere but would be invisible to the users until actually needed They would share information processing power and user interfaces thus enabling computing to take place anywhere anytime while remaining unobtrusive Virtual Reality VR where people are put inside a virtual compu ter generated environment is in some respect the opposite of Ubiquitous Computing where computers are forced to exist in a real environment Alan Dearle has taken the the ory of Ubiquitous Computing even further to include the whole world with the concept of Global Smart Spaces Dearle03 Global Smart Spaces would support location aware in teraction between people devices and places on a global scale Sales year Mainframe era e PCera Ubiquitous Computing era Time Figure 2 1 Graph of the evolution of computing showing the three separate eras Weiser91 A more recent study Tolmie02 however suggests that invisibility might be purely cogni tive and that the term relates more to how we perceive and use computers Once users have grown used to having computers and gadgets around them they become accustomed to them and no longer think of them as separate entities but rather as user interfaces and ac cess points for information Thus far it seems that ubiquitous computing has only advanced to the point of one user and many computers Current household items
327. tp www memsnet org J Mikkonen L Kaila J Vanhala Living Room project Towards User Activation in Proceedings of the OZCHI 2000 Conference Interfacing Reality in the New Millenium Sydney Australia 2000 pp 109 110 References Mikkonen06 MIT03 moteIV Mozer05 Mozer99 MS07 Myllym ki03 M kel 08 M kinen03 M yr 05 Nabaztag06 Nagel04 J Mikkonen RF network for smart environments Master of sci ence thesis Tampere University of Technology Electrical Engineer ing Department Tampere Finland 2006 Massachusetts Institute of Technology PlaceLab A House_n TIAX Initiative 2003 available at http architecture mit edu house_n placelab html 5 pages moteIV mote platform 2007 available at http www moteiv com M C Mozer Lessons from an adaptive house inSmart environ ments Technologies protocols and applications D Cook R Das eds Hoboken NJ J Wiley amp Sons 2001 pp 273 294 M C Mozer An intelligent environment must be adaptive IEEE Intelligent Systems and their Applications Vol 14 1999 pp 11 13 R Harper T Rodden Y Rogers and A Sellen Being human hu man computer interaction in the year 2020 Microsoft Research 2007 51 pages P Myllym ki Bluetooth in home environment Master of science thesis Tampere University of Technology Electrical Engineering Department
328. tructure An ad hoc net work can be formed between only two devices point to point or between multiple devices either static or mobile In a centralised network functionality is also usually concentrated in the centre node In a distributed network functionality and resources are scattered throughout the network and any node can take over if another node fails This type of network is much more fault tol erant than a centralised network and it will continue to function as long as a communica tion path exists between nodes However a distributed network vastly increases the complexity of devices and protocols because each node has to keep a record of nearby nodes routing tables their capabilities and location Each device must also be able to identify itself to anyone on request in order for the network to know what devices are present and what functions they offer User interfaces can connect directly to any node al so possible in centralised networks in a limited fashion if service advertisements are not implemented and they can communicate directly with a node or request a command to be sent elsewhere in the network In practice distributed networks require the use of some form of ad hoc networking which allows nodes to freely join and leave the network with out the need for further reconfiguration User Interface T User Interface a Figure 6 5 Centralised vs distributed network topologies 123 Hardware Aspects 6
329. ttman Service Location Protocol Automatic Discovery of IP Network Services IEEE Internet Computing Vol 3 Issue 4 1999 pp 71 80 U Hansmann L Merk M S Nicklous T Stober Pervasive com puting the mobile world 2nd edition Springer Verlag 2003 432 pages Home Audio Video Interoperability 2009 available at http www havi org S Helal W Mann H El Zabadani J King Y Kaddoura E Jansen The Gator Tech Smart House A Programmable Pervasive Space in IEEE Computer 2005 pp 50 60 J Hightower A LaMarca I E Smith Practical Lessons from Place Lab IEEE Pervasive Computing Volume 5 Issue 3 2006 pp 32 39 M Himanen Mik tekee kodista lykk n What makes the home smart in lyk s kaluste kaluste lykk ss tilassa Smart furniture furniture in a smart space J Ahola T Holm lund S Torkki eds Helsinki Finland 2003 pp 7 17 HIPERLAN 2 Global Forum 1999 available at http www hiperlan2 com HomePlug powerline alliance 2009 available at http www home plug org home Home RF Resource Center palo wireless 2003 available at http www palowireless com homerf HomeSeer PRO series products 2009 available at http www homeseer com products pro_series index htm HomeSoft lyk s talotekniikka ratkaisukuvaus Smart building technology product description 2004 25 pages J I Hong J A
330. tus of the microwave oven standby or in use with a current consumption sensor Sometimes the lack of a network connection does not matter for ex ample remote control of the kitchen stove would serve little purpose as someone has to be present when cooking anyway and modern stoves already have automatic safety timers However in this case monitoring the temperature around the stove and movements of the person can prove more valuable information Networks in the Smart Home have allowed devices to be connected either wirelessly or with wired networks using a standardised connector Even if the device and network con figuration can be changed rather freely the major challenge is reconfiguring the Home Controller and especially UIs Adding a new device requires an XML description and pos sibly a rewrite of all UIs since certain new UI elements might be needed Thus modifying the functions UIs and Home Controller software requires some amount of manual work making the process cumbersome and slow A dynamic self configuring UI and a modular plug and play protocol would be required for a fully functional and adaptive commercial smart home network Further development is also needed of protocols and the way that new devices are introduced into the environment It is difficult to make flexible user inter faces for every device especially if they can be removed and added frequently which is why development on a service discovery protocol was started in
331. ucapccasscheseucsavapesassceas as SaSao 174 9 1 Findings From the Living ROOM 0 0 eee eeseeseceseeseceeeeeeeeeeeees 174 9 2 Findings From the Smart Home 2000 eee ee eeeseeeeceseeeeceeeeeeeeeeeees 175 9 3 Findings From the eHome 000 0 eee eeeeesee ce crseeeceseeeceseeeeeeeeeees 177 QA DISCUSSION Seinnear EE R EEN E E O E AEE 181 QS SUM EET p TE AE E E T T TETE 183 ECTE KE C A EA E EES T EEEN AA 184 10 1 Reasons Why Smart Homes Still Are Almost Nonexistent 184 10 2 F t re DireCtiOns renesansi aineita na a ei ETE eee 185 10 3 Final Thoughts venre neea iii EE EEA NEE 187 TOA CONCIUSION verie e e eR E E E GSE 188 References ccsccsssiesiasescsscecosssscosccvaiesescesacsesecesssolesscetsestecdsesseccesdsesssessessenssestendens 190 viii List of Abbreviations List of Abbreviations AC ADC A D AI AmI API ASCII BAN bps CAN CPU dBm DC DLNA DSL DVD ECG EEPROM EMC EPIS GPRS GPS GSM HCI HDTV HTML HTPC HomePNA HVAC I O I2C PC IC ICT IEEE IP IR IrDA ISM ISTAG IT IP ISO LAN Alternating Current Analogue to Digital Converter Artificial Intelligence Ambient Intelligence Application Programming Interface American Standard Code for Information Interchange Body Area Network Bits per second Controller Area Network Central Processing Unit Power ratio in decibels dB referenced to one milliwatt mW Direct Current Digital Living Network Alliance Digital Subscriber Lin
332. uld somehow cause them discomfort Others were enthusiastic and eagerly tried out the different functions and UIs The mobile phone UI was considered cool by friends and co workers and it was always demonstrated to people who were cu rious Guests who stayed overnight quickly grew accustomed to the light controls but young children were given a night lamp to prevent them accidentally turning on all lights in the house in the night Obviously a need existed for a more intuitive interface for light ing control although people commonly need time before learning which switches control particular lights when entering any unfamiliar space such as a hotel room At the beginning of the testing period the tenants had understandable reservations about the system and its reliability It took several weeks for them to get used to using the devices and learning how to operate the various user interfaces New UIs and functionality were gradually introduced to the eHome under controlled conditions Whenever a new device or function was introduced the same learning process was repeated and user feedback was collected However when there were problems it could take some time for the tenants to regain trust in the system For example when timer settings and user profiles were lost dur ing an update of the Home Controller software there was a considerable delay before the 179 Findings tenants started using it again There were also other ways in whic
333. ural UIs computerised UIs and mobile UIs although the distinction is not always clear and sometimes an UI can belong to several categories VTT03 Natural UIs i e traditional ways of using devices and equipment such as buttons levers handles etc offer users a tangible UI and a physical way of interacting with equipment VTT03 The name derives partly from the way that equipment has been used for gener ations For example a hammer a door handle and a vacuum cleaner have remained rela tively unchanged through the years and their use is generally well known Their UIs have evolved to a point where the most logical and suitable designs have become de facto stand ards Light switches and home appliance controls have also remained mostly unchanged but there are also instances where these have been radically changed to offer new ways of control Completely changing the way of controlling everyday items might lead to confu sion and frustration among users at least until users are more familiar with the technology On the other hand changes are required if progress is to be made Computerised UIs such as graphical UIs and touch screens depend on some form of infra structure such as displays buttons or computers If designed properly these are easy for people to use without recourse to user manuals but less visual UIs buttons panels etc can be less intuitive Mobile UIs are usually located in personal portable devices and they can t
334. user forget The flower pot monitor was designed to monitor the health of the plant by measuring the water level in the tank beneath the flower pot with two strips of wire measuring the resistance between them The purpose was to alert the user if the flower became dry or had too much water Depending on the potted plant a suitable schedule and profile could be used to ensure optimum health and growth Users could monitor the plant s condition and measurement graphs from the graphical UI running on the PC 68 TUT Smart Home Research Figure 5 11 Flower pot sensor Motorised Curtain The artificial balcony in the Living Room had a large window with two bright lights be hind to simulate daylight The artificial sun rose in the morning becoming gradually lighter and set again in the afternoon The quality and amount of ambient light makes users subconsciously aware of the time of day and can also be used to set a suitable atmos phere in the apartment A motorised curtain was installed to control the amount of light entering the room The curtain motor was connected to a controller card which was con trollable from the tabletop PC Infrared Tags Identifying the user is a primary requirement for personalisation and customisation in smart environments When the user is identified the information can be used for access control personal greetings etc In the Living Room coin sized infrared tags were used for identifying user
335. uted over a specific area This might be a factory building with environmental or process relat ed sensors Connolly05 a forest with humidity and temperature sensors mounted onto trees or a glacier with GPS modules that measure its movements Martinez05 Sensor net works such as TUT wireless sensor network TUTWSN could also be used for domestic applications In general sensor networks utilise small battery powered sensor modules with some processing power an energy module and a radio transceiver to allow them to communicate with each other or a main network node In order to keep power require ments and complexity low the range of the transceiver might not be more than ten metres As described earlier in order to cover greater ground sensor nodes can send messages through each other with the message hopping from one node to another until it arrives at its destination This multi hop network topology enables network nodes to communicate with each other without the need for a rigid network infrastructure either wired or wire less Nodes can communicate directly with each other or with a remote node using other nodes as relays The network protocol becomes rather complicated as it has to be able to find an optimal route for the message taking into account the complex and non fixed structure of the ad hoc wireless network In addition since nodes are fairly independent and there is no central node that governs traffic it has to be possible to
336. uting in the future changes from a one user one computer scenario where users are implicitly sitting in front of the computer and using it to a pervasive computing space with embedded computers everywhere and multiple users in the space our perception of computers and the way we use them will completely change The physical location of the computer becomes irrelevant but the identity and location of the user again becomes sig nificant A Mediated Space also requires sensors for recording and understanding speech gestures user identification and data analysis The last requirement is fundamental and it practically means that a Mediated Space has to understand what computations the comput ers are performing in order to understand what people in the space are saying doing or pointing at This again implies that the space must also be aware of what devices the space currently contains their capabilities properties and also their orientation All these requirements make it extremely difficult to implement a Mediated Space using today s technology Making computers understand speech and gestures locating and iden tifying devices and people who can freely move around in a space and understanding all interactions between these is something that current technology is not able to do at least unobtrusively and in real time 2 7 Intelligent Environments Intelligent Environments is a more general term stemming from interactive environments by bring
337. ve and describe quantities In this case 15 C would be rather cool 22 9 C warm and 32 C rather hot Some sensors on the other hand only report on off type binary information which is relatively simple to use directly in user in terfaces for example A magnetic door sensor can detect if the door is open or closed a passive infrared sensor can detect whether people are present or not and so on Sensors that measure continuously or periodically gather large amounts of data in a short while This data can be analysed to detect patterns repeated events or other information of importance Pattern recognition is thus an important factor in machine learning and data categorisation Pattern recognition is used in speech processing medical applications and image analysis Data collected from all sensors in a smart home can rapidly use consider 129 Hardware Aspects able amounts of memory capacity amounting to thousands of megabytes When such amounts of data are being processed on a computer transformation to another form might be required to make processing easier and faster A paper by M Galushka Galushka06 describes methods of transforming and analysing data recorded from smart environments This approach is important especially with context aware applications where data is con stantly being collected analysed and stored 6 7 Sensor Networks The concept of sensor networks usually covers large concentrations of sensors distrib
338. ve can only be controlled by the parents and balcony doors are not left open and certain functions are not accessible from the home UIs For elderly people and the handicapped smart homes can enable them to enjoy life at home without requiring special assistance The benefits of new UIs in this area are particularly apparent because users may be inca pable of using all appliances from their own UIs reaching switches or reading small font text Reminders auditory and visual UIs can bring the functions of the home close to hand 31 Smart Homes for such users In general control interfaces should be close to the location where they are needed a sophisticated control tablet mounted on the hallway wall will probably only be used when people are entering and exiting the house in other cases its accessability can be a hindering factor Another important issue is the difference between solutions that are use ful in a work environment versus those that are useful at home Something useful in the office may not be so useful in the kitchen or at home with children present Aldrich03 Unfortunately most computing equipment and the way we interact with computers are de signed and optimised for office use Consistency and predictability are also important factors for ensuring smart home owners feel secure and relaxed Users should be able to anticipate the actions and functions of their homes so that they feel comfortable with the various events an
339. ve computer systems are restricted so they operate at the same performance level as humans but when this limitation is removed the computer system can 14 Definitions freely be let to run ahead of the user anticipating and predicting possible near term ac tions and enabling faster response times New challenges arise when UIs for proactive fast er than human systems are developed and according to the author computer science should be directed towards the physical world and everyday reality 2 6 Mediated Spaces A Mediated Space is a kind of a smart space that understands and participates in multiper son interaction Mark99 In other words it differs from other points of view by concen trating not only on human computer interaction but also on the interaction between people The Mediated Space is constantly gathering information about the users and from the out side world and it is monitoring the users and analyses their contexts for example through speech and gestures For example a mediated office room would assist a group of workers in carrying out their project work by providing relevant information and proactively pro viding data that could be useful during the workflow A Mediated Space requires both be hind the scenes technology that coordinates activities and exchanges data between devices and at the interface technology that relates to how people perceive the space and how multiperson interaction is managed As comp
340. vironmental control Remote controls Interoperability resource sharing Figure 2 3 Classification into building automation home automation and smart homes Where home automation concerns computers and devices that contain information tech nology building automation is more related to the physical building e g building man agement security monitoring and environmental control lighting HVAC controls A home automation system would for example turn down the heat inside the house when users leave the home for work but it would not be able to adapt to the users routines or predict what time people come home A smart system would learn that users come later than normal on the first Wednesday of each month because they go to visit their grandpar ents for example and it would also take into account the variations in outside temperature 21 Definitions and the effect of sunlight Thus the goals of home automation and smart homes can be considered to be the same at a basic level making life easier and more comfortable How ever the number of tasks the automated home is able to perform and their impact are in ferior to those of a smart home Furthermore an automated home brings with it new controls settings and functions that users need to learn whereas a smart home aims at making control of the home easier and simpler Most homes already contain some form of automation such as lamp timers that turn on lights when it
341. were interesting and the concept was developed to include a fingerprint scanner and motorised door The front door to the Smart Home was thus motorised and fitted with an electric lock The door could be opened in the conventional manner with a key or from the inside by pressing a button A door controller unit was also constructed to allow the door to be opened remotely through the home net work and a USB fingerprint scanner was installed to control access to the room The scan ner which is physically located next to the front door on the outside is connected to a PC in the Smart Home which runs fingerprint scanner software The database contains fin gerprints of all persons that are authorised to access to the room and once a positive match is made the software sends a signal to the server software to open the door The fingerprint software also sends out a text string containing the person s name which the server outputs to the speaker outside the door The individual person can then hear a personalised wel come message on entering the room such as Welcome Lasse it s a nice day today Figure 5 24 Fingerprint scanner near the front door of the Smart Home Controllable Lights Lighting was constructed using halogen spots in a similar way to the Living Room The Smart Home is equipped with 45 bright 50 Watt halogen lights that can be individually controlled Currently however the lights are grouped according to the rooms they
342. wireless transceiver mod ules and this address is used in a similar way as with wired devices Serial devices RF at Master gt RF slaves ttt 4 1 2 14 gt Serial HUB Figure 5 28 RS 232 network hub used in the Smart Home Wired devices have addresses 1 14 wire less devices from 20 onwards x When the server needs to send a command to a device it opens the communications port to the hub transmits a command and device address and waits for a reply The hub redi rects this command to the appropriate port waits for a reply and relays this data back to the PC This solution makes it impossible for a slave device to open a connection to the server instead all connection attempts have to be opened from the server side The wireless alternative to the wired serial network uses matchbox sized transceiver mod ules and a master unit to replace cumbersome serial cables Wireless RF is used whenever cabling is too difficult to implement the distance too great or if greater flexibility or mo bility is required The wireless transceiver module plugs directly into the serial port of any smart home device and connects to the Smart Home network via the master module The transceiver consists of an RF board with a single chip transceiver and PCB antenna and a microcontroller board with external interfaces The nRF401 radio module has sufficient range to cover the apartment without the need for external antennas The transmitted R
343. ww irda org Report from the Information Society Technologies Advisory Group ISTAG Scenarios for Ambient Intelligence in 2010 K Ducatel M Bogdanowicz F Scapolo J Leijten amp J C Burgelman eds 2001 available at http cordis europa eu ist istag reports htm Report from the Information Society Technologies Advisory Group ISTAG Ambient Intelligence from vision to reality 2003 available at http cordis europa eu ist istag reports htm Report from the Information Society Technologies Advisory Group ISTAG Shaping Europe s future through ICT 2006 available at http cordis europa eu ist istag reports htm W S Jeong B S Cho P R Kim An Analysis of the Economic Ef fects for Launching the Ubiquitous City in proceedings of Portland International Center for Management of Engineering and Technolo gy PICMET 2007 Portland Oregon USA 2007 pp 1154 1159 195 References Jini Jojic00 Kaila01 Kaila05_1 Kaila05_2 Kaila05_3 Kaila05_4 Kaila07 Kaila08 Kaila09 196 Jini org the community resource for Jini technology 2009 availa ble at http www jini org N Jojic B Brumitt B Meyers S Harris T Huang Detection and Estimation of Pointing Gestures in Dense Disparity Maps in Pro ceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000 8 pages L Kaila J Mikkonen P
344. xample of the dialogue between the user and SCARS could be a case where the user controls the lights in the living room The user commands living room lights on to which the computer responds living room lights turned on and takes the appropriate ac tion in reality both commands and audio feedback are in Finnish 81 TUT Smart Home Research Figure 5 25 SCARS microphones and charging station top left ceiling mounted speaker and infra red diodes bottom left and receiver unit right Hyv nen03 Location Sensors As with other sensors the floor sensors in the Living Room proved that this kind of infor mation is highly relevant to smart home applications Since the Smart Home had remova ble floor tiles it was possible to test different ways of measuring movements and presence in the apartment The sensors used in the Living Room test apartment were EMFi film sen sors but using these throughout the Smart Home would have been extremely expensive Thus alternatives were sought and first test were made with strain gauges one placed be neath each corner of a floor tile Strain gauges are conductors whose resistance changes as their physical dimensions change e g when they are stretched These sensors were able to detect when a person was stepping onto a tile but covering the whole apartment with strain gauge sensors would have been a major undertaking involving a lot of wiring and sensors Koskinen03 Other altern
345. xperiences Press release 2001 pp 5 6 available at http www futurelife ch V Sundramoorthy H Scholten P Jansen P Hartel Service dis covery at home University of Twente Publications 2003 5 pages References Sundramoorthy06 V Sundramoorthy P Hartel J Scholten On consistency mainte Sunspot Tannenbaum02 TATU04 Tennenhouse00 Tieranta05 TinyOS Tiresias08 Tolmie02 TUTOO TUTWSN Tuulari05 nance in service discovery 20th International Parallel and Distrib uted Processing Symposium 2006 15 pages SUN small programmable object technology SUN microsystems 2009 available at http www sunspotworld com A S Tannenbaum Computer Networks 4th ed Prentice Hall USA 2002 912 pages Tekniikan amp arjen tutkimus TATU Research on technology and everyday life lyk s koti piloteista massatuotteeksi The smart home from pilot to mass product Tampere University of Technology 2004 96 pages D Tennenhouse Proactive computing in Communications of the ACM No 5 Vol 43 2000 pp 43 50 T Tieranta J Hyv nen L Kaila amp J Vanhala Inductive data link for body area network applications in Proceedings of IASTED In ternational Conference on Networs and Communication Systems NCS 2005 Krabi Thailand 2005 TinyOS an open source OS for the networked sensor regime 2009 available at
346. y digital information New innovations in the lab are 53 Related Work structures created inside the Open Source Building Alliance OSBA OSBA which in clude easily deployable chassis beams that have ductwork plumbing power and signal ca bling and mechanical attachments integrated in the structure Such structures make it easier and faster to deploy and modify future homes The House_n project is managed by the Department of Architecture at MIT Figure 4 11 PlaceLab and an OSBA prototype element Photos MIT 4 2 12 MIT aire Another project at MIT is the aire Agent based Intelligent Reactive Environments AIRE which was devised to investigate the design of pervasive computing systems and applications The aire project includes topics such as augmented spaces speech recogni tion perceptual sensing and distributed agents During the course of the project several aire spaces or intelligent environments have been designed ranging from pocket sized computers to large conference rooms A variety of applications has been built around these environments for example different kinds of visualisations input methods and contextual reasoning The aire project forms part of MIT s pervasive computing project Oxygen Oxygen and there are currently two laboratories used for research into pervasive technologies E21 is a conference room equipped with ubiquitous technologies such as walls that function as both white boards a
347. yards Active sensors such as those using radar Zakrzewski05 or ultrasound Makel408 trans mit signals into the space and measure the strength of reflected signals from objects or people in the space Reflected signal levels can be quite low and signal processing is re quired in order to detect people or movement in the space and some kind of scanning is also required if a larger area is to be monitored Ultrasound sensors are commonly used in burglar alarms and in medical applications Video cameras are also a viable alternative since standard webcams and miniature video cameras have become cheap enough to be placed anywhere By processing the video signal it is possible to detect people and movement in the space but this requires even more processing power and it is heavily dependent on ambient lighting and similar conditions However a properly set up camera system could optimally identify users gestures move ment and other actions Other novel methods have also been researched ranging from powerline positioning the system uses tone generators that transmit a signal into the powerlines and tags that locate these signals to monitoring pressure changes in HVAC systems when people move about or doors are closed Patel08 Another use of location information is related to devices themselves there are situations where the location of the device is important even if it is not mobile or moveable Context aware applications for exampl
348. ype has to be changed A wired device can be changed to using wireless RF or Bluetooth simply by removing the serial cable and attach ing a wireless transceiver module into the connector instead Addressing is implemented by mapping the physical port on the serial hub that it is plugged into to a hardware address wired devices or from the ID code on the wireless transceiver module wireless RF de vices As well as the serial port additional communication interfaces or I O ports may be re quired to control external equipment such as sensors and relays Some devices also have a kind of a user interface to allow manual control for example with motorised blinds and projector screens each which has pushbuttons for manual control Sensors typically use I2C RS 232 or SPI buses Fig 6 11 below shows the structure of a typical device in the Smart Home Sensors Drivers relays transistors ATmega8 8 bit RJ 12 microcontroller connector Power regulation Displays supply buttons LEDs Figure 6 11 Block diagram of a typical smart home device Devices in smart home projects at TUT have undergone several years of development and evolution The first device created a flower pot monitor used a simple 8 bit Atmel AT90S1200 microcontroller with 1 kB of flash memory Programming was done using Assembly language and the components used were all through hole mounted A little later the programming language was changed from assembly to C
349. ystem state of the art entertainment with audio and video avail able in every room motorised locks and lighting control It was designed to function not only as a showcase but as a living laboratory with periodic user experiments Families were interviewed before during and after their stay in the smart house The smart house contained seven different UIs including wall panels PDAs voice commands mobile phones and computers The user groups ranged from people who were familiar with tech nology to those with little experience The aim of the project was to gather information on how useful and usable users found the technology with the ultimate goal being the launch of such technology on the global market Results from the Orange at Home project Randall03 are interesting and there are many similarities with the eHome project presented in Chapter 5 Users in the orange house liked the ability to remotely control the home and the feeling of being in control was also highly rated Sometimes however the feeling or illusion of being in control and the feel ing of losing control can sometimes overlap depending on the design of the UI Users had reservations about being monitored it was considered intrusive and there were also ques tions concerning control priorities who is allowed to override the adjustments made by others and how will personalised settings be affected if someone else is using the system One significant result from the

Smart Home - Tampereen teknillinen yliopisto

Contents

Download Pdf Manuals

Related Search

Related Contents