Evolution of Smart Homes

I. Introduction

Smart homes, the next gigantic leap in the field of home automation, have become an emerging research field in last few decades. Research on smart homes has been gradually moving towards application of ubiquitous computing, tackling issues on device heterogeneity and interoperability. A smart home adjusts its function to the inhabitant's need according to the information it collects from inhabitants, the computation system and the context [1].

By 2050, approximately 20% of the world population will be at least 60 years old [2]. This age group is more likely to suffer from long-term chronic diseases and will face difficulties in living independently. According to World Health Organization (WHO), 650 million people live with disabilities around the world [3].The most common causes of disability include chronic diseases such as diabetes, cardiovascular disease and cancer; injuries due to road traffic crashes, conflicts, falls, landmines, mental impairments, birth defects, malnutrition, HIV/AIDS and other communicable diseases. It is not possible and logical to support all these patients in the medical center or nursing homes for an uncertain period of time. The solution is to accommodate health care services and assistive technologies in their home environment which is the main objective of smart homes.

Sensors, multimedia devices and physiological equipments are core components to perceive information from home environment Infrared (IR) sensors, pressure sensors, magnetic contacts, passive and active Radio Frequency Identification (RFID) tags are used to track inhabitant location detection. Electrocardiogram (ECG), photoplethysmograph(PPG), ,temperature, spirometry, galvanic skin response, colorimetry and pulse measurement equipments are used to get physiological information from the patient. Camera and microphones provide audiovisual response from home user. Inhabitant can access the system through display panel.

Power line communication protocols are widely used for the connectivity of home appliances. Public telecommunication network with voice and text messaging service is involved to provide telecare facility from remote location. Videoconferencing is used as an interactive communication media between caregiver and the client. TCP/IP protocols of Ethernet network provide data connectivity for local and remote sites and locations. Ethernet protocols are also used to connect health-monitoring equipments and to provide data repository service.

Algorithms from machine learning, data compression, statistics and artificial intelligent are employed to predict user behavior, detect activities of daily life (ADL) and location. C4.5 algorithm from machine learning is utilized to build spatiotemporal context of user. C4.5 algorithm is developed by Quinlan in 1993 which classify the data to construct a decision tree according to data attributes [47].Active LeZi from data compression algorithms is used to predict inhabitant's next behavior. Active LeZi by Gopalratnam et al..in 2007 builds a decision tree utilizing similar methodology of LZ78 data compression algorithm and predict next event using Prediction by Partial Matching (PPM) algorithm[22].Statistical predictive algorithms like Bayesian filtering, dynamic Bayesian network algorithms classify the information and recognize ADL of home client[34][41][44]. Different flavors of AI algorithms extended for smart home data processing. Markov model, Hidden Markov model, Artificial Neural Network can detect the living pattern of user and can also predict the user [7][13][14][38]. Fuzzy Logic is used for home appliance control [36].

Smart home is mainly dedicated to provide health care, safety, security and monitoring service for patient and elderly. The house is equipped with sensors, cameras to track people and can trigger an alarm to a remote heath care service provider in the case of emergency. Sophisticated physiological devices monitor heart rate, blood pressure, body temperature, ECG record and the patient is being observed from a distance location. Telecommunication service is used for communicating with service provider, relatives or neighbor and as a redundant acknowledgement method from the patient. For home comfort system, lighting, heating, doors, windows and home appliances are automatically controlled by ambient intelligence of smart home. Smart home also has significant contribution towards energy conservation by integration of energy meter with smart home [4].

Home automation is the initial state of smart home where electronic technologies are used to provide an easy access to household devices. Rapid development of sensor technology accelerated the growth of smart home that involved more data processing. Improvement of information and communication technology make possible to develop easy and cost effective methods for data repository and exchange. Smart home is a growing concept, efficient and lower cost solutions for general people are the main idea to promote it.

II. Smart Home defination

Smart home is an extension of modern electronic, information and communication technologies. The main objective of smart home research is to provide smartness to a dwelling facility for comfort, healthcare, security and energy conservation. Remote monitoring system is a common component of health smart home where telecommunication and web technologies are used to provide quick and proper medication to the patient from specialized assistance centre.

The first formal definition of smart home was published by Intertek in 2003, which was involved to Department of Trade and Industry (DTI) smart-homes project in UK [5]. According to Intertek a smart home is a dwelling incorporating a communications network that connects the key electrical appliances and services, and allows them to be remotely controlled, monitored or accessed. A home needs three things to make it smart:

1. Internal network - wire, cable, wireless
2. Intelligent control - gateway to manage the systems
3. Home automation - products within the homes and links to services and systems outside the home

III. Review of Smart Homes

Smart homes projects are being conducted for last several decades and they convey different ideas, functions and utilities. It is growing to different brunches of specialization focusing the interest of the researchers and user requirements and expectations. This article is a study of the evolution of smart home according to time.

Adaptive Control of Home Environment (ACHE) system is developed by Mozer in 1998 in USA. ACHE monitors user device usage pattern utilizing different types of sensors and builds an adaptive inferential engine for neural network to control temperature, heating and lighting. ACHE can control three main components of a home while trying to maximize user comfort and conserve energy [7].ACHE is one of the early smart home projects which is able to partially automate home environment via controlling lighting, temperature and heating components.

CarerNet is an architectural model of integrated and intelligent telecare system proposed by Williams et al. in 1998. Its core components are sensor set, a sensor bus, intelligent monitoring system and a control unit. ECG, photoplethysmograph, spirometry, temperature, galvanic skin response, colorimetry, and pulse measurement tools used to collect physiological data. The communication network within the client's local environment is an integration of HomeLAN and Body Area Network (BAN) which is responsible to carry real-time data, event data, command and control data. It has a distributed intelligence system in the form of smart sensors, smart therapy units, body-hub, Local Intelligence Unit (LIU) and Client's Healthcare Record (CHR). Home emergency alarm system, community health information and ambulatory monitoring service can be provided by the system. [8]. CarerNet is an abstract model of health smart home and interconnecting components. No prototype of the model has been developed. Only a hypothetical case study is of an individual who had undergone brain surgery after suffering from a subarachnoid is discussed.

Barnes et al. in 1998 have evaluated life style monitoring data of elderly using infrastructure of British Telecom and Anchor Trust in England. The system detects inhabitants' movement using IR sensors and magnetic contacts on the entrance of the doors. To measure temperature it uses a temperature sensor in the main living area. An alarm activation system is developed which detects abnormal behavior and communicates to remote telecare control center, the clients and their carers[9]. The researchers presented a lower cost solution for smart telecare. The limitation of the system is it can identify only abnormal sleeping duration, unexpected inactivity, uncomfortable home temperature and fridge usage disorder. Moreover, it uses a special new telecom protocol named 'No Ring Calling' which demands modifying existing telecom protocols.

TERVA is a health monitoring system developed in Finland by Korhonen et al.(1998). TERVA processes physiological information like blood pressure, heart beat rate, body temperature, body weight to draw graphical representation of wellness condition of the subject[10].Research goal of the TERVA system is to develop a real time visual monitoring system but it is unable to provide long-term trend of certain physiological information. It cannot detect physiological problems and no assistive service is deployed to provide health care.

The intelligent home (IHome) project at the University of Massachusetts at Amherst has developed an intelligent environment (Lesser et al.1999) IHome is a simulated environment designed with Multi Agent Survivability Simulator (MASS) and a Java Agent Framework (JAF) as tools to evaluate agent behavior and their coordination. The focus of the project is to model agent interactions and task interactions so that the agent can evaluate the tradeoff between robustness and efficiency [11].IHome is a simulation only solution, the project never build a practical smart home to evaluate their model.

The Aware Home Research at the Georgia Institute of Technology developed a smart home, which equipped with monitoring facilities to study human behavior (Kidd et al. 1999). To build a model of user behavior pattern, it uses smart floor to sense footsteps. Hidden Markov models, simple feature-vector averaging and neural network algorithms are applied on these data to create and evaluate behavioral model [12]. The aim of the project is to study user behavior, which is the primary stage of smart home research. The project never developed home intelligence which is a big shortfall of the research.

The EasyLiving project at Microsoft Research based on intelligent environment to track multiple residents using distributed image-processing system (Krumm et al. 2000). The system can identify residents through active badge system. Measurements are used to define geometric relationship between the people, devices, places and things [13][14].The system is workable in single room only and can track upto three peoples simultaneously.

SELF (Sensorized Environment for LiFe), is an intelligent environment, which enables a person to maintain his or her health through 'self-communication' (Nishida et al. 2000). SELF observes the person's behavior with distributed sensors invisibly embedded in the daily environment, extracts physiological parameters from it, analyzes the parameters, and accumulates the results. The accumulated results are used for reporting useful information to maintain the person's health. The researchers constructed a model room for SELF consisted of a bed with pressure sensor array, a ceiling lighting dome with a microphone and a washstand with display[15].SELF describes a self-assessment system of human health but measuring only respiratory system and sleeping disorder, which is not sufficient to monitor health condition.

The ENABLE project was set up in 2001 to measure the impact of assistive technology on the patient suffering from mild or moderate dementia (Adlam et al. 2004). The researcher installed two devices (cooker and night light) in the apartment of several patients in different locations to evaluate the efficiency of the system [16]. The research scope is limited to only two household devices but to assist this type of patient the whole house must possess some kind of intelligence.

Health Integrated Smart Home Information System (HIS) is an experimental platform for home based monitoring (Virone.et al. 2002). IR sensors are used to track inhabitant activities and the information is transmitted via Controller Area Network (CAN) to a local computer. The system generates alerts according to some predefine zones [17]. The research is only limited to single inhabitant monitoring.

In 2002, Guillén et al. developed a system composed of two parts: home station (HS) and caregiver medical center (CMC) connected via integrated service digital network (ISDN) backbone. The home station is equipped with vital signs recording module to monitor physiological data like blood pressure, temperature, ECG, pulse oximetry. Caregiver medical center is like a call center designed specially with patient monitoring software. An interactive communication system between home and caregiver center is developed using videoconferencing technology [18]. Figure 1 shows functional modules of multimedia smart home. The system requires high Internet bandwidth for videoconferencing, which needs expensive equipments and high maintenance cost.

Functional module of multimedia platform [18]

At University of Tokyo, Noguchi et al (2002) designed an intelligent room to support daily life of the inhabitant. The system has three main components: data collection, data processing and integration of processed data. The system learns current state of environment from sensors attached to bed, floor, table and switches. A summarization algorithm is used to track any changes in the system. The algorithm segments the collected sensory data at the points where sensor outputs changes drastically (i.e. pressure data appears suddenly or switch sensors are changed). It labels the segment with the 'room state'. It joins a state of each segment to quantize the accumulated data and ties up the changed situation. The algorithm also tries to eliminate and reduces situations that changes slightly [19].The proposed summarization algorithm can detect user activities which is tested for single room only. No home automation method discussed utilizing the algorithm.

MavHome (Managing an Adaptive Versatile Home) first introduced by Das et al. in 2002 at the University of Texas, Arlington [20]. Figure 2 describes MavHome architecture in brief. MavHome use multi disciplinary technologies: artificial intelligent, multimedia technology, mobile computing and robotics. It is divided into four abstract layers: physical, communication, information and decision. X10 protocol is used to control and monitor more than sixty X10 devices plugged into the home electric wiring system [21]. Active LeZi algorithm is developed that makes a decision tree based on kth order Markov model and predict next action calculating probability of all actions applying prediction by partial matching method [22]. Although MovHome utilize algorithms to make accurate prediction and decision, it only predicts the behavior of single inhabitant [23].

concrete architecture of MavHome[21]

The Rehabilitation Engineering Research Center on Technology for Successful aging (RERC-Tech-Aging) at the University of Florida introduced 'House of Matilda' (Helel et al. 2003, 2005)[24].The home is inhabited by a dummy called Mutilda. The main aim of this research is to perceive user location using ultrasound technology. After two years, in 2005 they designed the second generation of this home named 'GatorTech'[25]. GatorTech is actually integration of smart device with sensors and actuators to optimize the comfort and safety of older peoples. The system is not user friendly because it requires wearable device for user tracking.

In 2004, Mihailidis et al. developed a computer vision system in pervasive healthcare systems. The vision system consists of three agents: sensing, planning and prompting. Statistics and physics based methods of segmenting skin color in digital images are used for face and hand tracing [26]. Only hand and face tracing is not sufficient to make an efficient smart home system, the system should include body tracking and hand gesture reorganization.

Multimedia Laboratories, NTT DoCoMo Inc. in Japan, has developed a system for modeling and recognizing personal behavior utilizing sensors and Radio Frequency Identification (RFID) tag (Isoda et al. 2004)[27]. C4.5 algorithm is used to construct decision tree from the data obtained from the sensors and RFID tags. The user's behavioral context at any given moment is obtained by matching the most recently detected states with previously defined task models. The system is an effective way for acquiring user's spatiotemporal context but no intelligent system is developed for home appliances control.

Andoh et al. in 2004 developed a networked non-invasive health monitoring system analyzing breath rate, heart rate, snoring and body movement. Researchers adopted Ethernet network for breath monitoring system implementation. The system can estimate sleep stages analyzing data using the algorithm developed for the purpose [28]. The system cannot summarize long term observation of patient's sleeping disorder.

In 2005, Masuda et al. have developed a health monitoring arrangement using existing telecommunication system for home visit rehabilitation therapists. Researchers used an air filled mat to measure heartbeat and respiratory condition. When the patient lies on the air mat, his heartbeat and respiratory movement cause significant change in air pressure inside the mat, which is measured by pressure sensor and analyzed by appropriate filtering process [29]. The interesting part of the project is the usage of an air bag as monitoring equipment but its limitation is, it can only measure heart rate and respiratory condition.

In 2005, Ma et al emphasized on context awareness to provide automatic services in smart home. They used case-based reasoning (CBR) to provide more appropriate services. CBR technique relies on previous interactions and experiences to find solutions for current problems. The system can adopt any manual adjustment done by modifying case data [30].This is the initial state of the project where few scenarios like AC, TV, lamp interaction is evaluated. Their future plan is to add more contexts and enrich the features of case tables.

The House_n group at MIT designed PlaceLab a new "living laboratory" for the study of ubiquitous technologies in home environment (Intille et al. 2005). PlaceLab deployed with numerous wire, light, pressure, temperature water, gas, current sensors with video and audio devices to create vast amount of real life data from single volunteers as well as couples [31].The goal of the project is to study human behavior, influence of technology on the people and how technology can be used to simplify user interaction with home appliances. Their main contribution is an open online database of smart home sensor events and a well featured analyzing software [48].Researchers never implemented the study to build an autonomous intelligent home.

Yamazaki (2006) constructed "Ubiquitous Home," a real-life test bed, for home context-aware service. It is a housing test facility for the creation of useful new home services by linking devices, sensors, and appliances across data networks. Active and passive RFID tags located above the ceiling and at the entrance of the door are used to detect and recognize inhabitants. Pressure sensors are used to track user movement and furniture. The system is occupied with plasma panels, liquid crystal display and microphone for better interaction with the users. A network robot is employed to perform certain home services. Researchers concluded that the goal of smart home is not to design an automated home but to develop an environment using interface technologies between human and the system [32].Although, the researchers installed enough sensors and interfacing devices , the system is only sensible to few task automations like TV program selection, cooking recipe display and forgotten property service.

Ha et al. (2006) presents a sensor-based indoor location-aware system that can identify resident's location. Researchers used an array of Pyroelectric Infrared (PIR) sensor and proposed a framework of smart home location aware system. An algorithm is developed to process the information collected from PIR sensors for inhabitant location detection. Their next step is to design an algorithm to determine location and trajectory of multiple residents simultaneously [33]. The project in dedicated to user location detection system which is an essential part of smart home. No system is developed to provide intelligence to the house employing user location.

In 2007, Rahal et al. at DOMUS laboratory, Universit´e de Sherbrooke, Canada, utilized Bayesian Filtering methods to determine location of the inhabitants. Bayes filters are efficiently used to estimate a person's location using a set of fixed sensors. In this method, the last known position and the last sensor event are both used to estimate a new location. The algorithm based on Bayesian filtering shows a mean localization accuracy of 85% [34].This project also deals with user location detection algorithm, no home automation is developed using the processed information.

De Silva et al. (2007) have implemented an audiovisual retrieval and summarization system utilizing multimedia technology for human behavior tracking. Using a large number of cameras a hierarchical clustering of audio and video handover used to create personalized video clips. An adaptive algorithm is used for complete and compact summary of the video retrieved. Basic audio analysis methods are applied for accurate audio segmentation and source localization. An interface allowed users to incorporate their knowledge into the search process and obtain more accurate results for their queries [35].The system can track people, extract key frame, localize sound source, detect lighting change but cannot distinguished different people.

At Tampere University of Technology, Vainio et al.(2008) developed a proactive fuzzy home-control system. An adaptive algorithm applied to evaluate the test on obtained results. The goal of the research is to help elderly people live independently at home. Developed system can recognize routines and also recognize deviations from routines. The system can provide information to caregivers about living rhythm, sleeping disorders, and medicine taking of inhabitant [36]. But the system works sensibly only for lighting control.

In 2008, Swaminathan et al. proposed an object reorganization system using visual image localization and registration. Appliances are first registered in the image processing system. According to the voice command of the user, appropriate object is selected using an environmental map [27].It is actually a home automaton project using speech reorganization to receive user command and commands are executed to the objects which already known to the system.

Growing Self-Organizing Maps (GSOM) used a self-adaptive neural network to detect and recognize activities of daily life addressed by Zheng et al in 2008 [38] [39]. The GSOM follows the basic principle of the Kohonen self-organizing map with a special focus on adaptive architecture. The learning process of the GSOM is started by generating an initial network composed by four neurons on a 2-dimensional grid, followed by iteratively presenting training data samples. The system is tested in single room apartment for about two weeks where it can recognized user pattern of 22 distinct activities. Like other Self Adaptive Neural Networks (SANN), the system is depends on several learning parameters to be determined in advance such as initial learning rate and the size of the initial neighborhood. Other machine learning method must be utilized in parallel to determine optimum parameter for best performance.

In 2008, Perumal et al. from Institute of Advanced Technology of University Putra Malaysia (UPM) have presented a design and implemented Simple Object Access Protocol (SOAP) based residential engagement for smart home system's appliances control [40]. An appliance control module based on SOAP and web services developed to solve the interoperation of various home appliances in smart home systems. Fifteen feedback based control channels implemented with residential management system through Web Services. If the residential management system experiences server downtime, the home appliances can still be controlled using alternate control mechanism with GSM network via SMS Module locally and remotely. This system offers a complete, bi-directional real-time control and monitoring of smart home systems. No security mechanism is used to protect the web server from unauthorized access.

Virone et al. present a dozens of statistical behavioral patterns obtained from an activity monitoring pilot study. The pilot study examined home activity rhythms of 22 residents in an assisted living environment with four case studies. Established behavioral patterns have been captured using custom software based on a statistical predictive algorithm that models circadian activity rhythms (CARs) and their deviations (Virone et al. 2008). The system cannot differentiate multiple inhabitants [41].

Yoo et al. examined web-based implementation possibility of a central repository to integrate the biosignal data arrives from various types of devices in a remote smart home. Medical waveform description Format Encoding Rule (MFER) standard is followed for communicating and storing the biosignal data in ubiquitous home health monitoring system. The web-based technology allowed ubiquitous access to the data from remote location. The paper presents a common data format for all types of sensor (Yoo et al. 2008)[42].Figure 3 describes functional architecture of web based data retrieval system. Information security, which is a burning issue for any web based system is not considered in this research.

A web-based architecture for transferring the measured biosignal data from the u-House to the remote central repository.

A snow-flake data model is designed by Zhang et al. in 2008 to represent the activities' data in smart homes [43]. Sensor data are stored in the homeML structure. A new algorithm is proposed on the prediction of class labels for variable person and activities of daily life (ADL) indicating who is doing what, given the observed episode and time information. Accuracy is calculated as the proportion of the number of correctly predicted class over the total number of episodes in the evaluation dataset. The learning output in the form of a joint probability distribution is also assessed by the distance to the true underlying probability distribution, using the Euclidean metric. The smaller the distance is, the closer the learned model to the true situation. The algorithm is based on probabilistic distribution and able to predict ADL of more than one inhabitant. The result given is based on simulated data and the example shows only one task identification (‘making drink' activities).

In 2008, Park et al. proposed a method for recognizing ADL at multiple levels of details by combining multi-view computer vision and RFID based direct sensor [44]. A hierarchical recognition scheme is proposed by building a dynamic Bayesian network (DBN) that encompasses both coarse-level and fine-level ADL recognition. Their methodology combines the two tracking technology. The system requires wearable RFID tag which is not comfortable for users.

Rashidi et al. developed CASAS at Washington State University in 2008. CASAS is an adaptive smart home that utilizes machine-learning techniques to discover patterns in user behaviour and to automatically mimic these patterns. The goal is to keep the resident in control of the automation. Users can provide feedback on proposed automation activities, modify the automation policies, and introduce new requests. In addition, CASAS can discover changes in resident's behaviour patterns automatically. Frequent and Periodic Activity Miner (FPAM) algorithm mines this data to discover frequent and periodic activity patterns. These activity patterns are modelled by their Hierarchal Activity Model (HAM), which utilizes the underlying temporal and structural regularities of activities to achieve a satisfactory automation policy. User can provide feedback on proposed automation activities, modify the automation policies, and introduce new requests [45].To make a system more interactive smart home should be equipped with voice reorganization facilities which is absent in this system.

Raad et al. developed a cost-effective user-friendly telemedicine system to serve the elderly and disabled people. An architecture of telemedicine support in smart home that consists of web and telecom interface is considered in their research (Raad et al. 2008)[46]. This system also suffers from information security issues.

PRIMA (Perception, recognition and integration for interactive environments) research group of the LIG laboratory at the INRIA Grenoble research center in France has defined a model for contextual learning in smart homes (2009). The authors developed a 3D smart environment consisting cameras, a microphone array and headset microphones for situation modeling. It relies on 3D video tracking and role detection process regarding activities of the person. Roles are learned by support vector machines (SVM). It is also capable to learn speed of the inhabitant and distance to the interacting object. Proposed system can identify situations like introduction, presentation, aperitif, game and siesta. Its error rate is very high [49].

Kim et al. developed a pyroelectric infrared (PIR) sensor based indoor location aware system (PILAS) in 2009.The system uses an array of PIR sensors attached with the ceiling and detects inhabitant's location by combining overlapped detected areas. PIR sensors construct a virtual map of resident location transition. To improved accuracy, they applied Bayesian classifier using a multivariate Gaussian probability density function to determine the location of an inhabitant. PILAS is unable to detect multiple residents [50].

Wang et al. have developed a smart home monitoring and controlling system(2009). The system can be controlled from remote locations through an embedded controller. They have developed different GUI for mobile devices and PCs. Each device has a unique address. A new command format to control the devices is introduced. It is a complex system and not compatible to previous smart homes architectures [51].

Yongping et al. have developed an embedded web server to control equipments using Zigbee protocol (2009). For this purpose they used S3C2410 microprocessor which was programmed with Linux 2.6 kernel. To provider online access a small web server (only 60 Kbytes) named Boa is installed. An interface had also been designed to communicate with Zigbee module (MC13192).The system do possess any type of intelligence [52].

Hussain el al. have developed inhabitant identification system using wireless sensor network (WSN) and RFID sensors (2009). The system can identify user location by the intensity of the Radio Signal Strength Indicator (RSSI) of WSN. A person is recognized by attached a RFID tag. The combined reading of RSSI signal and RFID receiver can successfully identify specific location of a resident in the home. The system is limited to single person tracking [53].

At Industrial Technology Research Institute (ITRI) in Taiwan, Chen et al.