Machine Learning for City Infrastructure Development

Abstract:

Urbanization led by industrialization has made the cities the center for human Due to the advancement of technology, there are many smart devices which have become a basic need of humans and it generates a large amount of data can be studied and reviewed by the humans. The access to data through human smart devices and the sensors with help of high speed internet helps in collection the large amount of data provided. Based on these data generated, Machine learning algorithms for the adaptive solution holds the key. Urban development. This paper reviews a couple of infrastructure in a city where machine learning is used and further advancement in the same will make the make the city a better place to live.

Introduction:

Industrial Revolution marked the rise of the urbanization by not only creating job opportunities to the people but also providing them the opportunity to grow economically. By the end of nineteenth and the beginning of the twentieth century, the world saw a rapid industrial growth due to the setting up of multiple large and medium scale industries which resulted in the beginning of the Urbanization.

Since the International Conference on population and Development, held in Cairo in 1994, the World’s population has grown from 5.7 to 7.2 billion with, three quarters of that growth occurring in Asia and Africa[1]. All though the efforts has been made to reduce the speed of population growth, its estimated that by 2050 the population may reach the around 9.6 billion be 2050. Data from the recent past show that much of these population rise has happened in cities of the countries who are the emerging market and hence the estimation goes to around 6.5 billion people calling cities their home by 2050.

With this exponential amount of population growth and mobility into the city, many complex issues need to be dealt with. Urbanization needs more innovative solution to tackle and manage the complex issues like overcrowding, energy consumption, resource management and environmental protection. The other complex challenges include traffic congestion, unemployment, poverty comes are the bi-product of unemployment, climate change, insecurity, demand for public places and the list goes on. As cities grow and evolve they generate technical, social, economic and organizational pressures that put economic and environmental sustainability in jeopardy[2]. The mitigation plan to tackle these problems need to be at its place which will make the cities Smart. Smart Cities are complex system of interconnection between all the resources like transportation, communication, services, utilities and people.  Internet of Things (IoT) offers smart solutions for future cities to address these challenges with minimum human intervention[3]. Data can be picked from either the personal devices used by the people or from the sensors added at different infrastructures like the traffic signals, buildings, sewage drainers, water supply pipelines, electricity supply lines etc.

 Traffic solution in Smart Cities

With the growth of a city and its population it is natural that the congestion in traffic. Congestion in traffic is one of the biggest problems that affects the daily lives of billions of people in most countries across the world. Over at least the past 30 years, many attempts to alleviate this problem in the form of intelligent transportation systems have been designed and demonstrated. Among these different approaches, some use real time traffic information measured or collected by video cameras or loop detectors and optimize the cycle split of a traffic light accordingly[4]. Studies show that the increase in the number of vehicles increase with an average of 1:1 ratio in the present world. Recent research has shown that current driving conditions and the driver’s driving style have a strong influence over a vehicle’s fuel consumption and emissions[5]. As Many studies are conducted couple of cities by the urban development to find a solution for this traffic problems. The data is captured from the sensors installed at the traffic signals and the monitoring cameras. Using the data collected, One potential AI algorithm is deep reinforcement learning (DRL), which has recently been explored by several groups[6]. These results showed an improvement in terms of waiting time and queue length experienced at an intersection in a fully-observable environment. Supervised machine learning with the data available from these sources give good understanding on the pattern of people using the roads and the mode of transportation.

E.g.: According to a study with Highways in Huston, Texas, the expansion of the road is not the solution for the traffic reduction. With the existing data, the study says that the if the capacity of the road is doubled, the amount of the vehicles coming on road will be doubled because the drivers will see that the roads are wider hence the traffic is lesser and hence move on to driving instead of using the public transportation and so on and so forth. Hence the ratio to road size to congestion remains to 1:1 ratio and the study showed that the idea widening the road is clearly not the solution for traffic congestion. The study on traffic shows that the traffic slows down exponentially, with the last couple of cars getting added on the road which gives exponential curve and hence the 20th car on the traffic slows done smore significantly than the 15th, hence a small addition of cars at the end makes the traffic makes the congestion go up exponentially. This study hence helped in identifying the problem which was used to find a solution. As a solution, there are ramp meters setup on the highways that will restrict the number of cars entering into the highways, they usually let one car on every 5 seconds, since number cars entering into the highway is less, the traffic congestion was reduced on that highway. To test this negatively they tried to shut down the Minnesota Ramp meters for 8 weeks and the experiment resulted in highways capacity decreasing by 9%, travel time increasing by 22%, speed drops by 7% and crashes increased by 26%[7].

Though in the above example, the traffic is under control, if the number of vehicles increase on the highway or at the exit ways,  there is solution for the dynamic change in traffic, hence to make the system learn from the dynamic data and work on its won, Q-Learning Algorithm can be taken into consideration.

Q-Learning from Watkins[8] refer to for a detailed explanation of general reinforcement learning and Q-learning but we will provide a brief review in this section. The goal of reinforcement learning is to train an agent that interacts with the environment by selecting the action in a way that maximizes the future reward. At every time step, the agent gets the state (the current observation of the environment) and reward information (the quantified indicator of performance from the last time step) from the environment and makes an action. During this process, the agent tries to optimize (maximize/minimize) the cumulative reward for its action policy. The beauty of this kind of algorithm is the fact that it doesn’t need any supervision, since the agent observes the environment and tries to optimize its performance without human intervention[4].

Q-value which is denoted by Q(st, at), is the function of st(observed state) and at(action) which gives the future reward of expected cumulative. Discrete time index is denoted by t. The expression goes by:

Q(st, at) = rt + γrt+1 + γ 2 rt+2 + γ 3 rt+3 + …[4]

Here, rt is the reward at each time step, the importance of which should be indicated by the real issue, and γ < 1 is the markdown factor. At each time step, the specialist refreshes its Q work by an update of the Q esteem:

Q(st, at) := Q(st, at)+α(rt+1+γ max Q(st+1, at)−Q(st, at))[4]

In majority of the cases, which includes traffic control, as there are much complexities with state space and action space, approximation of Q function can be done using neural networks. The value which is used:

Q(st, at) + α(rt+1 + γ max Q(st+1, at) − Q(st, at))[4]

 as the output target of a Q network and do a step of back propagation on the input of st, at.[6]

Though the above algorithm helps in study of the dynamic traffic, a lot of complexity is involved. In addition to this, the sensors and such intelligent traffic control schemes are expensive and, therefore, they exist only at a small percentage of intersections in the United States, Europe, and Asia. Recently, several cost-effective approaches to implement intelligent transportation systems were proposed by leveraging the fact that Dedicated Short-Range Communication (DSRC) technology will be mandated by US Department of Transportation (DoT) and will be implemented in the near future[4]. These sensors can be installed should be installed in all the major smart cities to pick up the vehicles travelling into the cities or the highways which will give the actual data. Majority of the population already use GPS and Google Maps for their navigation, this data also can add up in combination of the data from sensors and  Machine learning on these data could be used to predict the future traffic and come up with the solutions on the same. This problem is pretty complicated as there multiple variables which are affecting the traffic, hence there has to be Multivariate Regression which needs to be applied on the data available to get the result which probably near the fact.

ENERGY IN SMART CITIES

Another major challenge which needs to be addressed in a city to make it smart is the energy consumption because of the growing demand all over the globe. Home appliances consume a considerable amount of energy. In a Smart home environment Home Energy Management plays a critical role in finding a suitable and reliable solution to curtail the peak demand and achieve energy conservation[9]. Recent developments in the area of information and communication technologies have provided an advanced technical foundation and reliable infrastructures for the smart house with a home energy management system[10].  Further, realization of smart, energy-efficient and green home infrastructure would allow the development of “livable” interconnected communities, which will form the backbone of a futuristic green city architecture[11]. Hence, energy management in smart homes is a key aspect of building efficient smart cities [12]. In any industrialized nation, commercial and home buildings consume a significant amount of electric energy in a urban area. Hence a need of high energy efficient is in atmost need.

 The IoT devices used in each house help in capturing the data from the houses and the buildings. This can also help in monitoring the power level of each devices used in the houses. Adding a deep machine learning to there data can help in understanding the work paradigm of the devices.

Q-learning are good from a conceptual point of view and are very successful when applied to problems with small, discrete state spaces. However, for more realistic systems, the “exploration overhead”, stochastic approximation inefficiencies and stability issues cause the system to get stuck in sub-optimal policies.

Better solution can be resulted in using the Reward matrix Computation algorithm. The states s(t) defined in the Algorithm 1 are different combinations of power levels derived from the peak power rating of the appliances. Apart from refrigerator all other appliances can be turned On/Off in a smart home as the refrigerator needs to continuously run throughout the day and should not be stopped. Usage pattern of all other appliances vary throughout the day and can be controlled through the MCCU. Therefore, in total there are 10 appliances and a 2n − 1 transition states depicting various combination of power levels (n = 9) which results in 511 states. Lets depict each appliance in ascending order of their peak power level from Table-1 with level “pl”. Thus Lighting will be symbolized by p1 and WasherDryer by p9. The power values are coded as binary states i.e., 0 represents the Off state and 1 represents On state. For example, 001001010 means Microwave, Heater–2 (Bedroom) and Stove are in On state and rest all are in Off condition. The total power consumed at that instant t is 7600 Watts if every On appliance is operating at peak load.

Table 1 : Home appliances with these power ratings.

There are four different actions that can be performed based on the states. Turning the appliance Off, turning it On, pausing the operation and postponing the operation. For the case of simplicity, turning the appliance Off is considered to be a required action. Also pausing and postponing the operation of the appliance can be selected for the symbolic Off state through the MCCU control based on the situation. The representation remains the same but power level changes. Moreover, we also define User Input Preferences (UIP) as a user input control which changes the decision of the MCCU controller algorithm as desired by the user at a certain time interval. After the scheduling task is over, the control is shifted back to MCCU algorithm. Agent can move from one state to another state after performing an action. The user inconvenience is modeled in the reward matrix and the goal of the strategy is to minimize the user inconvenience[9].

Machine learning with IoT-enabled solutions can be used here for making optimized choices in terms of food availability. On the other hand, the transportation of the food can also be optimized by incorporating intelligent means of transportation. IoT devices are generally battery operated and have limited storage space. Concerning these fundamental limitations of sensors, it is difficult to realize the IoT solutions with prolonged network life. In order to efficiently utilize the limited sensor resources, an optimized energy-efficient framework is of paramount importance. It will not only reduce energy consumption, but also maintain the minimum QoS for the concerned applications[13].

With the increased amount of human population, there is an obvious increase ins industries and vehicles and this in-turn leads in the environmental pollution. The pollution is not only restricted to Air, but also to water and soil too. This makes the life of the people miserable by making the air not breathable, water not drinkable and also the environment not liveable. Many major cities in the world are already experiencing the fact that the air inside the city is not breathable due to the concentration of the toxic elements in the air added because of pollution. According to WHO, the potential illness and deaths will increase from hundreds of thousands by 2050. Mobile operators are helping smart city governments to reduce air pollution using predictive Internet of Things (IoT). Sensors provide updates on air pollution in real-time, enabling accurate monitoring. Artificial Intelligence-assisted (AI) monitoring platforms assess big data feeds and enable smarter analysis using machine learning of weather and traffic commuter information to help predict areas of poor air quality[14].  

The information is then used by local governments to reduce pollution by reducing traffic and output from factories. Predictive IoT is applied in areas such as disease control, weather and natural disaster management, and traffic optimization in smart cities[13].

The Scottish city of Glasgow has been using Sensing the City pilot project for a more efficient and economical way to monitor air quality and reduce emissions. Advertisement sensing the City uses the Libelium IoT Sensor node to reduce pollution by monitoring using mobile technology. This low-cost solution is used in addition to Glasgow’s high-cost, static sensing stations[14]. 

The project is a collaboration with the University of Strathclyde Institute for Future Cities and the industry-led Center for Sensor and Imaging Systems (CENSIS).

The two systems complement each other. Low-cost systems’ advantage is in flexibility and rapid mobile configurations whereas the static stations provide highly accurate data. The problem with the high-cost solution is the limitation in the quantity of deployments due to, precisely, the cost[14].

The two solutions working together provide indicative IoT air-quality data in areas of low or no coverage and support identification of pollution sources[14].

Sewage treatment hold the key to maintain the public health and sustainable development of a city. Sewage treatment if not done appropriately obviously leads to the Water pollution. This is of at-most importance as the water is the second most important element after air for a life to sustain on the planet. The growing cities has already seen the worse degrees of water pollution due to non technical and unhealthy sewage treatment. On a contrary many cities in the world have already implemented ways to learn from the existing data. The sensors equipped inside the houses and the drainage pipelines helped in collecting the data. Studies were conducted on this data with the help of Machine learning algorithms along with business intelligence and Artificial intelligence which has led not only for the better treatment of the sewage, but also to handle the future sewage abundance.

Going the smarter way, today Kansas city has the world’s largest smart sewer network that is estimated to save $1 billion in the coming years[15].

The sewer system is equipped with nearly 300 sensors on the base of the rugged manhole covers along the 2,800-mile sewer pipe network covering 318 square miles. The city is deploying sensors that are a part of the Internet of Things (IoT) for monitoring and controlling sewer and stormwater flows. These sensors act as a kind of flow metre that operates like sonar, mapping the flow and depth of water in any given location[15].

Moreover, the city is also using deep data sets of approximately 5 terabytes that have enabled coordinated rehabilitation of 140-year old water and sewer pipes that are inclined to frequent breaks and leaks[15].

Additionally, tech company EmNet has provided a real-time decision support system that actively controls the flow of water – preventing the combined sewage from the Missouri River. The system increases the storage capacity in the sewer systems by using the in-line gates during heavy rains – similar to the smart traffic lights working during peak hours[15].

This smart sewer system, worth $1.2 million will help in deterring the construction of deep tunnels and pumping stations that call for millions of investment. The smart sewer is a successful project deployed in Kansas City that acts as an example showing the capabilities of[15] Internet of Things, Machine learning, Artificial Intelligence, and data mining technology.

There are many other fields and infrastructure related to smart cities in which the machine learning implementation. There can be either supervised or unsupervised learning which can be used in according to the need of that problem. There are couple of algorithms mentioned below in table 2 which can be used as one of the resources with machine learning for the solution.

Table 2 – Machine learning algorithms and use.

Conclusion:

Cities have been the heart of human survival and seeing at the rate at which they are growing it is indispensable challenge to make it smart and sustainable. Study of the available data on the cities using machine learning has helped and will help in predicting the future growth which will be helpful in making the cities a better place to live. This review paper gives an overview of challenges faced in a city and has looked into how a couple of those challenges that have been handled. The examples used to show how Machine Learning was implement at certain areas making it better in a pave of building a smart city. There are many such areas and all the cities on this planet which needs certain studies and infrastructure to making it a much better place to live in.

References:

[1] “Economic and Social Council – PDF.” [Online]. Available: https://docplayer.net/51394350-Economic-and-social-council.html. [Accessed: 07-Oct-2019].

[2] D. Bennett, D. Pérez-Bustamante, and M.-L. Medrano, “Challenges for Smart Cities in the UK,” in Sustainable Smart Cities: Creating Spaces for Technological, Social and Business Development, M. Peris-Ortiz, D. R. Bennett, and D. Pérez-Bustamante Yábar, Eds. Cham: Springer International Publishing, 2017, pp. 1–14.

[3] W. Ejaz and A. Anpalagan, “Internet of Things for Smart Cities: Overview and Key Challenges,” in Internet of Things for Smart Cities: Technologies, Big Data and Security, W. Ejaz and A. Anpalagan, Eds. Cham: Springer International Publishing, 2019, pp. 1–15.

[4] “Zhang et al. – 2018 – Intelligent Traffic Signal Control Using Reinforc.pdf.” .

[5] Jungme Park et al., “Intelligent Vehicle Power Control Based on Machine Learning of Optimal Control Parameters and Prediction of Road Type and Traffic Congestion,” IEEE Trans. Veh. Technol., vol. 58, no. 9, pp. 4741–4756, Nov. 2009.

[6] “Genders and Razavi – 2016 – Using a Deep Reinforcement Learning Agent for Traf.pdf.” .

[7] How to Fix Traffic Forever. .

[8] “Watkins-Dayan1992_Article_Q-learning.pdf.” .

[9] C. Mahapatra, A. K. Moharana, and V. C. M. Leung, “Energy Management in Smart Cities Based on Internet of Things: Peak Demand Reduction and Energy Savings,” Sensors (Basel), vol. 17, no. 12, Dec. 2017.

[10] B. Zhou et al., “Smart home energy management systems: Concept, configurations, and scheduling strategies,” Renewable and Sustainable Energy Reviews, vol. 61, pp. 30–40, Aug. 2016.

[11] T. Mendes, R. Godina, E. Rodrigues, J. Matias, and J. Catalão, “Smart Home Communication Technologies and Applications: Wireless Protocol Assessment for Home Area Network Resources,” Energies, vol. 8, no. 7, pp. 7279–7311, Jul. 2015.

[12] W. Ejaz, M. Naeem, A. Shahid, A. Anpalagan, and M. Jo, “Efficient Energy Management for the Internet of Things in Smart Cities,” IEEE Commun. Mag., vol. 55, no. 1, pp. 84–91, Jan. 2017.

[13] “IEEE AGAZINE. January 2017, Vol. 55, No. 1. A Publication of the IEEE Communications Society – PDF.” [Online]. Available: http://docplayer.net/55307541-Ieee-agazine-january-2017-vol-55-no-1-a-publication-of-the-ieee-communications-society.html. [Accessed: 07-Oct-2019].

[14] “Reducing Air Pollution in Smart and Sustainable Future Cities.” [Online]. Available: https://interestingengineering.com/reducing-air-pollution-in-smart-and-sustainable-future-cities. [Accessed: 07-Oct-2019].

[15] smartcity, “How US Smart Cities Are Deploying Smart Sewer?,” Smart City, 26-Oct-2018. .