Introduction: Measurement Issues in State-Level Energy Performance Predictors

Measurement Issues in State-Level Energy Performance Predictors: Building a Transparent, Reliable, and Valid Construct Measurement Instrument

Abstract

An 1895 paper submitted to the Stockholm Physical Society by the Swedish Chemist Svante Arrhenius  proposed that the combustion of fossil fuels would disrupt our planet’s atmosphere creating the condition we now call Global Warming (Somerville, 2006).  Numerous scientific studies including a 2017 study on average annual temperature over the contiguous United States indicates a 1.2°F (0.7°C) increase for the period 1986–2016 relative to 1901–1960 and by 1.8°F (1.0°C) using linear regression analysis for the period 1895–2016 (very high confidence) (Vose, Easterling, Kunkel, & Wehner, 2017). In the last several years, the public, watchdog groups, government agencies, and the media establishment have paid considerable attention to the global warming phenomena, the environment, and sustainable practices, with measurement predictors of environmental performance becoming increasingly important. Most of the work up to this point has focused at the corporate environmental performance level with little analysis at the state environmental performance level. To meet this demand for environmental performance information related to states, an accurate measurement tool to capture the elements of state environmental performance needs to be created.

In theory sustainability targets for state governments are being achieved using public policy, incentives, and regulations designed to improve energy efficiency and a resultant reduction of carbon emissions into the atmosphere. Despite this heightened interest in sustainable practices, a major hurdle in the understanding of what drives the state environmental performance (STEP) output, is an accurate understanding and measurement of the predictors of environmental performance that will operationalize the construct. We’ve found little or no measurement tools that analyze this relationship. In addition, data used to analyze these relationships include extensive primary data collection, proprietary databases, and information that has limited replicability.

Verification of successful strategic policy is necessary to validate methods for increased renewable energy usage & energy efficiency, and their contribution to sustainability. This deficiency is addressed in the development of a transparent state energy performance (STEP) predictive measure, with an explicit coding scheme, developed entirely from available public data. The model will test well for internal consistency and inter-rate reliability, as well as convergent and discriminant validity.

Introduction

The concept of “Global Warming” took on heightened visibility in contemporary society with Vice President Al Gore’s 2006 film “An Inconvenient Truth”, and contributed to him winning a Nobel Peace Prize in 2007.  This was the second noble prize on this theme as Wangari Mathai won the award in 2004 for her work on reforestation efforts in Kenya and “The Green Belt Movement”.  Her working championing the planting of 35 million trees while creating a geopolitical agenda that impacted government policy and environmental conservation on a global level (Hayanga, 2006).  The recent 2016 United States Presidential Election created a substantial focus on the controversial existence of “Global Warming” and the future direction of environmental policy.  Greenhouse gas emissions in the atmosphere have been shown to affect the global climate (Lee, Kim, & Chong, 2015).

Studies have shown that all energy systems emit some form of greenhouse gases, and that they  contributing to anthropogenic climate change (Weisser, 2007).  Significant changes to physical and biological systems primarily in elevated temperatures have been observed since at least 1970 that cannot be explained by natural climate variations alone.  The Intergovernmental Panel on Climate Change (IPCC) Fourth Assessment Report ‘s attribute this condition to observed increases in anthropogenic greenhouse gas concentrations (Rosenzweig et al., 2008).

The largest source of greenhouse gas emissions from human activities in the U.S. is from burning fossil fuels for electricity, heat, and transportation.  Electricity production generates the largest share of greenhouse gas emissions.  Approximately 67 percent of our U.S.  electricity comes from burning fossil fuels, mostly coal and natural gas (EIA, 2016).  Environmental policy at the Federal, State and Local levels including government regulations and incentives as well as government investment have been enacted to reduce these greenhouse gas emissions in the United States. Each state has its own dynamic set of “regulations” and “incentive policies” to address energy use & supply, as well as Federal programs that benefit the individual states.  Tracking and analysis of programs including failed policies that often lead to termination, and continued access to funding from those successful.  Greenhouse gas reduction strategies include: 1. Increasing renewable energy usage, 2. Improving energy efficiency.

Meyer & Rowan (1977) postulated that organizations work to incorporate practices and procedures defined by current rational concepts of organizational work institutionalized by society.  Adhering to this position increases legitimacy and probability of an ongoing business. Many of the stances, programs, policies, and processes of modern business are shaped by public sentiment, views of important parties, any knowledge validated by the educational system, social prestige, and by the laws, and interpretations of the courts.  These institutional rules that function as myths bind organizations to the system (Meyer & Rowan, 1977).  “The basic tenet of economics, much of strategic management, and a great deal of sociology and organization theory is rationality or functionalism-that the structures, concepts, and social arrangements which evolve are the “rational” or “efficient” solutions to the problems of production, coordination, and change. These functional structures are either designed, selected, or otherwise evolve (Rumelt & Teece, 1994).” DiMaggio and Powell (1983) stated, “The distinguishing contribution of institutional theory rests in the identification of causal mechanisms leading to organizational change and stability on the basis of preconscious understandings that organizational actors share, independent of their interests(DiMaggio & Powell, 1983).”  The “triple bottom line” that sustainable practices will provide value for all the stakeholders including people, profitability, and the planet instead of solely focusing on shareholder wealth maximization has become popular(Swallow & Furniss, 2011).

Seungtaek Lee, Yeowon Kim, and Wai K. Chong’s paper on “A Statistical Analysis of Effectiveness of Energy Policy in the United States: Incentives vs. Regulations paper analyzed how influential various policies would be in promoting renewable energy use while improving energy efficiency(Lee et al., 2015).  The authors proposed to identify the key factors of a successful energy policy.  They systematically analyzed the effectiveness of environmental policies, identifying factors that influence energy policy.  Most importantly, policymakers can utilize the factors that influence their energy policies.  Greenhouse gas emissions in the atmosphere have been determined to affect the global climate. Federal, state and local governments have developed and/or implemented environmental policies to reduce greenhouse gas emissions to minimize environmental risks in the United States.  The increased use of renewable energy improved energy efficiencies combats greenhouse emissions.  States have their own set of regulations and incentive policies to address energy supply and demand, changing in a dynamic world.  Failed policies are often scrapped, with successful policies being extended if funds exist.  Determinations of programs that increase renewable energy usage and energy efficiency are critical to successful planning.

Cadoret & Padovano’s (2016) “The Political Drivers of Renewable Energy Policies” empirically analyzes how political factors affect the deployment of renewable energy sources and compares their explanatory power to that of other economic, energy and environmental drivers.  The study presents a strategic plan for a sample of EU countries that are projected to achieve 20% share of gross final energy consumption 2020 target.  Panel data analysis was employed to demonstrate that lobbying by the manufacturing industry has a negative impact on renewable energy deployment, whereas standard measures of government quality show a positive effect.  Political factors are postulated to have correlation to countries’ renewable energy deployment decisions.  Renewable energy deployment is a good indicator of country commitment to the promotion of environmentally friendly energy policies.

They also explored the explanatory power of political determinants with economic, energy and environmental drivers. Political economy analyses of energy and environmental policy decisions have mainly focused on two types of determinants:

1. the quality of government, including the institutional framework where energy and environmental policy decisions are implemented; and

2. the incumbent government ideology(Cadoret & Padovano, 2016). 

The inverted Kutznets curve provides theoretical framework to describe the relationship between economic performance, quality of governance, and quality of the environment.  The people of poor countries value material well-being more than environmental amenities; and upon reaching sufficiently high per capita country income, citizen’s greater percentage of attention to the environment.  Policies reflecting people’s preferences, resulting in poor countries having a higher propensity to sacrifice the environment at the expense of development, with an inverse relationship with rich countries(Cadoret & Padovano, 2016). 

Empirical results of the study demonstrated the role played by political factors in the deployment of renewable energy, manufacturing industry’s lobbying efforts inhibit the deployment of renewable energy, with standard measures of governance quality showing a positive impact, and that left-wing parties promote the deployment of renewable energy more than right-wing parties, regardless of the level of concentration of the governing majority and the institutional decision-making framework.  The political factor of government ideology also potentially affects environmental quality and stringency of energy policies.  The authors’ highlight that market oriented and right-wing governments have been more active at deregulating product markets, including the market for energy. 

Empirical evidence shows that right-wing governments do in fact promote energy market deregulation.  Additionally, they provide empirical evidence that left-wing governments favor regulation in the energy sector, with the fragmentation of government offsetting.  Greater institutional constraints favor deregulation of the sector, and conversely market-oriented, right-wing governments endorse energy deregulation, although the link between environmental policy and government ideology are less evident than for left-wing positions (Cadoret & Padovano, 2016).

Research on renewable portfolio standards often focuses on the expected costs and benefits of adoption.  Occasionally macro level risk reduction, and environmental impact are included (Chen, 2009).  Studies have found that renewable policy standards implementation isn’t a significant predictor renewable energy generation percentages of total energy mix.  Political institutions, natural resource endowments, deregulation, gross state product per capita, electricity use per person, electricity price, and the presence of regional renewable energy policies are significantly related to renewable energy deployment (Carley, 2009).

Research recently has focused on the effectiveness of state policies on the generation of renewable electricity, with mixed results.  While some studies have found positive results (Bird et al., 2005; Delmas & Montes‐Sancho, 2010; Vachon & Menz, 2006; Yin & Powers, 2010; Zarnikau, 2003), others haven’t found significant results (Carley, 2009).  Also, findings on the impact on financial incentives are mixed (Carley, 2009).

A 2016 study conducted using multinational panel data from 85 countries during the years 2002 to 2012 investigated the relationship between government ideology and environmental quality, creating comprehensive indicators for an empirical environmental quality study. They used the bias-corrected least square dummy variable (LSDVC) method analyze the relationships and control for potential endogeneity. Government ideology and GDP were used to study impact on environmental performance through its influence on economic development (Wen, Hao, Feng, & Chang, 2016).  Evaluating environmental sustainability performance is extremely complex. Environmental indices are needed to assess the environmental sustainability. Current simplistic, generic environmental indices currently require deeper analysis to create accurate evaluations of genuine environmental sustainability credentials(Olafsson, Cook, Davidsdottir, & Johannsdottir, 2014).

In transparent open societies citizens should have the right to know whether their elected representatives have successfully addressed environmental issues. GDP growth, unemployment rates, and economic performance allow voters to make determinations of political performance. However, environmental performance is difficult to measure and understand. Simple measure tools for environmental policies face numerous obstacles, including the complexity of the policy field “environment”, data lacking in both accessibility and quality, absence of scientific and political consensus on the relative importance of sub-fields like climate change, waste, and biodiversity, and the preponderance of micro level detail analysis by expert communities that fail to communicate the big picture (Jesinghaus, 2012). Natural resource consumption, emissions, effluents, and waste are examples of current environmental challenges.

To overcome these issues, literature points to environmental management strategies, based on voluntary administrative instruments, which are described as “Environmental System Analysis Tools” (Maceno, Pawlowsky, Machado, & Seleme, 2018). Although there are some studies on the factors impacting the achievement of green growth, they are limited in quantity with most focusing on specific influencing factors such as political factors, environmental regulation, and technological innovation (Guo, Qu, & Tseng, 2017). Though these analytical tools evaluate environmental performance, they weren’t created with a specific focus on the assessment of state level environmental policies, and therefore doesn’t support environmental outcomes. Studies attempting to compare the sustainability performance of countries have found ratings of performance highly variable based on the sustainability indices engaged. Empirical comparison across the triple bottom line of economy, environment and social impact has been difficult(Buys, Mengersen, Johnson, van Buuren, & Chauvin, 2014).

Measuring environmental sustainability is a complex task that has relied on comparative evaluations of national performance using ranking lists and generic policy targets (Cook, Saviolidis, Davíðsdóttir, Jóhannsdóttir, & Ólafsson, 2017). At the state level, environmental policy studies engage many empirical strategies to measure state effort. Four categories are typically observed: indices of state programmatic indicators, government expenditures, pollution abatement costs, and regulatory enforcement actions(Konisky & Woods, 2012). The evaluation of environmental management is an important measure for monitoring, analyzing, and evaluating the environmental management system of government in obtaining degree of achievable outcomes or value related to aim, objectives, and results of any action, providing for future decision-making related to environmental management (Panya, Poboon, Phoochinda, & Teungfung, 2017). The concept that there are three pillars of sustainability: social, economic,  and environmental has evolved in the last 30 years from a vague, qualitative notion to more precise quantifiable specifications, creating a need for a wide variety of indicators(Moldan, Janoušková, & Hák, 2012).

My research has found no indicators of a state level environmental measurement tool that currently exists, other than magazine rankings for being “Green”. The contribution of this paper is the creation of a State Environmental Performance (STEP) measurement instrument that captures the most relevant features of STEP surpassing convergent validity hurdles, discriminant validity, internal consistency reliability, and inter-rater reliability. All the data assimilated in the instrument can be obtained from publicly available information, with specified data sources, and delineated coding schemes, that provide complete transparency.

The paper will be divided into five sections. First will review STEP as a concept and construct. We’ll look at both the strengths and weaknesses of existing STEP measures providing guidance on improvements that can be universally adopted.

Second, we’ll build on a review of current state environmental performance tools to develop our measurement tool. The tool will provide a unified framework in which to construct a policy indicator and to study its determinants through a latent regression approach. Similar studies on country rankings have used the effect of economic and institutional observables to understand the affect of policy design (Galeotti, Rubashkina, Salini, & Verdolini, 2018).

The third section will examine and provide explanation for the methodology for testing the validity and reliability of our measurement tool. The process of evaluation gives assertions on reliability, effectiveness, and efficiency improving all aspects of society. Analyzing organizations from a system perspective or System theory is a critical concept for the analysis and evaluation of private and public organizations (Panya et al., 2017).  We will test for internal consistency reliability and inter-rater reliability of the instrument using Cronbach alpha(Rahman & Post, 2012). Using Forbes’ America’s Greenest States ranking methodology based on six equally weighted categories: carbon footprint, air quality, water quality, hazardous waste management, policy initiatives and energy consumption we’ll look for evidence of convergent validity. We’ll also investigate discriminant validity using the Forbes score. Our results will be provided in the fourth section. The fifth and final section will discuss theoretical and applied contributions of the proposed STEP measurement tool, recognizing limitations, and opportunities for future research (Rahman & Post, 2012).  Ultimately the question how do we measure state-level energy performance needs to be answered? The data captured in answering this question is needed to provide more reliable, consistent, and accurate information to those in government making key strategic decisions that will affect the planet we all live on.

References

Bird, L., Bolinger, M., Gagliano, T., Wiser, R., Brown, M., & Parsons, B. (2005). Policies and market factors driving wind power development in the united states. Energy Policy, 33(11), 1397-1407.

Buys, L., Mengersen, K., Johnson, S., van Buuren, N., & Chauvin, A. (2014). Creating a sustainability scorecard as a predictive tool for measuring the complex social, economic and environmental impacts of industries, a case study: Assessing the viability and sustainability of the dairy industry. Journal of Environmental Management, 133, 184-192. doi:https://doi.org/10.1016/j.jenvman.2013.12.013

Cadoret, I., & Padovano, F. (2016). The political drivers of renewable energies policies. Energy Economics, 56, 261-269. doi:http://dx.doi.org/10.1016/j.eneco.2016.03.003

Carley, S. (2009). State renewable energy electricity policies: An empirical evaluation of effectiveness. Energy Policy, 37(8), 3071-3081. doi:https://doi.org/10.1016/j.enpol.2009.03.062

Chen, C. (2009). Weighing the costs and benefits of state renewables portfolio standards in the united states: A comparative analysis of state-level policy impact projections.

Cook, D., Saviolidis, N. M., Davíðsdóttir, B., Jóhannsdóttir, L., & Ólafsson, S. (2017). Measuring countries’ environmental sustainability performance—the development of a nation-specific indicator set. Ecological Indicators, 74, 463-478. doi:https://doi.org/10.1016/j.ecolind.2016.12.009

Delmas, M. A., & Montes‐Sancho, M. J. (2010). Voluntary agreements to improve environmental quality: Symbolic and substantive cooperation. Strategic Management Journal, 31(6), 575-601.

DiMaggio, P., & Powell, W. W. (1983). The iron cage revisited: Collective rationality and institutional isomorphism in organizational fields. American Sociological Review, 48(2), 147-160.

EIA, U. (2016). Energy information administration (2016). Sources of greenhouse gas emissions.

Galeotti, M., Rubashkina, Y., Salini, S., & Verdolini, E. (2018). Environmental policy performance and its determinants: Application of a three-level random intercept model. Energy Policy, 114, 134-144. doi:https://doi.org/10.1016/j.enpol.2017.11.053

Guo, L. l., Qu, Y., & Tseng, M.-L. (2017). The interaction effects of environmental regulation and technological innovation on regional green growth performance. Journal of Cleaner Production, 162, 894-902. doi:https://doi.org/10.1016/j.jclepro.2017.05.210

Hayanga, A. J. (2006). Wangari mathai: An african woman’s environmental and geopolitical landscape. International journal of environmental studies, 63(5), 551-555.

Jesinghaus, J. (2012). Measuring european environmental policy performance. Ecological Indicators, 17, 29-37. doi:https://doi.org/10.1016/j.ecolind.2011.05.026

Konisky, D. M., & Woods, N. D. (2012). Measuring state environmental policy. Review of Policy Research, 29(4), 544-569. doi:10.1111/j.1541-1338.2012.00570.x

Lee, S., Kim, Y., & Chong, W. K. (2015). A statistical analysis of effectiveness of energy policy in the united states: Incentives vs. Regulations. Procedia Engineering, 118, 1282-1287. doi:http://dx.doi.org/10.1016/j.proeng.2015.08.483

Maceno, M. M. C., Pawlowsky, U., Machado, K. S., & Seleme, R. (2018). Environmental performance evaluation – a proposed analytical tool for an industrial process application. Journal of Cleaner Production, 172, 1452-1464. doi:https://doi.org/10.1016/j.jclepro.2017.10.289

Meyer, J. W., & Rowan, B. (1977). Institutionalized organizations: Formal structure as myth and ceremony. American journal of sociology, 83(2), 340-363.

Moldan, B., Janoušková, S., & Hák, T. (2012). How to understand and measure environmental sustainability: Indicators and targets. Ecological Indicators, 17, 4-13. doi:https://doi.org/10.1016/j.ecolind.2011.04.033

Olafsson, S., Cook, D., Davidsdottir, B., & Johannsdottir, L. (2014). Measuring countries׳ environmental sustainability performance – a review and case study of iceland. Renewable and Sustainable Energy Reviews, 39, 934-948. doi:https://doi.org/10.1016/j.rser.2014.07.101

Panya, N., Poboon, C., Phoochinda, W., & Teungfung, R. (2017). The performance of the environmental management of local governments in thailand. Kasetsart Journal of Social Sciences. doi:https://doi.org/10.1016/j.kjss.2017.03.001

Rahman, N., & Post, C. (2012). Measurement issues in environmental corporate social responsibility (ecsr): Toward a transparent, reliable, and construct valid instrument. Journal of Business Ethics, 105(3), 307-319.

Rosenzweig, C., Karoly, D., Vicarelli, M., Neofotis, P., Wu, Q., Casassa, G., . . . Seguin, B. (2008). Attributing physical and biological impacts to anthropogenic climate change. Nature, 453(7193), 353-357.

Rumelt, R. P., & Teece, D. J. (1994). Fundamental issues in strategy: A research agenda: Harvard Business Press.

Somerville, C. (2006). The billion-ton biofuels vision. Science, 312(5778), 1277.

Swallow, L., & Furniss, J. (2011). Green business: Reducing carbon footprint cuts costs and provides opportunities. Montana Business Quarterly, 49(2), 2-9.

Vachon, S., & Menz, F. C. (2006). The role of social, political, and economic interests in promoting state green electricity policies. Environmental Science & Policy, 9(7–8), 652-662. doi:https://doi.org/10.1016/j.envsci.2006.07.005

Vose, R., Easterling, D. R., Kunkel, K., & Wehner, M. (2017). Temperature changes in the united states.

Weisser, D. (2007). A guide to life-cycle greenhouse gas (ghg) emissions from electric supply technologies. Energy, 32(9), 1543-1559.

Wen, J., Hao, Y., Feng, G.-F., & Chang, C.-P. (2016). Does government ideology influence environmental performance? Evidence based on a new dataset. Economic Systems, 40(2), 232-246. doi:https://doi.org/10.1016/j.ecosys.2016.04.001

Yin, H., & Powers, N. (2010). Do state renewable portfolio standards promote in-state renewable generationʔ. Energy Policy, 38(2), 1140-1149.

Zarnikau, J. (2003). Consumer demand for ‘green power’and energy efficiency. Energy Policy, 31(15), 1661-1672.