Riordan Frost: How Green is Your State? Constructing a modern state environmental index

>> All right. I think we'll go ahead and get started. Good morning and thank you for coming out today. I'm Sarah Mills, I'm a Post-Doctoral Fellow and Lecturer in the Center for Local State and Urban Policy here at the Ford School Center for Local, State and Urban Policy. CLOSUP is one of research centers here. And this event is actually part of our CLOSUP in the classroom initiative. That is project that's funded by the Provost Office third century initiative to help us better integrate the research that's going on in our center with what's happening in the classroom here. And so as part of that grant we were able to develop two different courses. This is a course on environmental policy research. And so it really links up with CLOSUPs efforts in, we have an environmental, energy and environmental policy initiative. And so many of the students in the class are engaging with that data and others are doing work through the class that's directly implacable to the type of stuff that we're doing in the center. In addition, to CLOSUP sponsoring this, it's also being co-sponsored by the program and the environment, the School of Natural Resources Environment and the Energy Institute. So I want to thank those partners and she slipped out of the room, but I also wanted to thank Bonnie Roberts, my colleague at CLOSUP for pulling all of the details together for this. Because this is part of the class, there are as I said, a number of students from that particular class here. And pedagogically what I really wanted to present them with was the nuts and bolts of research. How do you pull together a policy research process? And while I was in D.C. for a conference last year, I met Riordan Frost. And he was telling me about this project and really met making pragmatic decisions on what, what categories do you include if you're trying to measure the states against each other in terms of their level of environmental policy? And so that's really what I hope that we'll be able to get out of this, an introduction generally to his environmental index. Riordon is finishing up his PhD in public administration at American. Hopefully it will be done by the end of the year. In addition to this work, he's also, his dissertation is also looking at urban sustainability and urban policy. And I understand he's blogger on urban policy and planning, specifically in D.C., so I look forward to your presentation.

>> Excellent. Thank you very much. Thank you Sarah, thank you to Barry and Brian as well for hosting me. This is a project that I'm working on with the Center for Environmental Policy at American University which is within the Department of Public Administration Policy there. And as Sarah mentioned, this is, it's really about trying to create a modern state environmental index. And I'm going to go over both the results from that index but also how we're trying to construct that index, the challenges that are inherent in constructing an index. So just to justify why we're doing what we're doing off the bat, we're focusing on the state level. William K. Reilley, not to be confused with Bill O'Reilly, is the former admin, EPA administrator under George H.W. Bush. He's kind of our Center for Environmental Policies figure head. And he gave a speech to the environmental council of the states back in 2015 in which he said climate action and adaptation has been most notable and imaginative at the state and local levels. And if the clean power rules overturn we will continue to be. So he said this, about a year before the 2016 election and the Clean Power Rule now is I would say on even shakier ground. So the state and local levels are really going to be I think the new focus for these types of policies, especially as they are either ignored or worked against on the federal level. So if we agree on the state level then, why create a ranking? I understand that the students in this class will be creating somewhat similar things but not necessarily exactly like this. But we wanted to do a ranking because it's great for benchmarking, you can see the progress that you've made or lost, especially over the years. You can see if you're, you know, in the leading states or if you're at the bottom of the ranking room for improvement. You compare yourself to your neighbors, everybody loves to do that. And then also, everybody loves and hates rankings. I put a logo of US News and World Report college ranking. I went to undergrad at Connecticut College and while I was there, they signed onto a list of colleges that were boycotting the US News and World Report College ranking, because they felt as though the methodology was unfair or maybe not transparent enough, whatever. And I'm sure it probably had nothing to do with the fact that you know, we slipped rankings or something that year. But the fact of the matter is as I'll get into later, rankings really can rile people up. As to you know, we're really proud that we're at the top, we're really mad that we're at the bottom. Why did you have this methodology this way? So that's what I'm going to explore. So when it comes to environmental rankings, there's quite a few on the national level. This is Yale University's environmental performance index. They've been producing this for a while. These are the 2016 results of environmental performance blue being good, red being bad. Basically it looks like a developed country, developing country map. But this is just to show that you know, there's a lot of work being done here on the national level. On the city level there's a lot of work to be noticed as well. Kent Portney's, "Taking Sustainable Cities Seriously" is a great look at it. There's an organization called Sustain Lane that was ranking cities in the U.S. There's a lot of global city rankings for instance, from ARCADIS here. But on the state level, the most recent one that we could find was the financial services blog, WalletHub for some reason has decided to rank the states on environmental policy and performance. But the most recent comprehensive ranking that we could find is the 1991-1992 Green Index which was the Institute for Southern Studies, I believe, Bob Pauley and Mary Lee Kerr. And it had about 256 indicators in it and was somewhat surprisingly to us one of the most recent we could find. So 1991-1992 is quite old when it comes to data, not when it comes to people but when it comes to data, it's very, very old. And so, scholars have been looking at what have people been using in lieu of a good modern index? So, Konisky and Woods in 2012 wrote an article that categorized what environmental scholars have been using since then, because there haven't really been any good ones since then. So they have it in four categories, state government expenditures on environmental protection, private sector pollution abatement, state environmental regulatory enforcements and then of course environmental indices. And the funny thing is that some people as recently as 2009, including the scholars who wrote this very article, have been using the 1991-1992 Green Index even though it's almost, you know, as of 2009, 20 years old. So, what has happened since then in terms of indices, is one that appeared in Forbes in 2007. It really didn't have its own dedicated website or anything. It was just kind of an article that appeared on it. America's Greenest States, there was six indicators listed there. Carbon footprint, air, water, etcetera. The states that led in that ranking were Vermont, Oregon and Washington, the states that are at the bottom Alabama, Indiana, West Virginia. And then more recently as I mentioned in the financial services blog, WalletHub which gives you credit scores and other things has decided to rank the states and surprisingly in depth actually as well with three categories and 17 indicators total, 5 in environmental quality, 8 in friendly behaviors and 4 in climate change contributions. The states that lead up on there are Vermont, Washington and Massachusetts. And the states that are at the bottom are Montana, North Dakota and Wyoming. So, we set out at the Center of Environmental Policy to create a new modern, comprehensive state policy index. And so we were trying to mirror the 1991-1992 Green Index which as I mentioned has 256 indicators, 179 condition indicators and 77 policy indicators. So that seems like a very high bar, but the fact of the matter is that some, it is a fairly high bar but some of their variables were somewhat questionable. For instance, they had things like, what percent of your state is federal land? That's not really anything the state has any control over necessarily. What percent of your state is forest land? Some states are more forest than others. You know, the climate, the geography is vastly different across the country. You know, you could argue that you should punish more agricultural states that have torn down their forests or urbanized states have torn down. But you know, there's also states that are more deserted in general. So you know, there's just some interesting things you could take issue with. The number of motor boats was the one that I thought was most peculiar, it was in their kind of fun and lifestyle part of their index where they are trying to see how much leisure, how good the leisure and recreation is in a state. But you know, I'm from Minnesota and there's going to be a lot more motor boats in the land of 10,000 lakes than there are in say, Nevada. So, so double counting as well, that they're using their methodologies so it wasn't as rigorous as it could be. And then the biggest problem though that I would say is that replicability is very difficult with this index. So if I just wanted to just look and say okay, I just want to make a 2016-2017 Green Index, I'll use all their same indicators. I actually can't do that because a lot of their indicators were public available government data, but a lot of their indicators were other indices made by other organizations. And the big problem with that is that those organizations stop making those indices. And so if I wanted to replicate it then I'd have to go, you know, to the World Wildlife Fund and say, hey remember back in you know, 1990 you know, when you made this index, can we make it again? You know, so it quickly becomes an unmanageable task. So, when we set out to create our index, we are looking both at policies and conditions. So policies are the things that you know, the state legislatures pass, they funding conservation, water nutrient standards, carbon cap and trade, like the Regional Greenhouse Gas Initiative in the northeast. And then conditions are kind of the results we would say of those policies and of also things that the policies haven't yet addressed. But carbon missions, energy consumption, so on and so forth. So we started looking at collecting in both of those. And our guiding principles were, especially for conditions to normalize the data to be able to compare states to one another because otherwise comparing you know, Rhode Island and California doesn't make a lot of sense. So the most common ways of normalizing that is by GDP or by population. We chose to do it by GDP because it answers the question, you know, how much air pollution do you need emit to generate a $1 million of GDP? And that's what we like to call ecological efficiency. But it actually has pretty similar results when you do it by population as well which I'll show you. We also wanted to collect data from publicly available sources that were very easily accessible, very transparent, very replicable. So we're just looking for kind of like off the shelf data from the Department of Transportation, the EPA, the Department of Energy. We're not really looking for anything where you would need if you needed to replicate it to like go to every state and say hey you know, what is your data on this specific thing? And that's a problem when it comes to the policy variables. So you know, where a lot of those policy variables in order to see which states have these policies, you need to go to each state's legislature website, so on and so forth. So what we decided to focus on first was the conditions, the ecological efficiency side normalized by GDP to compare. And we came up with 8 to 9 indicators in these different categories. So in energy and climate we have carbon dioxide emissions. We get that from the Energy Information Administration, energy consumption from the same source. Air, we have criteria, air pollutants, criteria air pollutants are 6th, air pollutants that the Clean Air Act requires the EPA to regulate. And so we have those which is from the EPA. Transportation vehicle miles traveled, this is from the Department of Transportation. That's kind of a proxy variable to get it the numerous effects of the transportation sector from you know, the carbon emissions from it, air pollution from it, run off from it as well. And then in the water category, again, kind of proxy variables, fertilizer purchased, we were trying to get it run off from that, some nutrients from that. Water withdrawals, surface or ground, we can split that into two variables which is why this is 8 or 9 indicators. And then in the waste variable, we have toxic release inventory and hazardous waste generation. And these variables aren't perfect and I'll talk about that as well, how we got quite a bit of feedback on these variables. But these are the results here from the index including all those variables that were just listed. And as you can see, just so that you can actually read it, I just put the top eight and the bottom eight here. But the top eight are mostly small northeastern states. And the bottom eight are a little more geographically spread out but in general more resource extractive states or fossil fuel dependent states, a little less developed than the kind of small northeastern states in terms of urbanization. So and then this is also showing you per GSP, just means Gross State Product, so it's the same thing as GDP. But comparing that with population and you're seeing that, you're basically within 1 to 3 ranks on other per GDP or per capita. So, the challenges that we experienced with this particular index were data availability and data quality. Since we were looking for things that were right off the shelf, easily available, data availability was a big problem. And it was basically a big problem in anything except for energy. There's great data in energy. If you're doing a project on energy, congratulations it's going to be much easier than for instance water. If you're going to do a project on water, reconsider. Because water is just, a very interesting thing, it's not very available and the data quality itself is not high. And additionally, sometimes you'll just have to use variables that are proxy variables, so vehicles miles traveled as I mentioned is a proxy variable. It's not directly measuring what you're interested in but it gets added through you know, just saying this is the vehicle miles and then that's associated with the different effects of the transportation sector. But data quality as I mentioned with water, one of the things that we were considering for a policy indicator was you know, what percent of your water in a state have you assessed? And we thought okay that might be a good proxy variable for, how good of a handle do you have on your water quality? How committed are you to addressing your water quality? What we noticed in that data set which is reported to the EPA, is that some states would say oh how much percent of our water bodies have we assessed? 120 of course, 119%. And that is impossible so we asked the EPA about it and they basically just kind of shrugged and said you know, if we send it back to the states and say no do it again, we get a call from a congressperson that says hey, stop bothering my state. So, we just kind of tossed that variable out the door. Surface water withdrawals is another thing that has data quality issues. Because surface water withdrawals, we are trying to use as a proxy for like water use in a state. And for one, there's the issue of some states are going to depend more on their surface water, some states are going to depend more on their ground water. But surface water withdrawals also includes water that is being drawn out into hydro power damns despite that it's then being put right back. So, you know, we don't really want to punish states for using hydro power in that variable. So that's an issue that we ran into as well. There's also the issue of extreme outliers. One of the most important things you could do in projects like this, is really kind of delve into your data and see what's happening in terms of the range, the outliers etcetera. And one of the things that we noticed is that Alaska for instance was a crazy high outlier on toxic release inventory. So, number 1 on toxic release inventory is Vermont, releases about 271,000 pounds a year of toxic releases. And Alaska on the other hand is at 970 million. And Alaska is also much higher than anyone in you know, 49th or 48th place as well. And we don't really know why this is. We've hypothesized that it could be because of the mining operations in Alaska, it could be the injection wells and so on are you know, that's included in toxic releases. So you know, it's unclear but this is something that we've seen in the data and it troubles us, because we're not sure if it's a reporting error or if it's just something that maybe we don't want measured. And when it comes to things that we don't want measured, we also notice that in carbon intensity there are places like Alaska that had very high carbon emissions, much, much higher than anybody else even when we're you know, controlling our GDP. And I noticed when I delved into the data that the miscellaneous tier had a lot of the carbon emissions. So I emailed the EPA and that's another piece of advice is that you know, emailing these agencies directly and just asking them can be very helpful. And one of the things that the EPA told me was oh yeah and some of the earlier years you know, like besides our most recent data, we were including emissions from forest fires in your carbon emissions. And that's not really something you know, that I want to punish a state for. You know, because it's not from their direct industry. And it's not necessarily something that they have control over. So, those are two issues that we had. And then eco-efficiency indexed the feedback we got. People critiqued the appropriateness of the variables including like the fertilizer variable, water variable withdrawals, proxy variables people sometimes take issue with. And we also had some critiques about the fairness of comparisons. We presented this index to the environmental council of the states which is an organization of all of the heads of the environmental departments in all the different states. They were gathered in DC because they were actually considering making their own ranking. And one of the things that a representative of North Carolina told me was you know, I'd be comfortable compared to Tennessee, I'm not comfortable being compared to New York State. So, that's another thing that you could do. You can do a nationwide comparison like we've doing. In this project or you know, you could kind of break things out by region to see more specifically how people are doing against the neighbors. People also critiqued how it was kind of unfair against certain economic industries. You've got small northeastern states that have a lot of financial services which don't have a lot of direct environmental impact and then you have other states like North Dakota, Wyoming, Montana that are all more dependent on natural resource extraction and this was kind of punishing them. It was also pointed out to us that fossil fuel dependent states do poorly in this index, but you know, that's actually pretty intentional because of your fossil fuel dependent state. Then I think you shouldn't do well in an environmental index. So, additionally people wanted us to show performance beyond rankings because rankings don't tell you as much. So, that's one of the things we changed first is that you know, rankings are great, you can say great I got first place, I got second place whatever. But it's impossible to tell, you know, what's the distance between 22nd and 23rd place? It could be very, very close or it could be you know, a wide chasm that is there. So what we decided to do was start assigning state scores. And this is something that a lot of indices and rankings and everything, like a lot of places assigned scores. But people are like strangely proprietary about their score formulas. You know, they say oh they oh we base it on this, this and this, but they never show you the actual calculation. So just in transparency and replicability was important to us, we came up with a very, very simple formula that I'll show you on the next slide. But you know, this is an interesting thing as well because there are more decisions to make here. Do you base it on the average? Do you base it on the median? Do you base it on the standard deviation? When you base it on the average, it's susceptible to outliers. The median, to a certain extent is also susceptible to outliers, that there's enough outliers to bring it up. Standard deviation is something that was suggested to us at APPAM the policy conference when we presented this there in the fall and so we actually ended up using that, because standard deviation is based on the total distribution of the state's performance. And so it's not as susceptible to outliers and it works a little better for us we've found. The other things that we did with the feedback is we saw that you know, there's definitely issues with appropriateness, with data quality on several of the variables like toxic release inventory or the water withdrawals, so we decided to focus on a sub-index and really focus on the variables that have the highest quality data, that have the most favorable feedback on them and that were also thematically linked. So we created something called the Air Climate and Energy sub-index which we'll just call the ACE index. And that has energy efficiency, carbon intensity, vehicle miles traveled and criteria air pollutants, so things that are all connected to air climate and energy. And the things that we also have the most confidence in, as I mentioned with energy, a lot of confidence in their data quality and data availability and that's two of the four variables here. So, our scoring formula, as I mentioned it's based on standard deviation, its super simple. I don't even have you know, any Greek on the board. It's basically just standard deviation divided by performance times 10 so that you can read it a little easier is the times 10. But and then we weighted everything in the ACE index except for the vehicle miles traveled variable and our justification for that was that vehicle miles traveled was more of a proxy variable. The other ones are more direct measurement variables, so we weighted vehicles miles traveled a little less. So the advantage of this is that you know, the magnitude between the states is revealed, the rankings don't tell you that but the scores do. The results are comparable across indicators, because the performance and the standard deviation are both within that measurement but then they produce a score that you can prepare you know, across energy sufficiency and carbon intensity. So then just as an example on how simple this formula is and how it works. Michigan consumed about 2000 megawatt hours per million dollars GDP in 2013. And Alabama which was you know, the first state in the alphabet, consumed about 3000 megawatt hours per dollars GDP. So the standard deviation is about 806 for energy efficiency, for energy consumption and then you see the results from that, the higher the score, the better you are. So very easy to understand, pretty simple based on standard deviation, not as susceptible to outliers. So you know, it's, you could do more complicated things that might give you more interesting results, but we really wanted to kind of keep it simple and have something that was easily understandable. So now we are closer to the results which is, which is something that people always like to see. And when it comes to results, there's a lot of different ways to display your results. So obviously there's tables, the classic way, but as you saw earlier I'm only really showing the top 8 and the bottom 8 of the states in these tables. And so you know, you can't show all 50 states to an audience like this. So then there's maps, great for comparing the states but less specific information known on that. Radar charts, I'll show you one of those for Michigan which kind of shows within state performance. And then scatter plots as well can show you good trends. So on the ACE index, again this is just the four variables, carbon intensity, energy efficiency, vehicle miles traveled and criteria air pollutants. These are the results, so the results of the scores and the rankings there. So top 8 again, mostly small northeastern states, but we also have California now which is not even in the top 8 on the eco-efficiency index is second here. We have scores that range up to 47 and down to 2. On the bottom we have Montana and North Dakota and Wyoming, more natural resource intensive states as well as Alaska. Mississippi is not dead last so they would be happy to see that. But it's still on the bottom 8 there and some other states there as well. So this is then showing again per GDP versus per capita for the ACE index, not the eco-efficiency index which I showed earlier. And as you can see it's pretty similar across the board. There is a one bigger difference that I'll point out with Mississippi, they have I think a below average GDP, so they do a little worse in the GDP and a little better in the population moving from 46 to 49th place in that particular ranking. So this is what I mentioned earlier with the different ways that you could do your scoring formulas. So on the left I have standard deviation which is what I just showed, top 4 and bottom 4 here. And these are the scores that are produced when you base it on standard deviation. These are the scores in the middle that are produced when you base it on average and the rankings. The top 4 are the same there with 3 and 4 have switched so New York moves up. And then the bottom 4 are almost the same except Alaska does a lot better. And this also, you see in the median, the top 4 stay the same, bottom 4 you know, Montana, North Dakota, Miami holding down the fort across the board, same rankings. But Alaska moves to 38th place and 40th place when you base it on average and median. And I'm thinking that that's because Alaska is benefitting when they're skewing up the average or the median because it's a higher, it's closer to them when the average and median are both closer to them when they're doing that when they're biasing it upwards. Whereas, standard deviation it's less susceptible to that. So, everybody loves maps, I say that because I love maps. I'm not sure it's actually true, but there's a bunch of maps that I can now show and this is the overall ACE index. So we have again, the variables on the right for anyone who's interested in the criteria air pollutants. The 6th are ozone, particulate matter, carbon monoxide, sulfur dioxide and nitrogen dioxide and lead is actually a criteria air pollutant. But since the successful legislation in the 90s, since 2005, lead has been so negligible that it's been hardly reported which is really good news. So this is what you see, darker green is better, gold is worse. And this is, this is something that you know, you don't think that colors on a map is necessarily going to make that much of a difference. But one of the most common feedbacks that we got was that it was you know, be nice to the states that aren't doing well. So, at one point and time I listed the bottom states as something like you know, middling performance or something and I got quite an earful for that. So now you know now we don't use green and red or anything like that. It's you know, you're just kind of a yellowish color if you're not, if you have room for improvement. So you know, it's something that I did not run into but it's definitely a useful thing to keep in mind. So again you have here the overall results and then you can specify this to each variable. So I used a free mapping application, I used, called [inaudible] public. I highly recommend it and I can, you can make these great maps pretty easily. Carbon intensity, you have a pretty similar to your last slide, but there's a couple of states like Idaho that are doing better than usual because they've got their hydro power, that's what we think and then the northeastern states doing well and everything. And one of the reasons that we're making this ranking, this index is so that researchers will look at these maps and look at these results and say oh like I wonder why this state is doing well or poorly or whatever. I wonder why these results are the way they are. And that's something that the index can then you know motivate, is further research on these things. That's one of the reasons that we wanted to create it is that researchers would be able to use it you know as an indicator of their own if they want or to just get into the reasons for these different levels of performance. So criteria air pollutants is a little more interesting. There's outliers on the other side in criteria air pollutants. So there's people who or states rather who are doing really, really well on criteria air pollutants and then other states they are just doing you know, pretty well. So you know the middle of the country in general is a little lower on these rankings, northeast and west doing a lot better. And again those are all the criteria air pollutants that are included here. So then energy efficiency states in general do better on this one, but again you're seeing the same pattern on the northeast and the west coast. But you also have some states like North Carolina, Virginia, Georgia, Florida that might be doing a little better than you might expect. North Carolina in general actually in this index does better than you would expect and a little bit in the Midwest there as well. And then vehicle miles traveled, you'll notice that Alaska on all of these was doing poorly. Alaska is number 1 in the country for vehicle miles traveled per GDP. And the reason I think that this is because there aren't many roads in Alaska. Intercity travel in Alaska is mostly done by you know, ferry and plane. And so it's not like a place like for instance in Montana where if you want to get to anywhere else in Montana you're going to be driving long stretches. So you know, that's what we're thinking but you know, it could be easily interesting to get more into that, same thing with Hawaii. But you see a lot of the kind of southeastern states, more driving dependent states not doing as well on that. And I think too that I should mention is that you know, some people register a surprise for instance that Texas is doing better on this variable and there is a little bit of bias giving to states that have very high GDPs since we're controlling by GDP. Texas has very high GDP as does California. So in general you know, they have more room to have like higher vehicle miles traveled for instance because of their very high GDP. So, specifics on Michigan, we have the, they're basically, Michigan's basically in the middle of the pack but a little better than 25th place on most of these variables. These are its scores. Again, the ACE index is with weighting and so that's why it's you know not bigger than the sum of the others. But 23rd in ACE index, 28th in carbon intensity, 31st in vehicle miles traveled, 22nd in criteria air pollutants, 26th in energy efficiency. So this is, on the left an example of a radar chart. And this is something that in our experience with the feedback that we've received, states like a little more, because they can see where they're doing better and where they're doing worse. But basically as I mentioned before, the scores, higher scores are better. And what you can see here is you know, like if higher scores are better than a bigger shape is better. And Michigan is doing pretty well on criteria air pollutants. Not as well as on vehicle miles traveled, a little better on carbon intensity and energy efficiency. And you know, Michigan and Ohio I wanted to compare. And I don't want to get kicked out for this or anything, but they're neck and neck and Ohio may be better in some areas but you know, for the most part across the rankings they're one rank away usually. But this is just another way too of showing this is just stacking the scores of all the variables on top of each other which you could do because the scores are comparable across indicators and so then you can just kind of see how that looks. And when you know, the states are slightly more different for instance Alabama and Michigan, you can see the difference better. But this basically looks like the same two bars there. So another thing that you could do is you could start to look at variable correlations and so you know, this is something that I just plugged into Stata but you could probably also do this is Excel. But it's just seeing how much these variables are correlated and how much they move together and everything. And so as you can see, vehicle miles traveled criteria air pollutants across the board you know, we're always higher than 0.5. These are like pretty strong correlations. Energy efficiency and carbon intensity are very highly correlated at 0.93 and that's actually a bit of a concern. We like to see some level of correlation with all of these variables you know, Dan Fiorino who heads the Center for Environmental Policy makes, uses this to make the argument that you know, when you prove one thing, you're improving these other things across the board. But one of the concerns is that carbon intensity and energy efficiency are so strongly correlated that we might be double counting a little bit there. Because you know, energy related to carbon emissions and so that would be a concern that we could kind of explore more. So one of the other things that we could explore more is what are the reasons for the variations in state performance? So this is just again the overall ACE index map. But you know, why is it that West Virginia is not doing well or Montana is not doing well. And as I mentioned this is the type of thing we want our index to motivate research on, but we can come up with our own hypotheses as well. So if you list it here in natural resource extraction, I mentioned fossil fuels, I mentioned. But two interesting things that I want to mention as well is that there's what's called a race to the bottom among states. And that's something where states will race with each other to cut their environmental regulations to attract businesses to their state. Vogel, the scholar called this the Delaware effect. Not because of Delaware's environmental regulations but they basically used like their tax regulations and cutting those to attract more corporations to their state. And so that's something that's studied a lot and studied a lot especially with southeastern states that are trying to create business friendly environments by cutting a lot of those regulations and making it really cheap to setup there but there's also a thing called race to the top. And that's where states that are already dark green are basically trying to race to be the best in all of the you know, environmental rankings and everything. California comes out with a policy and other states are trying to come up with similar policies because they're also you know, not only businesses, but that residences are looking to locate in states with good environmental policies. And there's also the fact that occasionally the feds will come in and say we want to create a new policy, we're going to base it on for instance California's policy. And so that's a bit of a race to the top, because then California's very stringent policy brings everyone else up as well. So you know or it could be you know as much as we're conjecturing about all those other things, maybe it's just that red states are not as good and blue states are better or maybe you know, it's a policy thing, so we'll get into some of that here. So natural resource extractive industries unsurprisingly on the top three states on the ACE index have less than 1% of their GDP and their employment in the extractive industry. And then the bottom three states have much more than that with Wyoming having the most 22% of it's GSP is from the extractive industry. This is all from the U.S. Extractive Industry Transparency Initiative which is led by the U.S. Interior Department. These numbers though were still kind of surprisingly low on the bottom three in terms of like percent of employment especially. But this is just an interesting thing you can see you know, maybe that's the reason that Montana, North Dakota and Wyoming are always 48, 49 and 50. So it could be a policy thing. So one of the things that we did that is instead of including other indices with in our own index is that we could just compare our index to there's. So our ACE index is based again on performance, on conditions. So it's like the conditions that are actually occurring. Whereas the Council for an Energy Efficient Economy has created a whole index on the policies that each state has passed specifically relating to energy. And when we compare our index on the bottom there and the ACEEE index, it's a pretty up and down positive correlation. Correlation scores specifically of 0.67 and this is something that you know, we can use to kind of confirm our results if this looks right. Our air climate and energy index is matching up pretty well with the energy policy index. And so this is something that you know, if we wanted to then we could get into even further with regressions of trying to see how strong that relationship holds up. So, red states and blue states and those in between, we noticed that we saw this article by Jacob Hacker and Bob Pierson in the New York Times in July of 2016 a couple of months before the election. And they color coded the states on a zero through 4 scale based on how many times they had voted democratic in the past four presidential elections. So again, this was before the 2016 presidential election. So, the past four presidential elections you get dark blue if you went democrat all four times. You get light blue if it's three times, purple if it's twice, light red if it's once and then dark red if you didn't vote democrat at all in the past four elections. So then they put this and ranked the states at a variety of different indicators. So, we have like on the right, that's education, we've got on the left income innovation, life expectancy, so on and so forth. And the point there I'm making is that you know, the blue states are pretty much at the top and the red states are pretty much at the bottom of these rankings. So we anxiously stole this and gave them you know, credit for it and everything. But this is what it looks like with our indicators. And so to compare that I would argue that it's even more or strongly divided with blue states at the top and red states on the bottom. And I updated the color code to include the 2016 and exclude the 2000 election, so the past four elections which actually turned more states blue than red because of what happened in 2000. But this is the result of that and the, it's probably hard to read but this is carbon intensity, vehicle miles traveled, criteria air pollutants and energy efficiency and the overall index over on the right. And as you can see with vehicle miles traveled, Alaska up there. The you know, only kind of, Alaska and Texas are the deep red at the very top of the rankings there. So I also did a simple correlation with this. So carbon intensity and all of my variables with that color coded variable just the zero through four variable. And this is what I found, pretty strong correlations across the board. The overall index is almost 70% there. Energy efficiency and carbon intensity are very high as well. So, in conclusion my advice, especially for the students in the room, but for anybody whose you know, looking at these index, indices or wants to create their own indices, seek out feedback and listen to it. It can definitely be harsh but it's really very useful and you know, I've often said you know with these types of projects and with life in general, never take yourself too seriously. You know, if someone says I you know, I'm not really a fan of that particular way of ranking it, you know, it's like okay they're right. You know, maybe it's not an insult to your character type of thing. So, so you know, it's a really interesting thing. Anybody can give you feedback on these types of things, because everyone has kind of an intuitive sense of what makes sense and what doesn't. So you know when it's talking to your friends or in my case I talked to the Center for Environmental Policy's advisory board you know, all the professors and so forth. So looking into alternative approaches also really helps also calculating for GDP and per capita because the most common feedback we got is why didn't you just control in population and then I can show I did and here's what it looks like. Knowing your variables and indicators is very helpful obviously. Keeping good notes to remember why you did what you did. And then being patient, you know, I often kind of chuckle at myself when there's news articles about oh this is the era of big data and everything like that. But for a lot of the data that we're trying to work with, it's not that great yet. You know, we're not really, we've moved quite a long ways but we're still working with a lot of data that has a lot of quality issues, that has a lot of availability issues and things like that. So you know, just being patient with that and understanding that you maybe have to use proxy variables or you maybe have to use a variable that doesn't report exactly what you want it to. You know, these are just limitations and every project has limitations. So you talk about them and you acknowledge them and you just try to do the best you can with the data that's available. So, that's it, I'll be happy to answer any questions.

[ Applause ]