Building an integrated model
The model developed for this study includes several variables across sectors and thematic domains. This can be achieved by ensuring data consistency (within and across sectors), which allows for the creation of a more comprehensive analysis that in our case includes energy demand, supply and emissions, as well as estimations of the investment required to reach desired emission reduction targets, resulting energy savings and employment creation. In practical terms, we have performed “knowledge integration” in a single framework of analysis for low carbon interventions.
Data Selection
Several data sources are used to customize and calibrate the model. These span across sectors, (including variables such as population, GDP, energy demand and emissions) and consider time series (to better understand historical trends and validate the model).
Specifically, the main data sources used to customize and calibrate the model at the country level include the World Bank’s World Development Indicators (WDI), United Nations Population Statistics, the World Economic Outlook (WEO) of the International Monetary Fund (IMF), and World Energy Balances published by the International Energy Agency (IEA). Several additional studies are used to better customize the model to national circumstances, including specific research on labor intensity and technology assessments, as well as national reports on policy and resulting energy and emission outlooks (e.g. National Communications to the UNFCCC) and of course, the INDCs published ahead of the COP21 meeting in Paris.
These data, which are checked for consistency across sectors as well as against national databases, are used to populate a model that generates projections from the year 2000 to 2040. These projections, both the historical (2000 – 2014) and future (2015 – 2040) portion, are carefully analyzed and validated.
Exploring causality
This model is used to assess the outcome of policy interventions. Targets can be introduced, and the investment required to reach them is estimated by considering the cost of technology (e.g. depending on the sector and the energy source affected) and the timing of the target (e.g. a learning factor is included in the analysis). As a result of investment, energy demand changes (e.g. due to an improvement in efficiency) and the energy mix deviates from the baseline scenario (e.g. due to the expansion of renewable energy). A different level of energy consumption and a new energy mix result in a change in the energy bill (e.g. leading to savings on energy expenditure), which compared with the investment required, gives us an indication of the economic performance (or desirability) of the interventions analyzed. Further, the combined effect of changes in energy demand and energy mix leads to a reduction in emissions, which are then compared to the desired target, against a base year (e.g. 2005) and relative to GDP and population.
Since the implementation of the investment requires labor (e.g. for the construction, installation and operation and maintenance of power generation capacity), the model also estimates employment. It estimates employment creation in two ways: based on the construction, stock and discard of fixed capital related to energy supply (e.g. coal power plants) as well as based on the investment, and resulting energy saving in the case of energy demand management. While employment creation (and relative salaries and wages) is generally considered to be a cost in a conventional cost benefit analysis (CBA), it is normally accounted as an added benefit by governments.
Validation
Models can be classified in many different ways and assessed according to different criteria, such as physical versus symbolic; dynamic versus static; deterministic versus stochastic, etc. As it relates to the notion of validity, a crucial distinction must be made between models that are “causal-descriptive” (i.e., theory-like or “white-box”) and models that are “correlational” (i.e., purely data-driven or “black-box”).
In correlational models, since there is no claim of causality in structure, what matters is the aggregate output behavior of the model; the model is assessed as valid if its output matches the “real” output within a specified range of accuracy, without any questioning of the validity of the individual relationships that exist in the model. This type of “output” validation can often be cast as a classical statistical testing problem. Models that are built primarily for forecasting purposes (such as time-series or regression models) belong to this category.
On the other hand, causal-descriptive models (such as the one developed for this study) make statements about how real systems actually operate in some aspects. In this case, generating an “accurate” output behavior is not sufficient for model validity; what is crucial is the validity of the internal structure of the model. A causal-descriptive model, in presenting a “theory” about the real system, must not only reproduce or forecast its behavior, but also explain how the behavior is generated, and possibly suggest ways of changing the existing behavior.
The performance of the model presented in this study is checked against historical data (i.e. behavioral validation), and several additional tests are done to validate the variables and equations used (i.e. structural validation). The result is a state of the art model that is consistent across sectors (e.g. energy demand and supply) and dimensions (e.g. employment, economic investment and environmental emissions), capable of replicating historical trends and projecting outcomes consistent with the best available research.
Structural Validation
Direct structure tests were performed to assess the validity of the model structure by direct comparison with knowledge about the structure of the real system, equation by equation. Examples of direct structure tests include: (1) structure confirmation tests; (2) parameter confirmation tests; (3) direct extreme-conditions test; (4) dimensional consistency test; (5) behavior sensitivity tests; and (6) phase-relationship tests.
Direct structure tests can be classified as empirical or theoretical. Empirical structure tests involve comparing the model structure with information (quantitative or qualitative) obtained directly from the real system being modeled. Theoretical structure tests involve comparing the model structure with generalized knowledge about the system that exists in the literature.
Behavioral Validation
Each country model includes 230 variables across sectors. Of these 230 variables 54 are constant, and several others represent conversion factors. In total, 80 to 100 variables per model (depending on country data availability) were tested against historical data from 2000 to 2014. This indicates that while the model is initialized using historical data for the year 2000 it is not fully driven by data until 2014. As a result, behavioral validation needs to be carried out to assess whether projections (resulting from the simulations of over 200 equations per country, using a yearly time step) reproduce observed historical trends. The analysis includes pattern prediction (periods, frequencies, trends, phase lags, amplitudes, etc.) as well as point (event) prediction.
Dealing with Uncertainty
While the model has undergone several validation tests to be able to generate solid scenarios, there are several elements of uncertainty that depend, rather than on structural soundness, on the availability of valid model inputs as well as on the presence of external factors affecting a country’s performance.
Sensitivity analysis was carried out for selected parameters for each country model. Monte Carlo simulation techniques were utilized to estimate the variability of model outputs to changes in model inputs. This provided a deeper understanding of the potential range of results that could be obtained from the model (e.g. on emissions and the payback time) when alternative assumptions are utilized (e.g. on the response of energy demand to changes in energy prices).
The following two types of sensitivity analysis were performed:
- Numerical sensitivity exists when a change in assumptions changes the numerical values of the results. It is an inherent property of models to exhibit numerical sensitivity; testing is to assure responsiveness consistent with the functions and feedbacks of the model.
- Behavior mode sensitivity exists when a change in assumptions changes the patterns of behavior generated by the model. For example, if plausible alternative assumptions changed the behavior of a model from smooth adjustment to oscillation the model would exhibit behavior mode sensitivity.
Customizations
The model is easily customized through modifying several scenario inputs. Some of these potential customized scenarios are policy-related, such as the energy efficiency or renewable energy target, while others have to do with external conditions (more properly defined as scenario drivers, as opposed to policy drivers). A full list of these customizable variables can be found here.
Given the high degree of model customization, alternative simulations can be generated to better match a client’s expectation of the future. This is relevant because, for instance, investments in energy efficiency would be a lot more attractive under a scenario of high energy prices than in a case characterized by declining energy prices. When thinking of recent trends, alternative scenarios could be tested on the impact on emissions if oil prices suddenly returned to above $100 per barrel. What emission reductions would this lead to, and to what extent would the investment required to reach a desired emission reduction decline?
An additional option for customization is a more marked modification of the analysis. For instance, if budget were the constraint to emissions reductions, the model could be modified to use investment as an input and estimate the emission reduction as an output (along with avoided energy costs, employment, etc.). Further, customized scenarios could be created to re-direct avoided energy costs to specific energy demand management interventions (e.g. as a way to share the cost between public and private sector) as well as to other economic activities, to estimate the indirect and induced impact of energy savings (e.g. on productivity and GDP).