Scenario Analysis And Sensitivity Analysis

Historical Scenario Analysis

Historical scenario analysis (also known as historical stress testing) is a form of backtesting where we investigate the risk and return of an investment strategy during various regimes, while taking into consideration the various structural breaks that are present.

Two typical examples of regime changes are when volatility changes, or when the economy switches from downturn to expansion, as described here:

1. Recession versus expansions. It is normal for an economy to occasionally enter periods of economic decline; one relatively recent example is the “Great Recession” that followed the global financial crisis of 2007 and lasted until 2009.

One point to note is that the beginning and end of such a downturn are defined retroactively; it is common for the beginning or end of a recession to not be formally identified by economic research organizations until a year or more later.

1. Low- and high volatility periods. We might define a high volatility period as one where the VIX is above its five year moving average, versus a low volatility period where the VIX is below its five year moving average.

(The VIX index was created by the Chicago Board Options Exchange [CBOE] to measure the volatility implied by the prices of options on the S&P 500 Index.)

Many other situations lend themselves to scenario analysis; for example, we might simulate the performance and risk of investment strategies in both “trade agreement” and “no trade agreement” environments.

The motivation for considering various regimes is that the risk and return of various strategies tends to vary depending on the regime. For example, thinking about the risk parity factor portfolio we considered earlier in terms of Sharpe ratios, the RP strategy is likely to be more resilient in a recession compared with the BM strategy. Furthermore, the RP strategy is likely to perform well in both a low volatility or high- volatility regime, while the BM strategy may not perform well in a low volatility environment.

We can also use a probability density plot to further explore performance in various regimes. For example, return distribution for the BM and RP strategies has been found to be more dispersed in an expansionary environment than in a recessionary environment. We can also evaluate non normality, for example, the BM strategy exhibits negative skewness and fat tails in both an expansion and a recession, though with a lower average return in a recessionary environment.

(The RP strategy performs similarly, though with lower kurtosis and volatility.)

Simulation Approaches

Asset allocation strategies are based on the general idea that the future will somewhat resemble the past.

However, another major assumption is that asset returns will take the shape of a multivariate normal distribution. In reality, return distributions frequently exhibit negative skewness and excess kurtosis (i.e., fat tails). Excess kurtosis indicates that a strategy is more likely to generate extreme returns (i.e., surprises), while negative skewness warns that these surprises are more likely to be negative than positive.

Both of these characteristics are considered undesirable by risk averse investors.

For this reason, conventional asset allocation methods that use covariance (e.g., mean− variance optimization) may at times yield misleading outcomes. Furthermore, rolling- window backtesting can only take into consideration things that have already happened in the recorded past, and it thus may not accurately characterize potential downside risks that result from changes in the financial markets or other unforeseen events.

For these reasons, we supplement our rolling window backtesting with (1) scenario analysis and (2) simulation.

Simulation Analysis Backtesting is based on the concept of going back to the past, implementing our investing strategy, maintaining the portfolio over time according to a set of predetermined rules, and then evaluating the resulting performance. This process has a number of benefits, for example, it is easy to understand. However, we are implicitly assuming that history will repeat itself.

With ever changing financial markets, this is often not a valid assumption: never before seen risks and opportunities occasionally arise.

Consider two primary classes of simulation: historical simulation and Monte Carlo simulation.

In a historical simulation, we select past returns at random from a long historical period, without regard to the order of occurrence. In an asset management application, this means that we randomly sample (with replacement) from the past record of asset returns, where each set of past monthly returns is equally likely to be selected. This methodology is similar to rolling window backtesting, except that we abandon the time sequence in which the returns actually occurred.

Financial institutions, and in particular banks, make extensive use of historical simulation to evaluate risk.

A downside of historical simulation is that we only have actual past data to draw from, and the past only occurred in one particular way. Monte Carlo simulation overcomes this issue by assigning a statistical distribution to each relevant variable, and then drawing random observations from this distribution. Monte Carlo simulation randomly samples from an assumed multivariate joint probability distribution, and the parameters of the multivariate distribution are calibrated using the past record of asset returns.

The Monte Carlo approach is very flexible, allowing a variety of distributions. In this way, we can allow actual characteristic such as non normality, tail dependence, fat tails, et cetera in our key variables, rather than forcing a normal distribution to the data or limiting ourselves to actual historical results.

One possible downside of Monte Carlo simulation is its complexity, which necessitates significant computing power. A particularly useful application of simulation is to try to evaluate an investment strategy’s downside risk. Simulation can also help us to validate the results of backtesting.

Steps typically used to perform a simulation analysis are as follows:

1. Select the target variable that we want to analyze. Usually, we are trying to evaluate the return of a particular strategy, as well as its return distribution. 2.

Determine the key decision variables. These include the returns of the underlying assets, and the portfolio weights of each asset as indicated by the strategy.

1. Select the number of simulation trials (“N”) to run. Choosing an optimal number of trials is complex; however, a number in the range of 1,000 to 10,000 is typically used.

Generally, the greater the number of trials, the more stable the results will be. 4. Specify a distribution for each key variable. Here, historical simulation and Monte Carlo simulations differ.

For a historical simulation, we randomly select samples from historical data. For Monte Carlo simulation, however, we instead choose an appropriate statistical distribution (e.g., normal, uniform, lognormal) for each key variable. We can also take tail dependence and correlations between variables into account. 5.

Draw “N” random numbers for each key variable. A random number generator is used to select these values. 6. Compute the values of the target variable.

Using the values of the key decision variables determined in the previous step, we calculate the corresponding value of the target variable (such as portfolio return). These results are compiled for use in the final step.

1. Repeat random number draws and calculate target variable. Steps 5 and 6 are repeated for “N” iterations. 8.

Calculate metrics. Using the “N” simulated values of the target variable, we compute measures such as average return, variance, Sharpe ratio, etc. We can also calculate various downside risk metrics, such as maximum drawdown and CVaR.

Simulation Inputs and Decisions

Historical Simulation

Historical simulation shares some similarities with rolling window backtesting; they both use the past to try to forecast the future. However, they differ in that historical simulation introduces a non deterministic element: historical simulation selects random pieces of history, rather than simply using it chronologically.

Historical simulation can be performed with or without returning the samples for further selection. We use the term bootstrapping to refer to sampling with replacement; this technique is useful when the number of simulations is large relative to the size of the data set.

Monte Carlo Simulation

Monte Carlo simulation has some commonalities with historical simulation. However, a significant downside of historical simulation is that the data is limited only to things that have actually happened in the past, which may not be a useful representation of the future. Monte Carlo simulation can overcome this issue: Monte Carlo simulation is a non deterministic technique that does not use historical data directly.

The first step in Monte Carlo simulation is to select an appropriate distribution for each key decision variable. This step requires us to consider not only the mean and standard deviation of each variable but also other higher order moments such as skewness and kurtosis, as well as tail dependence.

The tail dependence coefficient measures the correlation (i.e., co movements) between the tails of two random variables. Two variables with a high tail dependence exhibit tail probabilities that increase and decrease together, and a plot of the joint distribution of returns for the two factors will show peaks in the tail. We are likely to underestimate downside risk if we fail to account for positive tail dependence.

Various analytical tools and techniques can be used to fit a distribution to each of the key decision variables. Model calibration is then performed using regression and distribution fitting techniques to estimate the parameters (such as standard deviation, mean return, kurtosis, and skewness) underlying the statistical distributions of the key decision variables. Programming languages such Python, R, and Matlab can fit these variables with relative ease.

In order to produce useful results from a Monte Carlo simulation, it is vital that we select statistical distributions that correctly fit the historical data. Asset returns are often approximated by distributions such as the Student’s t distribution or the normal distribution. However, it is important to factor in any correlation that may be present between variables.

When we are simulating multiple assets or factors that have returns that are correlated, it is critical to specify a multivariate distribution rather than modeling each asset on an individual basis. A multivariate distribution allows us to represent this correlation, unlike standalone modeling of each factor.

It is important to recognize the tradeoff between having too few parameters versus making a model too complex. A model that is highly complex is likely to describe the historical data well; however, it will contain many parameters that must be estimated using historical data. If the historical data is limited, our estimates of these parameters are likely to be imprecise, leading to large estimation errors (despite low specification errors).

Conversely, a model with few parameters may not fit the data well (because it is misspecified), even though it has low estimation errors.

Interpreting Simulation Results

We use several measures to interpret the results of simulations:

• Sharpe ratio
• Downside risk measures CVaR

To interpret simulation results, it is useful to examine the difference between the simulated performance of the benchmark versus our portfolio.

Historical simulation and Monte Carlo simulation serve as useful complements to (and validation of) backtesting because they model randomness using different approaches and therefore provide us a richer analysis of performance.

Different approaches (i.e., backtesting, historical simulation, Monte Carlo simulation) are likely to produce different outcomes. Varying specification of parameter estimates, distribution characteristics, et cetera will produce varying estimates of the Sharpe ratio, CVaR, and downside risk.

Sensitivity Analysis

Sensitivity analysis can be used to gain a better understanding of the risks and returns of an investment strategy. Specifically, sensitivity analysis helps us to understand how our target variable (typically portfolio return) varies due to changes in one of the various input variables.

The result of a Monte Carlo simulation depends heavily on whether we have accurately specified the distributions of underlying variables. In the presence of fat tails and skewness, the joint distribution of returns will rarely be multivariate normal, so means and variances of these returns and the correlations between them will be insufficient to describe the joint return distribution.

To quantify the risk of misspecification, we can conduct a sensitivity analysis by calibrating a different distribution to our factor return data and then running the simulation again.

The process for performing a sensitivity analysis is similar to the process described earlier for performing a Monte Carlo simulation; however, we fit the data to a multivariate skewed t distribution rather than to a multivariate normal distribution. (The multivariate skewed t distribution is similar to the multivariate normal distribution, but it allows for fat tails and skewness.)

After generating (for example) 1,000 simulations using this new distribution, we evaluate the strategy risk and return indicated by the results. We can then compare the results of backtesting and simulation, using measures such as Sharpe ratio or conditional VaR. We may find that the various simulation methods support each other, or that they paint different pictures of, for example, left tail risk.

Untitled