For years, Phase 5 and 6 firms captured by the Uncleared Margin Rules (UMR) have been anticipating that the regulation’s backtesting-benchmarking requirements were to be dropped. Now, those firms have to act, as the draft Regulatory Technical Standards (RTS) on SIMM (Standard Initial Margin Model) Validation, published last week by EBA (European Banking Authority), states that backtesting and benchmarking are here to stay.

We chat with Andy Shaw, Founder of Links Risk and 25-year veteran of backtesting risk models, and Cassini’s Head of Product, Thomas Griffiths, to discuss the implications that the draft RTS will have on firms captured by UMR. 

For more information on Cassini’s backtesting service, view our product sheet or get in touch with one of our experts. 

EBA have released their draft RTS on SIMM Validation. What are the implications for firms impacted by UMR?

Andy Shaw (AS):  Model validation is here to stay in relation to SIMM and UMR, and firms need to either self-build or use vendor backtesting and benchmarking methods if they are to stay on the right side of this regulation.

Thomas Griffiths (TG): I totally agree, the main implication is that no firm is going to get relief from this requirement – there is an expectation that some form of backtesting will always be required for all in scope UMR firms.

The industry expectation was that backtesting for Phase 5 and Phase 6 firms was likely to be dropped from the regulatory guidelines. Is backtesting still mandatory based on the EBA Draft RTS?

AS: Yes – either a static backtest or a dynamic backtest.  Practically speaking, backtesting (and benchmarking) is all about the process of ensuring that your initial margin (IM) levels continually meet regulatory set thresholds – 99%  confidence level using a 10-day margin period of risk (MPOR).  To do this, while a dynamic backtest can help and sometimes is enough, to ensure the most efficient margin levels are maintained, users of SIMM will need to rely on a static backtest.

TG: It is still mandatory, and although some consideration has been taken to the size of the firm, the EBA is still expecting there to be governance processes and technical analysis on the performance of SIMM from all firms. This means that every firm in scope for UMR, in the forementioned regions, needs to consider backtesting in their UMR solution.

Is the EBA suggesting any change in their approach towards the model validation for phase 5 and phase 6?

AS: No, while regulators draw a line between counterparties using an Average Aggregated Notional Amount (AANA) calculation, allowing ‘smaller’ notional counterparties to pursue a slightly simpler approach to model validation, the need to backtest remains for both counterparties exposed to the ‘standard’ and ‘simplified’ model validation processes.

TG: I agree, very simply these are two things that firms now need to ensure they have in place, with their UMR solution – on top of the basic legal, process and calculation requirements that firms were already planning for.

What should phase 5 and 6 firms do now?

AS: Firms should ensure their uncleared derivative operating models more broadly, and in particular, their model governance frameworks, meet all aspects of the required regulation.  This includes having a backtesting approach that is fit for purpose and that, most importantly, can be effectively utilised to maintain IM at levels that reflect regulatory set thresholds on an ongoing basis.

TG: Two things. Firstly, set up a documented governance process for their SIMM calculations  and secondly, ensure they can calculate a robust backtest of their portfolio.

Briefly, how is backtesing used?

AS: Backtesting output is used to ascertain if a model is performing as required, including whether or not it is failing.  If a model failure occurs, perhaps as a result of increased market volatility, backtesting output will be used to re-set IM levels before SIMM is re-calibrated.  Put simply, this output can set the “cost of holding risk”.  Given backtesting output is very sensitive to the parameters used to calculate it, failure to be able to calculate and understand your own backtesting output could lead to a loss of control over your cost of risk.

TG: The Cassini backtesting output consists of a suite of documentation on governance, model risks and backtesting and benchmarking reports that are standardised for a firm to take to their regulators. This provides our clients with immediate comfort that they have met their regulatory and compliance obligations. As Andy points out, one of the main non-regulatory advantages to this is that if the SIMM IM amount is at any point considered to have failed due to an increase in market volatility, a firm with a backtesting output is able to negotiate any change in IM levels on an equal footing with their counterparty, rather than being subject to enforced increases in margin due to a lack of backtesting information

What does the future hold for backtesting?

AS: Backtesting IM models has been a routine Central Counterparty (CCP) risk process for several decades now.  Techniques used by CCPs have slowly increased in sophistication, particularly as a result of the regulatory changes brought about by the Lehman default.  Regulation in the US and Europe is now starting to align on backtesting for uncleared derivatives, and if the experience of CCP’s is anything to go by, not only is backtesting here to stay, but its standards may also well slowly creep higher as these techniques become established for users of uncleared derivatives.

TG: Backtesting looks to be a core requirement for regulators now with regards to SIMM, and this RTS is proof that it is not an item that regulators are comfortable with simplifying away for smaller firms.

Thomas has almost two decades worth of experience in the capital markets and financial services sectors. Starting his career at ABN Amro bank in the early 2000s, Thomas has since held numerous senior management roles including Head of Equity at RBS and Head of Business Management at TriOptima. Most recently, Thomas can be attributed to TriOptima TriCalculate’s success, in his role as CO-CEO. Now, Thomas is Head of Product at Cassini Systems, and the primary driver of the Product vision; overseeing the development and management of the product’s roadmap and creation of solutions that deliver value to both customers and business goals.

Andy Shaw founded Links Risk Ltd. in 2008 and is Links Risk’s Managing Director. He is a veteran of risk analysis work, and specialises in cutting-edge financial product valuation and best practice risk management. Andy was selected as a derivatives expert witness, alongside leading economists and industry professionals, to represent JP Morgan in one of banking’s largest litigation cases. More recently he has become regularly involved in the uncleared margin space, advising clients how to prepare for and capitalise upon the significant changes influencing the management and trading of financial market bilateral risk.