Key Control Indicators or KCIs also referred to as Control Effectiveness Indicators are metrics that provide information on the extent to which a given control is meeting its intended objectives (in terms of loss prevention, reduction, etc. In so doing they can be used to measure the effectiveness of particular operational risk controls at a particular point in time. In order to provide such information, the control effectiveness indicator has to have an explicit relationship to both the specific control and to the specific risk against which the control has been implemented. Examples of operational risk related control effectiveness indicators include the number of cases of customer identity misrepresentation detected (which may indicate deficiencies in customer information security controls), the number of network user access rights not reviewed within a specific period (indicating weaknesses in user access security controls) or the number of business continuity plans not tested/updated within the specified review period (indicating weaknesses in continuity planning controls).1
A Key Control Indicator quantifies how effectively a specific control tool, approach, or methodology is working.2
Key Control Indicators (KCIs) are used to define the company wide controls to and monitor the achievement of the set objectives. Managers define the related desired tolerances for controls before measuring. The KCIs´ role is to ensure that adequate responses and monitoring have been provided to a risk situation identified by KRIs. Control verification is a key component of a KCI, and it usually includes auditing, quality assurance and improvement programs. Typical KCIs cover the reliability of financial reporting, number of audit issues or product quality assurance ratios.3
Best practice organizations also introduce Key Control Indicators (KCIs) into the overall indicator universe. KCIs are indicators that are used by an organization to help define its controls environment and monitor levels of control relative to desired tolerances. KCIs play an important role is managing the execution of strategy and management of risk as they enable the effectiveness of controls to be monitored and proactively managed. This in turn helps create an environment within which decisions can be effectively implemented. A robust controls environment also helps create a “no surprises,” culture; thus enabling the organization to remain focused on delivering their strategic objectives. KCIs are used to answer the question “Are our internal controls effective? Are we, as an organization, ‘in control’?”4A Methodical Approach to Key Control Indicators5
When it comes to KCIs, you need to plan, execute and monitor your control infrastructure in a managed way. But what is the best way to do that?
KCI Vs. KRI
- Examine Your Environment: Similar to key risk indicators (KRIs), it all begins with critical introspection. Where (or what) are your "crown jewels" that you need to protect? From there, what are the compliance/control boundaries?Take the time to consider these two questions carefully, as the answers you provide - which will be unique to your organization - will change how you architect your environment, as well as how you protect it. It is helpful to consider using asset classes of devices, networks, users, data and applications, and breaking each down into the cyber defense categories defined by the National Institute of Standards and Technology (NIST):
This five-by-five grid will give you a solid foundation for your defensive strategy.
- Anticipate Control Complexities: As a baseline, most organizations have policies in place to ensure compliance, procedures that enforce or put those policies into practice, and audits that confirm that those procedures are being followed. Again, as with the concept of the KCI itself, this can seem simple and linear. In reality, the waters are easily muddied when you take the different variables into account. You might perform an audit to confirm that all of your compliance boxes are checked appropriately. But, during the course of your business operations, you may have purchased many different systems and products to help prop up your security infrastructure. This can introduce a level of complexity that makes any measurement of what is actually going on quite difficult.
- Measure Control Indicators: When it comes to your controls, in order to have the right policy, procedure and audit processes in place, you need to have that higher-level, comprehensive understanding of your company’s security and compliance environments. This is particularly true, given the fact that often what you are doing in these cases is seeking to detect the unexpected - whether that is a misconfiguration or a security incident. While often these adverse impacts are analyzed as performance indicators, they can also fall under the umbrella of your control indicators, in the event that a control that should have been in place was not. By adopting a more measured and methodical approach to how you form those control processes in the first place, the better you will be able to understand your environment, remain in compliance and protect what matters most.
Key Risk Indicators (KRIs) measure the potential for risks to occur by identifying metrics that indicate a raised risk profile. For example, a significant rise in employee staff turnover can indicate an increased likelihood of loss of key staff. KCIs have a strong relationship with KRIs, simply because if a KCI indicates the failure or weakness in a control, then it makes it likely that the level of
risk is increasing. For example, a KCI that monitors the effectiveness of staff supervision. If this activity is reduced, then it is likely that the risks mitigated by the control will become more likely to occur. As the following diagram shows, KCIs are more focused than KRIs, in that they are specifically related to the controls that mitigate a risk. Moreover, KCIs can apply to multiple controls,
which themselves can mitigate multiple risks.Typical relationships between KCIs,Controls and Risks
Source: XactiumThe Benefits of KCIs6
There are a number of benefits of KCIs:
- Better Focus: KCIs focus on ensuring that internal controls are effective in a measureable way. Rather than using broad definitions of control effectiveness, they provide a more empirical means of assessing the potential for control failure. This is something that can be quickly and easily determined in a systematic way. However, this obviously relies on the KCI metrics providing an accurate measure of potential failure.
- Wider Impact: Because Controls can mitigate multiple Risks, effective KCIs have the potential to positively impact multiple risk areas within an organisation. KRIs on the other hand tend to just focus on individual Risks, which can make them less widely applicable.
- Early Warning: KCIs can often be viewed as leading KRIs in the sense that failure of a Control is an early warning signal for the failure of a Risk. For example, a KCI that flags the potential failure of IT security Controls is likely to determine the potential of a security Risk sooner than KRIs that measure security failures directly.
- Audit Friendly: Many organisations will have an audit function as part of their “three lines of defence” governance model. A core part of this activity is an audit of existing controls to ensure that they are working effectively to reduce Risk. Obviously, KCIs can assist in this activity by providing Auditors with clear metrics with which to assess the effectiveness of controls.
See AlsoRisk ManagementEnterprise Risk Management (ERM)Key Performance Indicators (KPI)Business ContinuityBusiness Continuity Planning (BCP)Disaster Recovery PlanningKey Risk Indicator (KRI)ComplianceIT Governance