White paper

Best Practices For Implementation Of Detective Controls And Minimization Of Residual Risk

Together, AIRPs and Similarity Match provide a complementary, multi-layered detective control framework.

November 1, 2025 · Behavox

November 2025

TABLE OF CONTENTS

Definitions

Introduction

Inherent Risk In Business

Quantifying Inherent Risk

Implementation Of Controls To Manage Risk

Overview of Approaches to Detective Controls

Random Sampling

Lexicon

Artificial Intelligence

Measuring Effectiveness Of Implemented Controls

Methodological Approach

Best Practice Is to Combine AI, Similarity Match, and Lexicon

1. AI Risk Policies (Primary Control for Inherent Risk)

3. Lexicon (Exact-Match Safety Net)

Conclusion

Definitions

Inherent risk refers to the level of risk that exists in the absence of any controls or mitigating actions. It is the natural level of risk that arises from the nature of the business, process, or activity.

Residual risk is the level of risk that remains after controls and mitigating actions have been implemented. It is the risk that is left over after management's efforts to mitigate inherent risk. Similarity Match is recommended as the principal mechanism to control residual risk following AIRP implementation.

Lexicon approach is a traditional method used in communication surveillance systems to detect non-compliant behavior by scanning communications data for specific keywords or phrases indicative of potential risks.

AI Risk Policies (AIRP) are computer applications developed by Behavox that utilize machine learning algorithms, natural language processing (NLP), and other AI technologies, including large language models (LLMs), to detect non-compliant behavior by scanning communications data in surveillance systems.

Similarity Match A Behavox-developed enhancement to the AI Risk Policies (AIRPs) that allows compliance users to input risk-related sentences. The system converts these sentences into vector embeddings and compares them against embeddings of communication content to identify semantically similar expressions of misconduct. Similarity Match operates as a parallel workflow within AIRPs, designed to increase recall on known risk patterns while maintaining production efficiency.

Introduction

Traditionally, compliance teams have relied on methods such as random sampling and lexicon-based approaches to monitor communications for signs of non-compliance. While these methods have been instrumental, they suffer from well-known limitations in recall, precision, and scalability. The advent of artificial intelligence (AI) has transformed communication surveillance, enabling far more accurate and efficient detection of misconduct across larger populations, more languages, and a greater variety of communication channels.

However, even state-of-the-art AI systems are not infallible. AI Risk Policies (AIRPs) detect the majority of inherent risk, but some institution-specific or highly contextual expressions of misconduct may not appear in the model’s training corpus. These missed detections form residual risk - the remaining exposure after AI has already removed the overwhelming majority of inherent risk.

To address this residual risk, Behavox introduced Similarity Match, a governed and user-controlled capability that allows compliance teams to explicitly encode sentences they consider definitive examples of risky language. The system then identifies semantically similar statements in employee communications using a multilingual Text Encoder and high-dimensional similarity scoring. This approach enables compliance teams to extend AIRP coverage to institution-specific phrasing, paraphrased risk expressions, and emerging language patterns - without rebuilding models or maintaining brittle keyword lists.

Together, AIRPs and Similarity Match provide a complementary, multi-layered detective control framework:

AIRPs serve as the primary, high-recall AI control that detects the majority of known risk types.
Similarity Match acts as a secondary safety layer that captures institution-specific or phrased-differently expressions of risk, significantly reducing residual risk.
Lexicon-based rules remain useful for the narrow subset of exact-match items that neither AIRPs nor semantic similarity models are designed to target.

This paper introduces a unified framework for managing inherent and residual risk using AI, Similarity Match, lexicon-based methods, and random sampling. It explains how to quantify inherent risk, how to implement detective controls using modern best practices, and how Behavox validates these controls to meet regulatory expectations. By combining advanced AI models with governed customization and structured audit processes, institutions can significantly enhance their ability to detect, mitigate, and continuously manage compliance risks across their communication channels.

Inherent Risk In Business

Inherent risk refers to the level of risk present in the absence of any controls. It is the natural exposure to potential losses and regulatory fines that exists due to the nature of business operations and activities. In financial services inherent risks cannot be ignored as they form the baseline level of risk that needs to be managed.

Employees regularly communicate with clients and colleagues using electronic means of communication, such as emails and messaging platforms. This activity inherently carries the risk of collusion, market abuse, and financial crime.

To mitigate these risks, employees need to be trained on compliance policies, which should be published and regularly updated. Additionally, surveillance controls should be implemented to monitor communications for any signs of non-compliance.

Quantifying Inherent Risk

One of the ways of quantifying inherent risk that is potentially present in employee communications involves identifying and creating exemplary communication phrases that represent potential risks.

Here are example phrases that represent Circular Trading (Market Abuse) and illustrate the inherent risk that could be present in communications data:

"I'll buy it back at a slightly higher price."
"Back to collect my items."
"Hold this for me."
"Rebuy tomorrow for the same price."
"Eventually I want them back."
"Inflows and outflows between us."
"Need to stay flat."
"Help us with the rebound."

When creating the dataset of example phrases, the compliance team must address several quality risks and common pitfalls:

Misunderstood or Undefined Target Risks: For example, generating phrases that address "inappropriate language" but neglecting to include elements of market abuse when compiling data for market abuse risk.
Inconsistent or Poor-Quality Data: Such as using phrases in the Vulcan language instead of English, which may be unrealistic or irrelevant to the risk being modeled.
Lack of Diversity in Data Sources: Datasets might be compromised if all phrases are generated by a single individual. Ideally, multiple team members contribute phrases and help review each other's contributions.
Bias in Data Selection: For instance, if all phrases disproportionately include references to Chinese names, this could skew the dataset.
Non-Representative Data: Creating phrases for equity sales in capital markets when the AI model is intended for use in the commodities trading sector could result in ineffective or misleading outcomes. Also, having the number of sentences uniformly distributed across each risk policy would make the data more representative.

It is recommended to have at least twenty true positive phrases for each risk type. If the institution is looking to ensure a comprehensive evaluation, this baseline should be considered a minimum requirement.

The dataset of inherent risk example phrases should not be static. As new risks emerge and communication styles evolve, the dataset must be updated and expanded to reflect these changes. This includes adding new seed sentences for emerging risks and updating variations to include new types of evasion or communication trends.

By adhering to these guidelines, Institutions can develop a dataset of inherent risk example phrases that provides a rigorous benchmark for evaluating the effectiveness of their controls.

Implementation Of Controls To Manage Risk

Institutions must implement various types of controls to mitigate inherent risk in their business operations. These controls can be categorized into four main types:

Directive Controls: Policies and procedures, codes of conduct.
Preventive Controls: Training programs, access restrictions, and segregation of duties.
Detective Controls: Continuous monitoring and surveillance systems.
Corrective Controls: Incident response plans and disciplinary actions.

The controls should be implemented using an onion layer approach, where multiple layers of controls work together to provide comprehensive risk mitigation. One control alone is not sufficient and will not be considered effective by regulators. However, if each control is implemented according to industry best practices, the overall risk management framework will be robust and effective.

Overview of Approaches to Detective Controls

Detective controls play a central role in mitigating inherent risk present in employee communications. Over the past three decades, technological advancement has dramatically changed what is possible - and what regulators expect - within communication surveillance. Today, institutions can choose from a spectrum of approaches ranging from traditional sampling-based techniques to advanced semantic-AI systems. Each method offers different strengths and should be deployed according to its capabilities and limitations.

Historically, compliance teams relied heavily on random sampling and lexicon-based detections. These methods were simple to implement and easy to explain, but they lacked scalability and struggled to detect paraphrased, contextual, or emerging forms of misconduct. As communication volumes increased and regulatory expectations rose, these traditional approaches became insufficient as primary detective controls.

The introduction of Artificial Intelligence (AI) - and specifically Behavox AI Risk Policies (AIRPs) - marked a transformational improvement. AIRPs deliver high recall, low false-positive rates, and multilingual support across large populations. They detect the majority of inherent risk and form the foundation of modern communication surveillance programs.

However, it is now recognized across the industry that even the most advanced AI models cannot capture every institution-specific phrase or every linguistic variation of misconduct. These missed detections form residual risk - the remaining exposure after primary controls are applied.

To address this gap, Behavox introduced Similarity Match, a governed, user-configurable capability that allows compliance teams to encode verified true-positive sentences and automatically detect linguistically similar communications using a multilingual Text Encoder. Similarity Match operates in parallel with AIRPs, significantly reducing residual risk by capturing paraphrased or institution-specific expressions that do not appear in the model’s training corpus. As a semantic system, it is far more flexible and maintainable than custom lexicon rules and is now Behavox’s recommended secondary control for managing residual risk.

As a result, the modern hierarchy of detective controls can be understood as follows:

AI Risk Policies (AIRPs) – the primary, high-recall AI control that detects the majority of inherent risk with precision and multilingual robustness.
Similarity Match – the secondary semantic-matching control that reduces residual risk by capturing institution-specific or paraphrased expressions known to compliance officers.
Lexicon-based rules – a tertiary safety-net control appropriate only for exact-match items that neither AIRPs nor Similarity Match are designed to detect (e.g., fixed codes, identifiers).
Random Sampling – a periodic audit mechanism for validating that no new, unknown risks are escaping detection (“unknown-unknowns”).

Regulators do not prescribe specific technologies, but they do require that detective controls be demonstrably effective and aligned with the firm’s risk profile. In practice, this means institutions must adopt the most effective controls available. When deployed together, AIRPs and Similarity Match dramatically reduce both inherent and residual risks, while lexicon and sampling remain valuable supporting layers in a comprehensive surveillance framework.

Regulators demand the implementation of effective systems and controls to protect the integrity of financial markets. They do not mandate specific technologies. For example, see the most recent notice from FINRA, where the regulator acknowledges that AI presents promising opportunities and reminds member firms that FINRA’s rules are intended to be technology neutral.

Therefore, compliance teams must review and evaluate new technologies, especially those that are orders of magnitude more effective in protecting the integrity of the financial system. Adopting advanced technologies like AI ensures that the systems and controls in place are not only up-to-date but also highly effective in mitigating risks.

Random Sampling

Random sampling was initially used to evaluate the effectiveness of controls by randomly checking communications. In the early 1990s, with email being the primary communication channel and low communication volumes, random sampling was considered the most appropriate method for implementing detective controls. It was affordable and reasonably effective, and FINRA still recommends its use to their members.

Benefits:

Simple to implement.
Provides a baseline for compliance monitoring.
Can be effective for detecting previously unknown examples of risks.

Drawbacks:

May miss less frequent, but significant non-compliant activities.
Requires a broad scope to catch any risks, which will be expensive to operate and generate a huge volume of alerts (e.g., 1% of 20,000,000 emails results in 200,000 alerts per day, an impractical number for manual review without significant cost).
Specific risks and specific phrases will not be picked up, leading to gaps in detection and potential non-compliance slipping through unnoticed.

Lexicon

As communication volumes grew and new channels emerged in the early 2000s, compliance teams began supplementing random sampling with lexicon-based detections. This approach involves scanning communications for predefined keywords or phrase patterns associated with known risks. For many years, lexicon was the industry standard because it was simple to configure, easy to understand, and straightforward to audit.

While lexicon remains a familiar and transparent tool, its role in a modern surveillance program has changed. With the introduction of AI Risk Policies (AIRPs) as the primary detective control - and Similarity Match as the recommended secondary control - the lexicon method is now best viewed as a special-purpose control rather than a broad detection mechanism.

Benefits:

Easy to Set Up and Understand - The lexicon approach involves creating a list of specific keywords or phrases that are indicative of risky or non-compliant behavior. This list can be easily developed and updated, making the implementation straightforward. Employees and compliance officers can quickly understand how the system works, facilitating training and adoption.
Effective for Detecting Known Risks - Lexicons are particularly effective at identifying risks that have been previously identified and documented. By using established keywords and phrases associated with known risks, the system can reliably flag communications that match these criteria. This makes the lexicon approach a valuable tool for monitoring well-understood compliance issues.
Easy to Audit - Because the lexicon approach relies on predefined keywords and phrases, it creates a clear, auditable trail. Compliance teams can easily demonstrate to regulators how their monitoring system works, providing evidence of the specific terms being monitored and the results of these efforts. This transparency simplifies the auditing process and helps ensure regulatory compliance.

Drawbacks:

High False-Positive Rate - because lexicon matches based on literal text, it often flags benign communications containing the same terms. This results in unnecessary alert volume and reviewer fatigue.
Poor Recall on Real-World Language - lexicon is not equipped to detect paraphrases, synonyms, or contextual variations. Slightly rephrased misconduct (e.g., “rebuy tomorrow” vs. “buy it back tomorrow”) is typically missed, leaving material risk unaddressed.
Expensive to Maintain - Keyword lists must be manually curated, validated, tuned, and updated. Over time, they tend to grow noisy, inconsistent, and operationally burdensome.

With the introduction of Similarity Match, most use cases that previously required complex or customized lexicon rules can now be handled more effectively through semantic comparison. Similarity Match is easier to maintain, aligns more naturally with true positives, and provides significantly broader language coverage with far lower maintenance overhead.

As a result:

AIRPs should be used to detect the majority of inherent risk.
Similarity Match should be used to detect institution-specific or paraphrased residual risk.
Lexicon should be used only for exact-match cases where literal string detection is required and where neither AIRPs nor Similarity Match are intended to operate.

In modern, AI-enabled surveillance environments, lexicon is no longer the primary control, nor even the secondary one - it is a specialized, edge-case detector that completes the layered control framework but does not serve as the core method for risk detection.

Artificial Intelligence

In the 2020s, the advent of advanced AI models fundamentally transformed the effectiveness of detective controls used in communication surveillance. Behavox’s AI Risk Policies (AIRPs) represent the most significant improvement to date, delivering high recall, multilingual coverage, and low false-positive rates across large, complex communication populations. AIRPs are trained centrally by Behavox on extensive, curated datasets and work “out of the box” for clients, allowing institutions to deploy cutting-edge AI detection without building or maintaining models internally.

Benefits:

Higher Accuracy and Recall: AIRPs identify the overwhelming majority of inherent risk in employee communications. They evaluate entire sentences semantically, enabling detection of misconduct phrased in numerous ways - not just through literal keywords. This reduces both false negatives and false positives, strengthening overall risk mitigation.
Robustness to Obfuscation: Unlike lexicon methods, AI models understand context and can recognize meaning even when individuals attempt to evade detection through misspellings, abbreviations, code words, or inconsistent phrasing.
Adaptability to New Risks: AIRPs are retrained periodically using expanded datasets that reflect emerging regulatory cases, client feedback, and evolving language patterns. This continual improvement ensures that the system adapts over time without requiring manual rule creation.
Scalability: AI supports surveillance across tens of thousands of employees, multiple communication channels, and more than a dozen languages, all within a unified pipeline.

However, even state-of-the-art AI systems are not perfect. AIRPs depend on the training data provided to them, and there may be cases - particularly institution-specific phrasing or highly contextual expressions - that do not appear in the training corpus. These missed detections represent residual risk, which must also be controlled.

To address this, Behavox introduced Similarity Match, a semantic detection capability designed to work alongside AIRPs. While AIRPs detect the majority of inherent risk automatically, Similarity Match allows compliance teams to specify exact sentences they consider true positives, enabling the system to detect paraphrased or semantically equivalent versions in communications. This ensures that institution-specific risks and nuanced expressions of misconduct are captured without requiring custom lexicon rules or model retraining.

Together, AIRPs and Similarity Match provide a comprehensive AI-based surveillance solution:

AIRPs serve as the primary detective control, minimizing inherent risk with high recall and precision.
Similarity Match serves as the secondary semantic control, reducing residual risk with governed, user-defined examples.
Lexicon-based rules are reserved for exact-match edge cases.

Behavox’s approach allows institutions to adopt AI as the main detective control while retaining the flexibility to enhance coverage with targeted, user-driven semantic matching - ensuring both effectiveness and transparency within a regulated environment.

Measuring Effectiveness Of Implemented Controls

The dataset containing example phrases is crucial for evaluating the effectiveness of detective controls. These phrases are used as planted true positives in the communications data to test the detection capability of the compliance surveillance system.

The effectiveness of detective controls is measured by the number of planted phrases caught by the system. Phrases missed by the system constitute residual risk. The goal is to implement controls that address both inherent and residual risks. While it is reasonable to accept residual risks after careful evaluation, firms should strive to minimize these risks as much as possible.

The size of the residual risk, measured as the percentage of missed risk phrases, determines the effectiveness of the detective controls. If residual risk remains high, it indicates that the inherent risk is not adequately mitigated.

Behavox has conducted extensive testing in client environments by planting risk representative phrases (true positives) in communications data and employing various methodologies to detect these phrases in alerts. The following statistics are generalized illustrations of what was consistently observed across all customers:

	Detection rate	Residual Risk	Alert volume	Effective?
Random Sample	0%	100%	200,000	No
Lexicon	2%	98%	25,000	No
AIRPs	80%+	20%	500	YES
AIRPs + Similarity Match	95%+	5%	700	YES

Methodological Approach

To conduct the extensive testing and obtain the numbers above, Behavox performed outcomes analysis following the Federal Reserve Bank’s guidance on model risk validation. Each phrase from the testing dataset was incorporated into an email and sent to a monitored inbox specifically set up for this purpose. Each email had to be original - neither a reply nor a forward - and should originate either from a compliance officer or a designated test email address. The recipient should also be a compliance officer or use a test email address. The targeted inbox must be part of a monitored employee group designated for testing within the scenario release bucket and the Scenario Testing Lab (STL).

Additionally, the target inbox should receive regular communications, including daily market updates, corporate announcements, market insights, and other general information to simulate a realistic environment.

After the emails have been sent, it is crucial to evaluate the generated alerts in terms of both quality and quantity. To assess whether alerts would have been triggered, customers can utilize the Scenario Testing Lab feature of the Behavox Platform. This functionality allows users to apply the AI Risk Policy to communication data for a specific date to generate hits. It is important to note that these hits are not production alerts and do not interfere with reviewer workflows.

Best Practice Is to Combine AI, Similarity Match, and Lexicon

Given the significant advances in communication-surveillance technology, institutions no longer need to rely solely on traditional controls such as lexicon or random sampling. AI Risk Policies (AIRPs) now detect the overwhelming majority of inherent risk with high accuracy, low false-positive rates, and multilingual scalability. However, no AI model - regardless of its sophistication - can detect every possible linguistic variant or every institution-specific phrasing of misconduct. This remaining exposure is residual risk, and it must be mitigated through additional controls.

The modern best-practice approach is therefore a three-layer detective-control framework, where each layer addresses a different dimension of risk:

1. AI Risk Policies (Primary Control for Inherent Risk)

AIRPs serve as the main detective control and should be applied across the full monitored population. They use transformer-based models and domain-specific training to identify the majority of risk-relevant content. In practice, AIRPs consistently detect 80%+ of inherent risk in client environments, drastically reducing alert volume compared to legacy approaches.

2. Similarity Match (Primary Control for Residual Risk)

Similarity Match is Behavox’s recommended method for addressing residual risk - the fraction of true positives that AIRPs may miss because the exact phrasing did not appear in model training.

Using a multilingual Text Encoder and semantic similarity scoring, Similarity Match:

Detects paraphrases and linguistically varied risk expressions
Captures institution-specific language based on user-defined sentences
Boosts detection by 10–15 percentage points beyond AIRPs alone
Provides governed, auditable customization aligned with SR 11-7 expectations
Eliminates the need to build and maintain large volumes of brittle lexicon rules

Because Similarity Match operates semantically rather than literally, it offers far broader coverage and far lower operational maintenance than custom Advanced Scenarios or lexicon-only detection.

For most clients, Similarity Match now replaces the bulk of custom lexicon rules and becomes the standard secondary control layer.

3. Lexicon (Exact-Match Safety Net)

Lexicon rules still have value - but their purpose has narrowed.

They should be used only for:

Literal phrases that must always trigger an alert (e.g., fixed identifiers, deal IDs)
Highly specific wording where semantic variability is not expected
A small number of cases where both AIRPs and Similarity Match are not meant to operate

Lexicon is no longer well-suited as a general-purpose detector due to its high false-positive rate, inability to generalize, and high maintenance burden.

Instead, it serves as a precision safety net layered beneath AIRPs and Similarity Match.

When implemented together, AIRPs and Similarity Match routinely achieve 90–95% total detection in client environments, with lexicon covering only a narrow remainder.

Random sampling then serves as an audit mechanism - rather than a primary control - to ensure that no new, unknown risks are escaping detection.

This layered approach aligns with regulatory expectations for effective systems and controls while minimizing operational burden and maximizing risk coverage.

Conclusion

Inherent risk is an unavoidable aspect of business operations, particularly in financial services where employees communicate continuously with clients, counterparties, and colleagues. Effective management of this risk is essential to prevent financial losses, regulatory breaches, and reputational damage. By identifying and quantifying inherent risks in employee communications, institutions can build surveillance frameworks that are both robust and defensible.

Detective controls play a central role in mitigating this exposure. Traditional methods such as random sampling and lexicon-based surveillance were historically useful, but they are limited in accuracy, scalability, and adaptability. These approaches struggle to keep pace with the evolving complexity of financial communication, increasing volumes, and the sophisticated ways in which misconduct can be concealed.

The introduction of Artificial Intelligence (AI) marked a transformational improvement in the effectiveness of detective controls. Behavox AI Risk Policies (AIRPs) provide high recall, low false-positive rates, and broad multilingual coverage, allowing institutions to detect the majority of inherent risk with unprecedented precision. However, even state-of-the-art AI models can miss institution-specific or linguistically varied expressions of misconduct - creating residual risk that must also be mitigated.

To address this residual risk, Behavox introduced Similarity Match, a governed and user-controlled detection capability that enables compliance teams to encode verified true-positive sentences and automatically detect semantically similar communications. Similarity Match operates alongside AIRPs and significantly reduces the remaining blind spots that AI alone cannot eliminate. It offers a far more scalable, maintainable, and semantically rich alternative to custom lexicon rules, while remaining fully auditable and aligned with SR 11-7 governance requirements.

Lexicon-based methods continue to play a role, but that role is now narrower and more targeted. Lexicons serve as a precise safety net for exact-match expressions that should always trigger an alert - such as deal identifiers, product names, or literal regulatory red flags. Finally, random sampling supports periodic audit processes, ensuring that “unknown unknowns” are surfaced and that the surveillance framework remains effective as communication patterns evolve.

Behavox’s extensive testing in production-scale environments demonstrates that the combination of AIRPs + Similarity Match + Lexicon + Random Sampling delivers the strongest and most comprehensive risk-detection architecture. AIRPs remove the majority of inherent risk; Similarity Match eliminates most of the remaining residual risk; lexicons provide targeted exact-match coverage; and random sampling supplies periodic validation.

In conclusion, institutions that adopt this multi-layered, modernized surveillance framework - not just AI, but AI paired with Similarity Match and supported by lexicon and sampling - are best positioned to meet regulatory expectations and maintain effective, future-proof compliance controls. As new risks emerge and communication styles continue to evolve, ongoing enhancement of these systems remains essential to protecting the integrity of financial markets and ensuring resilient risk management.

Download the paper

← All white papers & blog posts