Balancing False Positives and False Negatives
Every fraud detection system eventually hits the same question: at what score should we raise an alert? Set the threshold too low and investigators drown in false alarms. Set it too high and real fraud slips through. In this post we build a synthetic payment-fraud problem, find the cost-minimizing threshold, check it against Bayes decision theory, and visualize how the optimum shifts as the business context changes.
1. The Problem Setup
We score 200,000 transactions with a model that outputs a fraud score $s \in [0,1]$. The setup is:
- Fraud prevalence: $\pi = 0.01$ (1% of transactions)
- Cost of a false positive (manual review of a legitimate transaction): $C_{FP} = 5$ USD
- Cost of a false negative (undetected fraud, average loss): $C_{FN} = 300$ USD
- Legitimate scores follow $\text{Beta}(2, 8)$, and fraudulent scores follow $\text{Beta}(6, 3)$
A transaction raises an alert if $s \ge t$. For threshold $t$, the error rates are
$$
\text{FPR}(t) = \frac{FP(t)}{N_{0}}, \qquad \text{FNR}(t) = \frac{FN(t)}{N_{1}}
$$
where $N_0$ and $N_1$ are the numbers of legitimate and fraudulent transactions. The total operating cost is
$$
\text{Cost}(t) = C_{FP}\cdot FP(t) + C_{FN}\cdot FN(t)
$$
and the goal is
$$
t^{*} = \arg\min_{t \in [0,1]} \text{Cost}(t)
$$
A Theoretical Check
Bayes decision theory says to raise an alert whenever the expected cost of alerting is lower than the expected cost of staying silent:
$$
(1-p(s)),C_{FP} \le p(s),C_{FN} \quad\Longleftrightarrow\quad p(s) \ge \frac{C_{FP}}{C_{FP}+C_{FN}}
$$
where the posterior fraud probability is
$$
p(s) = \frac{\pi f_1(s)}{\pi f_1(s) + (1-\pi) f_0(s)}
$$
Here $f_1$ and $f_0$ are the score densities of fraudulent and legitimate transactions. We will compare the empirical optimum with this theoretical one.
Sensitivity to the Cost Ratio
Define $r = C_{FN}/C_{FP}$. The cost in units of $C_{FP}$ becomes
$$
\frac{\text{Cost}(t)}{C_{FP}} = FP(t) + r \cdot FN(t)
$$
We will explore this as a surface over $(t, r)$ in 3D.
2. The Full Source Code
The naive approach re-evaluates every transaction for every threshold, which costs $O(N \cdot T)$. The fast approach sorts the scores once and uses binary search, which costs $O(N \log N + T \log N)$. Both are included and compared.
1 | import time |
3. Code Walkthrough
Data generation
We draw a Bernoulli label with probability $\pi = 0.01$ for each transaction, then draw a score from $\text{Beta}(6,3)$ for fraud and $\text{Beta}(2,8)$ for legitimate transactions. The two distributions overlap, which is exactly why a trade-off exists. If they were perfectly separable, a single threshold would produce zero errors.
Naive evaluation
naive_counts loops over 1,001 thresholds. In each iteration it builds a Boolean alert vector across all 200,000 transactions and counts false positives (alert & ~is_fraud) and false negatives (~alert & is_fraud). It is easy to read, but it touches every transaction for every threshold.
Fast evaluation
fast_counts sorts the legitimate and fraudulent scores once. For any threshold $t$:
- The number of legitimate transactions with $s \ge t$ is $N_0$ minus the count below $t$, which is exactly
len(s0) - searchsorted(s0, t). - The number of fraudulent transactions with $s < t$ is
searchsorted(s1, t).
Using side="left" makes the boundary behave identically to the naive scores >= t rule, so the assert confirms the two methods agree exactly. The sorting cost is paid once, and each threshold then costs only $O(\log N)$.
Cost minimization
With fp and fn arrays in hand, cost = C_FP * fp + C_FN * fn is a single vectorized expression, and np.argmin returns the best threshold. We also report the result at the conventional default of $t = 0.5$ to quantify the improvement.
Bayes threshold
We compute the posterior $p(s)$ on a fine grid using the exact Beta densities, then find the first score where $p(s) \ge C_{FP}/(C_{FP}+C_{FN})$. Because the likelihood ratio $f_1/f_0$ is monotonically increasing in $s$, the posterior crosses the cutoff exactly once, so this is the theoretical optimum. The empirical minimum should land very close to it.
Sensitivity analysis
The 2D analysis uses 80 cost ratios on a log scale from 1 to 300 and reuses the same fp and fn arrays, so the entire ratio-by-threshold matrix is built with broadcasting and no extra passes over the data. The 3D surface uses a restricted threshold range of $[0.2, 0.95]$ and caps the cost at 1,500 USD per 1,000 transactions so that extreme values at very low thresholds do not flatten the interesting region.
4. Execution Results
Console Output
==============================================================
Fraud Detection Threshold Optimization
==============================================================
Transactions: 200,000 (fraud: 2,065 / legitimate: 197,935)
C_FP = 5 USD, C_FN = 300 USD
--------------------------------------------------------------
t = 0.50 optimal
Threshold 0.500 0.450
False positives 3,971 7,784
False negatives 317 188
FPR 2.0062% 3.9326%
FNR 15.3511% 9.1041%
Total cost (USD) 114,955 95,320
--------------------------------------------------------------
Saving vs. t = 0.50: 19,635 USD (17.1%)
Empirical optimal threshold : 0.450
Bayes theoretical threshold : 0.442
--------------------------------------------------------------
Naive loop : 1.4651 sec
Fast method: 0.000544 sec
Speedup : 2,692x
Graph Output
5. Interpreting the Graphs
Score Distributions (top left). The legitimate scores pile up near the low end, while the fraud scores concentrate at higher values. The overlap in the middle is the region where every threshold choice necessarily trades one error type for the other. The red dashed line marks the cost-optimal threshold, which sits well to the left of where the two histograms would cross if the classes were equally likely, because a missed fraud is far more expensive than a wasted review.
FPR and FNR vs. Threshold (top center). As $t$ increases, FPR falls and FNR rises. This is the core trade-off. Raising the threshold reduces investigator workload but lets more fraud through.
Cost Curve (top right). The false-positive cost decreases with $t$, the false-negative cost increases, and their sum forms a U-shaped curve. The minimum is the optimal operating point. Notice how flat the curve is near the bottom. Small deviations from $t^*$ cost little, but moving far away, especially toward high thresholds, is expensive because every missed fraud costs 60 times as much as a review.
ROC Curve (bottom left). The optimal operating point is not the point closest to the top-left corner, nor the point that balances FPR and FNR. It sits where the slope of the ROC curve equals the cost-and-prevalence-weighted ratio $\dfrac{(1-\pi),C_{FP}}{\pi,C_{FN}}$. In practice this puts the operating point at a noticeably higher true positive rate than the default $t = 0.5$, accepting more false alarms in exchange for catching more fraud.
3D Cost Surface (bottom center). The two horizontal axes are the threshold $t$ and the cost ratio $r$, and the height is the cost per 1,000 transactions. The surface forms a valley whose floor shifts as $r$ grows. The red line traces the valley floor, the optimal threshold for each ratio. As fraud becomes relatively more expensive, the valley moves toward lower thresholds, and the walls on the high-threshold side rise steeply. This view makes it clear that the optimal threshold is not a universal constant. It is a function of the business cost structure.
Optimal Threshold vs. Cost Ratio (bottom right). The empirical optimum, found by brute-force search over the data, follows the Bayes-theoretical curve closely across the whole range. The small deviations come from sampling noise in the 2,000 or so fraud cases. The gray dotted line marks our scenario of $r = 60$. As $r$ grows, the optimal threshold decreases steadily, which means a business facing larger fraud losses should be willing to review more alerts.
6. Conclusion
Threshold selection in fraud detection is a cost-minimization problem, not an accuracy-maximization problem. With heavily imbalanced classes and asymmetric costs, the default threshold of 0.5 is almost never optimal. Three lessons stand out:
- Define the costs explicitly. The optimal threshold depends on $\pi$, $C_{FP}$, and $C_{FN}$, and changes whenever any of them change.
- Use the theory as a sanity check. The Bayes rule $p(s) \ge C_{FP}/(C_{FP}+C_{FN})$ gives a principled target that empirical search should match.
- Vectorize the search. Sorting once and using binary search makes evaluating thousands of candidate thresholds essentially free, which makes sensitivity analysis like the 3D surface above practical even on large datasets.
In production, you can extend this framework with per-transaction loss amounts, a limit on the number of alerts analysts can review per day, or threshold schedules by customer segment. The same cost-minimization logic carries over directly.
