For teams evaluating or re-evaluating a screening vendor
How do you test a sanctions screening vendor before you buy?
Every vendor shows you the demo that works. A test kit shows you the ones that do not. This page describes a reproducible way to test a sanctions screening vendor on the dimensions that decide whether its output is defensible: recall, false positives, transliteration, coverage honesty, and whether each result carries the evidence a decision needs. Run it on your own data, a vendor's own benchmark is marketing, not measurement.
Primary sources
- EBA/GL/2024/14, screening methodology and data quality
EBA guidelines (applicable from 30 December 2025) set the control expectations a screening system must meet: appropriate methodology, data quality, completeness and alert handling, i.e. the properties this kit tests for.
- Screening Decision Record, the evidence a result must carry
The open reference for the fields a screening result needs to be reconstructable. The evidence test below checks a vendor's response against it.
The five things to test
| Dimension | Test | What a pass looks like |
|---|---|---|
| Recall | Feed known listed parties, including aliases and mis-spellings. | It surfaces the real match, not just the exact string. |
| False positives | Feed adversarial clean names that resemble listed parties. | It does not drown you in unactionable alerts, and it reports a confidence you can tune. |
| Transliteration | Feed the same name in Latin, Cyrillic, Arabic and Greek. | It matches across scripts, or states plainly which scripts it cannot screen. |
| Coverage honesty | Simulate an in-scope source being unavailable. | It returns partial coverage and withholds clearance, it does not present a clean no-hit. |
| Evidence | Inspect a single response. | It carries sources, versions, coverage, candidate completeness, decision and an integrity reference, not just a score. |
The test that most vendors fail
The coverage-honesty test is the one that separates a screening tool from a search box. Give the system a case where a source it depends on is down, and watch what it returns. A defensible tool marks coverage as partial and refuses to call it clear. A search box returns an empty match list that looks exactly like a clean pass, the false clear that reaches a customer.
You cannot see this in a happy-path demo. You have to force the failure.
Building the test set
- 1Assemble known positives from public listings, real listed entities, plus their aliases and common transliterations.
- 2Assemble adversarial negatives, clean names deliberately close to listed ones, to measure false positives.
- 3Include multi-script names and at least one name in a script the vendor may not support.
- 4Prepare one coverage-failure case (a source forced unavailable) and one exact-only case, to test the honesty and mode boundaries.
- 5Score each dimension separately. A single blended accuracy number hides exactly the failure you are buying to avoid.
Limitations
- A vendor's published benchmark is self-administered on data it chose. Treat it as a claim to verify, not a result, and always re-test on your own population and languages.
- Matching is probabilistic. No screening system reaches perfect recall with zero false positives; the honest question is where the vendor sets the trade-off and whether it tells you.
- A good test result is evidence of capability at test time, not a guarantee of future coverage as lists, languages and volumes change.
- This page is an implementation reference, not legal advice or a vendor endorsement.
Build the evidence, not just the alert
Questions
Can I just trust the vendor's accuracy number?
No. A single self-reported accuracy figure is chosen by the vendor on the vendor's data. Re-run the dimensions above on your own names and languages, and force the coverage-failure case.
What is the single most revealing test?
Force an in-scope source to be unavailable and see whether the tool returns partial coverage or a clean-looking no-hit. That one test predicts whether it will ever produce a false clear.
Does Verifex publish a benchmark?
Yes, with its methodology and confidence intervals, and it is labelled self-administered. Use it as a starting claim to verify with this kit on your own data.
This page is an implementation reference for engineering and compliance teams. It is not legal advice and does not certify any regulatory outcome. Regulatory obligations, their interpretation, and the decision to treat any result as clear remain yours. Verifex supports the workflow and preserves the evidence; it does not make the compliance decision.