∞Direct answer
Direct answer
Use AI to sort leads. Do not let it choose an investment. Define two key errors. They are false positives and false negatives. Test both with old deals. Check sources before you contact a firm. A person should make the next call. NIST says teams should measure both error rates on real test sets. [1]
Define the two errors before you screen
A false positive is a lead that passes a screen. A later check says it breaks a rule. A false negative fails the screen. A later check says it met the rule.
Use these labels only for the screen. They cannot rate a future return. Good screening finds work to check. It cannot prove a company is good.
Write rules in plain words. Include stage, place, sector, cheque range, and conflicts. Do not guess unknown facts. A missing fact does not prove poor fit.
- Pass: the linked evidence meets a stated screen rule.
- Review: the lead may fit, but a key fact is missing or conflicts.
- Fail: linked evidence shows a stated mismatch.
- Escalate: a conflict, sensitive fact, or source issue needs an owner.
Build a test set that includes near misses
Use deals that people checked before. Add clear fits and clear misses. Add close cases too. A close case is near a stage, sector, or place rule. It may show a weak rule.
Let a person read source evidence before labelling a test deal. Save some test deals. Do not use them to tune a prompt or rule. Use them to test a change.
NIST says test sets should be like live work and represent it. It says to record the method. Record results by useful data groups too. [1]
- Use the same source types that the live screen will use.
- Keep the source link, check date, and reviewer note with each label.
- Sample both passed and failed leads for human review.
- Check stage, sector, geography, and source type where useful and lawful.
Ask the tool for evidence, not a verdict
Ask for a brief card for each lead. It needs the matching rule. It needs a source link and date. It needs missing facts. Set out each rule. Do not use one total score to hide gaps.
AI can make up a source or citation. NIST calls this confabulation. It says false content in important decisions needs care. [2] Open the source. Check that it proves the fact.
Send weak proof to a person. Do not auto-pass or auto-fail it. The reviewer must be able to fix a result. They must be able to change a rule or stop the process.
- Exact rule: what the screen tested.
- Evidence: source link, short quote, and date checked.
- Limit: what the source cannot show.
- Next step: pass, review, fail, or escalate, with a reason.
Measure errors and change one thing at a time
Count the leads people review. Count false passes and misses. Count weak sources. Count reviewer changes. Record the rule, tool, version, sources, and date.
Set a reply rule first. More misses could mean a check of some failed leads. More false passes could mean the proof rule is too loose. Test one change on the saved set before wide use.
ESMA says investment firms should test and watch AI used in investment services. It says they need to check input data and keep records. IOSCO highlights separate validation. It also lists alerts and records for human review and overrides. [3] [4] These help teams set controls. Legal duties vary by firm, service, and place.
- Set a named owner for the screen and its changes.
- Review error samples on a planned schedule and after a material change.
- Pause the workflow when sources fail or results shift without a clear reason.
- Keep the final meeting and investment choice with people.
FAQ
Frequently asked questions
What is a false positive in deal screening?
It is a lead that passes a screen. A later check finds it breaks a set rule. Bad data, a bad source, or a loose rule can cause it.
What is a false negative in deal screening?
It is a lead that fails a screen. A later check finds it met a set rule. Check a sample of failed leads. It can show rules that are too tight.
Should we try to remove every error?
No. Tight rules can cut false passes. But they can cause more misses. Loose rules find more leads. But they use more review time. Set the trade-off for your mandate. Then test it.
Can an AI score decide who gets a meeting?
No. A score can sort research. It must show facts, sources, gaps, and conflicts. A person should decide if a lead moves forward.
REF
Sources
- 1
National Institute of Standards and Technology
Artificial Intelligence Risk Management Framework (AI RMF 1.0) ↗ - 2
National Institute of Standards and Technology
Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile ↗ - 3
European Securities and Markets Authority
Public Statement on Artificial Intelligence and Investment Services ↗ - 4
International Organization of Securities Commissions
Supervisory Toolkit for AI Use in Capital Markets ↗