AI due diligence tools: test known answers before you buy

Choose one task, not a full deal verdict

Due diligence means checks before you invest. This guide helps teams test tools used to read startup files. It is not a way to grade a founder or predict returns. Start with one task, such as answering cash questions from a file pack.

Write what a useful reply must show. For a cash question, ask for the cash date and unit. For runway, ask how long cash will last. The reply must state the net cash burn used. That is cash spent each month after cash coming in.

NIST warns that AI can state false facts with confidence. It can invent sources too. OpenAI urges tests built for the task, not just broad scores. A neat reply can still be wrong. [1] [2]

• Name the task and the person who checks it.
• State the facts, dates and units required.
• Say what the tool must not decide or do.

Build an answer key from approved files

An answer key is a list of test questions and checked replies. Write it before you test the tool. For each reply, name the source file, page and date. Ask a second team member to check the key. Fix gaps in your key before judging the tool.

Include plain facts, sums and hard cases. Add a question whose answer is not in the files. Add two files that disagree without a clear reason. The expected reply should flag the gap or conflict, not guess which fact is true. OpenAI calls for normal, edge and hostile cases. [2]

Use made-up files or approved copies with private facts removed. Match the stages, sectors, languages and file types you plan to use. A test built only from clean English text says little about scanned files in other languages. It also says little about real deal quality.

• Clear fact: find the cash balance and its date.
• Sum: use the stated cash and net burn to find runway.
• Missing fact: say the files do not give the answer.
• Conflict: show both sources and ask for a check.

Freeze the inputs and keep the test safe

Give each tool the same files and questions. Keep the checked answers out of its input. Save the tool version, settings and test date. Record any web search or extra files it may use. Without this, you may be comparing different source packs, not tools.

Before any upload, check the vendor's terms and data settings. Ask who can access the files. Ask how long files and logs stay stored. Check whether they may be used to train models. Do not upload private deal files until your firm approves that use.

Run in a read-only space. Do not give the test tool rights to trade, send mail or share files. Security tests need their own safe scope. NIST's profile is voluntary guidance, not proof a vendor meets local law. Seek qualified local advice on data and contracts. [1]

Score the answer and the source apart

Check each reply against the key. Then open its cited source. Does that passage support the claim? Is the date right? Is the unit right? A correct number with a false source is still a failed source check. Do not fold all faults into one average.

Google separates finding the right source from using it well. If a reply fails, check which step went wrong. A faithful summary of an old file may still miss your question. Support in a founder's file does not prove that file is true. Verify key deal facts by other means. [3]

Here is a made-up test, not a vendor result. Of ten replies, eight pass the answer check. Six pass the source check. Those groups may overlap. One reply gets cash wrong. If that could change the deal view, stop use for that task. The other passes do not cancel that fault.

• Answer: does it match the checked fact?
• Source: does the cited page support the claim?
• Date and unit: do they match the question?
• Unknowns: does it flag missing or conflicting facts?

Repeat the test without teaching the final answers

Try the same inputs more than once. Record each reply, not just the best one. AI may give different outputs from the same input. Count how often a major fault returns. Do not choose a good run and hide the rest. [2]

Use some cases to fix prompts and settings. Keep other cases aside for a final test. Do not use those final answers to tune the tool. Add fresh cases when new file types or failure modes appear. A small test is a start, not a sound estimate of all future errors.

If AI helps score replies, check its scores by hand. A second AI can miss the same error. OpenAI says model scoring needs checks against human labels. Have a named reviewer settle disputes. Score support and meaning, not just matching words or longer prose. [2]

Approve a narrow use and keep a stop rule

Set pass and stop rules before you see results. Your firm must choose them for the task's risks. There is no single safe pass rate for all deals. A major cash, ownership or data-access error should block that use until fixed and retested.

If the test passes, approve only the task you tested. Keep a human check on facts that affect a deal. Retest after changes to the model, prompt, search or source pack. Log new faults and who can stop use. NIST calls for source checks before use and during ongoing review. [1]

This guide used AI support and the linked sources. Its test example is fictional. No vendor was tested for this article. This is not investment or legal advice. Full deal checks and local data rules still apply. A tool test cannot prove a startup is a good investment.