Use NIST and NCSC as review frames

Public frameworks give you a list of topics to review. They do not give you a verdict.

The NIST AI Risk Management Framework

NIST published the AI Risk Management Framework (AI RMF 1.0) in January 2023. Use of the framework is voluntary. Its core has four functions: Govern, Map, Measure and Manage.

The site's guides treat the framework as a way to structure the review of context, measurement and response. The framework treats risk management as an ongoing activity. It is not proof that a vendor is safe.

The NCSC guidelines

The UK National Cyber Security Centre published the Guidelines for secure AI system development. They cover four stages: secure design, secure development, secure deployment and secure operation and maintenance.

Use the guidelines to inspect system boundaries and permissions. Use them to ask the supplier how it monitors a deployed system and how it responds to incidents.

What these sources are not

Neither source is a universal legal compliance check. Neither one replaces advice from a qualified local adviser. Recheck local law before launch and when the use changes.

Keep a risk record

Make one row for each risk. Write down:

• The risk, in one sentence.
• Where it appears: the task, the data, the users.
• How you will measure it.
• The owner.
• The response.
• The status: open, reduced, accepted or closed.

Record unresolved risks before you choose a supplier. Record the decision to proceed or to stop, with the date and the person who decided.

Sources

• NIST, AI Risk Management Framework (AI RMF 1.0)
• NCSC, Guidelines for secure AI system development
• AIAgencies guide: How to vet an AI agency (this site)
• AIAgencies guide: AI rules, permits and data approval (this site)