Disclaimer: I’m not being super careful about my use of the term ‘AI’ in this article. Someone can tell me how this is bad in the comments.
I think I’m finally over the AI hype.
I’ve passed through all the stages of AI-Hype-Grief:
- Complete ignorance of what AI really is
- Realization of hype and rejection of technology
- Embrace of hype and attempt at complete adoption
- Challenges in implementing viable AI solutions and complete rejection of the technology
- Realistic recognition of the technology’s capabilities and expected global adoption.
Now, I need to seriously reckon with how generative AI will impact my niche: SOC 2 for SaaS.
Specifically, when I am auditing companies that are implementing LLMs within their product, what unique risks are posed and how should the SOC 2 report address them?
There are a handful of pieces by credible authors that have been published on this topic (see Sources at end), but very little of them are actionable. So I felt compelled to put something in plain English that normal people can use to improve their SOC 2.
I did a bunch of research (‘bunch’ is a technical, industry term):
- I started, of course with ChatGPT and put the question to it.
- That gave me a pretty good starting point and linked to some somewhat helpful articles from Schellman, risk3sixty, LBMC, and Linford.
- I also crowd-sourced, drawing on conversations with GRC-types and auditor-types I know who are seriously looking into this question.
- I reviewed the CSA STAR AI Controls Matrix, NIST AI RMF, ISO 42001 to get a sense of the risks they evaluated.
- I reviewed applicable SOC 2 authoritative guidance.
- I listened to GRC Uncensored’s podcast with Merrit Baer, because you can’t publish something in SOC 2 world without checking Troy Fine’s opinion about it.
Then I compiled all my findings in this whitepaper-blog-thing. And just in case you don’t like getting roped in to a limited set of choices, I’m going to give you 6 different possible options for how to describe use of LLM’s in SOC 2. Some options are better than others, but all are possible.
A brief note on Scope:
I wouldn’t be a CPA without inserting a healthy disclaimer:
- Included in my assessment: Systems that have implemented LLM’s as a meaningful portion of their core service offering.
- Excluded from my assessment: Companies that have enabled AI tools for internal use only.
- Also excluded: Agentic AI Stuff… and I’ll let someone else define ‘agentic’ for now, but if your product is doing agentic stuff then you’re introducing another level of risk.
How should implementing LLM’s impact the SOC 2 report?
Option 1 – Include AI governance and control disclosures in Section V only
Section V:
- Disclose any information that might be useful to the reader (not included under in auditor’s opinion), such as how the SOC 2 controls map to the ISO 42001 standard.
Option 2 – Modify the system description and test of controls to include Gen-AI-specific information and controls (relevant to applicable TSC only).
Section III:
- Ensure the system description articulates the specific ways that the company’s product offerings involve the use of LLM’s.
- Ensure any third party LLM’s or other model dependencies that are relied upon to perform key controls on behalf of the company are defined as subservice organizations
- Ensure that LLM-related third party products are described in Software or Infrastructure section
- Ensure that the description of controls addresses AI/ML risks as they relate to applicable TSC (see table below).
Section IV:
- Ensure that controls tested address AI/ML risks as they relate to applicable TSC.
Option 3: Include custom AI Criteria in the Auditor’s Opinion.
Section I and II:
- Adopt a set of custom AI Criteria statements into a SOC 2 report and include these within the auditor’s opinion and management’s assertion. Come up with a custom set of criteria and map controls to support.
- For example, risk3sixty has a set of Typical AI Criteria you could use, based off combining NIST AI framework and ISO 42001 principles: https://risk3sixty.com/whitepaper/soc-2-ai-compliance-pack?/
Section III:
- Everything described in Option 2 above, as well as:
- Ensure that the description of controls addresses AI/ML risks as they relate to the custom AI criteria defined.
Section IV:
- Everything described in Option 2 above, as well as:
- Ensure that controls tested address AI/ML risks as they relate to the custom AI criteria defined.
Option 4 – Include the Processing Integrity Category for SOC 2 and customize the scope of those controls to the consistency of LLM training or tuning inputs and outputs.
This could be relevant (to varying degrees) whether customers are simply relying on the LLM-feature’s output or whether the product is actively manipulating customer data.
Option 5 – keep the SOC 2 the same but check out CSA STAR for AI or get ISO 42001 certified
(albeit a different focus from SOC 2 criteria but accomplishes much of the same purpose):
- CSA STAR AI framework: https://cloudsecurityalliance.org/star/ai
- Learn about ISO 42001: https://learn.mastermindassurance.com/products/courses/iso-42001-lead-auditor
Option 6 – Complete a synchronized SOC 2+ ISO 42001 engagement
(very similar to Option 1 but using a more recognized framework).
Those are the options I see available for general treatment of AI implementation in the SOC 2 report.
Ok, so those are the options. How do I actually decide what to do?!
Hire me!
Just kidding, I’m an auditor – don’t hire me for consulting. But, I could point you to a few folks who could help you if consulting is what’s needed.
But you don’t need consulting – you’re still reading this blog because you’re a go-getter, DIY-type, you’re not going to spend your money on posh consultants from [Insert Large Firm]…. who’s probably going to just put this into ChatGPT anyway…
Don’t worry, I got you covered. Here are two principles for your consideration:
Principle 1:
The more fundamental your use of LLM’s is to your core customers relying on your SOC 2 report, you should consider the more in-depth options. If it’s used in an ancillary way or perhaps doesn’t touch sensitive/confidential data, consider the light treatment of Option 1. If it’s core to what you do, you might consider actually appending to the SOC 2 criteria to include AI-specific risks (Option 3) or getting a separate certification (Option 5).
Principle 2:
Make sure your bases are covered. I think there are a lot of SOC 2 reports out there right now that are “out of compliance”.
That’s not a technical term, but I just mean, remember when cloud implementation shot up, and all the SOC reports still described 20 different physical access controls like they were actually being done by the company getting audited?
Remember when CI/CD workflows arrived with agile development methodologies, and controls still were still focusing on how “developers don’t have access to production?”
Same thing with implementation of AI, SOC reports are lagging, but the implementation of AI is moreso creating additional risks rather than off-loading them to a vendor like AWS.
So, review the current scope (criteria and controls, as well as System Description) of your SOC 2 report to make sure you are telling an accurate story to your readers.
You might need to nudge your audit firm to do this. (We don’t like change).
Key AI/LLM Risks:
Traditional automation gave more binary outputs; sticking LLM’s in your product lifecycle introduces unpredictability and new kinds of vulnerabilities. Below is a running table of unique risks from AI implementation and controls that might apply to SOC 2:
Conclusion
In conclusion, if you are implementing LLM’s in your product lifecycle, go talk to someone smarter than me and to make sure A) you’re adequately addressing the risks, and B) your internal controls and compliance programs describe that accurately.
“Sources” (more like inspiration, but hey)
- https://risk3sixty.com/whitepaper/soc-2-ai-compliance-pack?/
- https://cloudsecurityalliance.org/star/ai
- https://learn.mastermindassurance.com/products/courses/iso-42001-lead-auditor
- https://www.schellman.com/blog/soc-examinations/artificial-intelligence-in-soc-reports?/
- https://www.schellman.com/blog/soc-examinations/how-to-incorporate-ai-into-your-soc-2-examination?/
- https://www.lbmc.com/blog/generative-ai-soc-2/
- https://linfordco.com/blog/soc-2-audit-considerations-ai-ml-platforms/
- https://www.compassitc.com/blog/achieving-soc-2-compliance-for-artificial-intelligence-ai-platforms
- https://www.cyberdefensemagazine.com/embracing-the-ai-revolution-how-to-incorporate-generative-ai-into-your-soc-2-compliance-plan/
- AICPA’s Attestation Standards, SOC 2 Guide, and ‘2017-trust-services-criteria-with-revised-points-of-focus-2022’




