📋 Let us dive into the v2 discussions …📋

We will look at the V2 from multiple POV in four sections.

1. OWASP T10 for LLM Applications v2 : Status

I am sure y’all remember the organized v2 candidate list from our June Newsletter [Here].

For those enquiring minds who want to know, the raw list is also available in our GitHub [Here]

Like we talked about earlier, the general focus for v2 is three fold:

1. Span Security (of the operator), Safety (of the user) and Trust (by the user) of LLM Applications

2. Be the one-stop-shop to answer the question "What are the top risks that I should worry about when I deploy my application that has LLM components?"

3. Make the artifacts approachable, accessible and consumable by a wide variety of audience - folks who have high information overload and a low attention span (current company included !)

In addition to the Top 10 for LLM list, we are pursuing multiple tracks to facilitate these goals.

Many interesting initiatives are being deliberated - Solutions doc, AI Red Teaming Best Practices, Data Collective and so forth. Lots of opportunities for all of us to participate.

Your turn to join the lists - voice your opinions; pick the ones you are passionate about and contribute !!

Of course I can’t hog the space 🙃 Scott has done an excellent job on the Solutions doc work. Outline and initial content here

2. v2 Voting Results

Back to the main feature - trends first, analytics next, and finally the gory details

Trends

1. Agents are rising in importance. We have a whole channel #team-llm_v2_agents_discussion led by our favorites John Sotiropoulos and Fabrizio Cilli. There are 2 candidates viz., Vulnerable Autonomous Agents and Agent Autonomy Escalation. Autonomous Agents bring together different aspects of of LLMs

   • Agents embody autonomy and inherently relate to the existing entry on excessive agency

   • Agents are a different way to use LLMs and change the application paradigm from the traditional almost client server model of LLM apps (and also the usual inference endpoint of traditional ML) to a distributed one. They bring new attack vectors and considerations including environment, adaptability, and new patterns such as reflection, and scenarios including emerging on-device agents and models, and challenge the notion of human-on-the-loop and how we scale it

2. The Gen AI security is maturing and separating itself from traditional ML and web applications. Agents, RAG all are newer paradigms (the ideas are not new, but the applications are) that figure prominently in the voting

3. As expected, Prompt Injection has very high priority

4. An interesting one is the system prompt exploits, like bypassing system instructions have risen in importance. There are 2 candidates that deal with this aspect - the C06:Bypassing System Instructions Using System Prompt Leakage and the C27:System Prompt Leakage

   • System Instructions is the superset that can contain system prompts, roles, response characteristics (like be succinct or be more detailed et al) and so forth) Not sure if bias or malicious code creation is blocked by system instructions.

   • The discussion chain in the slack channel, mentions two separate “vulnerabilities” - viz., the disclosure of sensitive information and second, use this information to jailbreak the LLM (may be straightforward, may be use the exposed information along with other information)

   • So probably a solution is to add to three items :

     • Add System Instruction as another internal information and methods to protect it in “LLM06: Sensitive Information Disclosure”

     • Add system instruction leakage as another source for prompt injection (basically modify system instructions to do harm) to “LLM01: Prompt Injection & other insecure input handling”

     • The System Instructions leakage would be another prompt injection vulnerability, adding to “LLM02: Insecure Output Handling” also is interesting. We filter any system instructions out ! I hope that itself is not yet another system instruction

Analytics

Steve has done a good job in coalescing the results:

Gory Details

• Steve’s summary slides are here, the raw voting results and the collated results all are here in our GitHub.

3. Notes from the Core Team v2 deliberations

The slack channel #team-llm-v2-brainstorm has lots of interesting discussions. I have a proposal mapping [here], based on the discussions from the v2 brainstorm discussions. We had a v2 core team conversation, which you can watch - video [here] and the slides [here].

Ads has captured the key points from the discussion:

4. OWASP T10 for LLM Applications v2 : Next Steps

The v2 is in development, evolving and still formative. It is up to all of us to shape the entries, their order and the write ups !

The Data Collective has a call out for data and their work is here. It is important to contribute as this will form the basis for v2.

🔗 OWASP LLM Documents & Links 🔗

• Previous newsletters (to catchup after marveling our eloquence 🙃) [Here]

• OWASP Top 10 LLM main site [Here]

• OWASP Top 10 for LLM Applications v1.1 [Here]

• LLM AI Security & Governance Checklist v1.1 [Here]

📱Stay Connected📱

Aubrey King has expanded our social media reach across various platforms including LinkedIn, Youtube, Twitter (Agree with Will, not going to say X), and soon even more! Follow and don’t be shy to re-share any content we post! BTW, if you're presenting anything related to our project externally, do let us know at [email protected] so we can review and blast out to our social media followers!

💭 Closing Thoughts 💭

As you can see, we are making good progress towards OWASP Top 10 LLM V2. And, we have more initiatives like the Solutions doc, AI Red Teaming Best Practices, Data Collective and so forth. An inflection point probably. So a good time to track, participate and contribute …

We look forward to seeing you in our Slack channels and at our upcoming meetings!

Till next time … Stay Secure & Stay Sharp …

Krishna Sankar
LinkedIn | Medium | Github