We, the Secure AI Futures Lab (SAFL) team, ran 5 sessions between 8 Aug and 3 Oct 2026 at the Secure AI Hub in Bengaluru to help technical professionals from different backgrounds upskill in AI red teaming.
We structured the sessions to go beyond paper reading and discussion by including exercises such as predicting results and using Colab notebooks to reproduce key results with smaller models.
Two guest talks.
Mohammad Taufeeque (FAR.AI) gave a talk for the workshop on 22 Aug, covering, among other things, what red teaming looks like at FAR.AI.
Pulkit Verma (IIT Madras) gave the first talk in our AI Futures Dialogue series on 21 Aug, on how users can assess AI agents after deployment.
20 different people attended, with 38 attendances in total. A core group of 6 came to 3 or more sessions.
Mean overall experience: 9.6/10; NPS +80 from 25 responses.
Main output:
A list of open problems in jailbreaking for participants to contribute to
5 sets of session notebooks and shared docs, available on request.
Top request from participants: a capstone research project or paper (8 of 15 suggestions).
What we set out to do
Most people in India who want to work on AI red teaming are on their own and don’t have a shared place to practice. The workshop was for technical professionals in Bengaluru, with or without a security background, who wanted to understand why model safeguards fail and how to measure that. Each session paired a key paper with hands-on work on small open models. By the end, we aimed for participants to be able to judge whether a reported jailbreak result is believable and test a claim themselves.
The plan was to have 8 sessions, one a week, 3 hours each. We made design choices such as making sessions standalone, requiring no dedicated setup for hands-on work; and designing sessions for active engagement.
Success criteria
We judge the workshop against two objectives, each with expected and stretch outcomes.
P50 is the outcome we expected to reach.
P90 is an ambitious outcome we expected to reach about one time in ten.
Objective
Level
Outcome
Result
O1. Skill. Participants can judge whether a reported jailbreak result is believable, and test a claim themselves.
P50
Every session runs a hands-on block with working material.
Met. All 5 sessions had hands-on material.
P50
Most of the core group can state a new paper's threat model, central claim, and weakest assumption.
Met. Every session asked these questions about the paper, and attendees could generally answer them.
P90
At least 2 participants run their own test of a published claim outside the sessions and share it.
Partly met. In the end-of-program survey, 1 of 5 respondents reported testing a published claim and sharing a write-up; a second shared results.
O2. Community. A Bangalore group that continues jailbreak research after the workshop.
P50
15 or more different attendees; a core of 5 or more at 3+ sessions; mean rating 8/10 or higher.
Met. 20 attendees, core of 6, mean 9.6/10.
P90
3 or more participants start a capstone project from the open-problems list, with a public output by early January 2027.
In progress. The list is ready. All 5 end-of-program survey respondents said they would join a capstone project.
Figure 1. Attendance per session, split into new and returning people.
The sessions did not run every week as planned due to operational constraints. There was a two-week gap before session 4 and a four-week gap before session 5, and sessions 6 to 8 did not run.
Mohammad Taufeeque, Member of Technical Staff at FAR.AI, gave a guest talk for the workshop on 22 Aug, before session 3. He spoke for 45 minutes over Google Meet and then took questions. He covered jailbreaking as a test of safeguards and misuse risk, manual attacks such as prefill and the instruction hierarchy, automated attacks framed as optimization such as GCG, and defenses such as data filtering and trusted-access programs. We worked with him to skip what sessions 1 and 2 had covered and to lead into session 3 on the refusal direction. Taufeeque said: “The crowd had great questions, so it was fun to present!... Great work on gathering this set of people.”
The day before, on 21 Aug, Pulkit Verma (Assistant Professor, Computer Science and Engineering, IIT Madras) gave a talk on “User-Aligned Assessment of Taskable AI Systems”. This talk was open to all, not just the workshop participants. He argued that checking an AI system against its designer’s intent misses what users need once it is deployed, and proposed an assessment module that queries black-box agents to tell users what they can do. See Pulkit's slides here.
Participants
Figure 2. Attendance by person (anonymized), grouped by first session.
Figure 3. Number of sessions each person attended.
Attendees fell into two groups: a core of 6 who attended three or more sessions, and 11 who attended once. Of the 8 people at session 1, 4 came back, and 3 were still attending at session 5. About 3 new people joined every session, which the stand-alone format made possible.
Most attendees were working professionals. Of the 20 attendees, 5 were software engineers, 4 were ML engineers or data scientists, 3 worked in research roles, 2 were security engineers, and 2 had founded startups. The other 4 were working independently or between jobs. None listed themselves as a student. Only 2 came from security, so most people came to red teaming from software or ML. The core group was as mixed as the wider group: it included a security engineer, a data scientist, a software engineer, an ML research intern, and two people working independently.
What worked
Stand-alone sessions brought in steady newcomers without losing the core group.
The hands-on notebooks worked well, and participants used the participant doc and notebooks more than anything else.
Participants found the Hub environment highly conducive to their work and interaction and expressed interest in more similar on-site future programs.
Participants also noticed the production quality (“Didn’t expect so much polish in the first and second workshop”) and the facilitation, which one described as “well structured… facilitator knowledgeable, welcoming and approachable”.
What we'd like to change
The schedule drifted: 5 of the planned 8 sessions ran, over 8 weeks. We didn't have a backup facilitator, so we couldn't run 3 sessions in between.
The survey collection also lapsed as the program went on. The response rate fell from 100% to 25% while attendance stayed level, so this was a collection problem rather than a drop in interest. Next time we will run the survey in the session, before people leave.
Figure 4. Survey responses against attendance, with the response rate.
The survey itself also had problems. Ratings hit the ceiling, with 18 of 25 responses giving 10, so they cannot tell sessions apart. It had two duplicate experience questions and no measure of learning. Next time, we intend to use a survey that asks for a before-and-now self-rating, value compared with other uses of time, the most and least valuable parts, pace, and one change.
Half of the attendees came only once, and although we accounted for participants joining the series in between, we'd ideally want participants to be regular attendees.
Figure 5. Distribution of recommendation scores across all sessions.
What participants asked for
Figure 6. Themes in free-text suggestions for future programs.
The most common request, in 8 of 15 suggestions and raised in every session, was a capstone research project or paper. The next most common, in 5 suggestions, was more hands-on or advanced technical work, such as an attack-versus-defence exercise, mechanistic interpretability, ARENA-style coding jams, and advanced courses. We responded with the open-problems list as a starting point for capstone projects.
Outputs
The main output is a paper of 7 open problems in three parts: measurement, defences, and attack surfaces. It is not public by default; we plan to share it as an invite-only doc with the participants who want to do self-directed capstones. The session notebooks and participant docs for sessions 1–5 are available on request.
End-of-program survey
We sent an end-of-program survey to all 20 attendees in October and, after several reminders, received 5 responses. Four came from people who attended three or more sessions, so the answers mostly reflect the core group. All five rated their ability to judge a jailbreak claim higher than before the workshop, from 2.8 to 4.2 on average out of 5. Four said the workshop was a much better use of their time than the alternatives, and one said it was somewhat better. Four have read further papers since; one has used the material in their work or study, one has tested a published claim and shared a write-up, and another has shared a write-up or results. All five said they would join a more intensive version and a capstone project. One asked for continuous classes that build on the previous session. With only 5 of 20 responding, we treat this as a signal from the regular attendees, not a measure of the whole group.
Limitations in our account
The survey was self-reported and anonymous, so we couldn't automatically link responses to attendance. The numbers are small: 20 attendees and 25 responses. We checked learning in each session’s discussion but did not measure it during the workshop. The end-of-program survey asked people to rate themselves before and after, and what they did afterwards, but only 5 of 20 attendees replied, mostly from the core group.
Who we are
The Secure AI Futures Lab (SAFL) is a research and capacity-building lab for advancing science of safe AI in emerging powers like India. We ran this workshop at our Bengaluru hub, one of our two physical workspaces in India for AI safety science (the other being in New Delhi). You can read more about us on our substack.
Summary
What we set out to do
Most people in India who want to work on AI red teaming are on their own and don’t have a shared place to practice. The workshop was for technical professionals in Bengaluru, with or without a security background, who wanted to understand why model safeguards fail and how to measure that. Each session paired a key paper with hands-on work on small open models. By the end, we aimed for participants to be able to judge whether a reported jailbreak result is believable and test a claim themselves.
The plan was to have 8 sessions, one a week, 3 hours each. We made design choices such as making sessions standalone, requiring no dedicated setup for hands-on work; and designing sessions for active engagement.
Success criteria
We judge the workshop against two objectives, each with expected and stretch outcomes.
Objective
Level
Outcome
Result
O1. Skill. Participants can judge whether a reported jailbreak result is believable, and test a claim themselves.
P50
Every session runs a hands-on block with working material.
Met. All 5 sessions had hands-on material.
P50
Most of the core group can state a new paper's threat model, central claim, and weakest assumption.
Met. Every session asked these questions about the paper, and attendees could generally answer them.
P90
At least 2 participants run their own test of a published claim outside the sessions and share it.
Partly met. In the end-of-program survey, 1 of 5 respondents reported testing a published claim and sharing a write-up; a second shared results.
O2. Community. A Bangalore group that continues jailbreak research after the workshop.
P50
15 or more different attendees; a core of 5 or more at 3+ sessions; mean rating 8/10 or higher.
Met. 20 attendees, core of 6, mean 9.6/10.
P90
3 or more participants start a capstone project from the open-problems list, with a public output by early January 2027.
In progress. The list is ready. All 5 end-of-program survey respondents said they would join a capstone project.
Program content
No.
Date
Topic
Attended
New
Survey
1
8 Aug
"Jailbroken: How Does LLM Safety Training Fail?" (2023)
8
8
8
2
15 Aug
"Do Anything Now: Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models" (2024)
6
3
6
3
22 Aug
"Refusal in Language Models Is Mediated by a Single Direction" (2024); guest talk from FAR.AI
9
3
5
4
5 Sep
"Jailbreaking Black Box Large Language Models in Twenty Queries" (PAIR, 2023)
7
3
4
5
3 Oct
"Large Reasoning Models Are Autonomous Jailbreak Agents" (Nature Communications, 2026); open problems
8
3
2
Figure 1. Attendance per session, split into new and returning people.
The sessions did not run every week as planned due to operational constraints. There was a two-week gap before session 4 and a four-week gap before session 5, and sessions 6 to 8 did not run.
Mohammad Taufeeque, Member of Technical Staff at FAR.AI, gave a guest talk for the workshop on 22 Aug, before session 3. He spoke for 45 minutes over Google Meet and then took questions. He covered jailbreaking as a test of safeguards and misuse risk, manual attacks such as prefill and the instruction hierarchy, automated attacks framed as optimization such as GCG, and defenses such as data filtering and trusted-access programs. We worked with him to skip what sessions 1 and 2 had covered and to lead into session 3 on the refusal direction. Taufeeque said: “The crowd had great questions, so it was fun to present!... Great work on gathering this set of people.”
The day before, on 21 Aug, Pulkit Verma (Assistant Professor, Computer Science and Engineering, IIT Madras) gave a talk on “User-Aligned Assessment of Taskable AI Systems”. This talk was open to all, not just the workshop participants. He argued that checking an AI system against its designer’s intent misses what users need once it is deployed, and proposed an assessment module that queries black-box agents to tell users what they can do. See Pulkit's slides here.
Participants
Figure 2. Attendance by person (anonymized), grouped by first session.
Figure 3. Number of sessions each person attended.
Attendees fell into two groups: a core of 6 who attended three or more sessions, and 11 who attended once. Of the 8 people at session 1, 4 came back, and 3 were still attending at session 5. About 3 new people joined every session, which the stand-alone format made possible.
Most attendees were working professionals. Of the 20 attendees, 5 were software engineers, 4 were ML engineers or data scientists, 3 worked in research roles, 2 were security engineers, and 2 had founded startups. The other 4 were working independently or between jobs. None listed themselves as a student. Only 2 came from security, so most people came to red teaming from software or ML. The core group was as mixed as the wider group: it included a security engineer, a data scientist, a software engineer, an ML research intern, and two people working independently.
What worked
What we'd like to change
The schedule drifted: 5 of the planned 8 sessions ran, over 8 weeks. We didn't have a backup facilitator, so we couldn't run 3 sessions in between.
The survey collection also lapsed as the program went on. The response rate fell from 100% to 25% while attendance stayed level, so this was a collection problem rather than a drop in interest. Next time we will run the survey in the session, before people leave.
Figure 4. Survey responses against attendance, with the response rate.
The survey itself also had problems. Ratings hit the ceiling, with 18 of 25 responses giving 10, so they cannot tell sessions apart. It had two duplicate experience questions and no measure of learning. Next time, we intend to use a survey that asks for a before-and-now self-rating, value compared with other uses of time, the most and least valuable parts, pace, and one change.
Half of the attendees came only once, and although we accounted for participants joining the series in between, we'd ideally want participants to be regular attendees.
Figure 5. Distribution of recommendation scores across all sessions.
What participants asked for
Figure 6. Themes in free-text suggestions for future programs.
The most common request, in 8 of 15 suggestions and raised in every session, was a capstone research project or paper. The next most common, in 5 suggestions, was more hands-on or advanced technical work, such as an attack-versus-defence exercise, mechanistic interpretability, ARENA-style coding jams, and advanced courses. We responded with the open-problems list as a starting point for capstone projects.
Outputs
The main output is a paper of 7 open problems in three parts: measurement, defences, and attack surfaces. It is not public by default; we plan to share it as an invite-only doc with the participants who want to do self-directed capstones. The session notebooks and participant docs for sessions 1–5 are available on request.
End-of-program survey
We sent an end-of-program survey to all 20 attendees in October and, after several reminders, received 5 responses. Four came from people who attended three or more sessions, so the answers mostly reflect the core group. All five rated their ability to judge a jailbreak claim higher than before the workshop, from 2.8 to 4.2 on average out of 5. Four said the workshop was a much better use of their time than the alternatives, and one said it was somewhat better. Four have read further papers since; one has used the material in their work or study, one has tested a published claim and shared a write-up, and another has shared a write-up or results. All five said they would join a more intensive version and a capstone project. One asked for continuous classes that build on the previous session. With only 5 of 20 responding, we treat this as a signal from the regular attendees, not a measure of the whole group.
Limitations in our account
The survey was self-reported and anonymous, so we couldn't automatically link responses to attendance. The numbers are small: 20 attendees and 25 responses. We checked learning in each session’s discussion but did not measure it during the workshop. The end-of-program survey asked people to rate themselves before and after, and what they did afterwards, but only 5 of 20 attendees replied, mostly from the core group.
Who we are
The Secure AI Futures Lab (SAFL) is a research and capacity-building lab for advancing science of safe AI in emerging powers like India. We ran this workshop at our Bengaluru hub, one of our two physical workspaces in India for AI safety science (the other being in New Delhi). You can read more about us on our substack.