The project, undertaken as Ilham’s MSc thesis Mind the Gap: Comparing Model- vs Agentic-Level Red Teaming with Action-Graph was developed in collaboration with Holistic AI; a London-based company founded by UCL Computer Science alumni, specialising in auditing, governance, and risk management of artificial intelligence systems. They received a $50,000 award as part of what organisers described as the largest hackathon ever hosted on Kaggle. The competition attracted more than 600 submissions and distributed a total prize pool of $500,000.
Ilham entered the hackathon as the lead author, working with researchers from Holistic AI, who provided co-authorship and technical supervision throughout the project. “Holistic AI acted as technical supervisor, providing valuable feedback and discussion on my work with their expertise in AI governance, safety, and audit,” he explained.
Ilham’s method focused on testing the model’s safety when used in agentic AI systems.
The winning entry, Mind the Gap: Model vs Agentic-Level Action Attack, identifies a critical “security gap” where traditional model-level red teaming fails to expose vulnerabilities in autonomous large language model (LLM) agents. Ilham’s framework demonstrates how “agentic-level” attacks can bypass standard safety mechanisms, revealing deployment-specific vulnerabilities.
At the core of the project is AgentSeer, a tool Ilham developed to provide observability into how agentic AI systems operate. Combined with a structured red-teaming approach, it offers what judges described as “an exceptionally rigorous and thorough investigation of jailbreaking, through a well-structured framework.”
Reflecting on the project, Ilham said:
“It was humbling to see how fast the AI safety landscape is changing, and that it is crucial to address it as a multidisciplinary problem requiring collaboration of research, engineering, and governance/policy expertise.”
Alongside the hackathon success, the work has been published in full paper format on arXiv and accepted for presentation at the NeurIPS 2025 Workshop on LLM Evaluation, one of the leading forums for AI and machine learning research.
Ilham also acknowledged the role of his UCL supervisor, Professor Philip Treleaven, in shaping the project:
“Support from Prof. Treleaven has been at the core of this success. He always motivates me to bring the thesis project beyond academic requirements, to publish and share the work with the world and make real impact.”
Looking ahead, Ilham hopes to continue in the AI safety and governance field:
“These achievements mark a significant milestone in my career, fuelling my momentum into the AI safety and governance space. I now aim to move into AI product engineering to deliver scalable AI safety solutions.”
For the official announcement of the competition results, visit Kaggle’s discussion page.
