MCA-Bench: A Multimodal Benchmark for Evaluating CAPTCHA Robustness Against VLM-based Attacks

Summary

This paper talks about MCA-Bench, a new testing system that checks how strong CAPTCHA challenges are against attacks from advanced AI models that understand both images and text. It brings together many different types of CAPTCHAs, like pictures, interactive puzzles, and logic questions, into one big benchmark using a shared AI backbone to test them all fairly and consistently.

What's the problem?

The problem is that CAPTCHAs, which are designed to tell humans and bots apart online, come in many forms and are becoming easier for smart AI models to solve. Existing studies only look at some CAPTCHA types separately, and there isn’t a unified way to test their security against modern AI attacks, making it hard to know which CAPTCHAs are truly secure.

What's the solution?

The solution was to create MCA-Bench, which combines over twenty different real-world CAPTCHA tasks from several categories and tests them with a shared vision-language AI model fine-tuned with special attackers. This lets researchers measure how well these CAPTCHAs hold up against cutting-edge AI across different challenge types using the same evaluation system.

Why it matters?

This matters because CAPTCHAs protect websites from harmful bots, and knowing which ones can resist advanced AI attacks helps make online services safer. MCA-Bench also provides guidelines for designing stronger CAPTCHAs that are harder for AI to bypass, supporting better security for millions of internet users.