If you use the Internet, at all, you’ve seen this checkbox:
One of the most frequently used tools for reducing website form spam are CAPTCHAs, like those used by Google’s reCAPTCHA system. The idea behind reCAPTCHA is that it can assess your behavior before and during the act of checking that box, and determine whether you’re a soulless bot whose sole purpose in life is to inform the website’s administrator about the merits of the product that they absolutely don’t want, or a living, breathing human being customer.
If the user is fortunate, all they’ll have to do is check the box, reCAPTCHA will be satisfied, and you can proceed with submitting your form. Or, if reCAPTCHA is feeling feisty that day, it may instead ask you to complete some annoying task or another, such as the following:
Inevitably, you find yourself asking the sorts of meditative and obviously insane questions that used to only occur to people who’d lived in the woods for too long, such as, “Does the edge of the wheel count as part of the motorcycle? What about the rider? What about the very top of the rider’s helmet?” Chances are you’ll be wrong and condemned to repeat the exercise.
You don’t enjoy this experience. No one does. And that’s been the case for a while. Five years ago, an investigative journalist at the tech news site The Verge wrote of CAPTCHAs:
At some point last year, Google’s constant requests to prove I’m human began to feel increasingly aggressive. More and more, the simple, slightly too-cute button saying “I’m not a robot” was followed by demands to prove it — by selecting all the traffic lights, crosswalks, and storefronts in an image grid. Soon the traffic lights were buried in distant foliage, the crosswalks warped and half around a corner, the storefront signage blurry and in Korean. There’s something uniquely dispiriting about being asked to identify a fire hydrant and struggling at it.
While this was the opinion of a single journalist a few years ago, even a casual glance at Google suggests that the CAPTCHA experience leaves something to be desired.
We both know CAPTCHAs do not improve the user experience, but we also know how incredibly annoying it is to get inundated with website form spam, and you’d do just about anything to stop it.
That’s why it’s worth taking a hard look at the question of whether setting up a CAPTCHA on your website form will block spam effectively enough to justify the annoyance to customers. To answer that complicated question, we’re going to take a look at what CAPTCHAs are, how effective they are at reducing spam, and the potential downsides of using them.
What are CAPTCHAs?
By the late 1990s, automated form spam was a serious issue, and computer scientists and software developers were actively testing a variety of software-based solutions to identify and stop spam bots from submitting forms.
One computer scientist interested in solving this problem was Luis von Ahn, who in 2003 coined the term “CAPTCHA,” an acronym that stands for, “Completely Automated Public Turing test to tell Computers and Humans Apart.” His concept of a CAPTCHA was that it would be a simple, automated test that could be easily solved by people, but not by the software powering spambots—they’d hit the sweet spot between being too difficult for bots, but not so difficult that they were overly difficult for people to solve.
In 2007, von Ahn released the first version of reCAPTCHA, which within a couple of years was deployed all across the Internet, with millions of users daily having to decipher distorted text like the following:
reCAPTCHA was only one of many competing CAPTCHA systems to be developed and sold, and over time, the increasingly distorted text-based CAPTCHAs of the late ‘00s and early ‘10s gave way to the increasingly complex and, yes, challenging puzzles seen today.
How effective are CAPTCHAs at stopping spam?
It may come as a surprise that it’s not well understood how effective CAPTCHAs actually are at blocking spam. If you try and find data on the topic, you’ll find writeups on studies and experiments from 10 or 15 years ago examining the performance of old, distorted text-style CAPTCHAs. But there’s very little information on how well more modern systems like reCAPTCHA perform.
On the other hand, what is well understood is how effective bots are at bypassing CAPTCHAs. Software-based approaches have been in development since CAPTCHAs were first implemented. Nearly a decade ago, researchers at Columbia University developed a bot-based system that could solve more than 70% of CAPTCHAs.
But in the era of ChatGPT, spammers have been empowered in ways never before imaginable. In 2023, it was found that Bing Chat, a free bot offered by the Microsoft search engine utilizing a version of ChatGPT called GPT-4, could be tricked into solving CAPTCHAs, even though it was specifically designed not to solve CAPTCHAs.
That’s just the tip of the iceberg. In 2023 researchers at the University of California, Lawrence Livermore National Laboratory, Microsoft, and ETH Zurich published a paper, “Empirical Study & Evaluation of Modern CAPTCHAs,” in which laid out their findings on how difficult popular CAPTCHA systems were to solve for both people and bots.
Their findings were alarming. As it turned out, bots are both more accurate and faster at solving CAPTCHAs. Humans had an accuracy rate of 50% to 85% for solving CAPTCHAs, depending on the type of CAPTCHA. Bots had an accuracy rating of 85% to 100%. And for every type of CAPTCHA tested, bots were faster than their human counterparts or were as fast as the fastest side of the spectrum of humans.
As of now, all the data at hand indicates that CAPTCHAs are failing to keep up with the technology wielded by spammers, especially now that sophisticated bot systems can be utilized at low or even no cost, making sophisticated, bot-powered spam attacks easier to deploy than ever.
What potential impact do CAPTCHA systems have on website customers?
Website owners and managers are typically focused on two factors when deciding to implement CAPTCHAs: how easy it is to implement, and how effective it is at blocking spam. However, they often fail to think about the potential negative consequences of using CAPTCHAs.
If you’re thinking about using them on a support page form, then the potential downsides are probably minimal. But using CAPTCHAs on a lead form, or any other marketing-related form, could be potentially disastrous. Is it possible that potential customers will bail on your website rather than deal with the hassle of solving a CAPTCHA to contact you?
To answer this, let’s look at three more key findings from the UC/Microsoft study mentioned above.
CAPTCHAs can take a long time to solve.
While the simplest checkbox-based CAPTCHAs took only 4 seconds for people to solve, anything more challenging took much longer. Depending on the type of CAPTCHA, average solve times ranged from 7 or 8 seconds for distorted text CAPTCHAs, to over 30 seconds for more difficult CAPTCHAs, such as the common image-based ones.
As marketers, we often stress to clients the difference that mere seconds can make to customers. Adding 4 seconds to the time it takes a customer to convert is discomforting. Adding 8 seconds is bad.
We would sooner set our hair on fire than do anything that actively annoyed our clients’ customers for 30 seconds.
Older users take longer to solve CAPTCHAs.
The study found that the average solve times for challenging CAPTCHAs increased by nearly a second for every 10 years older a participant was. For businesses that cater to older clientele, this means that CAPTCHAs would pose a higher risk of aggravating customers and sending them elsewhere.
Form abandonment rates are disastrously high.
If we knew nothing other than the average CAPTCHA solve times that the researchers measured in the study, our expectation would be that form abandonment rates could be significant.
What the researchers found was worse than we expected. When the study’s test subjects were told that they were being tested specifically on CAPTCHA completion, 25% gave up before completing a single CAPTCHA. But when subjects were told that they were being tested on another task, such as the creation of a website account, in which they had to complete a CAPTCHA, the abandonment rate was 50%.
From our perspective, we see optimizations that improve conversion rates by 10% at no additional cost to be huge wins and changes that decrease conversion rates by 10% to be absolute disasters. By that measure, losing 25% to 50% of customers is unimaginable.
Our takeaways on whether you should use CAPTCHAs on website forms.
The facts on the use of CAPTCHAs are as follows:
- There is no solid data currently available that indicates how effective CAPTCHAs are at reducing spam.
- There is a wealth of data indicating that modern bot systems can easily render CAPTCHA systems useless, often solving them more effectively than people can.
- CAPTCHAs can potentially add 4 to more than 30 seconds to the time it takes a potential customer to fill out a form.
- The use of CAPTCHAs potentially increases form abandonment rates by 25% to 50%.
- Older users take longer to solve CAPTCHAs, potentially resulting in higher abandonment rates with older customers.
Our recommendation: Based on the above evidence currently at hand, we would not recommend the use of reCAPTCHA or other popular CAPTCHA systems that require users to try to prove their humanity. At best, they frustrate users. At worst, they turn real customers away, while offering little resistance to bots.
In future articles, we will explore other forms of spam solutions, digging through the data to identify what works and what doesn’t.
