cross-posted from: https://programming.dev/post/37726760
- Guardrails can be bypassed: With prompt injection, ChatGPT agents can be manipulated into breaking built-in policies and solving CAPTCHAs.
- CAPTCHA defenses are weakening: The agent solved not only simple CAPTCHAs but also image-based ones - even adjusting its cursor to mimic human behavior.
- Enterprise risk is real: Attackers could reframe real controls as “fake” to bypass them, underscoring the need for context integrity, memory hygiene, and continuous red teaming.
It’s been obsolete for years. AI just makes it easy.
I mean the data from those has been used for AI training for ages so it’s hardly surprising it got good at it.
I posted this elsewhere, but CAPTCHAs have always been used to train models, and have always had to improve themselves even before LLMs blew up. This article was posted from a site with an .ai tld, and seems to be doing the whole Sam Altman “I’m scared of AI, AGI is right around the corner! I certainly don’t have a vested interest in making you think it does more than it actually does”
ChatGPT joins The Human League?
appropriate given they’ve been screaming “DON’T YOU WANT ME” at disinterested parties all this time
I’ve had a captcha solver browser extension for as long as captchas existed. Anyone believing reCaptcha’s claim that it was ever about bot prevention is a goddamn moron
reCapatcha exists soley for two reasons
- Free labor to train statistical models
- Punishing people who use VPNs, noscript, etc.
It’s about making headless bots unreasonably expensive to make massive requests with.
Which one do you use? I’ve used a handful over the years and they all stop working for me.
Currently it’s Buster: Captcha Solver for Humans (chromium)
Awesome, thanks! I’ll give that a shot. Although I use Firefox so I’ll have to see if they make it for non chromium.
Proof of work.
