What these articles never say is how many hallucinated bugs the LLM found that either weren’t real or were actually exploitable. The LLM didn’t find these with any confidence it highlighted areas of interest that actual security researchers then needed to investigate and confirm or rule out.
In the article it says the ffmpeg vulns were found by an “autonomous” agent and that it produced a proof-of-concept for each. So what do you base your claims on? They seem quite contrary to that.
Even if there was still a lot of human work involved, it seems that the LLM-Agents can help a lot with security research, as the number of (real) zero-days that are beeing found recently (with the help of AI) seems to spike (telling from what I read, e.g. here on Lemmy, or the number of security updates for my distro).
It’s states they were produced which I’m taking to mean found and it’s ambigously worded so it’s unclear if the article is actually claiming it generated PoC for all of them. It doesn’t say how many if any hallucinated results were produced or how much effort was needed to fully confirm, basically down played the human involvement.
It’s great that so many bugs are being found and squashed but it’s implied LLMs are doing all the work when what they are actually doing is assisting as a tool.
I agree that the wording is a bit ambiguous, I interpreted it the way it seems more natural to me. In the post by the researcher(s) themselves, it says in the tldr paragraph that the “agent produces concrete, reproducible PoC inputs to confirm its findings” but also that they (probably humans) “explored the exploitability of the issues and developed a PoC demonstrating a RCE exploit primitive”. Apparently it finds the vulnerabilities very concretely but humans were involved for the full-blown exploit. It also doesn’t say much about the number of false-positives.
I’m not in the business, so I can’t tell how much of the work such agents are actually saving. Since the articles don’t say much about the amount of human involvement, the imagination conveyed by them probably depends strongly on the (knowledge of the) reader. But in my opinion it is a bit of stretch to say this is downplaying it. It should be noted though, that the article probably sources its information from a post by the company selling that AI.
With that information, the “without any confidence” and “area of interest” parts of your previous post still seem misleading.
In my experience, when given real world data to run on, they don’t hallucinate that often. Its when you ask it to regurgitate stored info that its off by a wild amount sometimes. Fixing or comparing code is like AI 101, unlike code generation where it may be incorrect.
I strongly disagree. Every response longer than one line of code or longer than 1-2 sentences on a non-trivial task, has at least inaccuracies or mistakes. When working with something AI has created, I had to go in and fix it manually in 100% of those cases. That’s why I limit the use of AI to only type-ahead suggestions, as I can easily verify them and don’t waste more time than creating the result manually.
I’m frankly quite annoyed by the amount and extend of anti-AI hate in this community. It almost seems like this is a pure anti-AI rage community. The capabilities and the utility of LLMs are basically always denied or downplayed as much as possible without running into obvious contradictions.
It would be so nice to have some differentiated and insightful discussions here, about what can or cannot be done with AI, positive and negative impacts it has, possible new use cases, how AI should or should not be used, how the overall benefits of AI can be maximized and the overall negative effects minimized, what the world with AI could be like in the future, …
I was trying to have some insightful discussion on the actual capability of LLM which is difficult when the involvement of the human element is played down amd the role of the LLM is played up to feed the hype machine. It’s hard to acknowledge the real capabilities and weaknesses when the capabilities are always over reported and the weaknesses down played or denied.
It’s great that so many bugs are getting discovered but as I say there is no reporting on what effort was needed to sift and review the LLM output or how functional or understandable any PoC were… The article doesn’t directly even state the PoC were directly produced by the LLM and reads very ambigously.
What these articles never say is how many hallucinated bugs the LLM found that either weren’t real or were actually exploitable.
It literally wouldn’t matter if it did.
The fact that it found exploitable bugs means that these bugs need to be addressed. To be clear, I care much more about the security flaws and fixing them than how they were discovered.
I feel like you missed the forest for the trees.
The question is how many were made up?
I saw that, and you’re right, I wasn’t answering that question. What I was saying was that I thought the question was irrelevant and ignoring a bigger issue.
I disagree that its ignoring the bigger problem, which is that slop like this is overwhelming devs to get fixes out ASAP faster than they can fix.
So now we have AI big reports feeding AI big fixes in a lot of projects.
The assumption that what AI finds is correct in the first place is… Probably wrong.
It makes stuff up all the bloody time, so how many of these bugs were made up, or not actually bugs?
The term 0-day vulnerability in security has a specific meaning. It means that it was not found by security researchers and is instead being actively exploited on the day that it’s discovered. I’m pretty sure none of these by definition or zero days
What the fuck is a zero day in the context of ffmpeg?
Its not like its a system service that you can get ingress through…
“AI found 21 bugs in massive video project” sounds like junior developer shit hungry to get some shit on their resume.
Even if it wasn’t AI slop, this wouldn’t be impressive.
My understanding is that ffmpeg is the bedrock that all video streaming services use. I’m suspicious it’s a bigger deal than you think
Discovering bugs is not “AI slop”.
That term refers to something the AI made. This is just product testing, where a real human then fixes it as intended.
if you upload any image on lemmy, it goes through ffmpeg to be converted into a webp file.
now, you can feed arbitrary input to that ffmpeg process
I never thought of doing that with ffmpeg. Why ffmpeg, instead of imagmagick?
It just so happens that many video codecs are based on image formats, so ffmpeg already has a lot of the complex machinery to do so available to also implement these image formats - internally it can just handle it as a single frame of video with specialized formats for that.
Imagemagick (and other tools) also work, but why use multiple pieces of software if what you already have is adequate? ImageMagick is also software, and can also have vurnabilities.
ffmpeg is on any system, has a consistent user interface for many different conversions
From the article
Most are heap or stack overflows in parsers and demuxers, spanning components from the TS demuxer to the VP9 decoder. depthfirst says some already carry CVE identifiers; its writeup lists nine, CVE-2026-39210 through CVE-2026-39218, and notes the rest are fixed but not yet numbered. It also published a PoC.
Its not like its a system service that you can get ingress through…
With a competently crafted payload, you could perhaps get in via someone’s transcoding pipeline.
Does nobody isolate ffmpeg and friends from their application?
I can’t imagine you’d have much fun breaking into a container that terminates the moment the original ffmpeg stops, or over-runs its max execution time…
Container escapes do exist, and they have shared kernel with the host
Tell me you know nothing about the intricacies of media playback (especially hardware accelerated), without telling me…
I imagine there are many web services around the world which use ffmpeg to handle user submitted content.
FFMPEG in the command line generally has permission to access the entire non-sudo filesystem and delete files.
Yes but why are we allowing user input to be fed to an executable in that environment?
This is the environment that almost all user software is executed.
Indeed. AI agents cannot “discover” zero days, they can only discover vulnerabilities. The humans can create zero days by reporting the vulnerability in the media without disclosing it properly.
