This is not an AI problem, it is a Human problem.

Security Bug Bounty rewards those who find a security bug in an application. But what if I ask an AI chatbot to produce a report of a “new” vulnerability in an application and then send it to the application maintainer hoping to get the reward?

Actually, it seems that this has been going on for some time, see here for example, and is starting to overwhelm application maintainers.

AI tools can be very helpful in analyzing and discovering security vulnerabilities in applications, but they must be used as one of the tools in the security practitioner toolbox.