Unlocking Security Vulnerabilities: How GPT-4 Can Utilize Security Bulletins to Its Advantage

Scientists from the University of Illinois at Urbana-Champaign (UIUC) have concluded that the large language model (LLM) GPT-4 from OpenAI can successfully exploit real vulnerabilities if it “reads” a security bulletin about a CVE describing the issue.

“To demonstrate this, we compiled a dataset of 15 1-day vulnerabilities, including those categorized as critical,” explain the experts in their article. “Upon receiving the CVE description, GPT-4 could exploit 87% of these vulnerabilities compared to 0% for all other models we tested (GPT-3.5 and open-source LLMs), as well as open-source vulnerability scanners (ZAP and Metasploit).”

It is worth noting that the term “1-day vulnerability” typically refers to vulnerabilities for which information has been disclosed but the issue has not yet been fixed. CVE descriptions refer to NIST bulletins (e.g. a bulletin describing problem CVE-2024-28859).

The mentioned models that failed the task include: GPT-3.5, OpenHermes-2.5-Mistral-7B, Llama-2 Chat (70B), LLaMA-2 Chat (13B), LLaMA-2 Chat (7B), Mixtral-8x7B Instruct, Mistral (7B) Instruct v0. 2, Nous Hermes-2 Yi 34B, and OpenChat 3.5. Notably, the main competitors of GPT-4, Claude 3 from Anthropic and Gemini 1.5 Pro from Google, were not part of the research due to limited access, though the scientists hope to test them in the future.

The LLM agent could not exploit only two out of the 15 vulnerability samples: Iris XSS (CVE-2024-25640) and Hertzbeat RCE (CVE-2023-51653). In the first case, issues arose because the web application Iris had a complex interface, making it difficult for the agent to navigate. In the second case, the detailed issue description was published in Chinese, potentially confusing the LLM.

Eleven of the vulnerabilities tested emerged after the completion of GPT-4 training, meaning the model had no prior data on them during training. The success rate for these CVEs was slightly lower at 82% (or 9 out of 11).

The researchers’ work builds upon previous findings that LLMs can be used to automate attacks on websites in a sandbox environment.

As experts told The Register, GPT-4 can autonomously execute steps to carry out specific exploits that open-source vulnerability scanners cannot yet detect.

Researchers anticipate that soon LLM agents, such as those created by connecting a chatbot to the ReAct automation framework implemented in LangChain, will make vulnerability exploitation a simpler task. These agents will be able to follow links with CVE descriptions to gather additional information.

Forewarning, if the capabilities of GPT-5 and future models are extrapolated, they will likely be much more powerful than those accessible to script kiddies today.

On average, the researchers calculated that the cost of a successful LLM agent attack is only $8.80 per exploit, approximately 2.8 times less than the payment for 30 minutes of a pentester’s work.

If the LLM agent (GPT-4) is deprived of access to the corresponding CVE description, the probability of successfully exploiting the bug reduces sharply from 87% to 7%. However, the authors of the scientific article believe that restricting public access to vulnerability information is unlikely to be an effective way to shield against such LLM agents, as the principle of security through obscurity, according to the researchers, does not work at all.

The code of the described agent, experts say, consists of only 91 lines and 1056 tokens for queries. In their work, the researchers asked OpenAI not to publish their prompts, although they are willing to provide them upon request from the company.

0 / 5

Your page rank:


Subscribe: YouTube page opens in new windowLinkedin page opens in new windowTelegram page opens in new window

Leave a Reply

Your email address will not be published. Required fields are marked *

Post comment