Tag Archives: Vulnerabilities

Securing the AI software development tools: IDEsaster

Posted on by

I just quote two MaccariTA (Ari Marzouk) statements:

IDEs were not initially built with AI agents in mind. Adding AI components to existing applications create new attack vectors, change the attack surface and reshape the threat model. This leads to new unpredictable risks.

[…]

AI IDEs effectively ignored the base IDE software as part of the threat model, assuming it’s inherently safe because it existed for years. However, once you add AI agents that can act autonomously, the same legacy features can be weaponized into data exfiltration and RCE primitives.

and, as you can read in the original post, it is not only “risks” but also a quite long list of vulnerabilities (with 24 CVE) which affect, one way or or another, almost all AI IDE tools.

The issue is one we saw many times in the past: first features and functionalities, then we fix security. I agree that without features and functionalities any software product does not make any sense, but with security as a post add-on, there is the well known risk to have to pay a large security bill for a long time.

Latest AI Models can Autonomously Hack Websites

Posted on by

This research article is quite interesting and at the same time scary. It shows how the latest Large Language Models (LLMs) could be used to autonomously attack and hack Internet websites without human feedback or support.

The study shows that an AI model which

  1. can reach websites in Internet through tools and/or API
  2. can use the response of the websites as an input to itself to plan further actions
  3. can read documents provided a priori by humans as a support library of possible use

has in principle (and for GPT4, in practice) the capability to interact with the target website, identify vulnerabilities like SQL Injection, XSS, etc., and build and perform a successful attack.

The study also shows that, as of today, almost all AI models lack the three features to the maturity level required. Nonetheless, with the current speed of development of AI models, these features will become standard in very little time.

Due to the (future) ease and low cost of employing an AI model to hack a website, AI service providers face the critical task of preventing this type of abuse of their services, but owners of websites will need anyway to improve their security since sooner or later “AI hacking as a service” offerings will appear.

More Weaknesses of Large Language Models

Posted on by

Interesting scientific article (here) on new techniques to extract training data from Large Language models such as LLaMA and ChatGPT. The attack on ChatGPT is particularly simple (and for sure by now blocked by OpenAI): it was enough to ask it to “repeat forever” a word like “poem” and in the output, after some repetition of the word, it appeared random data and also a small fraction of training data, as for example a person’s email signature. This is a “divergence” attack on LLMs, in the sense that after the initial response, the model output starts to diverge from what is expected.

We still know too little about these models, their strengths and weaknesses, so we should take much care when adopting and using them.

On Open Source Software and the Proposed EU Cyber Resilience Act

Posted on by

I have not been following this, but I hear and read quite alarming comments about it (see eg. here).

If I understand it right (and please correct me if I don’t), the proposed Act starts from the absolutely correct approach that if someone develops some software, she or he is responsible for it and must provide risk assessments, documentation, conformity assessments, vulnerability reporting within 24 hours to the European Union Agency for Cybersecurity (ENISA), etc. This should work well for any corporation and medium/big size companies but requirements should be well balanced for example for open source distributed projects, or code released for free by single developers. Also taking into consideration that, as usual, not compliance with the Act will lead to fines.

Note added on December 10th, 2023: the final version of the CRA appears to address those concerns (see here for example).

Cryptography is still Difficult to Implement

Posted on by

This is a recurring theme (see eg. this older post), how difficult it is to implement Cryptography in the “right” way. It requires a lot of knowledge and expertise in Cryptography, programming skills are not enough.

Recently I read this article “PRACTICAL BRUTEFORCE OF AES-1024 MILITARY GRADE ENCRYPTION” by Kudelski Security Research, in which they show how it was possible to brute force what was called “AES-1024 military grade encryption“. AES is not broken (and “AES-1024” does not really stand for AES with 1024-bit keys) but the main problem was the custom procedure adopted to transform a user chosen password into an AES encrypting key.

Which again shows how it is difficult to implement Cryptography to attain the expected security goals.

CISA Catalogue of Known and Exploited Vulnerabilities

Posted on by

The Cybersecurity & Infrastructure Security Agency (CISA) has recently published the “Binding Operational Directive 22-01” which has the purpose of identifying the known and exploited vulnerabilities and address their resolution so to reduce the associated risks. 

In other words, CISA has identified the most risky and exploited vulnerabilities creating a catalogue (here) which can be used by everybody to identify the vulnerabilities which must be patched first. Indeed running a vulnerability scanner (or performing a penetration test) too often produces an extremely long list of vulnerabilities, classified by severity typically according to the CVSS-v3 standard: but which ones are really important / risky / even scary? A catalogue of vulnerabilities actually exploited by attackers can help to select the ones which really matter and that should be patched as-soon-as-possible.

Risks of Cryptography

Posted on by

Cryptography is too often considered as the “final” solution for IT security. In reality cryptography is often useful, rarely easy to use and brings its own risks, including an “all-in or all-out” scenario.

Indeed suppose that your long-lived master key is exposed, then all possible security provided by cryptography is immediately lost, which it seems to be what happened in this case.

Hardware Vulnerability of Some FIDO U2F Keys

Posted on by

Hardware Security Keys, like Google Titan Key or Yubico YubiKey, implementing the FIDO U2F protocol, provide what is consider possibly the most secure 2nd Factor Authentication (2FA) measure. Indeed the private key is protected in Hardware and should be impossible to copy, so that only the physical possession of the hardware token provides the authentication.

But a recent research (see here for the research paper, and here and here for some comments) shows that a class of chip (the NXP A700X) is vulnerable (CVE-2021-3011) to a physical hardware attack which allows to extract the private key from the chip itself, and then to clone the security key. To fully succeed, the attack requires to know the credential of the service(s) for which the security key works as 2FA and the physical availability of key itself for at least 10 hours. Then the security key is dismounted and the secret key is obtained by measuring the electromagnetic radiations emitted by the key during ECDSA signatures. Finally a new security key can be cloned with the stolen secret key.

From a theoretical point of view, this vulnerability violates the fundamental requirement of an hardware security key, that is that the private key cannot be extracted from the hardware in any way. But it should also be noted that the FIDO U2F protocol has some countermeasures which can be useful to mitigate this vulnerability, like the presence of a counter of the authentications done between a security key and a server so that a server can check if the security key is sending the correct next sequence number which would be different from the one provided by the cloned security key.

On practical terms, it all depends on the risks associated with the use of the security key and the possibility that someone will borrow your security key for at least 10 hours without anybody noticing it.  If this constitutes a real risk, then check if your security key is impacted by this vulnerability and in case it is, change it. Otherwise if this attack scenario is not a major threat, it should be reasonably safe to continue to use even vulnerable security keys for a little while, while keeping up to date with possible new developments or information from the security keys manufacturers. Even in this case, vulnerable security keys should anyway be changed as soon as convenient.