Category Archives: AI/ML

Are AI Coding Assistants Declining in Quality?

Posted on by

“AI Coding Assistants Are Getting Worse” is an intriguing article on IEEE Spectrum. According to the author (Jamie Twiss, CEO of Carrington Labs), the quality of the AI Coding assistants is declining contrary to what it may look at first sight.

Indeed, the author  noticed that some of the most recent AI models produce, more often than previous models, code which runs but which fails to perform as intended, even when given wrong instructions which cannot lead to a running code. In the opinion of the author, this can be due to the quality of the large volumes of training data needed by the latest models, and on the direct interaction of the AI Assistants with the users, which can “push” the AI models to produce code which runs.

Securing the AI software development tools: IDEsaster

Posted on by

I just quote two MaccariTA (Ari Marzouk) statements:

IDEs were not initially built with AI agents in mind. Adding AI components to existing applications create new attack vectors, change the attack surface and reshape the threat model. This leads to new unpredictable risks.

[…]

AI IDEs effectively ignored the base IDE software as part of the threat model, assuming it’s inherently safe because it existed for years. However, once you add AI agents that can act autonomously, the same legacy features can be weaponized into data exfiltration and RCE primitives.

and, as you can read in the original post, it is not only “risks” but also a quite long list of vulnerabilities (with 24 CVE) which affect, one way or or another, almost all AI IDE tools.

The issue is one we saw many times in the past: first features and functionalities, then we fix security. I agree that without features and functionalities any software product does not make any sense, but with security as a post add-on, there is the well known risk to have to pay a large security bill for a long time.

The “Bizarre” Case Involving ChatGPT, a Divorce and Coffee Cups Reading

Posted on by

I am not commenting on the case itself which you can read for example here, but just thinking about how good an AI, such as a chatbot, with a Human Interface is:

  • writes / talks as or better than an average human
  • is very convincing
  • answers questions on everything (knows it all!)
  • provides references (if asked) about anything, even imaginary!
  • is always available and answers within a few seconds
  • is very good at explaining
  • etc.

How can we not trust it?

Artificial Intelligence and “Artificial Science”

Posted on by

“A weird phrase is plaguing scientific papers – and we traced it back to a glitch in AI training data” is an interesting article about what can go wrong in training Machine Learning models. An error in scanning old printed scientific papers, and a similar error in translating from Farsi to English, made it so that the phrase “vegetative electron microscopy”, which is nonsense, became part of training datasets for many current advanced AI models and started appearing in published scientific papers.

The problem is how to get rid of this and similar other errors in AI training data.

Are these errors going to be our future “digital fossils”?

LLMs Still Not so Good at Math Reasoning

Posted on by

A recent study (here the paper, and here some comments) shows that the latest LLMs models, though are getting good in mathematical computations, still lack mathematical reasoning, that is the ability to provide a detailed and exact proof of a mathematical statement with rigorous reasoning (unless they have been already trained with the proof or have access to it). The researchers evaluated some of the top LLMs on the six problems from the 2025 USA Math Olympiad just hours after their release, assuring in this way that the detailed solutions were not known to the LLMs.

Anthropic: “Reasoning models don’t always say what they think”

Posted on by

Interesting article by Anthropic, it seems that there is still a lot to understand before reaching “Explainable AI”. Quoting: “our results point to the fact that advanced reasoning models very often hide their true thought processes, and sometimes do so when their behaviors are explicitly misaligned.“

AI and Professional Work

Posted on by

Apologies if I am late on these considerations but the implementation of the AI Act brings up an interesting aspect.

As ethically required, we will need to declare which parts of professional jobs are performed not by us humans, but by AI assistants/agents. But there are also free, or almost free, AI assistants/agents that can be directly used by the customers.

I am not a market expert, but I can imagine that this will lead to (and it reminds me of years ago when Google Search arrived): 1) professional jobs performed without or with little AI assistants/agents’ contribution, + 2) direct customers use of AI assistants/agents to perform the job; and very little in between.

A non-IT example can be the legal profession: asking an AI chatbot for legal advice is for free (or it can appear to be so), but it is not the same as paying a lawyer for legal support.

Image Recognition and Advanced Driving Assistance

Posted on by

“Securing the Perception of Advanced Driving Assistance Systems Against Digital Epileptic Seizures Resulting from Emergency Vehicle Lighting” is an interesting research study on the current status of image recognition for advanced driving assistance and autonomous vehicle systems. The study found that some standard Driving Assistance Systems can be completely confused by emergency vehicle flashers with the risk of becoming the cause of serious incidents. Machine Learning models can be part of the cause of this vulnerability, as well as part of the solution proposed by the researchers called “Caracetamol“.

Is the Reliability of LLMs Improving with their Dimension?

Posted on by

This article studies the reliability of increasingly larger LLM models (such as GPT, LLaMA, etc.) with respect to their correctness and ability to solve more complex problems. A priori it would seem that more powerful, larger, and “better” trained models would improve and become more reliable. The study instead shows that it doesn’t really seem so: even if the models become better at solving more complex problems as they grow, they also become less reliable, that is they make more mistakes.

A “Morris” Worm for Generative Artificial Intelligence

Posted on by

Almost every day there is a new announcement about Artificial Intelligence and Security, and not all of them look good. The latest (here) describes how it is possible to create a worm that propagates between Generative Artificial Intelligence models. For (understandable) historical reasons, it has been named “Morris II”.

The approach seems simple: abusing the Retrieval-Augmented Generation (RAG) capabilities of these models (that is the capability of retrieving data from external authoritative, pre-determined knowledge sources) it is possible to propagate adversarial self-replicating prompts between different Gen-AI models. In other words, through external shared sources such as email, a Gen-AI model can propagate the worm to another model. Notice that the effect of the input data (prompt) to a Gen-AI model is to replicate that prompt in output so that it can be picked up by another Gen-AI model.

This is only a research study and the authors intend to raise this issue in order to prevent the real appearance of Morris II-type worms.

But all this only means that we have still a lot to learn and a lot to do to be able to create and use Artificial Intelligence securely.