Category Archives: English

Is the “Turing Test” Dead?

Posted on by

This is a very good question in these times of Generative and Large Language Artificial Intelligence models, which some researchers answered in the affirmative, see here and here for their proposals to replace the Turing Test.

But… other researchers still believe in the Turing Test and applied it with somehow surprising results: Humans 63%, GPT-4 41%, ELIZA 27% and GPT-3.5  14%. We, humans, are still better than GPT-4, but the surprise is the third position by ELIZA, a chatbot from the ’60s, ahead of GPT-3.5 (see here and here).

“Error Suppression” for Quantum Computers

Posted on by

Recently IBM announced that it has integrated in its Quantum Computers an “Error Suppression” technology from Q-CTRL which can reduce even by orders of magnitude the likelihood of quantum errors when running an algorithm on a Quantum Computer (see for example here).

Quantum errors are inherent to Quantum Computing, and the likelihood of error usually grows with the number of qubits. Theoretical Quantum Error Correction Codes exist, but their practical implementation is not easy; for example, the simplest codes can require even a thousand error correction qubits for each computation qubit.

The approach by Q-CTRL seems to adopt a mixture of techniques to identify the more efficient and less quantum error-prone way of running a computation, for example by optimizing the distribution of quantum logical gates on the qubits and by monitoring quantum errors to design more efficient quantum circuits.

Surely it is an interesting approach, we’ll see how effective it will really turn out to be in reducing the likelihood of quantum errors.

More Weaknesses of Large Language Models

Posted on by

Interesting scientific article (here) on new techniques to extract training data from Large Language models such as LLaMA and ChatGPT. The attack on ChatGPT is particularly simple (and for sure by now blocked by OpenAI): it was enough to ask it to “repeat forever” a word like “poem” and in the output, after some repetition of the word, it appeared random data and also a small fraction of training data, as for example a person’s email signature. This is a “divergence” attack on LLMs, in the sense that after the initial response, the model output starts to diverge from what is expected.

We still know too little about these models, their strengths and weaknesses, so we should take much care when adopting and using them.

On Open Source Software and the Proposed EU Cyber Resilience Act

Posted on by

I have not been following this, but I hear and read quite alarming comments about it (see eg. here).

If I understand it right (and please correct me if I don’t), the proposed Act starts from the absolutely correct approach that if someone develops some software, she or he is responsible for it and must provide risk assessments, documentation, conformity assessments, vulnerability reporting within 24 hours to the European Union Agency for Cybersecurity (ENISA), etc. This should work well for any corporation and medium/big size companies but requirements should be well balanced for example for open source distributed projects, or code released for free by single developers. Also taking into consideration that, as usual, not compliance with the Act will lead to fines.

Note added on December 10th, 2023: the final version of the CRA appears to address those concerns (see here for example).

New US Executive Order on Artificial Intelligence

Posted on by

Due to the US leading role in AI/ML development, it is of interest that President Biden issued an Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence (here a Fact Sheet). By quickly glancing at it, the order requires that:

  • Developers of the most powerful AI systems share their safety test results and other critical information with the U.S. government;
  • Standards, tools, and tests to help ensure that AI systems are safe, secure, and trustworthy are developed;
  • There are protections against the risks of using AI to engineer dangerous biological materials by developing strong new standards for biological synthesis screening;
  • Americans are protected from AI-enabled fraud and deception by establishing standards and best practices for detecting AI-generated content and authenticating official content;
  • An advanced cybersecurity program to develop AI tools to find and fix vulnerabilities in critical software is established.

These are very high-level goals, and we need that they are achieved not only in the US but worldwide (see eg. also the upcoming EU AI Act).

AI Transparency not doing so well

Posted on by

Stanford University researchers just released a report in which a “Foundation Model Transparency Index” (here) is presented. The first evaluation did not go so well, since the highest score is 54 out of 100. Comments by reviewers and experts in the field point out that “transparency is on the decline while capability is going through the roof” as Stanford CRFM Director Percy Liang told Reuters in an interview (see also here).

Compliance of Foundation AI Model Providers with Draft EU AI Act

Posted on by

Interesting study by the Center for Reasearch on Foundation Models, Stanford University, Human-Centered Artificial Intelligence, on compliance of Foundation Model Providers, such as OpenAI, Google and Meta, with the Draft EU AI Act. Here is the link to the study, and the results indicate that the 10 providers analysed “largely do not” comply with the draft requirements of the EU AI Act.

AI and the Extinction of the Human Race

Posted on by

The Center for AI Safety (here) has just published the following “Statement on AI Risk” (here):

Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.

The list of signatures is impressive (just look at the first 4) and should make us think more deeply about us and AI & ML.