AI produces deceptive real research data

An incredible number of age statements end with the number 7 or 8

Various contradictions emerged. First, for many of the alleged participants, the gender mentioned did not match what one would expect based on their first names. Second, there was no correlation between vision test results and pre- and postoperative imaging. Third, Wilkinson and Law examined how numbers were distributed in some columns in the data set to find potential patterns that did not occur by pure chance. The eye imaging results passed this test, but some people’s age scores clustered in a way that was very unusual relative to the real data: a disproportionate number of participants’ ages ended in the numbers 7 or 8.

The study authors acknowledge that some weaknesses in their data set can be revealed upon closer inspection. However, “it’s hard to see that the data wasn’t collected by a human if you just glance at it,” Giannaker says.

Bernd Pulferer, editor-in-chief of EMBO Reports, is also concerned about this. The peer review process, that is, the evaluation of studies by colleagues before publication, “doesn’t actually check all the data again. “Sophisticated integrity violations are unlikely to be detected by AI,” the expert says. He believes that journals need to adapt their quality checks to identify data collected by AI.

In a collaborative project led by Wilkinson, researchers want to design statistical and non-statistical tools to investigate potentially problematic studies. “In the same way that AI can be part of the problem, AI-based solutions can also be found. “We can automate some of these verification processes,” Wilkinson says, warning that advances in generative AI may soon open up ways to bypass these screening protocols. He has the same opinion: “Once you know what is being looked for in these tests, artificial intelligence can easily be used to circumvent these tests.”