Human or machine? Distinguishing between real and AI-generated content
More and more frequently, we hear about AI-generated phishing attacks that are indistinguishable from real emails and deepfake content that looks like written by professional journalists. In recent years, AI-based systems are continuously improving so that they can now independently generate content such as text, graphics, images and videos that ever more closely resemble human-generated content. As technology advances, this content is becoming more sophisticated, making it not only more difficult to verify its authenticity but also to be recognized by security mechanisms. Below are some features that can be used to recognize AI-generated text that can be contained in documents or emails, for example.
Analysis of the writing style
To begin with, a check can be made by analyzing the writing style. AI-generated texts tend to have a monotonous and formulaic writing style. This includes recurring patterns, excessive neutrality or a lack of personal nuances. In addition, AI-generated ones tend to be free of spelling and punctuation errors. Especially in texts that are written quickly, as is usually the case with emails, the author often unintentionally makes spelling and punctuation errors. The texts may also contain unusual formulations or abrupt changes of subject. Such inconsistencies in context should not be present if the texts were written by a human or at least proofread by a human before publication. Longer document content in particular, which is supposed to be perceived as authentic and trustworthy, should contain comprehensible and trustworthy references. AI-generated texts usually do not contain any indication of sources.
In cases where the content also contains images, these are often lacking in realism in the details or have inconsistencies in light and shadow. Especially when people are depicted in the image, the color tones usually appear unnatural.
In addition to these features, which readers of a document or email can recognize easily without extensive knowledge, there are also methods for systematically determining texts or text content designed by an AI.
Methods for identifying AI-generated texts
- Entropy
Entropy is a concept from information theory. It measures the degree of disorder or uncertainty in the distribution of elements. In texts, for example, these are the letters, words or sentences. Entropy can provide an indication of how predictable or surprising the information used is. Entropy is also suitable for determining the authorship of a text, such as an AI. If a text contains extensive vocabulary, complex sentence structures, or original ideas, that text will have high entropy. This rather suggested that the text was not written by an AI. On the other hand, if the text contains frequent repetitions or general phrases, especially those that are unusual in the context, the text has a lower entropy, which is more likely to indicate that it was written by an AI.
Furthermore, the meaning of individual words or sentences can be examined in order to understand the overall content of the text. If there are many words among them that are less relevant for recognizing the overall message, the entropy is lower, which may again be an indication of writing using an AI. To determine the entropy, the probability of each character in the text is determined first. To do this, the frequency of occurrence of the character in the text is calculated. This frequency is divided by the total number of all characters in the text. This probability is then multiplied by the logarithm of the calculated probability. The logarithm to the base 2 is often used here. This then gives the entropy in units of bits. To determine the total entropy of the text, the sum of the entropy is divided by the number of characters. The result is the average entropy per character in the text. See also “Texte aus der Sicht der Informationstheorie” from Akademie Aktuell, 2007, issue 2, edition 21.
- Analysis of metadata
Metadata is structured data that contains information about other data ( incl. resources such as images, videos, documents, books, etc.). In the case of an email, this includes the sender address, the recipient address(es), the subject, send date and time, the message ID, a reply-to address, the presence of file attachments, user-defined email server-specific headers, all routing information, etc.
To generate a document with a longer text, for example, a human usually needs more time than an AI. If the creation time is included in the document’s metadata, a very short creation time may indicate an AI-generated document. Another metadata is the editing history. Especially with longer documents, people write more frequently and save them multiple times. If a multi-page document has only been edited once, this is more likely to indicate an AI-generated text, or the creator has copied the text completely from another source.
If the email contains photos or graphics, the metadata of the graphics can be analyzed. Photos in particular can contain a variety of hidden metadata based on the Exif format or IPTC standard. In this scope fall the camera used, date and time of recording, focal length, exposure program, image description or creator. If his metadata is not present, it does not automatically mean that the photos or graphics were generated by an AI. But it may hint the involvement of AI program in this metadata. If documents, photos or graphics with these metadata characteristics are attached to the email, this can also mean that the entire email was generated by an AI.
- Machine Learning
Machine Learning (ML) is a branch of artificial intelligence (see picture Definition of terms). The computer tries to use algorithms to learn from experiences and expand these experiences with new data without having to be explicitly programmed. Such algorithms can also be used to examine whether a text was generated by an AI. In this case, an AI tries to detect whether a text was generated by the same AI that created the text or by a different one.
Well-known algorithms are decision trees, neural networks or support vector machines. Neural networks can discover non-linear, hidden connections and patterns in texts and also learn in the process. These neural networks can understand not only individual words, but also complete text contexts and situations. To do this, it is necessary to collect large amounts of data in advance, from which the algorithms can then derive patterns (see picture Machine Learning). A typical example is the automatic translation of texts into other languages using such systems. The systems can also be used to find spam emails. In this case, the content of a meaningful email is compared with typical words from spam emails. This teaches the system to distinguish between a spam email and a meaningful email. See “Wie man KI-generierte Texte erkennen kann”.
The first systems that employ text analysis to ascertain whether texts were created by AI are currently available on the market. Some of these systems will be presented in a follow-up article.
Author: Dr. Rolf Kremer
