Many portions of Magnifica Humanitas appear to be AI-written
Magnifica Humanitas is a recent ‘encyclical’ by Pope Leo XIV, leader of the Catholic Church. It outlines a vision for how humanity should interact with artificial intelligence, emphasizing the importance of human dignity and ensuring that AI does not replace human relationships, among other topics. Interestingly, many portions appear to be written by AI. Why I thought to check this Friends of mine Linch Zhang and the Axolotl noticed that parts of the English text appear to be AI-generated, and twitter user kartr found that the Italian text had the largest fraction of AI-generated content out of all the translations published by the Vatican, speculating that it was the original copy, and translations by humans appear less ‘AI-generated’ to various tools. What I actually did I took the Italian text of Magnifica Humanitas, and ran it thru the Pangram AI detector software. Why the Italian text? As mentioned in the previous segment, it had previously been claimed to be the most AI-ish version. Also, given that the Vatican is in Italy, it’s a reasonable guess that the text was initially drafted in Italian (presumably by a few different people with the Pope’s input). Why Pangram? Pangram is a reasonably accurate AI text detector. In particular, it manages to achieve an extremely low rate of marking human-written text as AI-written, while maintaining a reasonable rate of flagging AI-written text as AI-written. My results The whole text is too long to fit into Pangram, so I ran it section by section. Note that for some paragraphs only part of the paragraph was flagged as AI-written, I’m trying to only flag paragraphs that are mostly flagged. The introduction 43% of the text was flagged as AI-generated. This included paragraphs 7-9 (on the images of the tower of Babel and Nehemiah rebuilding Jerusalem) and 13-16. Chapter 1 62% of the text was flagged as AI-generated. This included paragraphs 21-24, 26-29, 31-32, 34-36 (where large parts of 36 were flagged as being ‘AI assisst