Reporters may use AI tools vetted and approved for our workflow to assist with research, including navigating large volumes of material, summarizing background documents, and searching datasets. Even then, AI output is never treated as an authoritative source. Everything must be verified.
When we attribute a statement, a position, or a quote to a named source, that material comes from direct engagement with interviews, transcripts, published statements, or documents reviewed by the reporter. AI tools must not be used to generate, extract, or summarize material that is then attributed to a named source, whether as a direct quote, a paraphrase, or a characterization of someone’s views.
How do you control „Everything must be verified” specifically? I’m pretty sure similar rules must have been in place when they published a hallucinated interview.
tl;dr - don’t quote or link to Arse Technica because it could be made up entirely
So basically their journalists are cleared for using a tool that gives off radioactive levels of mutated LinkedIn-speak (not X but Y!), and in the best case scenario they think the proximity to these tools won’t leech into the style of their own writing?
And that’s at the very best of circumstances. Just a little toxicity in the article contents.
They lost an editor who used fake AI quotes before this new explicit allowance.
As far as policy guidelines go, I think they’re quite reasonable. I read the entire policy and the staff comments on the page.
They lay out expectations: Humans must write the articles and verify everything that an AI output claims comes from a source such as through transcribing interviews or summarizing documents. They lay out that not following the policy is a violation which leads to the authors or editors being on the hook for those failures, likely tied to Ars or Condé Nast’s disciplinary procedures. These would be the reactive controls through accountability for failures, which there may yet be.
Per the document and the subsequent comments, they confirm that editors and reporters both have to verify that reporting is accurate. That is a reasonable amount of proactive controls. If there becomes a pattern of failures either in the amount of them or a lack of accountability, then it would be fair to assume their output is AI slop, but I think that’s currently too early to claim.
You’re under no obligation to assume they’ll be successful or that they are sincere, but it’s a clearly written reader-facing policy.
They’re reasonable in terms of covering their butts. They do not ensure in any way, shape or form that they are actually followed. If they were followed (all source material read and understood by a human) then an LLM wouldn’t add anything of value because a human already summarised it. This is a lot of text about their dreams and aspirations, not an actual policy.
They do not ensure in any way, shape or form that they are actually followed.
There is no mechanism in all of human nature that automatically and definitively binds us to our word. Whether we’re writing code, or laws, or wedding vows, it is a choice at all times to adhere to them - and they are prone to error.
All Ars can do is write the rules and choose to enforce them. Whether they choose to or not is a trust exercise, which it’s only built on a long history of consistent good choices.
I meant that the only mechanism of verification of LLM output makes LLM pointless, hence if they’re using LLMs they won’t be really verifying it. If your use case has a risk appetite for being wrong 10% of time then go ahead. I’d rather get my news from a source that aims for 0% even if they make an occasional miss.
How do you control „Everything must be verified” specifically? I’m pretty sure similar rules must have been in place when they published a hallucinated interview.
tl;dr - don’t quote or link to Arse Technica because it could be made up entirely
So basically their journalists are cleared for using a tool that gives off radioactive levels of mutated LinkedIn-speak (not X but Y!), and in the best case scenario they think the proximity to these tools won’t leech into the style of their own writing?
And that’s at the very best of circumstances. Just a little toxicity in the article contents.
They lost an editor who used fake AI quotes before this new explicit allowance.
As far as policy guidelines go, I think they’re quite reasonable. I read the entire policy and the staff comments on the page. They lay out expectations: Humans must write the articles and verify everything that an AI output claims comes from a source such as through transcribing interviews or summarizing documents. They lay out that not following the policy is a violation which leads to the authors or editors being on the hook for those failures, likely tied to Ars or Condé Nast’s disciplinary procedures. These would be the reactive controls through accountability for failures, which there may yet be.
Per the document and the subsequent comments, they confirm that editors and reporters both have to verify that reporting is accurate. That is a reasonable amount of proactive controls. If there becomes a pattern of failures either in the amount of them or a lack of accountability, then it would be fair to assume their output is AI slop, but I think that’s currently too early to claim. You’re under no obligation to assume they’ll be successful or that they are sincere, but it’s a clearly written reader-facing policy.
They’re reasonable in terms of covering their butts. They do not ensure in any way, shape or form that they are actually followed. If they were followed (all source material read and understood by a human) then an LLM wouldn’t add anything of value because a human already summarised it. This is a lot of text about their dreams and aspirations, not an actual policy.
There is no mechanism in all of human nature that automatically and definitively binds us to our word. Whether we’re writing code, or laws, or wedding vows, it is a choice at all times to adhere to them - and they are prone to error.
All Ars can do is write the rules and choose to enforce them. Whether they choose to or not is a trust exercise, which it’s only built on a long history of consistent good choices.
I meant that the only mechanism of verification of LLM output makes LLM pointless, hence if they’re using LLMs they won’t be really verifying it. If your use case has a risk appetite for being wrong 10% of time then go ahead. I’d rather get my news from a source that aims for 0% even if they make an occasional miss.