Yoopya with Associated Press
Ask the new artificial intelligence tool ChatGPT to write an essay about the cause of the American Civil War and you can watch it churn out a persuasive term paper in a matter of seconds.
That’s one reason why New York City school officials this week started blocking the impressive but controversial writing tool that can generate paragraphs of human-like text.
The decision by the largest U.S. school district to restrict the ChatGPT website on school devices and networks could have ripple effects on other schools, and teachers scrambling to figure out how to prevent cheating. The creators of ChatGPT say they’re also looking for ways to detect misuse.
The free tool has been around for just five weeks but is already raising tough questions about the future of AI in education, the tech industry and a host of professions.
WHAT IS CHATGPT?
ChatGPT launched on Nov. 30 but is part of a broader set of technologies developed by the San Francisco-based startup OpenAI, which has a close relationship with Microsoft.
It’s part of a new generation of AI systems that can converse, generate readable text on demand and even produce novel images and video based on what they’ve learned from a vast database of digital books, online writings and other media.
But unlike previous iterations of so-called large language models, such as OpenAI’s GPT-3, launched in 2020, the ChatGPT tool is available for free to anyone with an internet connection and designed to be more user-friendly. It works like a written dialogue between the AI system and the person asking it questions.
Millions of people have played with it over the past month, using it to write silly poems or songs, to try to trick it into making mistakes, or for more practical purposes such as helping compose an email. All of those queries are also helping it get smarter.
WHAT ARE THE PITFALLS?
As with similar systems, ChatGPT can generate convincing prose, but that doesn’t mean what it says is factual or logical.
Its launch came with little guidance for how to use it, other than a promise that ChatGPT will admit when it’s wrong, challenge incorrect premises and reject requests meant to generate offensive answers. Since then, however, its popularity has led its creators to try to lower some people’s expectations.
ChatGPT is incredibly limited, but good enough at some things to create a misleading impression of greatness, OpenAI CEO Sam Altman said on Twitter in December.
Altman added that it’s a mistake to be relying on it for anything important right now.
It’s a preview of progress; we have lots of work to do on robustness and truthfulness.
CAN IT BE USED FOR WRITING SCHOOL PAPERS?
This is what ChatGPT said when The Associated Press asked it to answer that question in all caps from the perspective of a principal shouting a brief message through a school’s PA system:
DO NOT USE CHATGPT OR ANY OTHER AUTOMATED WRITING TOOL FOR SCHOOL PAPERS. THIS IS CHEATING AND WILL NOT BE TOLERATED. IF YOU ARE CAUGHT USING CHATGPT OR ANY OTHER AUTOMATED WRITING TOOL FOR SCHOOL PAPERS, THERE WILL BE SERIOUS CONSEQUENCES.
ChatGPT
But when asked to answer the same question on its own, ChatGPT offered this more measured warning: As a general rule, it is not appropriate to use ChatGPT or any other automated writing tool for school papers, as it is considered cheating and does not benefit the student in the long run.