Egregious Essay Expressions Exceed Expectations?

   
Old
vs
New

Egregious Essay Expressions Exceed Expectations?

Testy Test-Takers Too? AI vs AI

For as long as high schools and colleges have required take home tests and essays or even extensive reading, there have been companies focused on creating short cuts, study guides and clandestine “resources.” Young smarty-pants “entrepreneurs” have long since found ways to aggregate imported “essays” to “facilitate” students hell-bent on expediting their assigned homework burden. In my college years, Cliff Notes were a purported way to avoid reading dialect-heavy tomes, long lumbering assigned books or getting summaries of those science and math class complexities made easier to digest. Professors were on to those who tried. Tests focused on identifying writing styles without proper noun triggers, for example. Ugh.

But that seems to be such a primitive era today as content aggregators and sophisticated code writers have the self-teaching ability with increasing layers of complexity inherent in the growing world of artificial intelligence. Indeed, these self-honing programs and websites have become “resources” for homework-avoiders and terrified-test-takers. And while AI capabilities are not “all there” yet, they are self-learning very fast.

Writing for the January 26th CNN Business, Samantha Murphy Kelly writes: “ChatGPT is smart enough to pass prestigious graduate-level exams – though not with particularly high marks… The powerful new AI chatbot tool recently passed law exams in four courses at the University of Minnesota and another exam at University of Pennsylvania’s Wharton School of Business, according to professors at the schools.

“To test how well ChatGPT could generate answers on exams for the four courses, professors at the University of Minnesota Law School recently graded the tests blindly. After completing 95 multiple choice questions and 12 essay questions, the bot performed on average at the level of a C+ student, achieving a low but passing grade in all four courses…

“ChatGPT fared better during a business management course exam at Wharton, where it earned a B to B- grade. In a paper detailing the performance, Christian Terwiesch, a Wharton business professor, said ChatGPT did “an amazing job” at answering basic operations management and process-analysis questions but struggled with more advanced prompts and made “surprising mistakes” with basic math… ‘These mistakes can be massive in magnitude,’ he wrote.

“The test results come as a growing number of schools and teachers express concerns about the immediate impact of ChatGPT on students and their ability to cheat on assignments. Some educators are now moving with remarkable speed to rethink their assignments in response to ChatGPT, even as it remains unclear how widespread use is of the tool among students and how harmful it could really be to learning.

“Since it was made available in late November, ChatGPT has been used to generate original essays, stories and song lyrics in response to user prompts. It has drafted research paper abstracts that fooled some scientists. Some CEOs have even used it to write emails or do accounting work.” You can be sure that the bots will find a way to correct these deficiencies. But if AI can be used as a homework/test substitute, can it also be used to detect AI-created essays and test results? The short answer is “yes,” but the AI vs AI battle is just beginning.

According to Steven Melendiz, writing for the January 25th FastCompany.com: “As concerns rise about students’ use of generative artificial intelligence like ChatGPT to complete schoolwork, a pair of education nonprofits have created a free system to help teachers detect AI-assisted essays.

“The tool, called AI Writing Check, was developed by the writing nonprofits Quill and CommonLit using an open-source AI model designed to detect the output of ChatGPT and related systems. It enables teachers (or anyone else) to copy and paste text and within a few seconds receive a determination on whether the work in question was written by ChatGPT.

“AI Writing Check, which the nonprofits began to develop in December, comes as surveys indicate growing concern among teachers over machine-generated essays. Other tools, including one called GPTZero, have also been released recently to detect automated writing… ‘We need to give teachers the tools so they can preserve academic integrity in lessons, because you’re only a seventh grader once,’ says CommonLit CEO Michelle Brown. ‘It could really, really harm students if they go through even one school year not getting a chance to do the heavy lifting we did in seventh grade.’

“Concerns about AI-generated academic work include worries about honesty and integrity, with a recent Stanford Daily article explaining the unauthorized use of the tool to do student work is likely a violation of the university’s honor code. Some K-12 school districts, including New York City’s, the largest U.S. public school system, have taken steps to block ChatGPT on school devices and networks, though there’s little they can do to prevent students from accessing the tool and others like it elsewhere. ChatGPT, a free AI tool built by OpenAI, can accurately answer a wide variety of questions in a matter of seconds—including the types of prompts given to students for papers and take-home exams.

“AI Writing Check isn’t perfect: The nonprofits estimate that it’s between 80% and 90% accurate, based on a test set of about 15,000 essays, and they emphasize that teachers should take additional steps like comparing suspect work to previous student assignments before assuming a given essay is plagiarized.

“It’s also not likely the last salvo in what’s shaping up to be an arms race of sorts between those using who are using AI to generate text and those, like teachers, employers, and editors, who are looking to spot unauthorized use of such tools. Quill founder and executive director Peter Gault suggests future detection tools might look into the revision logs generated by modern word processing software like Google Docs or Microsoft Word, revealing how essays were crafted rather than just the end product. That’s something recent reports say some educators are already starting to do manually, in some cases even requiring more handwritten work.” Ready player one? Your teacher/professor might be too.

I’m Peter Dekom, and while accessing targeted content via AI is in itself a valuable skill, not really learning but rely on technology denigrates both the student and the resulting “education,” … buy it sure smacks of the same conspiracy-theory-enabling bot-explosion that is attacking our very democracy.