• ChatGPT: opportunities and threats
• Assessment not banning ChatGPT but avoiding mindless copy pasting: challenge accepted
• Going into the details of the setup
  • 1. One in-class assignment per teaching session
  • 2. Assignment directly addressing the content just seen in class
  • 3. Powerpoint to the rescue (who would have guessed…)
  • 4. Grading 2 out of 4 assignments
  • 5. ChatGPT and access to Internet not forbidden during the assessment
  • 6. A last insurance: a codicil in the evaluation grid
• Example: The file of an in-class assignment
• About me

A new academic year is starting, the first one since ChatGPT has been released. I share the file of an assignment I will use this term and that should meet a number of constraints that ChatGPT has brought to the teaching scene.

ChatGPT: opportunities and threats

I personally consider that ChatGPT (short for generative AI in general) is indeed a fundamental, important innovation which will bring radical changes in the short term (see my blog post from Dec 2022, also in French here).

ChatGPT makes it easier to find answers on the web, draft content, explore a problem. These are fascinating opportunities.

source: https://www.nea.org/nea-today/all-news-articles/-enters-classroom-teachers-weigh-pros-and-cons

But at the same time, it can become a substitute for learning, which I would list as a threat. I have witnessed it first hand when students submitted ChatGPT-generated answers to my assignments in Spring classes. Many (luckily, not all) answers demonstrated that skills of copy-pasting were indeed well acquired, but I doubted that the assessment proved anything in terms of the acquisition of the skills listed on the syllabus. With this experience, I hope to do better this term.

Assessment not banning ChatGPT but avoiding mindless copy pasting: challenge accepted

The goal is to find ways to provide the best of both worlds of ChatGPT to my students:

• allowing for the use of ChatGPT to leverage its potentialities
• all while avoiding the situation where disengaged students would do some mechanical copy pasting to pass the exam with absolutely zero knowledge on the subject.

First, let me say that I considered many options, and that oral examinations first had my strong favor. But at group sizes around 40 students, and literally a dozen of groups to teach, I would have spent days and weeks sitting doing these oral examinations (or grading them in asynch mode). I’ll definitely keep oral examinations for resits though.

So, the setup:

1. one in-class assessment per teaching session. One assignment lasts 30 minutes, at the end of a 3 hours session.
2. the in-class assignments are simplistic scenarios that leverage and put to work the concepts seen just before in class
3. the assignment is formatted as a Powerpoint and avoids the format of a Q&A.
4. I grade only 2 of the 4 in-class assignments - but students don’t know in advance which 2 of the 4 will be graded
5. ChatGPT and access to Internet are not forbidden during the assignment.
6. added a codicil to the evaluation grid: generic answers can be discounted at the liberty of the grader.

Going into the details of the setup

I’ll go through each of the 6 points above as they all matter very much. Skip and download the file of an example of an assessment at the bottom if you prefer!

1. One in-class assignment per teaching session

I owe this idea to my colleague Lisa Buchter, which she uses in her class. In 3 hours teaching sessions, the last 30 minutes are usually a struggle for everyone. Turning these last minutes into an assignment brings back concentration and focus in an astonishing way. Students basically don’t want to leave the classroom when the bell rings! Too busy finishing their assignments :-) And the fact that each teaching session has one assignment allows for shorter assignments, taylored exactly to the class.

In the context of a ChatGPT world, staying in-class also helps the professor monitor how students use it. Just walking through the aisles, you see which student is basically working with ChatGPT open in front, and those who try to answer the assignment directly. It gives some empirical feedback on the practices.

2. Assignment directly addressing the content just seen in class

Having an assignment dealing directly with what was discussed in the 2 hours right before, tends to give more confidence to the students in their capacity to answer correctly to the questions: it is still fresh in memory. This gives a better chance for the students to try and reason by themselves (they have it in mind!) instead of offloading the task to ChatGPT in a mindless way.

3. Powerpoint to the rescue (who would have guessed…)

The questions are scenarios which give some contextual elements, scattered on different pages, which obliges students to provide contextualized answers, not direct follow-ups to a generic prompt. In the same spirit of discouraging a mechanical process of prompt > answer, the document format is not an MS Word but a Powerpoint with tables and unusual diagrammatic designs, because this spatial configuration does not lend itself to a direct solving through ChatGPT. The goal is not to prevent the use of ChatGPT, but to encourage students to have a strategic use of it - and opening up the possiblity that a non ChatGPT aided answer might be simpler and make more sense.

4. Grading 2 out of 4 assignments

This one is very operational: sophisticated ways to improve the type of evaluations in the age of ChatGPT often falter because of the load of grading they induce. I almost forgot about that and was preparing a hell of a term for myself in terms of grading, just imagine: grading 40 documents after each class. Luckily, an adjunct professor working with me pointed that it was just out of scale, and that prompted a rethink.

The solution is pretty elegant I find: the students will be graded on 2 assignments out of the 4 they will produce and they will not know in advance which ones are going to be graded. The obvious effect we want is that they maintain their level of effort, even when the assignment is not graded. This is a new process so we’ll see how it goes but I am pretty happy and confident with it.

5. ChatGPT and access to Internet not forbidden during the assessment

Given everything that precedes, it seems that the conditions are met for a reasonable, productive use of ChatGPT and any other resource the student might want to search for. The scenario based assessment prevents that a “ready made, mindless” answer can be searched, found or generated, copied and pasted in the assignment. The unusual, slightly cumbersome ppt format discourages the linear Q&A flow that would immediately call for ChatGPT. Some participants will surely try to reconstruct a prompt from the instructions to feed ChatGPT with, but in this case I believe that it falls in the category of “creative, mindful use of ChatGPT”, where a cognitive effort and learning experience has effectively taken place. Experience will tell.

6. A last insurance: a codicil in the evaluation grid

All the above makes me hopeful that the assessments will lead to thoughtful answers by the students, aided by ChatGPT or not. But in case I was very wrong and I end up with most exam papers filled with answers that I immediately recognize as pale, unoriginal content produced at the chain with ChatGPT, failing to demonstrate any reasoning or knowledge acquisition? (which is, to recall, the nightmare scenario that prompted this whole assessment redesign!). Well, I have inserted this paragraph in the evaluation grid:

Responses which feel generic and impersonal will be downgraded at the liberty of the grader, because the expectation is that participants should be able to provide a personalized, thought-through response to the case.

This will be discussed in frankness with the students at the opening of the course. When content creation becomes so cheap and generic, there is an urgency or at least a benefit to develop one’s singular personality and reasoning capabilities. This is true from a pragmatic perspective (to find a job one would better be distinguishable from the dozens of other applicants), and from a humanistic perspective (is a life truly lived if we are replicants of each others?).

Example: The file of an in-class assignment

… for the class on “definitions of data and how does it create business value”.

Available for download here: class activity 1 - identify data sources to support business decisions.pdf

For context:

• the course is on data literacy and is meant to introduce engineering students to business studies, and business students to data and digital technologies.
• students have 30 minutes to do the assignment, which comes at the end of their first lecture of the course.

About me

I am a professor at emlyon business school where I conduct research in Natural Language Processing and network analysis applied to social sciences and the humanities. I teach about the impact of digital technologies on business and society. I build nocode functions 🔎, a click and point web app to explore texts and networks. It is fully open source. Try it and give some feedback, I would appreciate it!

• my email: analysis@exploreyourdata.com 📧
• or on Twitter: @seinecle 📱
• you can also read the other articles of this blog 👓, where I write about the process of developing the app.