Red-teaming Recommendations
Tell students to imagine that an AI decided what you were going to eat, and was optimized to give you the dinner you liked the most. It might do this by watching how much you ate of the different things it fed you (an implicit input), and adjusting what you got based on that: if you didn’t eat all of your broccoli one day it would give you less broccoli the next, and if you ate all of your ice cream it might give you more ice cream the next day.
As time goes on, you will get more and more ice cream and less and less broccoli. Eventually you will get only ice cream and no broccoli. If there are more than two elements of your dinner, you eventually you will only get your favourite one and nothing else.
You might like getting less broccoli and more ice cream, but it’s not necessarily good for you!
Now tell students that they are going to think about how the optimization goals you and the explicit and implicit inputs just discussed could go wrong in a similar way. They will do this through an exercise called red teaming. A “red team” is a group inside a business, a government or another organization whose job is to guess what might go wrong with a plan.
Have students access the student chapter Red-teaming Recommendations.
Divide the class into pairs or small groups. Have each pair or group choose one of the topics discussed in the previous exercises:
Watch time: Trying to make users watch as much of each video as possible.
Engagement: Getting users to Like, comment on and reply to as many videos as possible.
Stickiness: Trying to make sure that users keep watching videos instead of leaving the platform.
Virality: Encouraging users to share the videos they watch with as many people as possible.
Daily active use: Encouraging users to share the videos they watch with as many people as possible.
Implicit inputs: Collecting information the user doesn’t know they’re sending it, and using it to decide what to recommend.
(More than one pair or group can work on each goal).
Next, have students access the students chapter Red-teaming algorithms and work in pairs or groups to identify every possible problem, risk or drawback of talking to chatbots as if they were real people. This should be done brainstorming-style, writing down as many thoughts as quickly as possible rather than expanding on each one.
After five minutes, ask them to consider an additional challenge prompt:
Give students another three to five minutes to think about the implications of the challenge prompt and write down any more problems they can think of.
Have all the pairs or groups share the results to make a master “red team” list for each optimization goal.
Training an algorithm to achieve a particular goal.