Sketchnote: Rethinking User Study Design for Evaluating Model Explanations

•

Attributions are often claimed to possibly improve human "understanding" of the models, although little work explicitly evaluates progress towards this aspiration. But are they successful in this progressing towards this aspiration? (1/10)

Today's paper is : Rethinking User Study Design for Evaluating Model Explanations; @Sid_Arora_18,@danish037, @NormSadeh,@professorwcohen,@gneubig & @zacharylipton. Mixing explanation, evaluation & HCI is my area of research, & papers in that intersection are 💙. (2/10)

This research sketchnote is a special version, an @AmongUsGame edition. All papers in this area tie in beautifully with the storyline of Among Us: tasks, deceptions, sabotages (adversaries), crew, imposters, and a futuristic setting (akin to general AI). (3/10)

Researchers have proposed hundreds of techniques for attributing predictions to features that are deemed important to "explain" predictions of machine learning models. But, the main question remains, are these techniques helpful for simulation or manipulation? (4/10)

In this paper, the authors look at 3 different models, BERT ( Integrated Gradients & LIME for explanation), Bag Of Words (Linear Feature Coff. as explanation) and a linear student model (global cues & feature coff. as explanation), trained for Deception Detection. (5/10)

Participants are asked to guess the prediction of the model, in presence (train) or absence (test) of the explanations, and then, manipulate the sample to flip the prediction. Simulation Acc, % Flips and Confidence drops are used as evaluation metrics. (6/10)

In conclusion, the study implies that simulation accuracy in absence of explanation is no better than random guessing, and linear feature coefficients including global weights, along with token highlights help effective manipulation. (7/10)

If you want to take a look at the latest hot-take in the area, go to the thread here: (9/10) twitter.com/zacharylipton/status/1552771951923326976

If you want me to sketchnote a summary for your paper, please reply here or DM me your paper details. If you have any suggestions or comments, or ideas for either research paper choices or design choices, I would be delighted to hear them. (10/10)