Monday, October 31, 2016

10 reasons why 'Leaderboards suck in learning

Gamification has been around for years (over 30) but I fear it has recently descended into a superficial, Pavlovian box of tricks, overhyped by vendors keen to prove how ‘cool’ and ‘contemporary’ they are . In practice, much of it is a bit hokey, and insensitive to learning theory and, I fear, the whole point of the bigger game – learning.
Don’t get me wrong, I like gamification, in the sense of games’ techniques grounded in good learning theory. These include: levels of attainment, catastrophic failure and repeats, effortful learning, mini-sims and deliberate practice. What I witness, however, is the superficial buzz of the arcade; leaderboards, points, board games, sound effects, noise and animated piffle.
Gamification only works when the cognitive load, that gamification always entails, results in greater learning outcomes. It you go for approaches that are superficially buzzy, yet work against learning, in that they draw the learner away from actual learning, realism, context and transfer, you will have failed.  You may even have demotivated, rather than motivated most of your learners.
This is especially true of the one gamification technique I do not like, and do not recommend - the leaderboard. The word betrays itself, embedding as it does the the word ‘Leader’, but there are many other problems:
1. Decoupled from standards
Leaderboards are inappropriate for personal development. Learning is not a race, not the survival of the fittest. Any league table that ranks people into a linear list decouples the learning from real measures of competence. You can list all you want but that list is not anchored in competences just because it is ranked. Let’s say you have 100 learners in five leagues of 20. The entire 100 may be failing but you can still sort them into a table.
2. Leaders problem
One of the problems with leaderboards is something I experienced on a project for a major bank some years ago, where we set up product quizzes on kiosks (like fruit machines) around the country. The leaderboard was quickly dominated by a few fanatics, one site in particular, who would do anything, including cheat, to stay at the top. Leaderboard encourage people to cheat. These ‘leaders’ became a problem. We had to take it down. This is something I’ve seen elsewhere.
3. Zero sum game
In general, social challenges, where individual learners are exposed to group comparisons and scores, can demotivate those who lag behind. The downside is obvious, seeing people move ahead of you makes you think you’re not good enough. For every person that moves up a place there’s someone who moves down so the motivational spur may, in total, be zero. For every leader there’s a laggard. In fact, it may be worse than this, as even those above the median point may feel disappointed. In fact the leaderboard technique tends to ignore the majority in favour of the few at the top.
4. Self determination theory
Far better to let the learner use themselves as the benchmark, set self-targets and self-goals, then encourage progress in that way. This avoids the danger of disaffecting entire groups who do not see their learning in terms of games and learning. Gamification is often best when it is almost invisible and embedded in the learning strategy. This ‘self determination’ theory is well researched and relies more on encouraging intrinsic, as opposed to extrinsic motivation.
5. Short-term gain but no long-term gain
Over time the above effects become exacerbated. You get a short-term rush and gain, followed by a longer tail of disappointment, then disillusionment.
6. Good and bad failure
Visibility of ranked failure may also decrease that learner’s willingness to learn through good failure. It’s like cholesterol, there’s good failure and bad failure. Bad failure is illusory ranking, good failure is that which you learn from. Ranked lists make learners more risk averse.
7. Tests not behaviours
It is rare that assessment within a learning programme or experience matches the actual required behaviours in the real world. The tests are often simplistic, too text based often simple multiple-choice. You can do well on leaderboards by focusing relentlessly on the types of answers required in text-based tests. See my list of ways to cheat on MCQs.
8. Lower retention
When the score or position in a league table become the goal, there is less chance that the content will actually be remembered and recalled. Deeper processing and reflection is less unlikely when your goal is a score and not competence. It’s a case of in one ear, get the score, then out the other.
9. Conflict
Leaderboards may induce conflict rather than co-operation. When winning or moving ahead of your colleagues becomes the goal, collaborative or team effort becomes secondary. So this individualistic approach may destroy any semblance of collaborative or team behaviour that one would normally want to encourage in workplace learning.
10. Transfer
Transfer of skills to the real word is important and leaderboards set up an alternative, distinct and distractive world for the learner – the leaderboard. This diminishes the congruence between what is learned and the real target in the real world, that is the point of transfer.
Conclusion

All in all, leaderboards may seem like a good idea but for one upside, competitive motivation, but they have many identifiable downsides, statistically, motivationally and in terms of learning theory. The workplace is not a sporting venue, not a race for the top. Some areas may be driven by such competitive urges, arguably sales, but one should not universalise this phenomenon into learning. The learning game is not a penny arcade game, it’s about cognitive change – that’s hard and should not be trivialised. I must reiterate that I am NOT against gamification, only its superficial promises. For more see my Don't play Pavlov with gamification - 10 DOs and DON'Ts.

No comments: