Over the past 15 years of designing and implementing 360 Degree Feedback, we’ve seen a lot of different approaches to this basically simple process.
One of the key items to include in a Likert Scale for 360 feedback is a rating point for raters to choose if they are unable to rate their colleague, e.g. ‘Unable to Rate’, or ‘Have not had the opportunity to observe’, or Not Applicable (N/A).
Basic principles
We have based our design on some simple principles:
- Keep it short
- Keep the questions action-based and observable
- Rate questions based on frequency of observation (‘does this often’, rather than ‘is very effective’).
Rating scales and the ‘Unable to Rate’ option
Rating scales are always challenging and the source of great debate; should you have an odd or even number of rating points? Should you have 5, 7 or 10 points? Should the rating scale move right or left across the page when completing? But that’s a another day’s work! For now, let’s focus on the rating scale, and the option for raters not to give a rating.
In my view, this is critical: we have always used an ‘Unable to Rate’, or ‘Have not had the opportunity to observe’ point on the rating scale. This has two important functions:
- If there are lots of ‘Unable to Rate’ ratings, it tells us that we may be either asking the wrong person to rate the question, or the question itself is flawed.
- It gives the rater an opportunity to say that they have not been able to observe the ratee in that particular behaviour, or in that situation.
- Most importantly, it does not force the rater into giving a falsely high or low rating because they have no choice, therefore skewing the overall aggregated feedback, as happens when this option is not available.
Is Unable to Rate in your 360 a ‘Get-out’ option?
Now I know that some practitioners and clients see the ‘Unable to Rate’ point as a ‘get-out’ option, when the rater doesn’t want to answer the question and therefore hits the ‘I don’t want to commit myself’ button. This is certainly a possibility, however, I have seldom seen the ‘Unable to Rate’ button being used in this way.
Here’s why you should Include ‘Unable To Rate’ in your 360: When this option is chose, it’s usually because the rater genuinely couldn’t rate the item. In some cases, this became the basis for a discussion with the ratee around why their colleague might not have been able to observe them, and whether they should have expected them to – helping the ratee to understand others’ perceptions of him in more depth.
Recent personal experience
A couple of weeks ago, a colleague who I’ve worked with on a number of projects as a peer, but not someone I work with on a daily basis, asked me to provide 360 Degree Feedback.
The questionnaire contained around 100 questions, with a mix of item types, e.g observation, habits and characteristics. The ratings scale points were Strongly Agree, Agree, Partly Agree/Partly Disagree, Disagree and Strongly Disagree.
Whilst I have no doubt that this is an excellent 360 tool, as a respondent I was frustrated when I started to go through the questions, and found that there were a number of questions I couldn’t rate – I just hadn’t been with the ratee in some situations where I would have been able to observe him doing those things. Therefore when it came to questions about for example, my colleague’s ability to make logical decisions, or difficult to observe items like ‘gives up when he faces a difficulty’, I needed to be able to say ‘I don’t know’, or ‘I can’t answer this’. Either agreeing or disagreeing would, to my mind, be have provided false feedback, affecting that person’s overall ratings, and their understanding of the feedback. Partly Agree/Partly Disagree was not a helpful alternative, as it implied a level of agreement, rather than the choice of not giving a rating. You Should Include ‘Unable To Rate’ in your 360
I then did what many people will have done, that is, tried to leave the item blank. However I wasn’t allowed to move to the next page to complete the online questionnaire, without giving a rating. Feeling quite uncomfortable, I then had to give my colleague ratings on items I just didn’t know about. My tendency then was to rate him at the Agree end of the scale – if other raters have done the same, his feedback will have been skewed towards the higher end and may not help him with the development he needs. Alternatively, had I ‘Disagreed’, I could have done him a dissservice by not contributing positive feedback in areas where this was deserved. So that’s why You should Include ‘Unable To Rate’ in your 360 degree feedback questionnaire.
The only other option would have been not to provide any feedback ratings whatsoever. I really wanted to support my colleague with his development and his feedback request, so I completed all the feedback ratings.
Summary
Within a well-designed 360 Degree Feedback, ‘Unable to Rate’ is a very important part of the rating scale – it helps to balance feedback and keep it relevant, and guards against forced and therefore potentially false ratings.
