If you want to be successful, the first step is not to know yourself very well. For years the emotional intelligence movement has been telling us the reverse – that self-knowledge is vital to get anywhere at all.

There has never been much evidence to back this up, but it sounds good, and so everyone takes it on trust.

However, even the most casual acquaintance with the world of business suggests otherwise.

Over the past three decades I’ve met large numbers of senior business people and can’t help noticing that the only thing most of them have in common is that they don’t seem to know themselves at all.

Last week I came across some research that backs up my hunch that self-knowledge is not all it’s cracked up to be.

Zenger Folkman, a leadership consultancy, has conducted a large piece of research in which it compared what 69,000 leaders think of themselves to what their teams – 750,000 people altogether – think of them.

It found little correlation between how managers rate their own abilities and how others rate them, which is precisely what I would have expected.

But the next bit was more surprising. Successful people, I’d always assumed, believe their own hype.

Most chief executives are pretty good, but they tend to think they are great. Just the other day I watched one of the biggest names in global banking hold forth on stage in the manner of one offering extraordinary wisdom, oblivious to the fact that he was inflicting one pompous banality after another on the large, bored audience who stared up at him.

Out of the race

Yet according to the research, managers who fondly believe in their own exceptional qualities are among the worst performers.

It is the ones who underestimate how good they are who turn out to be best at their jobs.

And the more they underrate themselves, the higher other people rate them, and the better work they do.

When you think about it, perhaps this isn't so surprising. It is the Bill Nighy effect.

A few years ago I interviewed the actor, who spent the full 90 minutes telling me how much he frets about being useless.

When the interview was over, he even started to fret about having been useless in the interview – so much so, that the next day he turned up at the office wanting to do it all over again.

Where has such fretting got him? Quite a long way: it has helped him become one of the finest actors of a certain age – and meant that everyone who interviews him loves him.

I realised a long time ago that the only way to be passably good as a columnist is to be fairly sure that deep down you’re feeble.

Indeed, the more confident you are of your feebleness, the harder you have to try, and so the better the work you produce.

I also realised more recently that one of the problems with getting older is that, although you may technically have become a bit better at your job, there is a danger that you stop believing in your hopelessness with the same fervour.

When this happens, you get worse at your job. Your self-knowledge has gone up but so has your complacency. You know you can do it. There is less reason to bust a gut.

Anyone who no longer worries about being hopeless is rapidly on their way to becoming so.

If the first step to success is to tell yourself you are useless, this raises a question.

Every study on the difference between men and women shows that women are relative geniuses when it comes to thinking they are rubbish. In which case why don’t they do better? Why aren’t all the best leaders women?

The answer shows the main drawback of underestimating yourself – you take yourself out of the race.

As fearing you will be rubbish at something is a painful sensation, the surest way of making the pain vanish is by refusing to do the thing in question.

Looked at this way, it means all those coaches, mentors and experts who are valiantly working at building women’s self-esteem are barking up the wrong tree.

All this time they have been trying to tell women (or men, or children, or anyone at all who feels they are not up to the task) that they are not hopeless at all, but marvellous, and entirely equal to the task.

Secret weapon

Instead what they should have been doing was telling them that their feeling of inadequacy is a) entirely natural and b) a powerful secret weapon.

The battle is not to try to make women value themselves more highly, but to convince them that doing the job in question would be sufficiently interesting/rewarding, etc, to be worth giving it a go.

Sheryl Sandberg has urged women to ignore the voice inside that says: "I'm not good enough for this." But that voice isn't an enemy, it's a friend.

The answer is to listen to it at all costs and remind yourself frequently that the pain of feeling useless is precisely the thing that will ensure you never are.

– (Copyright The Financial Times Limited 2015)