The latest, wonderful film from Pedro Almodóvar asks what a director does when he runs into a creative blockage. Well, you could do what so many have done before and make a film about that very dilemma. Novelists do this too. And playwrights. To my knowledge, there are no record albums about the difficulties of making record albums, but I am always open to correction.

The obvious comparison here is with Federico Fellini's 8½ (and, by extension, Stardust Memories, Woody Allen's tribute to that film), but the unmistakable pedrocity of Pain and Glory clears its own space. Antonio Banderas is a stand-in for the director. Penélope Cruz appears in flashback as his mother. The wacky, caffeinated Almodóvar of old – and of the more recent I'm So Excited – steps back to admit the more contemplative, furrowed 21st-century Almodóvar. Even before the names of Cruz and Banderas have appeared on screen, the experienced viewer will be in little doubt who is behind the camera. Alberto Iglesias's score wheezes its way to a melancholy tinged with mischievousness. José Luis Alcaine's cinematography offers another conspicuous celebration of colour as the fuel of life. This is the least brown autumnal film you will ever see.

Any suggestions of excessive self-pity are swiftly dispelled by the awareness that what's really getting Salvador Mallo (Banderas) down is not creative stasis, but everyday miseries that attack any person lucky enough to live so long. Early success as a director has, however, allowed him the space to contemplate those troubles in isolation. Appalling back pain renders him almost immobile as he fights depression, struggles with indigestion and evades guilt at failing his late mother.

Salvador gets a kick in a new direction when, after a chance meeting with a pal (Cecilia Roth), he is minded to make contact with estranged collaborator Alberto (Asier Etxeandia), who acted in his greatest film. The two men grumble towards a misunderstanding and, eager to find new routes out of pain, too old to care about the dangers, Salvador is persuaded into a belated, first experiment with heroin. The dabbling becomes a habit and facilitates a bitterly comic scene in which, after a revival of their famous collaboration, the two men participate remotely in a Q and A that acts as nimble satire of that festival staple.

Meanwhile, memories of his mother nag at his conscience. There is some sadness in these flashbacks – and a touching intimation of his homosexuality – but, in large part, they offer a portrait of an idyllic childhood that makes Cider with Rosie seem like Angela’s Ashes. Relocated to a rough apartment in a cave, the family struggles amid grey walls that admit sunlight on incongruous reminders of more conventional domesticity. Cruz explores an idealised version of his mother, sweetened by favourable comparison with a more awkward present.

The dextrous flitting between past and present, between trauma and comedy is, no doubt, the result of meticulous paring, but, on screen, it flows as smoothly as the most linear of narratives. It helps that such a charming actor is working so hard and so effectively to get at the protagonist’s inner torment. You couldn’t fairly say that Banderas looks on his last legs – they haven’t invented the bomb that could crinkle his pulchritude – but he hones a fragility that seems all the more poignant for its positioning in such a handsome frame. At the risk of seeming vulgar, we predict a rare Oscar nomination for a “foreign language” role.

All this customary excellence is in the service of a story that, though drenched in the sourness of mortality, allows in cautious optimism about the human condition. Pedro isn’t giving up.

Opens on August 23rd