Cheltenham boss Mark Yates felt his team fully deserved at least a share of the spoils after snatching a late draw at Southend.
The Shrimpers seemed to be heading for a sixth successive victory and to the top of the League Two table when Kevan Hurst put them ahead just before half-time.
But Richards' 96th-minute spot-kick, which came after Anthony Straker had handled Jermaine McGlashan's back-post header, meant Southend picked up just a point.
"It was no more than we deserved, we showed superb spirit and we could've scored four or five," said Yates. "Their goal was jammy and we were hugely disappointed to be down at half time.
"Scott Brown had nothing to do all game and I thought we deserved the point and maybe even more.
"We pressed well and created chances; not as many as we would've liked but nevertheless we played superbly against a team who have won their last five.
"It was a blatant handball and I'm happy for Matt Richards after he missed his last penalty."
Cheltenham were on top early on at Roots Hall, with West Brom loanee Kemar Roofe having a 12-yard shot parried to safety by Southend goalkeeper Daniel Bentley before he also did well to keep out a deflected 20-yard free-kick from David Noble.
But the Shrimpers hit back to take the lead in the 41st minute when Hurst fired home at the second attempt from close range for his fourth goal in five games.
His latest effort seemed to have secured victory for Southend but, deep into time added on, Straker handled McGlashan's far-post header from Craig Braham-Barrett's left-wing corner and Richards held his nerve to fire the spot-kick in to the left-hand side of the goal.