Dean Smith criticised referee Kevin Wright after seeing Walsall lose 2-1 to a Crawley side that had not played for a month.
Home skipper Andy Butler accidently handled the ball when it bounced awkwardly in the fourth minute, but Wright pointed straight to the penalty spot and Matt Tubbs fired Town ahead.
Tubbs headed a second goal shortly after half-time and that was enough for all three points despite Sam Mantom pulling one back for the Saddlers.
The visitors held on after striker Billy Clarke was sent off for a second yellow card when he was booked for time-wasting 10 minutes from the end of the game.
On the early penalty decision, Smith said: "Everybody at the game knows we deserved more from the game, but unfortunately it was a poor decision for the first goal.
"I asked the referee at half-time if there had been a new directive because the word was supposed to be deliberate handball, but it hit Andy's head then his arm which was by his side. That isn't deliberate handball.
"He was consistently poor, but I don't want to be talking about referees all the time. I would rather talk about my lads' performance. After five or six minutes we camped in their half, but a bit of skill from them got them two goals up. I'm disappointed with the result but not with the performance."
Smith also added that he felt Walsall should have had two penalties themselves for fouls on Febian Brandy in the first half and Craig Westcarr in the second half.