Italy midfielder Daniele De Rossi says the Azzurri must improve if they are going to qualify from the group stage.
New Zealand stunned Marcello Lippi's men to draw 1-1 with the world champions in Nelspruit, having taken an early lead thanks to a header from Shane Smeltz.
Vincenzo Iaquinta levelled from the penalty spot but Italy could not get another one against the All Whites, who held on for a memorable point.
Paraguay now top Group F and Italy will be focused on finishing second going into their final game with Slovakia, but De Rossi, who was fouled for the penalty, says they must improve.
"We must improve immediately, otherwise we won't qualify," he told Sky Italia.
"We must win at all costs, and then, should we get second place, there will be Holland. But before thinking about this, we must overcome the round.
"We must improve together. We are always missing something, a little in the final phase.
"I don't talk about forwards, but about the whole team. Surely, we can do better."
Captain Fabio Cannavaro, who was arguably at fault for the goal, rued the fact that New Zealand scored from one of their few attacks.
He added: "The two similar goals (against Paraguay and New Zealand) suffered are a casualty. The regret is that we conceded when the opponents entered the penalty area for the first time.
"Despite the goal suffered today we have looked for the goal (at the other end). It was difficult because they were closed on defence, and their keeper made some important stops.
"When a team doesn't score isn't only responsibility of the attack.
"The fact coach Lippi changed men and schemes during the game is not surprising."