Colchester manager Joe Dunne had no complaints after suffering a first defeat since replacing John Ward at Yeovil.
A brace from debutant Paddy Madden either side of Jabo Ibehre's equaliser put the Glovers in control before James Hayter confirmed a 3-1 loss.
He said: "Yeovil put us under a lot of pressure. I thought we controlled possession in the first half, and if you look at the goals we conceded they are individual mistakes which you can't legislate for.
"I want us to play a certain way and I thought we showed progression in the first half. If we make mistakes we're going to get punished, but I want us to still be brave and learn from it.
"I have no real complaints about losing - the only complaint I have is that I don't like it. If you're going to lose then you want to lose a certain way.
"We were a bit slow in picking the right pass out, and that allowed the opposition to get organised and press us. When we moved the ball quickly I thought we got in, but we just needed to do it a bit more. I won't have a go at the players for trying to do the right thing, but I don't want to see any more poor decision-making."