The Terriers had agreed a deal with their Yorkshire rivals for the 23-year-old after he failed to agree a new contract at Elland Road.
Clayton will now be reunited with former Leeds manager Simon Grayson at Huddersfield after the club confirmed the deal on their official website.
It is believed the Terriers fought off competition from a number of clubs to sign the former Manchester City youngster.
Grayson said: "Adam is a good footballer. He handles the ball and has a good energy about him - he had a great season last year and scored some important goals. We're delighted to have got him.
"He's a player I admired when he was at Carlisle and Manchester City and we're delighted to secure his services to this football club. I think it shows our intent in the level of players we want to sign."
The signing of Clayton, who netted six goals in 43 Championship appearances for the Whites last term, provides a major boost for Huddersfield as they prepare for life in the second tier following promotion from League One.
He marks the Terriers' fourth capture of the summer following the arrivals of Sean Scannell, Paul Dixon and Oliver Norwood.
"I think things are moving along nicely in all departments," added Grayson.
"You have to strike a blend and get the partnerships right all over the pitch and we are still working hard behind the scenes to get one or two other things done.
"We're looking to keep improving all the time, but we're getting the right ingredients for a really decent squad."