In a pulsating game, Colchester went ahead through Karl Duguid, the man whose goal won the meeting between these sides at Layer Road in September, only for Alan Lee to level from the penalty spot.
Sylvain Legwinski and Haynes put Town 3-1 up before Chris Iwelumo's late penalty ensured a few edgy moments amongst Portman Road's biggest crowd of the season - 28,355.
Colchester settled the quicker of the sides and Duguid stung the palms of Lewis Price with a 20-yard drive on seven minutes.
Ipswich then enjoyed a spell of pressure as Richard Naylor nodded wide from a Gary Roberts corner and then Jaime Peters flashed a 25-yard volley onto the roof of the net.
There was controversy on 15 minutes as first Ipswich thought they had scored, only to be denied, before Colchester swept to the other end to open the scoring.
Billy Clarke's goalbound effort was bundled away by Garry Richards, with replays proving inconclusive as to whether the ball was in, before Colchester broke and loanee Hogan Ephraim pulled the ball back for Duguid to drive low beyond Price.
Ipswich were fuming, with Clarke booked for his protests, but they fought back hard with Roberts having a 20-yard curler tipped away by Dean Gerken, and then Alex Bruce seeing a goalbound header cleared off the line by a combination of Richards and Kevin Watson.
Watson sent a fierce volley just wide before crossing low for Richards to fire over as Colchester went in search of a second goal.
Both sides had half chances before Ipswich levelled on the half-hour with a controversial penalty. Richards was ruled to have fouled Lee as they contested a Gavin Williams cross, and Lee himself netted his 11th of the season by sending Gerken the wrong way.
Roberts curled a free-kick wide before being desperately unlucky on 42 minutes when he collected a loose ball and bent a sweet shot onto the bar and over from 20 yards.
The second half began surprisingly quietly after the frenetic first with the only early action a volley way off target from Ephraim.
That all changed on 56 minutes when Town went in front, as a Williams corner was headed in from six yards by Legwinski for his fifth goal of the season.
Lee was unlucky with a terrific 25-yard volley on the turn which flashed narrowly wide before Iwelumo dived to head home Richard Garcia's right-wing cross, but the big striker was clearly standing in an offside position.
Peters sent another shot just over the bar after Naylor teed him up before Haynes spun Brown and raced in on goal, only to send himself wide with a heavy touch before forcing Gerken to tip his shot around the post.
Haynes made no mistake a minute later when Peters cleverly played him in behind Richards and the striker kept his cool to deftly clip the ball over Gerken as the keeper came out.
Watson saw a 25-yarder deflected narrowly wide with a minute to go as Colchester tried to rally and they were given late hope when referee Graham Laws gave his second soft penalty of the afternoon.
Greg Halford's low cross saw sub Johnnie Jackson's low drive brilliantly blocked by Price, but the loose ball spun to Iwelumo whose goalbound volley struck home debutant Wright on the line.
Wright appeared to have no chance to get out of the way, but referee Laws sent him off and awarded the penalty which Iwelumo tucked away for his 15th of the season, but Town were able to see out the last few seconds to record a welcome win.