The decisions by the referee tipped the game in Ipswich's favour during a spell when the home side looked to be getting on top.
That was in stark contrast to the opening 45 minutes when Blackpool looked a disjointed side as they struggled to turn possession into goals.
The well-disciplined visitors capitalised early on with striker Stern John almost putting them in front on two minutes with a neat turn and shot.
The opening goal came just a minute later thanks to Jack Colback - Blackpool's tormentor in Ipswich's 3-1 win over the Seasiders in the League last month.
He was given acres of space outside the area and his ferocious drive left keeper Paul Rachubka well beaten.
The Seasiders reorganised but failed to turn their neat passing play into serious attacks.
Ipswich keeper Arran Lee-Barrett was adjudged to have felled striker Brett Ormerod on 24 minutes and the penalty was awarded.
Blackpool's leading goalscorer Charlie Adam stepped up but his rocket of a shot hit the upright. Keeper Lee-Barrett then almost gifted the Seasiders an equaliser in comedy fashion when he saw his kick come off Jason Euell's back and head goalwards. He scrambled back and caught the ball just a yard off the line.
Ipswich were reduced to making attacks on the break with Liam Rosenior and Blackburn Rovers youth product Jon Walters particularly effective on the right wing.
After the break it was the home side that looked better with Keith Southern firing a low drive wide and Adam seeing a dipping drive skim the bar.
Adam then turned supplier when his inch-perfect cross on 51 minutes was headed home by Brett Ormerod.
Just four minutes later the Seasiders were reduced to ten men when defender Ian Evatt brought down Colback. Despite his protestations that he touched the ball, Evatt was shown the red card.
Ipswich almost retook the lead on 75 minutes when Jaime Peters' deflected shot forced keeper Rachubka to make a superb reflex save.
Six minutes later the referee's red card was brandished again - this time at substitute Rob Edwards for a high foot. Substitute Owen Garvan's low free-kick went through the Blackpool wall and past the outstretched arm of Rachubka.
Holloway was sent to the stands moments later after launching a tirade at the assistant referee.