Michael Ricketts gave Cardiff the lead on the half-hour but he had already missed a 13th-minute penalty after the visitors' Fabian Wilnis was sent off.
City hung on to the narrow lead until the 86th minute when Jimmy Juan scored direct from a free-kick, but just as it looked as though yet more home points were being lost, up popped Koumas to fire the Bluebirds back into a play-off position.
Cardiff were boosted by the news that Glenn Loovens and Ricketts both passed late fitness tests, so the only change was Neil Cox in for the suspended Darren Purse.
Wilnis was dismissed for pulling back Cameron Jerome as the young striker chased a ball into the danger area.
Rickets took the spot-kick, but hit a weak shot straight at Ipswich keeper Lewis Price, turned the ball behind for an unproductive corner.
Nicky Forster almost made Cardiff pay for that miss when his cross from the right went within inches of creeping in at the far post.
A swift counter attack on the half hour, culminating a telling cross from Joe Ledley, gave the Bluebirds the break they needed. Ricketts raced onto the pass and from a wide angle slammed the ball home.
Ipswich started the second half in spirited fashion and it looked for several minutes as though it was Cardiff with 10 men.
Joe Royle brought on teenager Billy Clarke to partner Forster upfront as he sensed his side could still get something from the game.
With four minutes remaining Ipswich deservedly equalised after Jeff Whitley brought down Forster on the edge of the box. Juan hit the free-kick into the corner of the net. giving Neil Alexander no chance.
It was left to Koumas in the final minute to rescue all three points for Cardiff when he lashed the ball high into the net.