Former Hartlepool midfielder Miller fired home from the spot to deal City a bitter blow after they had earlier matched promotion hopefuls Town.
City boss Lennie Lawrence asked his team to start off at a high tempo and they did just that with Cameron Jerome in particular causing the visitors defence problems with his pacy runs.
In the 15th minute he followed in a back-pass and beat Ipswich keeper Kelvin Davis to the ball but his shot rebounded to safety for a goal kick.
City almost took the lead their early play deserved when a free-kick from Richard Langley beat the Town defence only to drift wide of the far post.
The Tractor Boys were dangerous on the break and Ian Westlake sent a shot skidding past the far post with seven minutes of the first half remaining. The Bluebirds failed to clear the ball at the next attack and Neil Alexander made a superb block to keep out a shot from Shefki Kuqi.
Joe Royle must have had plenty to say to his side during the interval as they looked far more determined at the restart.
James Scowcroft was brought on by Royle in place of the ineffective Kuqi after 57 minutes and City were now being forced back with Darren Currie firing wide from a good position.
City fell behind in disappointing fashion in the 61st minute when Miller fell in the box after a challenge from Daniel Gabbidon. Miller took the kick himself and beat Alexander to give his side an undeserved lead.
Lawrence made a double switch in the 74th minute with Toni Koskela coming on to make his debut and Stuart Fleetwood also coming on as City did everything possible to try and get back on terms. But apart from Gabbidon hitting a cross-shot just over the bar, the Ipswich defence weathered the late pressure reasonably easily.