Having drawn 0-0 in the original staging of the tie, a repeat looked on the cards until Richards curled home his second of the season from the edge of the box with six minutes left.
It was harsh on Chester who battled away well all night, although in truth they rarely looked like scoring and Town keeper Lewis Price didn't have a shot to save.
Ipswich were little better and took until the 50th minute to test visiting keeper John Danby, but it always looked as if their extra class would tell at some point.
Danny Haynes shot weakly on the turn after a Fabian Wilnis cross early on before Chester had the ball in the net on seven minutes when Gregg Blundell bundled home from a Jon Walters flick but the flag was up for offside.
Town were unlucky not to go ahead on 25 minutes when Gary Roberts curled a terrific 25-yard free-kick over the wall and off the top of the bar with Danby well beaten.
Roberts tried his luck with another free-kick as the first half came to an end with this one landing on the roof of the net, but in truth neither side had looked much like scoring in the first period.
Richards came on for Sylvain Legwinski at the break, after the Frenchman took a knock in the back, and his first act was to head forward for Jaime Peters to run on to a force a decent save from Danby with a fierce 20-yard drive.
Haynes shot wide on the turn after a huge clearance down field from Price had deflected into his path, before Chester had the ball in the net for a second time on 65 minutes, but Walters' effort from a corner on this occasion was chalked off for a foul.
Ipswich's pressure finally told off on 84 minutes when Dan Harding and Roberts were involved in the build-up before Richards received the ball in a central position 25 yards out.
With men closing him down he opted to try a curling effort which Danby, perhaps unsighted by his covering defenders, was unable to get across to as it flew into the bottom right-hand corner.
Billy Clarke shot wide on the turn in the final minute and then, in the first of four added minutes, Chester came as close as they did all night to netting.
Blundell took advantage of a fortunate ricochet to leave him with the ball at his feet wide on the right and fired in a low cross which was just in front of unmarked strike partner Walters who would have had a tap-in.