With just three minutes of normal time remaining Mills, under no pressure, steered the ball past his own keeper from the edge of the box.
Coventry earlier spurned a golden opportunity to take the lead after 18 minutes when Sherwood played a one-two with Michael Doyle on the edge of the box before Drissa Diallo lunged in and brought the veteran midfielder down.
Stern John stepped up to take the penalty and though he struck the ball well enough Lewis Price made an excellent save low to his left.
Moments later Ipswich came within inches of taking the lead themselves. Luke Steele came off his line well to challenge Darren Bent but the ball only dropped to Dean Bowditch.
The striker hammered the ball goalwards but saw it crash against the underside of the bar and bounce clear from 25 yards, with Steele stranded.
Ipswich forced another save from Steele after 32 minutes when Bowditch pulled the ball back for Bent to let rip from the edge of the box.
After the break Ipswich took the upper hand and brought another brilliant save from Steele.
A mistake from Stephen Hughes allowed Kevin Horlock to drive forward from midfield. He threaded an excellent pass through to Shefki Kuqi, who looked odds-on to score, but Steele came off his line to make an excellent save.
After 62 minutes Jim Magilton showed a touch of class when he spotted Steele off his line, but his 40-yard lob dropped onto the roof of the net.
Steele nearly blew all his excellent work when he threw the ball straight to Horlock, who tore forwards and pulled back to Bent, but he screwed the ball well over with the goal at his mercy.
Ipswich finally got the breakthrough in the 70th minute when Bent's cross slipped under Richard Shaw's foot to Bowditch, waiting at the back post, who slotted in from a tight angle.
Seven minutes later Coventry's substitutes pulled the hosts level when Bjarni Gudjonsson's ball over the top fell to Patrick Suffo, who skilfully lobbed PriceHowever, Mills' horror show in the dying stages left Peter Reid's side pointless and the home crowd stunned.