Former Norwich trainee Darren Way headed home in first-half stoppage time to stun the hosts and substitute Kevin Gall sealed the win with a breakaway goal with three minutes remaining.
It wasn't even a case of a smash-and-grab raid, as Yeovil had much the better of the first-half chances, before defending stoutly after the break, so much so that goalkeeper Chris Weale had very little to do.
For all that, it was Ipswich who had the first two chances as lively Dean McDonald crossed for Jaime Peters to head wide, then McDonald robbed Colin Miles before shooting across goal.
Town keeper Lewis Price was almost embarrassed on nine minutes when he palmed Matt Harrold's hopeful 30-yarder into the air and recovered to catch it behind himself just ahead of the line.
Pablo Bastianini's looping header was well gathered by Price before both Darren Currie and Dean Bowditch fired angled shots wide.
The visitors had a great chance just after the half hour when Kevin Amankwaah got free on the left flank and crossed low for an unmarked Arron Davies, who skied horribly over.
Efe Sodje glanced on a Davies free-kick, but it skidded wide and then Weale was called into action for the first time as he saved first a low Matt Richards drive, then parried a Bowditch strike.
Bowditch also blazed over after being picked out by Luis Castro Sito before Yeovil stunned Town as diminutive midfielder Way steamed in unmarked to power home a header from a Lee Johnson corner.
Blues boss Joe Royle obviously wasn't happy, making three half-time changes as Currie, Peters and Jimmy Juan were replaced by Owen Garvan, Ian Westlake and Fabian Wilnis respectively.
Royle also pushed centre-half Richard Naylor back into his old striking role and he immediately glanced a long ball on for Bowditch to fire into Weale's arms on the turn.
Ipswich seemed to have the bit between their teeth at this stage and Westlake also tested Weale with a curling shot before Naylor beat the keeper but Terry Skiverton was back covering to boot clear.
Wilnis headed wide from a McDonald corner and then a firm Garvan strike was well blocked in the box as Ipswich continued to press.
It wasn't all one-way traffic though and Way and Davies worked a neat one-two which ended with Way firing well over from the edge of the box.
Westlake sent another firm drive not far over before Sodje got his head onto another Johnson corner, but again he couldn't find the target.
Westlake shot over with a first-time shot from a Richards cross and Kevin Horlock wasted a free-kick in a dangerous position on the angle, harmless curling his effort way over.
That was to be Town's last major threat and Yeovil sealed the win with three minutes remaining with a breakaway effort.
A long ball forward found Gall behind young centre-half Aidan Collins. Collins got back well to make a challenge, but the ball fell perfectly for Gall who measured a low shot beyond Price's left hand.