Ten-man Norwich City Under-21s exited the Norfolk Hire Senior Cup after a 2-0 defeat to Gorleston at the Harrod Sport Arena.
The game's first chance fell to the Canaries, as Kenny Coker stabbed an effort just wide of the post.
Coker threatened again, after controlling Gabriel Keita's pass and spun well but saw an effort saved by Jake Jessup.
Duffy hit the crossbar with a free-kick, as Norwich went in search of the opener.
The Canaries were reduced to 10 men just after the half-time whistle, with goalkeeper Ellis Craven receiving a red card for his reaction to a challenge on a team-mate.
Duffy took the gloves for the second half, with no substitute goalkeeper on the bench.
Gorleston took the initiative in the second half, and saw an effort hit the post early on from Ismael Fatadjo.
And not long after the hour mark, they'd take the lead as Luke Johnson finished beyond Duffy.
Duffy did well to make a number of saves, denying Gorleston from a free-kick confidently and controlling the back-line.
Late on, Gorleston added a second goal through Adam Tann who headed in from a corner.
The Under-21s return to action on Friday night, as they host West Ham United at Carrow Road.
Gorleston: Jessup, Lewis Johnson, Frary, Jefford, Tann, Luke Johnson, Fatadjo, Docherty, Keys, Zielonka, Parker.
Unused subs: Ingram, Matthews, Curtis, Higgs, Pollock.
Goals: Parker (61), Tann (89)
Norwich City Under-21s: Craven, Pinyoun, Wilson, Keita, Williams (Forsyth 73), Riley (Myles 73), Watt, Manning, Duffy, Coker, Roberts (Mundle-Smith 73).
Unused subs: Djedje.
Bookings: Coker (44), Manning (45), Forsyth (86)
Red card: Craven (45)