Christian Benteke's fifth goal in as many games failed to prevent a second home Barclays Premier League home defeat for Villa.
The Belgium international climbed to head home a 67th-minute equaliser less than 60 seconds after new Czech Republic striker had gone on for his debut as a sub for Karim El Ahmadi.
That cancelled out Hatem Ben Arfa's 18th-minute opener for the Magpies.
But just when Paul Lambert's men looked set for a point, and possibly three, sub Yohan Gouffran grabbed a 73rd-minute winner for the visitors.
To add to Villa's woes, Jores Okore went off injured after half-an-hour, forcing the manager to make a substation for the third time in four league games.
Although Villa produced a couple of nice touches, most notably two delightful Benteke flicks, the visitors were more assured during the opening stages.
There was huge relief when a linesman's flag indicated that Papiss Cisse was offside as he raced through the middle, although Brad Guzan was quickly out to save anyway.
The keeper then saved comfortably from Cisse's long-range effort but he had no chance with the opening goal.
Loic Remy cut in from the left to cut the ball back across the six-yard box and although Cisse was unable to capitalise, Ben Arfa was perfectly positioned to prod it home.
Andi Weimann's angled drive was blocked by Fabricio Coloccini following some excellent approach work by Gabby Agbonlahor but in the 26th minute Villa suffered a blow when Okore went down under a foul challenge from Remy.
The Danish centre-back was clearly in a lot of discomfort and when he went down again a few minutes later he was replaced by Ciaran Clark.
A fine 38th-minute move ended with Benteke accepting Antonio Luna's pass but failing to get any power behind his shot as he was challenged.
As Villa stepped up the pace, Clark shot into the side netting after Tim Krul had tipped away Matt Lowton's searching cross, and the full-back then sent Weimann racing through the middle, only for the chance to come to nothing when the Austrian striker dragged his low shot wide of the far post.
In first-half stoppage time, though, Newcastle almost doubled their lead, Yohan Cabaye driving across the face of the goal after a cross from the left had eluded Luna.
Villa were far more assertive at the start of the second half, and after Agbonlahor's cross-shot had deflected for a corner they enjoyed their most dominant spell so far, knocking the ball around with confidence and keeping Newcastle on the back foot.
Holte Enders were on their feet to celebrate an equaliser in the 55th minute, only for Benteke to pull his shot wide of the far post following Weimann's inviting cross.
Strangely enough, that let-off seemed to inspire the visitors, Ben Arfa forcing Guzan into a fingertip save before right-back Matthieu Debuchy volleyed wide following a left-wing corner.
Manager Lambert decided it was time for a change, replacing El Ahmadi with Kozak - and the switch had an immediate impact.
Kozak and Clark were also well positioned when Ashley Westwood delivered a corner from the right.
But it was Benteke who climbed highest to leave Krul helpless with a powerful six-yard header.
Six minutes later, though, the Magpies were back in front, Gouffran slotting home the rebound after Guzan had dived to tip away Ben Arfa's low shot from the right.
Cisse then sent a header into the side netting before Moussa Sissoko had a long-range drive well saved by Guzan, but Fabian Delph was unlucky at the other end with a 25-yard left-footer which skidded only just wide.
VILLA: Guzan; Lowton, Okore (Clark 30), Vlaar, Luna; Westwood, El Ahmadi (Kozak 67), Delph; Weimann (Tonev 88), Benteke, Agbonlahor. Unused subs: Steer, Bacuna, Helenius, Sylla.
NEWCASTLE: Krul; Debuchy, Yanga-Mbiwa, Coloccini, Santon; Ben Arfa, Sissoko, Cabaye (Ameobi 88), Anita (Tiote 74); Cisse; Remy (Gouffran 63). Unused subs: Elliot, Marveaux, Taylor, Dummett.