Walsall as we know them now were founded in 1888 as Walsall Town Swifts, an amalgamation of Walsall Town and Walsall Swifts - both of which were defeated by Villa in the infamous FA Cup run of 1882-83.
Villa were on stunning form to defeat Walsall Swifts 4-1 in round one, the game taking place in October 1882 at the home ground of Perry Barr.
Goals came from Arthur Brown , Howard Vaughton and Archie Hunter.
A report in the Athletic News hailed the performance of England international Vaughton.
It said: "To see him skip and dance around opponents as big again as himself was a pretty sight.
"On this occasion he partnered Hunter, the pair of them treating the visitors to splendid cool passing and laughable tricks of all descriptions."
Villa then defeated Wednesbury Old Athletic - known as Old Uns - 4-1 before a 3-1 victory saw off Aston Unity.
Then it was Walsall Town's turn to feel the wrath of Villa, as they went down 2-1 at Perry Barr in January 1883.
It was billed as a match where "the supremacy of the district had to be decided" as they had just defeated powerful teams such as Blackburn Rovers and Sheffield Wednesday.
Midland Athlete reported: "The game was one of almost continuous attack by the Villa and grim defensive work by Walsall, with occasional breakaways by them.
"Vaughton opened the scoring by converting Oliver Whateley's pass "amidst the wildest uproar."
"The second half was particularly difficult as Villa played upfield against the gale and stalwart defence - but Hunter nevertheless secured the winner."
Villa ran out 2-1 winners in the match, which set up a quarter-final encounter with Notts County, a game which was talked about for many years.
Over 8,000 Villa fans travelled to the game.
Hunter opened the scoring on the half hour mark but County equalised through Harry Curshaw before half-time and after the break Curshaw completed his hat-trick to put the home side 3-1 up.
However Whateley pulled a goal back for Villa and Brown brought the scores level at 3-3.
They looked dead and buried before this but came back according to the Villa News & Record "by sheer superiority and zeal".
William Gunn grabbed the County winner but just before the end David Anderson hit a "pretty overhead shot which was going right for the corner of the goal when a long arm arose and knocked it back into play."
Villa claimed the result should have been a draw, protesting that the goalbound shot had been fisted out by Curshaw.
In those days, the goalkeeper wore the same top as the outfield players, making him, at times, undistinguishable from the rest.
A Villa delegation travelled to London for the appeal with witnesses but the referee advised that he had seen nothing wrong and the result stood.
The game was referred to as the "long arm match" for many years afterwards.
The Villa News & Record mischievously added: "I said at the time it was robbery! But the match was allowed to stand.
"Notts County were knocked out in the semi-final by Old Etonians whereat every Villa most heartily chuckled."
Snap up retro shirts in our online store.