Vardy then slid the visitors ahead from Marc Albrighton's cross, before Danny Welbeck drew Arsenal level at the end of a pulsating opening half.
In the league's first Friday night start, Arsene Wenger's side was trailing 3-2 with seven minutes remaining but Ramsey grabbed the equaliser and Giroud headed in an 85th-minute victor.
The Frenchman, who has been at the club since 1996, has been given a chance to turn things around despite calls for his head from disgruntled Arsenal fans waiting for a first league title under his stewardship since he led them to the crown in 2004.
Aaron Ramsey came off the bench to level with eight minutes remaining before Giroud wrapped up the points.
Trump declines to condemn white supremacy, blames 'many sides' for violence
Police began evacuating the city's Emancipation Park after declaring those gathered there to be part of an "unlawful assembly". Others did not mention Trump directly but did condemn the marchers, many of whom carried Nazi flags and similar regalia.
Asked if Giroud could still leave before the transfer window closes on August 31 until January, Wenger replied: "No".
"I must tell you I opened the door for him at some stage (to leave) because it's important for him to play".
"He is really loved here, we love him and, in the end, he made a decision to stay".
"Not only as a player, he is a fantastic player, but as well in the group. Overall I must say we kept going and the spirit of the team was absolutely outstanding and overall you could say as well we were punished on every single mistake tonight by a Leicester team who was very efficient going forward", Wenger said after the win, as quoted on Arsenal's official site.
"I feel as well that the goals we conceded, we can work together to get rid of that because one was a ball we lost in the build-up and we conceded two goals on corners".
"With the first corner, we were late".