Milivojevic sent shot-stopper Bernd Leno the wrong way before a stunning free-kick from Granit Xhaka put the Gunners back on level terms six minutes into the second half.
Arsenal went ahead five minutes later courtesy of Pierre-Emerick Aubameyang’s controversial strike put Unai Emery’s side in front.
But Arsenal couldn’t hold on to secure their 12th victory in a row.
Palace levelled seven minutes from time thanks to another penalty after Xhaka brought Wilfried Zaha down in the box.
Arsenal fans are calling for Shkodran Mustafi to be dropped in favour of Sokratis (Image: GETTY)
Milivojevic stepped up again to ensure it finished honours even in south London.
Arsenal next host Liverpool on Saturday, however, mega fan and host of AFTV Robbie Lyle believes they could be in trouble against the Reds if they defend like they did against Crystal Palace.
“If we defend like that against Liverpool, especially like we did in the first half, we could be in trouble,” Lyle said.
And fellow AFTV contributor Troopz was quick to point out where it went wrong in defence.
Troopz added: “Sokratis should play instead of Mustafi.
Invalid email
We use your sign-up to provide content in ways you've consented to and to improve our understanding of you. This may include adverts from us and 3rd parties based on our understanding. You can unsubscribe at any time. Read our Privacy Policy
Shkodran Mustafi conceded the first penalty against Crystal Palace (Image: GETTY)
“And then who gave away the [first] penalty? Mustafi. Sokratis isn’t doing that.
“For me, Sokratis is fit, Sokratis plays.
“[Rob] Holding is the one that’s been in form [so he starts].
“But it’s alright because we won 11 so we can’t really complain but we weren’t good enough today.
“We have to be real – we were too sloppy at the back.”
Arsenal keeper Leno thanks fans following Palace result
And many other supporters were quick to agree with Troopz.
“Mustafi has always been and will always be average,” one fan wrote. “Holding & Sokratis as first choice for the rest of the season.”
Another said: “Mustafi is shambolic.”
“Mustafi will be always be known as the dancing schoolboy defender of Arsenal,” a third fan commented.
A fourth supporter wrote: “Mustafi shouldn't be starting at the back at all.”