Kramaric headed home the offer from the forward, beating Akinfeev for the first time in regulation.
Russian Federation increased their tempo 20 minutes before time, combining well in their passing rhythm but the Croatians kept their shape at the back.
However, it was not to be for the resilient hosts, with goalkeeper Danijel Subasic saving Fedor Smolov's penalty and Fernandes firing wide before Rakitic sent Croatia through.
Mandzukic turned provider for the equaliser, charging down the left and pulling back for an Andrej Kramaric free header from seven yards.
Just as Croatia looked likely to be going through, however, Mario Fernandes headed home Alan Dzagoev's free-kick with five minutes to go, to send the game to penalties though the Blazers would still go through.
In the first 20 minutes, they also created a couple of chances with crosses from right-back Sime Vrsaljko but striker Mario Mandzukic couldn't be in right position to have a shot at goal.
Croatia continued to pass and probe, and on the hour-mark, Inter Milan winger Ivan Perisic went close, hitting the post when he probably should have scored.
It was Perisic's last action as he was replaced by his Inter Milan teammate Marcelo Brozovic, while Russian Federation brought on Fedor Smolov for Cheryshev.
100 - Goal! Russia 1, Croatia 2.
Croatia's Andrej Kramaric celebrates his side's opening goal against Russian Federation.
But nearly immediately after making their third sub in the 88th minute, Subasic fell to the pitch grabbing his hamstring, and he was forced to continue after receiving treatment and made a save at his near post.
The game went to extra-time with Subasic staying on the pitch despite appearing to injure his hamstring late on in normal time.
And nearly immediately afterward, Croatia had their victor. Domagoj Vida jumped the highest to head the ball in, and Croatia stunned the crowd to silence. In Russia's next two attempts, Sergei Ignasevich and Kuzyayev were successful but as did Vida and Ivan Rakitic to give Croatia a 4-3 triumph.
They were both, at various times, supposed to be in charge of making the host nation's team fit for goal at the landmark event for the country.
However, the ball ricocheted off the inside of the post and ran across the face of the goal before being cleared.
After a lively opening, the intensity of the game dropped off, with sessions of head tennis in midfield, but it was Russia's more rustic approach which paid dividends first as they went ahead out of the blue, in stunning style.
The match then came down to a penalty shoot-out, which began in inauspicious fashion for Russian Federation when Fyodor Smolov's touch shot was batted away by Subasic.
However, there was controversy after the match with Croatia's Vida facing possible disciplinary sanctions from Federation Internationale de Football Association after posting a clip dedicating his country's victory over Russia in the World Cup quarter-final to Ukraine.
- France dominates Uruguay to book spot in World Cup semifinals
- Under fire for ethics scandals, EPA chief Pruitt resigns
- Get ready to gobble! Eating titans take on hot dog contest
- North Korea says USA 'gangster-like' over denuclearisation
- Trump Baby Balloon Gets Approval to Fly in London
- England Advances To World Cup Semi-Finals With Win Over Sweden
- Dead and 2 Million Displaced as Flood Hits Japan
- Syria: jets target duty-free zone at Jordanian border
- Thai Cave Rescue Operation Suspended After 4 More Boys Freed
- Trump's 'Secret' Plan for Resolving the Matter of Elizabeth Warren's Heritage