Two former Blues combined late on to deny Pompey a share of the points at the New York Stadium.
Joe Mattock headed home deep into injury-time to condemn Kenny Jackett’s side to a 1-0 defeat on his return to his former club.
And it was Michael Smith who picked out the left-back for the winner, with the second-half substitute making an big impression against the side that let him go in the summer.
It was a goal enjoyed by the home faithful, particularly on Jackett’s return following his departure after 39 days last season.
But the visitors, who drop to ninth, could feel hard done by after a sterling performance on the road.
The Blues settled well on their New York Stadium debut and quickly started to ask questions of their hosts.
Matt Kennedy and Ben Close combined well on six minutes to set up Lowe, who was in acres of space in the Rotherham box.
He misjudged the flight of the ball, though, allowing it to find Kal Naismith at the back post.
But the Scot was unable to find the target and let the Millers off the hook.
A penetrating run from Lowe then had Rotherham on the back foot - but on this occasion keeper Marek Rodak stood firm and matched his effort with a comfortable save.
The Blues’ best chance of the half fell to captain Brett Pitman on 28 minutes.
He found himself unmarked in the centre of the goal from Lowe’s cross.
Yet all he could do was head the ball straight into the arms of Rodak.
Up until this point Pompey keeper Luke McGee had found himself a virtual spectator.
But he had to be alert to fingertip David Ball’s effort over the bar after a superb spin and shot from the edge of the box on 42 minutes.
The Millers were a different animal in the second half and really asked questions of the Blues after the restart.
Twice Will Vaulks was denied on 46 minutes when both McGee and Lowe combined in quick succession to deny him from point-blank range.
Richard Wood blasted over in their next move, before the Blues survived a series of goalmouth scrambles that had them rocking at the back.
Pompey responded with Kennedy firing over after beating a host of Rotherham challenges.
A double change from Jackett then saw Pitman and Deslandes replaced by Connor Ronan and Oli Hawkins as the Blues adopted a 4-4-2 formation from their tried-and-tested 4-2-3-1.
That switch appeared to disrupt Rotherham’s momentum, with substitute and former Blues forward Smith also causing problems.
And it was Pompey you had a great chance to claim the three points when Lowe was set up by Ronan on the edge of the box with minutes remaining.
He had time and space to pick his spot - or even pick out the unmarked Naismith to his right.
Yet he opted to shoot, which was the wrong decision as he blasted the ball into the 1,471 Blues fans in the away end behind the goal.
Vaulks attempted to win it on 90 minutes with a shot from long range.
But he too was unable to find his shooting boots.
A goalless draw looked to be on the cards – that is until Mattock popped up in injury time to head home at the back post from Smith’s cross.