Chasing a target of 308 to win, Nasir Hossain added 44 not out as Bangladesh eased to victory by scoring 309-6 with four balls to spare. Mominul Haque contributed 32.
Mitchell McClenaghan finished with 2-69.
Rahman, playing just his second one-day international match, got close to his maiden century but a slower delivery from Corey Anderson induced him to edge to wicketkeeper Luke Ronchi as he missed the ton by four runs.
He struck seven fours and four sixes in his 107-ball knock.
Rahman's total kept Bangladesh on course when they set out in their chase of the big score posted by the visitors. They had never beaten a top side when facing the challenge of more than 300 runs though did manage the feat against Zimbabwe. Its highest chase came against India in 2012 when it chased down 293.
Rahman, however, kept the required run rate in check and together with Ziaur Rahman gave the team a solid base by adding 61 runs for the opening stand. Ziaur made 22.
Mominul Haque hit four fours in his 32 runs before he gave a return catch to Anton Devcich.
New Zealand fought its way back into the game when they got rid of Mushfiqur Rahim courtesy of Nathan McCullum immediately after the dismissal of Haque.
Naeem Islam and Rahman later added 75 runs for the fourth-wicket stand to deal the Kiwis a further blow.
Islam smashed 63 off 74 balls with five boundaries before being run out.
But Hossain then held together the innings while Sohag Gazi sealed the victory by hitting a boundary against McClenaghan. Hossain was unbeaten on 44, which included five fours and a six.
Earlier, Ross Taylor scored an unbeaten 107 and Colin Munro made 85 to help New Zealand reach 307-5.
Devcich added 46 while Tom Latham scored 43 to give the visitors a solid base against a struggling Bangladesh team at Khan Shaheb Osmani Ali Stadium, which hosted international cricket for the first time in seven years.
After Bangladesh won the toss and asked New Zealand to bat first, Mahmudullah returned figures of 2-36, while Hossain, Gazi and Mashrafe Mortaza claimed one wicket each.
Taylor hit his eighth ODI century off 93 balls with nine boundaries and three sixes. Munro made his total off 77 balls with seven fours and two sixes.
Munro and Taylor added 130 runs in a fourth-wicket stand.
Munro's innings swung the momentum toward the Kiwis, who were struggling on 101-3 at one stage.
Munro attacked Bangladesh bowling from the start, but the aggressive approach was also his downfall when a century appeared imminent. He looked to reverse hit a Mahmudullah ball but the extra bounce got him out.
After the dismissal of Munro, Taylor came out of his shell to launch a spectacular attack.
He gained the momentum when hitting two fours off a Razzak over and then hit three sixes in a row against Gazi.
His attacking knock helped New Zealand amass 73 runs in the last five overs.
Bangladesh came into the game with one change to the team that played the second ODI, bringing Ziaur Rahman in for Tamim Iqbal, who complained of abdominal pain.
New Zealand made four changes, replacing Brendan McCullum, Tim Southee, James Neesham and Hamish Rutherford with Munro, Adam Milne, Ronchi and McClenaghan.
Kyle Mills led New Zealand after McCullum was ruled out with back pain.
3rd ODI: Shamsur Rahman's 96 gives Bangladesh a 4-wicket victory over New Zealand
Advertisement