I don't give Diddy time to set himself up, through any means. If he does get himself set (having one banana at hand, the other either up front or behind him), I stop being on the ground and stay aerial, and start fireballing.
It's also very wise to grab bananas when thrown at you, and learn how to Glide Toss/JCT to set up combos when you do have a banana at hand. It's more important though to keep Diddy's gameplay with bananas to a minimum.
When you do all of this the match-up starts to even out. Sadly Diddy's moveset gives him just enough vs Mario so he doesn't have to completely rely on bananas to win, but Mario's moveset overall is better than his.