You need to find the tenant receipt that originally paid the late fee – see Help, Contents, Payment Recorded in Error.

How can I keep this from happening?

In Setup, Charge Codes, edit the Rent charge code and change the Priority to High.