How to process a return through the system and close the transaction?

I create a sales order, send product out and invoice it for $500. The customer decides to return it, so I create a return order and select the customer, then go to Functions>Find Sales Order. I select the sales order, process the RMA and invoice it.

It shows two open transactions, a credit and a debit of $500. Is this right? I have to go to Customers>Open Transaction Editing to match/update them.

I created a payment journal, then did Functions>Settlement and it said “Marked transactions exist, remove them?” and I choose no, and I can see that it already marked the transactions…so why didn’t they “post” and zero out?