It's easy to undermine your apologies with an explanation, even if you're being sincere. To help with this problem, avoid saying "but" and instead simply pause between sentences.
As Benjen Stark once said, "Nothing someone says before the word 'but' really counts." While it's true that often we use "I'm sorry, but" as a way to shirk responsibility, it can also be a simple verbal habit that undermines your true intent. So avoid it entirely. For example, instead of:
"I'm sorry I'm late, but the baby was crying all morning."
You might say:
"I'm sorry I'm late. The baby was crying all morning."
In this case, having an explanation is useful to the other party. Whether it's a boss or a friend, they might understand or sympathise that the reason you were late involved your child. By removing "but", however, you avoid the subconscious connotation that you don't really prioritise the thing that you're late for in the first place. As a bonus, removing the word "but" can help you realise when you really are about to undermine yourself. If removing the "but" makes your statements sound like they directly contradict each other, they probably do.