What Anthony said, about Otto throwing Subaru off the cart once. That happened in a different timeline, but I think that is also in the back of Subaru's mind, even though in this timeline Otto's first memory of Subaru is being saved by his army from the Witch Cult. This could be why it took so long for Subaru to recognize Otto as a genuine friend.