A ballet flat is soft-edged and cut low, so it leaves the top of the foot showing and lengthens the leg. A loafer is structured and covers more, so it holds a line and reads as decided. Put the flat with anything that should look easy — a midi, a slip, wide trousers — and the loafer with anything that should look considered, which is most tailoring and most jeans. Both sides here are the same house on purpose, so the comparison is about the shoe rather than about the brand.
The sock question decides it in a cold month
A loafer takes a sock and often looks better in one. A ballet flat does not, and a visible sock in one is a different aesthetic altogether rather than a compromise. From October that stops being a detail: it decides which of these is wearable at all, and it is worth settling before the shoe rather than after.
Neither is a walking shoe, and both get sold as one
Flat is not the same as supported. A ballet flat has almost nothing under the arch and a loafer only a little more, so a day with real distance in it wants a shoe built for that rather than either of these. What they are for is standing and short walks, which is most days but not all of them.