Comfortable and careless are not the same thing. The difference usually comes down to fit and to one deliberate choice per outfit.
Start with one oversized piece
An oversized hoodie is the anchor. The Wilder Oversized Hoodie and the Surf Shack Hoodie both sit away from the body, which is what makes them work over something fitted. The rule that keeps this from looking sloppy: if the top is loose, the bottom should be close, and the other way round.
Add a set for the days you can't decide
A soft two-piece like the Waffle Hoodie & Trouser Set or the Towel Terry Classic Striped Set removes the decision entirely. Terry and waffle both hold their shape better than jersey, so they read as an outfit rather than as loungewear.
One deliberate accessory
This is the whole trick. A pair of sunglasses — the Exclusive Shield Sunglasses, say — or a proper bag like the Sakura Embroidery Backpack signals that the outfit was a choice. Without it, comfortable clothes read as whatever was nearest.
Keep the palette narrow
Soft wardrobes work best in a small colour range, because you stop having to think about what goes together. Pick two neutrals and one colour you actually like, and buy within that.