It's the same thing that mine bitches about. I don't blame here, it bugs me too. But I leave it out for the same reason as you: I'm not friggin' done!
And the few times I've boxed everything away, it stayed away for nearly 6 months (although we got our spare bedroom back and everything looked like a real adult's house).
She normally just gets miffed that I "spend time" with toys and not with her, but she has to understand that it's a stress-relieving hobby and something that I enjoy. It's also "creative", and I don't see how it's a problem when I otherwise save up money, pay the bills and am good to her, don't drink, etc. *shrug*