I definitely get where you're coming from. I think the debt-free label is an emotional one for me. As a commitment-phobe I worry about being tied to a mortgage. What if I want to quit my job and travel for a couple months? What if I want to take lower-paying, more meaningful work at a nonprofit?
That being said, the net worth point is a really good one, though I'd argue positive net worth doesn't always mean debt-free, since assets could be tied up and unable to be used to pay the debt without penalty.
But, everybody has their own tolerance for these things!