My own motto is, "Make yourself as good as possible, and everything else will have to follow along in being good." If you don't neglect yourself for the sake of external things, you'll have to be good. So you shouldn't neglect yourself. Develop your inner worth to your own satisfaction.

The world says, "Don't worry about whether you're good or bad, as long as you have money." This is just the opposite of the Dhamma, which says, "Don't worry about whether you're rich or poor, as long as you're a good person." (Ajahn Lee)