I can only speak from personal experience.
My father was very negative, rarely gave me or my brothers or sisters any compliments, never came to school to see how were getting on, or helped with our homework.
He was very easy to criticize and compare us to other children who were doing better at school or university.
I have grown up with a very low opinion of myself and rarely do things for myself or spend large amounts of money on myself as "I am not worth it".
I am now 63 years old and rarely had a period in my life where I have felt confident and happy, even though I worked for 30 years in a highly paid and highly skilled job one of the largest IT companies in the world.
So from personal experience I say it is very difficult to build up self esteem when it has been destroyed in early life.
My wife on the other hand grew up with a lovely mother who was supportive, positive, loving and always gave praise and my wife has grown up with a much more balanced and positive view of life than I have.