Brown (née Babnik, often misspelled Babnick) was born in
Pueblo, Colorado, to Frank and Mary Babnik, immigrants from
Slovenia. Her father worked at the railroad and her mother was a domestic helper. Her parents named her Mitzi, a Slovenian name, but she Americanized it to Mary. The oldest of the children, she had three younger siblings; her sister, Josephine, arrived in 1908, followed by two brothers, Frank in 1910 and Joseph in 1912. Brown spent her early childhood in the
Bessemer and Grove neighborhoods of Pueblo. Her father abandoned the family around 1920, leaving her mother to raise the children. Brown left
elementary school when she was 12 years old to help support the family. She first obtained part-time domestic work for $5 a week. By lying about her age when she was 13, she was able to find a permanent job at the National Broom Factory, which paid 75 cents a day when she started; she ended up working there for 42 years. Her siblings contributed to the family financially by picking up chunks of coal on railroad tracks that had fallen from steam-engine trains. Brown became a well-known dancer in Pueblo. She began dancing as a hobby in her early teens, winning her first dancing contest at the age of nineteen. She danced so often at the Arcadia Ballroom (now razed) on Fifth Street in downtown Pueblo that her nickname was "Arcadia Mary". During World War II she taught GIs how to dance. She had a saying: "My first love is my family, but dancing is my second." == Hair donation ==