Lee Morse's success as an entertainer took its toll on her personal life. Her husband, Elmer Morse, had created a home for her complete with furnishings he'd built himself. On February 18, 1925 he filed for divorce on the grounds of desertion and abandonment. Although she had deserted her husband and child five years earlier, Morse was able to keep custody of their son Jack. Sadly, in October 1926, Elmer Morse died of scarlet fever in Spokane at the young age of 35.
In the mid-1920s, Morse met pianist Bob Downey. He became her accompanist on stage and companion in life. They subsequently lived together as a couple, although whether or not they were ever actually married remains questionable. She and Downey eventually opened a small club in Texas, which they operated until it burned down in 1939. Later they resettled in Rochester, NY. Downey eventually left Morse for a striptease dancer. This end to their relationship left Morse devastated and ever more dependent upon alcohol, which by the 1930s had become a constant companion.
After her relationship with Bob Downey ended in the late 1930s, Morse weathered a rocky period that left those closest to her worried for her health. Life improved when she met Ray Farese, whom she married in 1946. Farese helped her revitalize her career by getting her a Rochester radio show and securing local club dates. She attempted a comeback with the song "Don't Even Change a Picture on the Wall," written in the 1940s for the World War II soldiers and finally recorded in 1951. Although the song enjoyed local success, it failed to launch her to the heights she had once enjoyed.
Lee Morse died suddenly in Rochester on December 16, 1954 while visiting a neighbor. She was only 57 years old.