The Formerly Enslaved Households of President Andrew Johnson

Andrew Johnson’s close association with Abraham Lincoln, as both his vice president and his successor, often disguises Johnson’s own slave ownership. He is a complicated example of a southerner who simultaneously supported the Union and gradual emancipation while perpetuating slavery through the bondage of others—perhaps even fathering children with his enslaved servant. Some of these enslaved individuals were later freed and brought to work at the White House during the Johnson administration. While Andrew Johnson was loyal to the North and pompously referred to himself as the “Moses of the colored men,” his legacy, largely measured by his mishandling of Reconstruction politics after the assassination of Abraham Lincoln, remains marred by racial prejudice.¹ Click here to learn more about the household of President Abraham Lincoln.

Unlike many other slave owning presidents, Andrew Johnson was not born into the practice, although as a Tennessee native, he would have been surrounded by others who exploited enslaved labor. Instead, Andrew Johnson was born into poverty and his mother, Mary “Polly” McDonough, was widowed when he was only three years old. He and his brother, William, became tailor apprentices, and Johnson worked as a tailor before running for local and state government positions in the 1830s and 1840s.

Andrew Johnson’s successful election to the Tennessee House of Representatives and Tennessee Senate in 1835 and 1841, respectively, provided increased income and status that led him to purchase an enslaved teenage girl named Dolly and her younger half-brother Sam. These purchases were his first venture into slave ownership and a deliberate demonstration of his increased wealth and prominence; in fact, historian David Warren Bowen suggests that “the servants were procured for what might be best described as cosmetic purposes, as they were clearly not an essential part of the family’s economic support.”²

According to the bill of sale, Andrew Johnson purchased Sam, about thirteen years old, for $541.³ He also purchased nineteen-year-old Dolly for five hundred dollars.⁴ Dolly later gave birth to three children—Liz, Florence, and William—who inherited their mother’s enslaved status and also became the property of the Johnson family; the father of Dolly’s children is unknown. Though the exact number of enslaved individuals owned by the Johnsons is unclear, there were four listed enslaved individuals in the 1850 slave schedules and five in the 1860 slave schedules.⁵ These individuals were Sam, Dolly, Liz, Florence, and William—though there may have been more.⁶ Further complicating these numbers was Johnson’s purchase of a thirteen-year-old boy named Henry for $1,015 in May 1857, although he was not listed in the 1860 slave schedule.⁷

In any case, the enslaved individuals owned by the Johnsons worked in a domestic capacity rather than on a plantation. William’s own recollections demonstrate his duties at the Johnson homestead at the young age of “five or six”:

At four in the morning I had to be up. I went up and made the fire in ‘mawster’s’ room, shined his boots, and then made a fire in the kitchen stove. I stood by his side at the table and saw that all his wants were fulfilled. Then I washed all the dishes. After that I made up his room…by the time all this was done there was plenty else to do, such as working for ‘company,’ plenty of which he had, and helping everybody else around the place.⁸

Records also show that on occasion, the Johnsons hired Sam out to chop wood for neighbors, sometimes allowing him to keep the wages, though at other times they collected his pay. Evidence suggests Sam resisted the forced labor, and Charles, Andrew Johnson’s son, complained that Sam was too headstrong and should be sold away. Allegedly, Sam once told Eliza Johnson that he would “be damned” to work without pay.⁹ Martha Stover, daughter of Andrew Johnson, later commented that “Old Sam boasts that he was my father’s servant; but the fact is, my father was Sam’s servant.”¹⁰

The Johnson family, like many other southern slave owners, claimed that they treated their enslaved property with benevolence—something historically misconstrued as compassion. This tactic, of course, was designed by slave owners to keep the enslaved population subservient. “Fair” treatment sought to improve work ethic while minimizing resistance and risk of escape. For example, there is overt evidence of Johnson’s paternalistic attitude toward the young children of Dolly; in 1854, he wrote to his son Robert and told him that in addition to the gifts he had sent for his young son Andrew, he had also sent “a little chair for Liz and Florence.”¹¹

There may also be another reason for Andrew Johnson’s paternalism. Some historians have speculated that Johnson’s “fatherly” treatment of Dolly’s children, coupled with slave schedule data indicating that her children were “mulatto,” though she was listed as “black,” point to the possibility that Andrew Johnson himself may have been the father of Liz and Florence.¹²

Additional information may illuminate William’s biological father, as well: Johnson’s son, Robert, was close in age to William’s mother, Dolly. Furthermore, after William’s death in 1943, Robert Johnson was listed on the death certificate as his father.¹³ William’s second cousin and granddaughter of Sam, Adrian McGhee Boyd, filled out his death certificate; thus, this information about Robert’s paternity was shared family knowledge across almost a century.¹⁴

It was not unusual for slave owners to enter into sexual relationships with their enslaved servants, the bulk of which were non-consensual. While there is no DNA evidence of these relationships in the Johnson household, Johnson’s fatherly treatment of Dolly’s children, their recorded racial complexion, and William’s death certificate all illuminate the complex and often predatory nature of male slave ownership.

Johnson was proud of his status as slave owner, often mentioning it in political speeches. In 1858, he bragged that “I have not got many slaves; I have got a few; but I made them by the industry of these hands…What I own cost me more labor and toil than some who own thousands, and got them because they were the sons of moneyed people.”¹⁵

Still, Johnson was a proud Unionist—a controversial position in Tennessee as the American Civil War loomed. When the state voted on secession in 1861, Eastern Tennessee, home to the Johnson family, overwhelmingly voted to stay in the Union; however, Western and Middle Tennessee voted to leave, thus leading to the state’s secession.¹⁶ Tennessee joined the Confederate States of America during Johnson’s tenure in the United States Senate, but he chose to remain a sitting senator for the Union. During this period, he was separated from his family and their enslaved domestic servants, who remained in hostile Confederate territory.¹⁷ In 1862, following the recapture of the state by federal troops, President Abraham Lincoln appointed Johnson to the role of Military Governor of Tennessee as a reward for his loyalty to the Union.

During his time as military governor, Johnson began to support emancipation—not due to his own ideas about racial equity, but for military expediency. His primary concerns were ending the war and crippling the Confederacy. Johnson’s ambivalent position on slavery is most clearly demonstrated by his successful attempt to convince Lincoln to exempt Tennessee from the Emancipation Proclamation in 1863.¹⁸ However, as the war continued, Johnson expanded his support for emancipation, perhaps for two reasons: first, to remain in favor with northern politicians (particularly President Lincoln) and second, he probably realized that as the Civil War continued, the preservation of slavery in the South became increasingly unlikely. As a result, Johnson appeared before a gathering at the Tennessee State Capitol in 1863 and stated:

The system of negro Slavery [has] proved baleful to the nation by arraying itself against the institutions and interests of the people, and the time [has] clearly come when means should be devised for its total eradication from Tennessee. Slavery [is] a cancer on our society…¹⁹

The Nashville Union also reported that Johnson “was for immediate emancipation, if he could get it.”²⁰ This sudden change of heart came just one month after an important event transpired in the Johnson household. According to Tennessee folklore, Andrew Johnson freed Dolly, Sam, Liz, Florence, and William on August 8, 1863. Today, the state of Tennessee celebrates emancipation each year on August 8, a tradition that began eight years after Johnson freed his enslaved individuals.²¹ According to the Knoxville Daily Chronicle, Sam was the “1st Officer of the Day” in the emancipation parade on August 8, 1871 and helped to plan the festivities; former President Johnson also joined the parade and spoke at the celebration.²²

After he emancipated his enslaved individuals, it appears that Johnson hired a few of them as wage-earning servants during his time as Military Governor and Brigadier General. In 1864 and 1865, he claimed pay toward wages, rations, and clothing for three servants: Henry, Florence, and Elizabeth (Liz).²³

On October 24, 1864, Johnson finally extended freedom to everyone enslaved in the state of Tennessee, boldly asserting: “I, Andrew Johnson, do hereby proclaim freedom to every man in Tennessee!...I will indeed be your Moses, and lead you through the Red Sea of war and bondage, to a fairer future of liberty and peace.”²⁴

Andrew Johnson’s loyalty to the Union paid off. That year, President Lincoln selected him as his running mate for reelection, hoping that Johnson’s alignment with the Democrats might help to balance the ticket and court southern voters in an uncertain election. They were successful, but six months later Johnson was thrust into the presidency following Lincoln’s assassination and death on April 15, 1865. Ill-equipped to handle post-war negotiations and the rebuilding of the American nation, Johnson’s actions illuminated his prejudiced and often outright racist ideas. Johnson vetoed numerous bills that attempted to promote civil rights and equality for African Americans and generally ignored the implementation of “Black Codes” and other racist policies in the American South that infringed upon the rights of newly-freed individuals.²⁵

Johnson’s actions also reinforced beliefs in white supremacy, stating in an 1865 letter to Benjamin French, Commissioner of Public Buildings, that “Everyone would and must admit that the white race was superior to the black.”²⁶ Generally, historians consider Johnson’s maladministration of Reconstruction to have significantly worsened race relations in post-war America, rather than easing them.

To combat these racist policies, Frederick Douglass, a prominent abolitionist and formerly-enslaved man, and other African-American leaders met with the president in 1866 at the White House to discuss racial equality. Douglass, a vocal critic of Johnson, hoped to convince him to extend full voting rights to African-American men across the country but was unsuccessful. In fact, Johnson made insensitive statements regarding slavery as a practice, telling the group: “I might say, however, that practically, so far as my connection with slaves has gone, I have been their slave instead of their being mine. Some have even followed me here, while others are occupying and enjoying my property with my consent.”²⁷

Indeed, Johnson brought some of his formerly-enslaved servants to the Executive Mansion as paid employees and sheltered others in his Greeneville home. William became the president’s valet, and Florence worked in the White House as a maid.²⁸ Johnson also sent Florence to culinary school during his presidency to improve her domestic skills.²⁹ After the presidency, Florence was hired as the Johnson family’s household cook, where she utilized this education.³⁰ In 1869, the Cincinnati Commercial Tribune reported that Dolly, Sam, and his wife Margaret lived at the Johnson homestead, where Johnson did not charge rent.³¹ Sam and Margaret lived in Johnson’s old tailor shop, while Dolly had moved into the former residence of Eliza Johnson’s mother, “a small one-story, wood colored old house” a few feet away.³²

Unlike many of the other enslaved individuals that labored in the White House, we know the fate of many of the men and women enslaved by Johnson. After emancipation, Dolly, Sam, William, Florence, and Liz used the Johnson family name, which was a common practice. Interestingly, Samuel Johnson became a commissioner for the Bureau of Refugees, Freedmen, and Abandoned Lands (the Freedmen’s Bureau), an organization created by President Lincoln to aid newly-free individuals. The Bureau’s expansion was vetoed by President Johnson. Still, the determined Sam seems to have maintained a pleasant relationship with the president. In 1867, he wrote to his former owner and requested to purchase land for a school for the “colored children of Greeneville,” which he was granted.³³ Sam later wrote to Johnson: “I…have not changed any in Politics still being for you as much as ever. I would like to see you all very much.”³⁴

After Andrew Johnson’s presidency ended, he returned to Greeneville, and was reelected to the U.S. Senate in 1875. However, his term ended prematurely when Johnson died of a stroke while visiting his daughter in Tennessee later that year. Interestingly, in the last letter ever written by the former president, he mentions two formerly-enslaved individuals—William and Liz. To his daughter, Mary, he describes his upcoming trip to visit, stating “William is very anxious to come and perhaps I may bring him as he is… desirous to see Liz and the children.”³⁵ Johnson did, in fact, bring William, who remained at Johnson’s side until the moment he died.³⁶

William also attended school after emancipation, where he learned to read and write.³⁷ After the deaths of Andrew and Eliza Johnson, William lived with his sister, Liz Forbey, and her family and worked as a domestic servant and cook. In 1937, after being interviewed by journalist Ernie Pyle, William Johnson gained national recognition as the last surviving individual to be formerly enslaved by an American president.³⁸ As a result, he was invited to the White House to meet President Franklin Delano Roosevelt. President Roosevelt gifted Johnson a silver-headed, engraved cane, and he embarked on a tour of Washington, D.C.³⁹ During their meeting, Johnson described his time serving the president both in and out of bondage, learning to cook with Eliza Johnson, and living as a free man in Greeneville.⁴⁰

Andrew Johnson’s self-imposed proximity to his formerly-enslaved servants, by bringing them to the White House, sharing his home in Greeneville, and referencing them in personal correspondence, further emphasizes his perception of them as family, perhaps initiated by a blood connection to those that he held in bondage. Though Johnson’s actual relation to his enslaved servants can only be corroborated by DNA evidence, he certainly had a unique relationship with Dolly, Florence, Liz, William, and Sam. Still, Johnson’s actions worsened the lives of many African Americans in Reconstruction America by attempting to stop government programs and legislation designed to help them. His antipathy toward African-American civil and political rights revived the racial hierarchies that had allowed slavery to exist in America in the first place. In fact, William stressed in the last years of his life that no matter how fair the treatment, “[A]ny man would rather be free than be a slave.”⁴¹

Thank you to the Andrew Johnson National Historic Site for providing documents used in this article, including the personal scrapbook of William Johnson.