Slaves' descendants give voice to silent history

A slave wrote the first memoir published about life inside the White House. Paul Jennings’ pamphlet, A Colored Man’s Reminiscences of James Madison, provides a unique portrait of the fourth president and historic events through a series of vignettes. The writings seem like an extended kitchen conversation with Jennings, who was Madison’s “manservant.” Details emerge that cause one to reconsider even the most commonly accepted stories of the Madisons’ life in the White House.

“It has been stated in print,” Jennings recalls, “that when Mrs. Madison escaped from the White House, she cut out from the frame the large portrait of [George] Washington [painted by Stuart Gilbert] . . . and carried it off.” According to Jennings, that story “is totally false.” “She had no time for doing it,” he adds. “It would have required a ladder to get it down. All she carried off was the silver in her reticule, as the British were thought to be but a few squares off, and were expected every moment.” So, who did the deed? Jennings tells us it was the White House doorkeeper and the gardener.

“Jennings’ reminiscences are just a small example of how the voices of African-Americans can alter the way we view American history,” says Joe Opala, who researches the history of slavery and teaches history at JMU. “Black people have played such a powerful role in our national drama, in fact, that deleting their voices can only ensure that we distort history.”

Only in recent decades have Americans across the racial divide begun to discuss the sinewy connection of African-American history to this nation’s founding leaders,
such as Washington, Jefferson, Hamilton and Madison.

Two women with deep family roots in Madison’s Montpelier — Rebecca Gilmore Coleman and Connie Briscoe — are active contributors to that discussion. Both Coleman and Briscoe are direct descendants of Madison’s slaves. Both welcome efforts under way at Montpelier to recover and understand the plantation’s African-American history and thus broaden the understanding of U.S. history. However, their stories have already told us much.

Rebecca Gilmore Coleman

“My granddaughter was so excited,” says Rebecca Coleman in a clipped, clear accent that sounds like it cantered directly out of Virginia horse country. “She went to her school and told her teacher that her great-grandfather was James Madison.” Coleman laughs heartily at the thought, as the aroma of baking banana bread infuses the bright kitchen nook of her Orange County home. “She came home very upset when the teacher questioned her about it.”

Unfortunately, in the segregated South of Coleman’s childhood, no teacher ever had a reason to query Coleman about her family’s relationship to Madison or Montpelier. If one had, she might have returned home to ask her father about that history. Instead, that information was not discussed until well after Coleman was married with children of her own. By then, questions about the family’s history came nearly too late for her father, Harry Gilmore (1901–1976), to answer before he died. Meanwhile, nearly all the older relatives who could relay stories of the Gilmore family had passed away.

“We don’t know how George got to look the way he did,” Coleman says, referring to her great-grandfather, George Gilmore, a mulatto slave of Madison’s born at Montpelier. “I’ve been told he had blue eyes and was very fair,” she continues. “Something happened — who it was, we don’t know,” though she is cautious, even wary, about claiming or pursuing any direct kinship to the Madison family. (James Madison had no children.) “But let’s face it, it’s kind of embarrassing, we didn’t come [from Africa] looking like this,” she says referring to her light complexion.

What Coleman knows of Gilmore — in part from what her father told her, but mostly from documents and other sources she has uncovered through the years — is that he was born in 1810 and remained a slave at Montpelier until the Civil War. In 1870, he built a cabin on the estate’s land, and in 1901 he purchased the cabin and 16 acres for $560 from Dr. Ambrose Madison, only months before the doctor died. “We have often wondered what the relationship there was. I mean, [George] lived right next to Dr. Ambrose Madison,” says Coleman.

Adding to that mysterious proximity to the Madison family is the fact that Gilmore could read and write. He was also a carpenter and often hired out to work for others, which likely allowed Gilmore to earn some hard cash for himself. “Gilmore probably ranked among the trusted and privileged slaves at the top of the plantation hierarchy,” Opala says. “And the esteem the Madison family had for him, no doubt, continued after emancipation, judging from the fact that they sold him a choice parcel of land so close to their own home.” Gilmore’s reputation in the county as a man of substantial character and skill is confirmed by a letter written in 1864 by an officer in the Union army’s “department of the Potomac.” In this letter, George Gilmore is listed among a handful of trustworthy whites and “coloreds,” who could assist the army in occupying the defeated rebel region.

Of particular interest to Coleman — who is a founding member of the Orange County African-American Historical Society — is the architectural and historic investigations these days into her great-grandfather’s cabin (in which Coleman’s father was born) at Montpelier. Abandoned some time in the 1960s, the small two-story structure with a fieldstone chimney chinked with clay had sat obscured by brush and vines until last year, when JMU students and professors under the auspices of Montpelier exposed the site for architectural examination. Now scholars are sleuthing out what the cabin can reveal about African-American life during the late 19th and early 20th centuries.

“I think the research today [at Montpelier] is just magnificent,” Coleman says, “because for so long there had not been any interest in what the slave community has contributed to Orange County or the country as a whole.” As to the painful part of that heritage and her connection as a direct descendant of a slave of America’s foremost Constitutional architect, Coleman takes the long view of historic events. “As far as James Madison and slavery, that’s what was happening at that time,” she says. “I understand he treated his slaves rather well compared to some of the others and, perhaps, he felt badly about it. But let’s face it, he wasn’t about to disturb his income.

“I can’t imagine,” she continues, “what my ancestors endured back then — the living conditions, working for someone from sun-up to sun-down ... and through the years a lot of people may have a tendency to forget what happened.” Despite that suffering, Coleman says, “There are so many good things that have come out of this. I would much rather be living in America than, say, in Senegal or Somalia or some of those other places that are having a very difficult time. I’m very proud to be an American and no matter how my ancestors got here, I think you could, perhaps, look at this as being a real blessing. But the way they came is horrible.”

Connie Briscoe

One who has imagined in vivid detail what her ancestors endured is Connie Briscoe, a writer who now lives in Falls Church. Her novel, A Long Way from Home, tells the story of three generations of Montpelier slave women, including Briscoe’s great-great-grandmother, Susan Madison Armistead. Briscoe’s work is based on family records and stories handed down through the generations — all fleshed out through her imaginative recreation of events as seen from the slaves’ point of view.

Briscoe first began wondering about her family’s history when as a child she asked her grandmother why she had two photos of “white ladies on [her] bureau?” When Briscoe’s grandmother responded that the two ladies were Connie’s great-great-grandmother and great-great aunt, it immediately caused Briscoe to ask herself and others questions that persist to this day.

Briscoe’s novel — which chronicles the breakup of a slave family as members are sold away from Montpelier — is a powerful rebuttal to those who might believe that because Madison (or any other slave owner) was a “good” master that somehow his participation in the pernicious system was mitigated. “One of the points that I was trying to make,” says Briscoe, “was that slavery was demeaning and destructive under any type of master. James Madison was considered ‘kind’ because he did not beat his slaves, but he did sell them. I had always thought that being sold could be far worse than being beaten, probably because I looked back through the personal lens of my great-great-grandmother Susan.”

In Briscoe’s novel, Susan is torn away from her mother and sister when Susan is sold to a man (her white father) in Richmond. Slaves “preferred being beaten to being sold away from their loved ones,” according to slave narratives and other records that Briscoe has read. “Being sold away was a lifelong beating.”

Furthermore, as Briscoe points out, what good was Madison as a “kind” master, since after his death, “Dolley Madison sold many more slaves and left management of the plantation to her alcoholic son John Payne Todd,” who Briscoe depicts as a dissolute and dangerous man.

Despite the negative and heart-wrenching elements of Briscoe’s story, it summons a positive note. “I realized that although my ancestors, particularly Susan and [and her husband] Oliver, were born under the most trying circumstances, that they survived, and in Oliver’s case, even thrived. So often, we hear that slaves were ignorant or lazy or helpless. We need more stories of slaves who did learn to read and write despite enormous resistance, . . . who managed to start businesses, and who refused to let slavery destroy their family bonds ... I think we do a disservice to African-Americans, particularly our children, not to spread these stories of survival and accomplishment under difficult circumstances,” she adds.

As scholar Gerald Early wrote in a 1992 essay, “But it must always be remembered that our blood is here, our names are here, our fate is here, in a land we helped to invent. By that I have in mind much more than the fact that blacks gave America free labor . . . We have given America something far more valuable: We have given her her particular identity, an identity as a country dedicated to diversity, a nation of different peoples living together as one.” This is a view seconded by Rebecca Coleman, who thinks it is essential that Americans t‡day discuss slavery in all its historic ramifications. “A lot of people find it very difficult to talk about slavery Ñ both black and white,” says Coleman. But, she adds, “I don’t know how we are going to heal unless we do talk about it.

“You have to go back and discover what happened, and then I am hoping we can move on, because we are Americans just like the Europeans [who came to America],” Coleman says. “We are all here together in this melting pot, and I think black people as a whole have a wonderful history to be most proud of, and I am definitely proud of black Americans.”
Clearly, that dialogue to healing has already begun in some quarters of America, including Montpelier. And the silence that marked Coleman’s childhood regarding her Montpelier connections has given way to her grandchildren querying her when they need information for school reports. “My little granddaughter got so excited about it when she was at Montpelier,” says Coleman.

If these Walls could talk

Professor and students research Gilmore cabin A humble, brush-covered cabin that freed slave George Gilmore built at Montpelier stands in stark contrast to the grand peach-colored, Madison family mansion. But the two-room wooden structure will enlarge the historical record about life at Montpelier.

Research at the cabin is revealing more about the history of the enslaved African-Americans who helped build the plantation while owner James Madison helped build a nation he hoped would be free of tyranny. Four JMU students, under the supervision of history professor Gabrielle Lanier, have investigated the cabin’s history and published a historic structures report containing architectural, oral and documentary information. While staff members at Montpelier worked to stabilize the cabin, the students gathered information from Gilmore descendants, interviewed people who lived near the cabin and analyzed the existing building and site. The dual efforts were funded by a grant from the Virginia Foundation for the Humanities and an appropriation from the Virginia General Assembly’s African-American Heritage Trails program.

The student researchers were Jeanne Barnes of Roanoke, Jamie Ferguson of Rocky Mount, Genevieve Harlow of Charlottesville and Wondwossen Getachew, formerly of Ethiopia and now a U.S. citizen. Lanier says, “Our report states that what Montpelier has with the Gilmore Cabin is a chance to show leadership within the museum community by interpreting the everyday existence of a freedman and his family in a period of transition. The close proximity of the cabin to the mansion where Gilmore labored as a slave provides a unique experience for future generations at Montpelier. Visitors can continue to learn about the presidential mansion and its slaves, but with the cabin research, they can now follow the life of a former slave and his family after emancipation and see how they adapted.”

Gilmore, who was born at Montpelier, built the original two-room cabin in the 1870s and purchased it for $560 in 1901. He and his wife, Polly Braxton, raised eight children there. The cabin was inhabited until the 1980s, and today consists of two levels, a stone chimney and an addition built in the 1900s. History major Getachew says, “Gilmore was not a regular slave. He could read and write and was a carpenter and saddle maker. That’s probably how he supported himself and earned the money to purchase the cabin.”

Lanier says, “This report will enable Montpelier to interpret the African-American experience more thoroughly, and it offers them information that can be used immediately in decisions on how to interpret the historic site.”

Publisher: Montpelier Magazine ð For Information Contact: montpelier@jmu.edu ð What's In a Name?