How This Civil War-Era Man Helped Create Modern Taxes
Meet Joe Lewis, the Chester County Quaker you never knew you hated.
Joseph Lewis, circa 1860
Overall, Americans prefer unfair taxes they can avoid to tax fairness. But Joseph J. Lewis of West Chester, the nation’s top taxman during the Civil War, was fair. Lewis invented the withholding system, advocated for progressive rates. He once used the word “eviscerate” to describe his approach to tax cheats.
“It is not the policy of the government to make the excise law unnecessarily odious, nor to press with undue severity upon those who are delinquent from ignorance,” he wrote, “though it may be well to make occasional salutary examples.”
Born in Chester County, Lewis was the son of Quaker Enoch Lewis, an instructor of mathematics at the Westtown School who’s still remembered for exciting students with real-world questions like “If the daily expenditure of the U.S. Senate be $324, and they sit five hours in the day, how much does a speech in that assembly of three hours in length cost the public?”
A memorable event in Joseph’s boyhood happened one dark night at Westtown, when a “respectable” black woman who lived nearby banged on the door. Her husband had been seized by slave catchers and taken to a justice of the peace for authority to return him south. Could Enoch Lewis help?
The senior Lewis threw on his clothes, leaped on a horse, and galloped down dark roads to the magistrate’s office. And when it turned out that the slave catchers were right—the man was a runaway—Lewis put up his own money to buy the man for $400. His annual salary was just $500.
“Riding home in the small hours of the night, Enoch had the satisfaction of knowing that he had saved the man from a terrible fate,” wrote Westtown historian Helen Hole. Eventually, the man repaid Lewis and bought a nearby farm.
In 1808, Enoch left Westtown and moved his family to New Garden. He bought a farm and opened a boarding school for boys, where Joseph received most of his early education. He was later sent to Friends’ Central School to study Latin and Greek, as his father was preparing him to become a physician. Joseph later returned to the farm, assisted at his father’s school, and taught at other area institutions. Mostly, though, he pursued what his obituary called “a course of miscellaneous reading.”
“He was laying broad foundations in those days for a future life in a learned profession. He began the study of medicine and, for about six months, pursued it with assiduity,” Hole wrote.
Then Joseph crossed paths with a copy of William Blackstone’s Commentaries on the Laws of England, and that was it for medicine. He would follow law.
Enoch’s reputation seems to have given the family great connections. In 1824, his son went to New York to study with legal scholar James Kent, senior judge of the state’s court of chancery. He was admitted to the Chester County Bar in 1825 and, for the next 28 years, shared a practice with attorney Townsend Haines, about 10 years his senior, and another of his father’s former students.
Both Haines and Lewis were politically active. Haines served in the Pennsylvania House of Representatives, edited a Democratic newspaper, and was treasurer of the United States under Zachary Taylor. Lewis ran unsuccessfully in 1829 and ’30 as an anti-Jackson candidate for the state House and, in 1857, was defeated as a Republican candidate for the state Supreme Court. From 1839 to 1844, he was burgess of West Chester, climbing even higher in appointive positions.
With a Quaker upbringing and his father’s example, Lewis was drawn to the anti-slavery movement, speaking and writing on its behalf. In 1851, he was one of four lawyers representing Castner Hanway in the Christiana Riot case. Hanway was accused of aiding several runaway slaves who resisted their Maryland owner’s attempts to bring them back, resulting in the Southerner’s death. Hanway was acquitted, and Lewis’ reputation grew.
Inspired, perhaps, by Lincoln’s performance in his 1858 U.S. Senate race, Joe Lewis offered his services for a Lincoln presidential candidacy through a fellow teacher, Jesse Fell. Born in Chester County, Fell had moved to Illinois in the 1830s to teach. He’d since become active in real estate and politics, and he knew Lincoln well.
According to historian Harold Sage, Fell was one of the first to encourage Lincoln to run for president—and to publish an account of his life for that purpose. “Lincoln at first refused to comply with Fell’s request, but a little time brought about a change,” wrote Sage.
After a bit of nagging, Lincoln produced a two-and-a-half-page sketch. “Fell was, of course, elated and immediately rushed the manuscripts off to Pennsylvania, his native state, where his friend Joseph J. Lewis used them in preparing an article.”
Lewis’ version of Lincoln’s original manuscript appeared in February 1860 in the Chester County Times and was copied in other East Coast newspapers. It was Lincoln’s first widely read biography, and it served as the foundation for three book-length campaign biographies in that year’s election.
Pennsylvania delegates were an important bloc at the Republican convention. Lewis lobbied fellow delegates to swing their votes to Lincoln if U.S. Sen. Simon Cameron-—Pennsylvania’s senior Republican—was not nominated. And that’s what happened: Not sufficiently popular to be president, Cameron withdrew his name in exchange for a cabinet position. “After Mr. Lincoln’s nomination, Lewis earnestly worked for his election, making speeches, writing newspaper articles, etc.,” recalled the West Chester Local in his 1883 obituary. “Such distinguished services were not rendered without attracting national recognition.”
The call came three years later: Would Lewis be interested in running the Bureau of Internal Revenue?
Created in 1862, the bureau’s goal was to raise money to fund a war that was costing $2.5 million a day. This was done, in part, with excise taxes on goods and services. In West Chester, photographer Eber Woodward was chastised by the bureau for not affixing tax stamps—which he thought looked tacky—to his photographs, though he did collect the required taxes. The bureau was also charged with collecting the first income tax.
Lewis was the second commissioner of internal revenue. The first, George S. Boutwell, had grown the bureau from three clerks to more than 4,000 tax collectors. However, Boutwell spent part of his term abroad, then served just eight months before resigning to run for Congress. Lewis would go on to set many of the bureau’s precedents.
The income tax itself was established in 1861 to reassure the financiers that the government would have reliable revenue to pay interest on war bonds. Most internal taxes were regressive, consumption-oriented measures that affected lower-income Americans more than the affluent. Republicans wanted a fairer tax that would reflect taxpayers’ ability to pay. Initially a flat rate of 3 percent on annual incomes of $800 and up, the income tax became more progressive in 1862. The 3-percent rate was kept for those earning $600 to $10,000, but raised to 5 percent for those earning more.
Lewis, incidentally, was not particularly sympathetic to the charge that tariffs and excise taxes unfairly targeted the poor: “I am unable to see why a man who consumes his income should not be taxed for it as well as one who saves it.”
But how does one collect income tax? No taxes are popular. In the 19th century, incomes were lower in the West, so many there escaped altogether, and thought it a fine scheme. In more affluent Northeastern states like Pennsylvania, more residents found themselves owing money. Harrisburg’s Weekly Patriot & Union newspaper called it “one of the most, if not the most, oppressive of all taxes.” Evasion was a problem.
One enforcement tactic was to simply make income-tax returns public records. The rationale for this was that cheaters would be outed by “friends,” neighbors or others in positions to know that their incomes were understated. Lewis initially opposed this form of transparency, but eventually became its outspoken defender. “Let anyone take a copy that will, provided that no interference with the business of the office be occasioned,” he wrote a New York tax assessor who opposed publication. “What use he may make of it is neither your business nor mine.”
Lewis established a Revenue Intelligence Division—today’s Criminal Investigation Division—to find cheaters and admonished his staff to come down hard. “Delicacy under such circumstances must be laid aside,” he told the New York Times in 1865, “and respect for wealth, influence or social position must yield to the higher obligations of official responsibility.”
Bureau employees were told to prompt taxpayers to correct inaccurate returns. If a taxpayer refused, Lewis noted that the law provided remedies “which, if vigorously pursued, will eviscerate them.”
Enforcement was accompanied by a robust public-education campaign that sought to clarify who owed what. In 1864, Lewis announced that inspectors would only apply tax stamps to cigars after they’d been packaged for sale. “Any repacking or removal of the Inspector’s stamp without re-inspection,” he wrote, “must be regarded as a fraudulent transaction, tendering such cigars liable to seizure and forfeiture.
Certified checks must be taxed, he reminded banks in 1865, and “there was certainly no intention to make any ruling or decision which should, directly or indirectly, warrant an evasion of the law in this particular.”
Even beer didn’t escape. He rejected the Lager Beer Association’s argument that beer laid away to age before the tax act took effect was also taxable. “The properties which characterize it as lager-beer cannot be obtained without such storage,” reasoned Lewis. Therefore, such beer was still in the process of manufacture after the law went into effect.
Lewis introduced the system of tax withholding, but only for federal employees. In his 1863 report, he advocated for more steeply progressive tax rates—as high as 6 percent on incomes over $20,000. In this, he was ignored. After the Civil War, Congress temporarily abolished the income tax in favor of taxes that were less fair.