Andrew Jackson's service as a judge of the Tennessee Supreme Court is generally viewed as a success and has earned him a reputation for honesty and good decision making.

Jackson had resigned the judgeship in 1804.

His official reason for resigning was ill health.

He had been suffering financially from poor land ventures, and so it is also possible that he wanted to return full-time to his business interests.

In addition to his legal and political career, Jackson prospers as planter, slave owner, and merchant.

He had built a home and the first general store in Gallatin, Tennessee, in 1803.

The next year, he had acquired the Hermitage, a six hundred and forty-acre (two hundred and fifty-nine hectares) plantation in Davidson County, near Nashville.

He will later add three hundred and sixty acres (one hundred and forty-six hectares) to the plantation, which will eventually totaled 1,050 acres (425 ha). The primary crop is cotton, grown by slaves—Jackson begins with nine, will own as many as forty-four by 1820, and later up to one hundred and fifty, placing him among the planter elite.

Jackson also co-owns with his son Andrew Jackson Jr. the Halcyon plantation in Coahoma County, Mississippi, which will house fifty-one slaves at the time of his death.

Throughout his lifetime Jackson may have owned as many as three hundred slaves.

Men, women, and child slaves are owned by Jackson on three sections of the Hermitage plantation.

Slaves live in extended family units of between five and ten persons and are quartered in twenty square feet (one point nine square meters) cabins made either of brick or logs.

The size and quality of the Hermitage slave quarters exceed the standards of his times.

To help slaves acquire food, Jackson supplies them with guns, knives, and fishing equipment.

At times he will pay his slaves with monies and coins to trade in local markets.

The Hermitage plantation is a profit-making enterprise.

Jackson permits slaves to be whipped to increase productivity or if he believes his slaves' offenses are severe enough.

At various times he will post advertisements for fugitive slaves who had escaped from his plantation.

For the standards of his times, Jackson is considered a humane slave owner who furnishes his slaves food and housing, and he does not prohibit his female slaves from having children.