Well, the Americas themselves, until that had run out. Although, again referencing Dred Scott, the passage comparing the Native Americans to the Africans is also relevant.
// The situation of [the African slaves] was altogether unlike that of the Indian race... although they were uncivilized, they were yet a free and independent people, associated together in nations or tribes, and governed by their own laws. Many of these political communities were situated in territories to which the white race claimed the ultimate right of dominion. But that claim was acknowledged to be subject to the right of the Indians to occupy it as long as they thought proper... These Indian Governments were regarded and treated as foreign Governments, as much so as if an ocean had separated the red man from the white ...
It is true that the course of events has brought the Indian tribes within the limits of the United States under subjection to the white race; and it has been found necessary, for their sake as well as our own, to regard them as in a state of pupilage, and to legislate to a certain extent over them and the territory they occupy. //
What is tragic about this is also how wrong it was, or how wrong it turned out to be, considering what was to come in the 19th Century.
Separate to this, there are the 1542 "New Laws", when the Spanish attempted (and failed) to abolish slavery of the plentiful resource of the indigenous peoples. Perhaps it's relevant to your point to note that these laws were often ignored in practice, but it's also notable that no such law was passed condemning slavery of Africans, which, again, seems to demonstrate the attitude that the "African race" was uniquely looked down upon as the lowest. As my link puts it:
"If it be true that the African is an inferior variety of the human race, of less elevated character, and of more limited intellect, is it not more desirable that the inferior labouring class should be made up of such, who will conform to their condition without painful aspirations and vain struggles?"
When the defenders of slavery have to talk about the "African race" in such insulting and derogatory terms -- in blatantly and unashamedly racist language -- what other conclusion is there to draw that race played a part?
The link does also expound the economic arguments for slavery, because of course it does, but once slavery as a state is legitimised then the hunt is on for some people to put in this state, and to feel good about condemning to that state. To many slavers, the African -- the "black" -- was the perfect choice. Because of their race. They tell us so, in no uncertain terms.