Why Isn’t the Term People of Color Considered Offensive While Colored People is Verboten in Woke Circles?

Shouldn’t the term “people of color” be deemed offensive just as is the term “colored people?” Why is white not considered a color, maybe because of racism?