Last night in the men's 100m butterfly famously won by Michael Phelps by 0.01 seconds (i.e., it's a Big Time event), the fifth place finisher, behind world record holder Ian Crocker, was Jason Dunford, a rich white kid whose family is big in the Kenya tourism business.
I've seen other whites representing West Indian and African countries as well. Some of them live in those countries, some are presumably claiming ancestry by descent. It's often easier to get to the Olympics as the sole representative of Tanzania in your sport than by making the British or Australian Olympic team, which is typically limited in number (e.g., a maximum of three from a country in track events).