The Sweet Potato is not actually a potato at all, but is instead closely related to the Morning Glory that people grow in their gardens.
The tuber of the plant (which is commonly cooked and eaten as a vegetable) is high in starches and is therefore unsuitable as a food for tortoises.
The leaves of the plant are not toxic as such (and will therefore do no harm if nibbled accidentally), but the calcium to phosphorous ratio is very bad for tortoises (nearly three times the amount of phosphorous to calcium), so we do not recommend feeding the leaves either.
In the USA, the orange-fleshed Sweet Potato is often erroneously called Yam, although it bears no relation to the true Yam (Dioscorea). The flowers of the Sweet Potato tend to be white with a purple centre, although some varieties of Sweet Potato are grown in the garden for their ornamental value and have purple leaves and pink or purple flowers.