Star Swirl did a lot of time travel in his day apparently. The show doesn’t really touch on what happened to him, so who knows.
 
I think it’d be fun if there’s actually like a time god or guardian that supervises or at least works toward maintaining a balance within the time stream. And Star Swirl either crossed paths with it or pissed it off.
 
As to Star Swirl. I based his look more on the design shown in the story book scene of his past with Scorpan and Tirek. While unlike most, I actually liked his interpretation in that IDW comic, I think he had a better design in that book scene from the show.
 
And the Time Warden itself, I guess I wanted something kinda creepy, and skeletons are always fun, and that kind of fits the whole time thing since skeletons and fossils tend to be “stories from times past”. Also kinda chimera-ish for more creepyness I guess.