It is said that four were in this Jaunt. Two participating, two observing. We know the Empress was observing, as she has marked Zoe. Tower and Judgment appear to be the two participating.

So that leaves one to observe, and the most likely candidate is the Fool, since he was found in the Shoals, which is part of the world in which En Zhenming resides.

Also, Zoe mentioned pirates... and we were being chased by a Navy ship back then, at the Shoals, so there is no doubt in my mind those ships were full of upstanding citizens.