When you click travel to remnant twice, then it sometimes backtrack you to the place where you've been before you used the travel to remnant.

It has happend to me twice.