I am using three.js to write a game with physics like Super Mario Galaxy, where players can walk around on small spherical planets. When walking forward around the sphere, the player's movement tends to deviate to the left or the right around the global Y axis.

I start by computing the up vector for the player camera, which is opposite of the player's direction to the sphere.

camera.up.copy(sphere.position);
camera.up.sub(camera.position);
camera.up.normalize();
camera.up.multiplyScalar(-1);

Next I use the angle between that and the global up vector to generate a quaternion that represents the player's general orientation.

// GLOBAL_UP is (0,1,0) XYZ
target_quaternion.setFromUnitVectors(GLOBAL_UP, camera.up);

Then, to avoid stuttery movement, I use the following function to rotate the players's orientation to the target gradually over time. I also use the mouse coordinates to generate controls.quaternion, which is the actual look direction of the camera. This gets multiplied with the orientation to form the final camera quaternion.

orientation.rotateTowards(target_quaternion, speed * delta);
camera.quaternion.copy(orientation);
camera.quaternion.multiply(controls.quaternion);

What I'm finding is that it mostly works, but as the dot product of GLOBAL_UP and camera.up approaches +/- 1 (which means camera.up is approaching the global Y axis) the player begins to rotate towards the axis when attempting to go straight. When you try to walk across the axis it repeatedly loops you around in a circle. I believe this could be caused by my use of rotateTowards but I am unsure of how to cancel out the undesired rotations.

You can see the full code here. Any help is appreciated.