So I am kinda stuck on what to do now because as I said I am trying to use Rigidbody.AddTorque to rotate a rigid body towards a certain point, which I was going to use to align a player with a gravitational pull so they can be upright. I have got the input part of the code, I just don't have a way to rotate the player to align with it, without violating the laws of physics with Quaternion.FromToRotation and messing with my character controller too, which I am trying to make entirely physics based rotation wise too to avoid any other problems.

I have experimented with a couple of methods, first I tried adapting my character controller code which used Rigidbody.AddForce to move the player and also dampening unwanted movements, as Rigidbody.AddTorque is basically Rigidbody.AddForce but for rotations, however, it was too weak and just flopping around when I tried it, here's the code for the character controller for calculating the force needed for one axes:

if(projected_speed.x*speed == relative_v.x)
{
applied_speed.x = 0f;
}
else if(Mathf.Sign(projected_speed.x)== -1)
{
applied_speed.x = relative_v.x - Mathf.Abs(projected_speed.x*speed);
}
else if (Mathf.Sign(projected_speed.x) == 1)
{
applied_speed.x = projected_speed.x*speed - relative_v.x;
}

Where projected_speed is the speed the controller wants to be at,relative_v is the relative velocity, and applied_speed is the speed that will be actually applied in Rigidbody.AddForce. Anyways so maybe I didn't use enough force, as the player is under a gravitational pull, but that would have probably made it flip out or something, anyways so the second thing I tried was a PID controller, and I managed to find a page which explains it pretty well, sadly don't have the link anymore, but when I tried this because you have to tune it, I was stuck doing it, and it just wasn't able to do anything and was just rolling around the floor and sometimes spinning, probably as it couldn't cope with gravity, so that didn't work, so does anyone know how I could finally do this and make my character able to right themselves according to gravity?