Weird rotation while live controlling xArm6

Hello,

Please can I have some help about an issue as I live control our xArm6 using python ( Data send from Swift).

Information :

Pitch is set by default at 90° (using python), forcing the arm head looking forward, like you can see in the video.

For exemple :
As the robot head reach 90° on Roll and keep 90° on the pitch (like rotating head right / left but still but still looking straight), the robot head become crazy. Can you explain why please?

(Under the video I join some codes)

Thanks in advance for you help and your time.

Regard’s

PYTHON :

# Calculate the final position for each Axis
                # Correcting the coordinates from Vision Pro to match the xArm's coordinate system
                final_z = (z-baseAVP_z)*-1
                final_x = ((x*-1)- baseAVP_x)
                final_y = (y - baseAVP_y)

                # Add 90° to pitch for looking froward
                final_pitch = 90+pitch

                # Limit the final position to a maximum / minimum of 150mm
                final_z = min(max(final_z, -150), 150)
                final_x = min(max(final_x, -150), 150)
                final_y = min(max(final_y, -150), 150)
                

                print(f"Final position: {roll}, {final_pitch}, {yaw}")
                # Send the command to the xArm (Using servo_cartesian for smooth live movements)
                code = arm.set_servo_cartesian(
                        [final_z, final_x, final_y, roll, final_pitch, yaw],                        
                        speed=SPEED,
                        mvacc=ACCELERATION,
                        wait=False, is_radian=False, is_tool_coord=False
                    )
                
                if code != 0:
                    print(f"Error moving arm, code: {code}")
                    return

SWIFT :

                let toDeviceTransform = pose.originFromAnchorTransform
                let devicePosition = toDeviceTransform.translation * 1000 // Convert meters to mm
                let deviceRotation = toDeviceTransform.upper3x3
                
                let orientation = rotationMatrixToEulerAngles(matrix: deviceRotation)
                print(orientation)
                if (accumulatedTime >= 0.05)
                {
                    let dataToSend : [Float] =
                    [
                        devicePosition.x,
                        devicePosition.y,
                        devicePosition.z,
                        orientation.pitch,
                        orientation.roll,
                        orientation.yaw,
                        0.0
                    ]
                    udpSender.send(values: dataToSend)
                    
                    accumulatedTime -= 0.05
                }

    func rotationMatrixToEulerAngles(matrix: simd_float3x3) -> (yaw: Float, pitch: Float, roll: Float) {
        let r00 = matrix[0, 0]
        let r10 = matrix[1, 0]
        let r20 = matrix[2, 0]
        let r21 = matrix[2, 1]
        let r22 = matrix[2, 2]
        
        let pitch = asin(-r20)
        let yaw = atan2(r10, r00)
        let roll = atan2(r21, r22)
        
        // Convert to degrees
        let pitchDeg = pitch * (180.0 / .pi)
        let yawDeg = yaw * (180.0 / .pi)
        let rollDeg = roll * (180.0 / .pi)
        
        return (yawDeg, pitchDeg, rollDeg)
    }

Nice project!
What’s the error prompt says on the Studio software?
And could you please reproduce the issue and then download the log by UFACTORY Studio, and then share a log to me by Email? My email is daniel.wang@ufactory.cc

Thanks for your answer.

In the video, for simplifying, I’ve used the Apple Vision Pro simulator, but in real use-case, I use the Apple Vision Pro HMD for moving the xArm, where I have placed a camera.

In the video the error is just the C23, but this error come after the weird rotation I got at the end.

Did you have any idea why, when the robotic arm head reach approximately 90° on roll and 90° on pitch simultaneously, the xarm become crazy and the rotation fail (like at the end of the video?)

I can share you the log, but the errors are mainly C23 or collision detection caused but the same reaction.

You will get gimbal lock while the pitch=±90°, check the details here

How about try the “set_servo_cartesian_aa()”, compare withe “set_servo_cartesian”, it controls rotation with the axis angle instead of the Roll/Pitch/Yaw.

Thank you for your answer.

I’ve tried with “set_servo_cartesian_aa()” instead, but the issue is, by the fact I force the 90° on the axe up/down for looking forward and not on the roof by default, if I turn 90* on the left/right, the arm look 45° down like you can see in the following video.

# Add 90° to pitch for looking froward
final_pitch = 90+pitch 
code = arm.set_servo_cartesian_aa(axis_angle_pose=[final_z, final_x, final_y, roll, final_pitch, yaw],                        
                        speed=SPEED,
                        mvacc=ACCELERATION,
                        wait=False, is_radian=False, is_tool_coord=False
                    )
    func rotationMatrixToAxisAngle(matrix: simd_float3x3) -> (rx: Float, ry: Float, rz: Float) {
        // convert to quaternion
        let q = simd_quatf(matrix)
        
        // quaternion to axis-angle
        let angle = 2 * acos(q.real)
        
        // Handle small angle case to avoid division by zero
        if angle < 1e-10 {
            // For very small angles, return zero rotation
            return (0.0, 0.0, 0.0)
        }
        
        // Extract axis components
        let factor = sin(angle / 2)
        let axis_x = q.imag.x / factor
        let axis_y = q.imag.y / factor
        let axis_z = q.imag.z / factor
        
        // Combine axis and angle to get axis-angle representation
        let rx = axis_x * angle
        let ry = axis_y * angle
        let rz = axis_z * angle
        
        // Convert to degrees 
        let rxDeg = rx * (180.0 / .pi)
        let ryDeg = ry * (180.0 / .pi)
        let rzDeg = rz * (180.0 / .pi)
        
        return (rxDeg, ryDeg, rzDeg)
    }

Looking at your link, I’ve tried (for the first time) “set_tool_position”, but the issue this function is only relative position, not good for live remote controlling.

How about try the rotate on UFACTORY Studio first.

  1. Set the orientation control and display to axis angle
  2. Set the live control panel to “Tool”, and then adjust the Rx and Ry to see if the rotation is what you need.

Here the values for arms if I look straight 90° to the left :

And here my values from my HMD :

Any idea if both have a mathematical relation ?

No. Where are the HMD data from, the IMU in the camera?

Please note the Rx-Ry-Rz is the tool coordinate orientation with respect to the base coordinate in Axis-
Angle representation. Compared to Roll-Pitch-Yaw(RPY) angles, sometimes it is not intuitive for the
axis-angle to express an absolute orientation status, because it is the 3D axis unit vector multiply by a
rotation angle.


For example, the above 2 representations are actually the same orientation. In the first representation, it
is not easy to figure out which is the single rotation axis that can achieve this orientation. However, it
can be intuitive to use axis-angle to represent a small relative orientation change, since it only involves
one angle in the known rotation direction.
Since you are controlling orientation in real-time, you may use rotation matrix to calculate the target
orientation. First calculate current tool rotation matrix (R) from R/P/Y or Axis-angle representation readable from Studio or our API. Say for example, you would like the tool to rotate around the Z axis (base coordinate) to rotate (theta) angles. The target orientation matrix would be calculated as:

Then you may convert this target rotation matrix back to target RPY or Axis angle, and use
set_servo_cartesian() or set_servo_cartesian_aa() to control the robot accordingly.

If relative calculation is not available for you. Then the correct tool angle-axis coordinate (relative to the robot base) has to be calculated from your headset IMU data. RPY may have discontinuity around gimbal lock.

For more details, refer to Basic 3D Rotations on Wikipedia.