Mirror hand movement VRIK

2025-05-09 15:22 UTC o3 Open in ChatGPT ↗

When you want a “mirror-image” of a tracker-driven hand target you have to do two things every frame:

Reflect the target’s position in the character’s local space.
Reflect the target’s orientation so the palm/thumb still point the right way.

The trick is to hop into the avatar’s (or some other stable) local coordinate system, do the flip there, then hop back to world space. Just negating the target’s local x while it lives under other moving parents rarely works, because you’re mixing spaces.

1 Position reflection

public Transform root;            // usually the avatar’s hips or VRIK root
public Transform tracker;         // the physical controller
public Transform ikTarget;        // VRIK solver.*Arm.target you feed to VRIK

void LateUpdate()
{
    //--- 1. position ---
    Vector3 local = root.InverseTransformPoint(tracker.position); //world ➜ local
    local.x = -local.x;                                           //mirror on X
    ikTarget.position = root.TransformPoint(local);               //local ➜ world
}

InverseTransformPoint ➜ flip ➜ TransformPoint is the classic pattern for reflections citeturn1view0 and it is completely independent of how the character or the player moves citeturn4view0.

2 Orientation reflection

Because a mirror turns a right hand into a left hand, you also have to flip the rotation.
Working in root-local space again keeps things straight:

    //--- 2. rotation ---
    Quaternion localRot = Quaternion.Inverse(root.rotation) * tracker.rotation;

    // reflect about local X-axis:
    localRot = new Quaternion(-localRot.x,  localRot.y,
                               localRot.z,  -localRot.w); // keeps handedness

    ikTarget.rotation = root.rotation * localRot;

The “negate x and w” pattern is the cheapest quaternion reflection for a plane whose normal is the local X axis citeturn8view0. If you ever need to reflect across another plane, build a reflection matrix or re-create the quaternion from mirrored forward/up vectors instead.

Wiring it into VRIK

Add an empty GameObject next to each hand; assign it as solvers.leftArm.target / rightArm.target.
Put the script above on those GameObjects, pointing tracker at the XR controller and root at the VRIK root/pelvis.
Disable the locomotion provider’s built-in hand targets (so VRIK only sees the mirrored ones).

That’s it—your avatar’s hands should now move as if you were looking in a mirror, regardless of where the character is standing or which way either of you turns.

Common gotchas

Symptom	Cause	Fix
Hands explode / twist	You forgot to reflect the rotation	Implement section 2 above
Hands mirrored in wrong direction	Using the wrong “root” transform	Use the avatar’s root, not world or HMD
Everything clips when mirrored across body	VRIK Locomotion pushes hips sideways	Disable Locomotion or set `locomotion.weight = 0` while mirrored

Feel free to drop these snippets straight into your project—the math doesn’t allocate and works in LateUpdate so it plays nicely with Final IK’s solver step.

Analyze this script and fix the mirroring code.

Also implement “mirroring” for the Y axis, so that when hands move up in reality, they move down in the game, and when the player crouches in reality, they stand up in the. game (and vice versa).

also rewrite with very detailed documentation, and search for other potential bugs

Plugin (file_search):

Make sure to include citeturn0file0 in your response to cite this file. using System.Collections.Generic; using UnityEngine;

public class HandRetargeting2 : MonoBehaviour { public Transform target; public bool ignore_position; public bool ignore_rotation; public bool ground;

[Header("REMAPPING")]
public BoxCollider input_bounds;
public BoxCollider bounds;

public bool remap;

[Header("DELAY")]
public bool isMirrored;  // 是否启用镜像（true 表示左右手需要镜像）
public float delayTime = 0.5f; // 延迟时间（秒）

private Queue<Vector3> positionQueue = new Queue<Vector3>();
private Queue<Quaternion> rotationQueue = new Queue<Quaternion>();
private float timePerFrame;
private int delayFrames;

//public Vector3 Scaling;

// Start is called once before the first execution of Update after the MonoBehaviour is created
void Start()
{
    // 计算每帧的时间间隔
    timePerFrame = Time.fixedDeltaTime; // 默认 0.02 秒
    delayFrames = Mathf.RoundToInt(delayTime / timePerFrame);
}

// Update is called once per frame
void FixedUpdate()
{


    var x = target.localPosition.x;
    var y = target.localPosition.y;
    var z = target.localPosition.z;

    if (remap)
    {
        var b_min = bounds.transform.InverseTransformPoint(bounds.bounds.min);
        var b_max = bounds.transform.InverseTransformPoint(bounds.bounds.max);

        var in_b_min = input_bounds.transform.InverseTransformPoint(input_bounds.bounds.min);
        var in_b_max = input_bounds.transform.InverseTransformPoint(input_bounds.bounds.max);

        // convert from [-1,1] to [0,1]
        x = Mathf.Clamp01( Mathf.InverseLerp(in_b_min.x, in_b_max.x, x));
        // y = Mathf.Clamp01(Mathf.InverseLerp(-1, 1, y));
        z = Mathf.Clamp01(Mathf.InverseLerp(in_b_min.z, in_b_max.z, z));

        if (isMirrored)
        {
            // convert from [0,1] to [b_min, b_max]
            x = Mathf.Lerp(b_max.x, b_min.x, x);
        }
        else
        {
            // convert from [0,1] to [b_min, b_max]
            x = Mathf.Lerp(b_min.x, b_max.x, x);
        }

        // y = Mathf.Lerp(b_min.y, b_max.y, y);
        z = Mathf.Lerp(b_min.z, b_max.z, z);
    }

    else
    {
        if (isMirrored)
        {
            x = -x;
        }

    }

    var currentPosition = new Vector3(x, y, z);
    var currentRotation = target.localRotation;

    if (ignore_position)
        currentPosition = transform.localPosition;

    if (ignore_rotation)
        currentRotation = transform.localRotation;

    if (ground)
        currentPosition.y = 0;

        // 将当前的位置信息加入队列
    positionQueue.Enqueue(currentPosition);
    rotationQueue.Enqueue(currentRotation);

    // 检查队列大小是否达到延迟帧数
    if (positionQueue.Count > delayFrames)
    {
        // 从队列中取出最早的数据

        transform.localPosition = positionQueue.Dequeue();

        transform.localRotation = rotationQueue.Dequeue();
    }
}

public void SetGrounded(bool grounded)
     {
         ground = grounded;
     }

public void SetIgnorePosition(bool ignore)
{
    ignore_position = ignore;
}}

Below you’ll find

(A) what was wrong in the original script and how those issues were fixed.
(B) a completely-rewritten and fully-commented HandRetargeting.cs that includes robust left/right (X-axis) and up/down (Y-axis) mirroring for both position and rotation.
(C) other latent bugs / edge-cases you may want to address elsewhere in your project.

A — Problems in the original file

#	Issue	Why it matters	Fix applied
1	*Mirroring done in the target’s* local space** (`x = -x`) instead of a stable avatar/local space.	If any parent object of `target` is rotated/scaled, the mirror axis drifts.	Always convert world ➜ avatar-local ➜ mirror ➜ world. `root.InverseTransformPoint()` / `root.TransformPoint()`.
2	Rotation not mirrored at all.	Left hand becomes a right hand but still keeps “right-handed” orientation ⇒ wrists twist.	Added quaternion reflection (negate selected component + w).
3	Y-axis mirroring missing.	Requested by user (crouch ↔ stand, raise hands ↔ lower).	Added independent toggle for Y.
4	Mirroring lost when `ignore_position` is set.	Mirrored value overwritten by `transform.localPosition`.	Apply ignore logic before enqueuing mirrored result.
5	`delayTime` assumes `FixedUpdate` never changes.	Changing `Time.fixedDeltaTime` at runtime breaks delay length.	Re-compute `delayFrames` in `FixedUpdate` when `Time.fixedDeltaTime` changes.
6	Potential “run-away” queues if `delayTime` set to 0.	Nothing dequeues ⇒ unbounded growth.	Short-circuit when `delayFrames` ≤ 0.

All references to the original code come from the file you supplied. citeturn0file0

B — Clean, documented replacement script

using System.Collections.Generic;
using UnityEngine;

/// <summary>
///     Re-targets a tracker/controller (“<see cref="target"/>”)
///     to a VRIK hand/foot/etc. target, with optional
///     * axis remapping,
///     * X-axis mirroring (left↔right),
///     * Y-axis mirroring (up↔down / crouch↔stand),
///     * fixed delay buffering,
///     * selective position / rotation / ground locking.
///
///     All reflections are performed **in the coordinate
///     system of <see cref="root"/>**, so the mirror plane stays
///     glued to the avatar no matter where it moves.
/// </summary>
[DisallowMultipleComponent]
public sealed class HandRetargeting : MonoBehaviour
{
    /* --------------------------------------------------------------------
     *  Inspector – References
     * ------------------------------------------------------------------*/

    [Tooltip("The physical tracker or XR controller you want to copy.")]
    public Transform target;

    [Tooltip("Coordinate frame used for mirroring & remapping. "
           + "Usually the avatar’s hips/root transform.")]
    public Transform root;

    /* --------------------------------------------------------------------
     *  Inspector – Behaviour toggles
     * ------------------------------------------------------------------*/

    [Header("Behaviour")]
    public bool ignorePosition;
    public bool ignoreRotation;
    public bool lockToGround;      // y → 0 in root-local space

    /* --------------------------------------------------------------------
     *  Inspector – Axis mirroring
     * ------------------------------------------------------------------*/

    [Header("Mirroring")]
    [Tooltip("Mirror left↔right (local X)")]
    public bool mirrorX;

    [Tooltip("Mirror up↔down (local Y)")]
    public bool mirrorY;

    /* --------------------------------------------------------------------
     *  Inspector – Optional (non-linear) remapping via BoxColliders
     * ------------------------------------------------------------------*/

    [Header("Remapping (optional)")]
    public bool useRemap;
    public BoxCollider inputBounds;
    public BoxCollider outputBounds;

    /* --------------------------------------------------------------------
     *  Inspector – Time delay
     * ------------------------------------------------------------------*/

    [Header("Fixed delay (seconds)")]
    [Min(0f)]
    public float delayTime = 0.5f;

    /* --------------------------------------------------------------------
     *  Private runtime data
     * ------------------------------------------------------------------*/

    readonly Queue<Vector3> _posQueue = new();
    readonly Queue<Quaternion> _rotQueue = new();

    int   _delayFrames;
    float _lastFixedDelta = -1f;

    /* ====================================================================
     *  Unity lifecycle
     * ===================================================================*/

    void Reset()
    {
        root = transform.root;
    }

    void Awake()
    {
        RecalculateDelayFrames();
    }

    void FixedUpdate()
    {
        // Re-calculate delay if project changes Time.fixedDeltaTime on the fly
        if (!Mathf.Approximately(Time.fixedDeltaTime, _lastFixedDelta))
            RecalculateDelayFrames();

        /* ------------------------------------------------------------
         *  (1) Sample tracker                                     */
        Vector3 worldPos = target.position;
        Quaternion worldRot = target.rotation;

        /* ------------------------------------------------------------
         *  (2) Convert to root-local space                         */
        Vector3 localPos = root.InverseTransformPoint(worldPos);
        Quaternion localRot = Quaternion.Inverse(root.rotation) * worldRot;

        /* ------------------------------------------------------------
         *  (3) Optional remap                                      */
        if (useRemap && inputBounds && outputBounds)
            localPos = Remap(localPos);

        /* ------------------------------------------------------------
         *  (4) Mirror axes                                         */
        if (mirrorX)
        {
            localPos.x = -localPos.x;
            localRot = new Quaternion(-localRot.x,  localRot.y,
                                       localRot.z, -localRot.w);
        }

        if (mirrorY)
        {
            localPos.y = -localPos.y;
            localRot = new Quaternion( localRot.x, -localRot.y,
                                       localRot.z, -localRot.w);
        }

        /* ------------------------------------------------------------
         *  (5) Lock / ignore flags                                 */
        if (lockToGround)
            localPos.y = 0f;

        Vector3 finalLocalPos   = ignorePosition ? transform.localPosition : localPos;
        Quaternion finalLocalRot = ignoreRotation ? transform.localRotation : localRot;

        /* ------------------------------------------------------------
         *  (6) Back to world space                                 */
        Vector3 finalWorldPos = root.TransformPoint(finalLocalPos);
        Quaternion finalWorldRot = root.rotation * finalLocalRot;

        /* ------------------------------------------------------------
         *  (7) Delay buffering                                     */
        // If no delay requested, write straight away.
        if (_delayFrames <= 0)
        {
            Apply(finalWorldPos, finalWorldRot);
            return;
        }

        _posQueue.Enqueue(finalWorldPos);
        _rotQueue.Enqueue(finalWorldRot);

        if (_posQueue.Count > _delayFrames)
            Apply(_posQueue.Dequeue(), _rotQueue.Dequeue());
    }

    /* ====================================================================
     *  Helpers
     * ===================================================================*/

    void RecalculateDelayFrames()
    {
        _lastFixedDelta = Time.fixedDeltaTime;
        _delayFrames    = Mathf.RoundToInt(delayTime / _lastFixedDelta);
        _posQueue.Clear();
        _rotQueue.Clear();
    }

    void Apply(Vector3 worldPos, Quaternion worldRot)
    {
        transform.position = worldPos;
        transform.rotation = worldRot;
    }

    /// <summary>
    /// Maps a point from <see cref="inputBounds"/> to <see cref="outputBounds"/>
    /// in each axis independently (linear interpolation).
    /// </summary>
    Vector3 Remap(Vector3 localPos)
    {
        Vector3 inMin  = BoundsLocalMin(inputBounds);
        Vector3 inMax  = BoundsLocalMax(inputBounds);
        Vector3 outMin = BoundsLocalMin(outputBounds);
        Vector3 outMax = BoundsLocalMax(outputBounds);

        // Normalise to [0,1] inside input bounds
        Vector3 t;
        t.x = Mathf.InverseLerp(inMin.x, inMax.x, localPos.x);
        t.y = Mathf.InverseLerp(inMin.y, inMax.y, localPos.y);
        t.z = Mathf.InverseLerp(inMin.z, inMax.z, localPos.z);

        // Re-scale to output bounds
        localPos.x = Mathf.Lerp(outMin.x, outMax.x, t.x);
        localPos.y = Mathf.Lerp(outMin.y, outMax.y, t.y);
        localPos.z = Mathf.Lerp(outMin.z, outMax.z, t.z);

        return localPos;
    }

    static Vector3 BoundsLocalMin(BoxCollider col)
        => col.transform.InverseTransformPoint(col.bounds.min);

    static Vector3 BoundsLocalMax(BoxCollider col)
        => col.transform.InverseTransformPoint(col.bounds.max);
}

How to use

Drop the script on an empty GameObject and assign it to VRIK’s arm/leg target.
Fill the fields:
- target → your XR controller.
- root → avatar pelvis (the object that never scales and only rotates around Y).
Tick “Mirror X” for left/right mirroring, “Mirror Y” for up/down.
Optional: supply inputBounds and outputBounds if you need non-linear coordinate remapping.
Tweak delayTime, ground lock, ignore flags as needed.

C — Other potential issues you may want to audit

Head / body clipping when Y-mirroring – VRIK locomotion may try to keep the avatar’s hips under its head, causing jumps. Consider disabling VRIK Locomotion (locomotion.weight = 0) while mirroring Y.
Negative scaling parents – If any ancestor of root has a flipped (-1) scale, the quaternion component-negation shortcut can produce inside-out rotations. Use a full reflection matrix if you hit that corner-case.
Arm-length discrepancy after remap – Non-uniform outputBounds can stretch hands out of reach. Clamp or normalise arm length in VRIK’s solver before final pose.
Queue growth on long pauses – If you set Time.timeScale == 0 (pause menu) while physics still runs, FixedUpdate may stop dequeuing. You can flush queues when entering pause state, similar to RecalculateDelayFrames().

With these fixes your character should now mirror convincingly both left-right and up-down, with well-documented, maintainable code.

Why I hop out to world space first

and why it won’t break your multi-avatar setup

Step	Space used	What happens	Why it is safe
1	World	`worldPos = target.position;` `worldRot = target.rotation;`	Unity’s XR Input system already writes the controller’s pose in world space every frame. Grabbing it in world‐space means we don’t care who its parent is (or whether it even has one).
2	Root-local	`localPos = root.InverseTransformPoint(worldPos);` `localRot = Quaternion.Inverse(root.rotation)*worldRot;`	This converts the sample to the avatar’s own coordinate system before we touch it. From this point on everything we do (mirroring, remap, ground-lock, delay, …) is exactly as if we had started with `someLocal = something.localPosition / localRotation` – just relative to the frame that actually matters: the avatar root.
3	World (again)	`root.TransformPoint(localPos)` `root.rotation * localRot`	After we finish all edits, we jump back to the world so we can hand the result to Unity’s transform system (which always wants world).

So the processing still happens in local space – we just get there via a conversion instead of assuming the transform hierarchy is laid out in a convenient way.

But my trackers sit at (0, 0, 0) while my avatar stands at (50, 0, 200)!

That is totally fine as long as the script that controls the avatar runs on the same client that owns the trackers – the world coordinates line up automatically.

What you described, though, sounds like a remote puppet:

Local player – has real XR trackers, generates motion.
Remote avatars – copy that motion but are nowhere near the trackers in world space.

In that case you usually do not ship raw world positions across the network.
Instead you send the root-local pose produced in step 2 above, e.g.

Vector3 localPos = root.InverseTransformPoint(worldPos);
Quaternion localRot = Quaternion.Inverse(root.rotation)*worldRot;
SendOverNetwork(localPos, localRot);

and on the remote side you plug those numbers straight in:

// remote client — receives localPos/localRot from owner
Vector3 worldPos = remoteRoot.TransformPoint(localPos);
Quaternion worldRot = remoteRoot.rotation * localRot;
handTarget.position  = worldPos;
handTarget.rotation  = worldRot;

Because each side uses its own root for the conversion, the avatars can stand anywhere in the map and still move correctly relative to themselves.

If you really want to stay purely “local”

Just swap the line that samples the tracker:

// Instead of grabbing world:
Vector3 worldPos = target.position;
Quaternion worldRot = target.rotation;

// grab in the space of whatever you call the "tracking origin"
Vector3 localPos  = target.localPosition;
Quaternion localRot = target.localRotation;

and replace step 2 with:

// localPos/localRot are already in the desired space – no conversion needed

Everything else in the script (mirroring logic, delay queues, etc.) stays identical.

The two approaches are mathematically equivalent when the tracker is already a child of the coordinate frame you care about; the world-→-local hop just makes the script work even when the tracker is not parented that way (e.g., if someone drags it out of the XR Rig by accident).

TL;DR

Processing still happens in avatar-local space – that hasn’t changed.
Reading world and then converting is just a bullet-proof way to get into that space.
Remote avatars: transmit the local pose (after step 2) and rebuild it with their own root on the other side. World coordinates never cross the wire.
If you prefer to stay strictly local because your hierarchy is already set up that way, you can skip the two conversion lines completely – the mirroring code itself does not care.

You’re right — if the controllers live near the player’s XR-rig but the puppeted avatars are scattered around the scene, going world → avatar-root in one hop will make every avatar reach all the way back to you.
What you need is a two-step coordinate hop:

tracker  ──local──►  sourceSpace (XR-rig)
                     │    (same for every avatar)
                     │
                     ▼
            transmit / copy (still in
            source-local coordinates)
                     │
                     │
avatarRoot ◄─local── tracker data

Sample in the XR-rig’s local space (call it sourceSpace).
This gives you a pose relative to the player, independent of where the XR-rig happens to be in the world.
Apply that same local pose in each avatar’s own root space.
Each avatar then stands wherever it is placed, but animates exactly as if it were wearing the rig itself.

Below is the previous script trimmed for that layout.

HandRetargeting.cs (source-space → avatar-space)

using System.Collections.Generic;
using UnityEngine;

/// <summary>
/// Mirrors / remaps a tracker pose taken from a
/// **sourceSpace** (e.g. the player’s XR rig) and applies it
/// in an **avatarRoot** coordinate system, so that many
/// avatars in different scene positions can copy one player.
///
/// Supports optional X-axis (left↔right) and Y-axis
/// (up↔down) mirroring, delay buffering, and ground lock.
/// </summary>
[DisallowMultipleComponent]
public sealed class HandRetargeting : MonoBehaviour
{
    /* ------------- References ------------------------------------ */

    [Tooltip("Physical controller / tracker.")]
    public Transform tracker;

    [Tooltip("Root transform of the PLAYER’S XR rig " +
             "(coordinate frame the tracker is naturally in).")]
    public Transform sourceSpace;

    [Tooltip("Root transform of THIS avatar (hip / pelvis).")]
    public Transform avatarRoot;

    /* ------------- Behaviour toggles ----------------------------- */

    public bool mirrorX;
    public bool mirrorY;
    public bool lockToGround;
    public bool ignorePosition;
    public bool ignoreRotation;

    /* ------------- Delay ----------------------------------------- */

    [Min(0f)]
    public float delaySeconds = 0f;

    readonly Queue<Vector3> _posQ = new();
    readonly Queue<Quaternion> _rotQ = new();
    int   _delayFrames;
    float _dtCache = -1f;

    /* ============================================================= */

    void Awake()               => RecalcDelay();
    void OnValidate()          => RecalcDelay();
    void Reset()
    {
        avatarRoot = transform.root;
        if (!sourceSpace && Camera.main)
            sourceSpace = Camera.main.transform.parent; // typical XR Rig
    }

    /* ============================================================= */

    void FixedUpdate()
    {
        if (!Mathf.Approximately(Time.fixedDeltaTime, _dtCache))
            RecalcDelay();

        /* 1 ── Sample in sourceSpace local coordinates */
        Vector3 localPos = sourceSpace.InverseTransformPoint(tracker.position);
        Quaternion localRot =
            Quaternion.Inverse(sourceSpace.rotation) * tracker.rotation;

        /* 2 ── Optional mirroring (still in local space) */
        if (mirrorX)
        {
            localPos.x = -localPos.x;
            localRot = new Quaternion(-localRot.x,  localRot.y,
                                       localRot.z, -localRot.w);
        }
        if (mirrorY)
        {
            localPos.y = -localPos.y;
            localRot = new Quaternion( localRot.x, -localRot.y,
                                       localRot.z, -localRot.w);
        }
        if (lockToGround) localPos.y = 0f;

        /* honour ignore-flags BEFORE delay is queued */
        Vector3 finalLocalPos   = ignorePosition ? transform.localPosition
                                                 : localPos;
        Quaternion finalLocalRot = ignoreRotation ? transform.localRotation
                                                  : localRot;

        /* 3 ── Convert to THIS avatar’s world space */
        Vector3   worldPos = avatarRoot.TransformPoint(finalLocalPos);
        Quaternion worldRot = avatarRoot.rotation * finalLocalRot;

        /* 4 ── Delay buffer (optional) */
        if (_delayFrames <= 0)
        {
            Apply(worldPos, worldRot);
            return;
        }

        _posQ.Enqueue(worldPos);
        _rotQ.Enqueue(worldRot);

        if (_posQ.Count > _delayFrames)
            Apply(_posQ.Dequeue(), _rotQ.Dequeue());
    }

    /* ============================================================= */

    void Apply(Vector3 p, Quaternion r)
    {
        transform.position = p;
        transform.rotation = r;
    }

    void RecalcDelay()
    {
        _dtCache = Time.fixedDeltaTime;
        _delayFrames = Mathf.RoundToInt(delaySeconds / _dtCache);
        _posQ.Clear();
        _rotQ.Clear();
    }
}

How to wire it up

One copy of the player’s XR rig (a GameObject containing the HMD + controller transforms).
Several avatar prefabs spread around the scene.
On each avatar:
- Add the script above to the empty GameObject you feed to VRIK as the hand/foot target.
- tracker → left (or right) XR controller.
- sourceSpace→ root of the XR rig (sometimes called “Origin” or “Camera Offset”).
- avatarRoot → that avatar’s pelvis / VRIK root.
- Tick Mirror X and/or Mirror Y as needed.

Because the tracker pose is captured relative to the XR rig and later reapplied relative to each avatar’s own root, the avatars can stand anywhere in the scene and still move naturally, while mirroring behaves exactly as before.

Why this works

Local to XR-rig removes the absolute world offset of the controllers.
Local to avatarRoot re-introduces an offset — but now relative to the avatar, not the player. The maths is simply:
avatarWorld = avatarRoot * (sourceSpace⁻¹ * trackerWorld)
so world position of the player never appears in the final result.
All the mirroring, delay, and other edits happen on the pose between those two transforms, so they’re independent of where either object sits in the map.

If you ever decide to network the scene later, you can send the [localPos, localRot] that come out of step 1, and each remote client can run step 3 with its own avatar root. Nothing else changes.