NVIDIA/IsaacTeleop v0.3.15

Release 0.3.15

Repository: NVIDIA/IsaacTeleop

Tag: v0.3.15

Published: 2026-02-10T23:34:04Z

Prerelease: no

Release notes:

Isaac Teleop (TeleopCore) Release Notes

Release 0.3.15 (vs 0.2)

*Release branch: release/0.3.x*

---

Highlights

• Project rename: TeleopCore is now Isaac Teleop — the unified framework for high-fidelity ego-centric and robotics data collection. Documentation and branding have been updated accordingly.
• Retargeting framework: Port of Isaac Lab retargeters with a type system, source nodes, and examples (gripper, SE3, dex bimanual, G1 tri-hand, locomotion, reordering). Retargeter tuning UI with IMGUI for live tuning.
• Camera & recording: Generic Camera Device Plugin interface; Oak-D plugin; MCAP recorder integrated into deviceio session; generic per-frame metadata in camera schema.
• Full body & locomotion: Pico full body tracking schema and testbed; locomotion schema and bindings compatible with Isaac OS Lower Body Controller.
• CloudXR: Runtime updates to 6.1.0 (pid3/pid4/rc2); tensor extension support; streamlined Docker setup and env handling; Quest3 as default device profile.
• Python & packaging: Python 3.10, 3.11, and 3.12 support; PEP 440 versioning; multi-version wheel builds in CI; git commit count in versioning.

---

New Features

• Retargeting
• Retargeter type system and interface; port of Isaac Lab retargeters (SE3, gripper, dex, combined).
• Source nodes and enums for source node identifiers.
• Examples: controller + SE3, gripper, dex bimanual, G1 tri-hand, locomotion, full bimanual reordering.
• Retargeter tuning UI (IMGUI) for live tuning.
• Optional retargeting dependencies in build.

• Camera & device plugins
• Generic Camera Device Plugin interface; Oak-D camera plugin (DepthAI) for recording.
• MCAP recorder integrated into deviceio_session with configurable recording path; Python example for session with MCAP recording.
• Generic per-frame metadata in camera.fbs (#88).
• hands.fbs for tracking both hands to align with hand tracker logic.

• Full body & locomotion
• Data schema and bindings for Pico full body tracking.
• Body tracking testbed for Pico WebXR body tracking extension (cxrjs/bodytracking example).
• Locomotion schema and bindings compatible with Isaac OS Lower Body Controller operator.

• Session & OpenXR
• Create OpenXRSessionHandles from Python (#81).
• Teleop session manager package: helpers, config, and teleop_session_example.
• Shared core::OpenXRSession reuse across plugins; OpenXR math helpers moved to oxr_utils.
• CXR tensor extension headers (Generic Tensor Collection / push tensor) for plugins.

• Device I/O & schemas
• Device output for Generic3AxisPedalOutput (e.g. Logitech-style pedals).
• Timestamps on device data; controller tracker uses FlatBuffer schema.
• Tensor and TensorGroup classes with type validation and tests; tensor types for retargeting engine I/O.

• Build & developer experience
• Devcontainer configuration for Isaac Teleop (#66).
• Teleop utils for building teleop apps.
• TELEOPCORE_DISABLE_CXR_ENV_CHECKS to disable CloudXR env checks when needed.

---

Improvements

• DeviceIO & trackers
• Tracker creation refactor: initialize → create_tracker; implementations retrieved from session; removed is_initialized() from tracker interfaces (#33, #34).
• Device plugin reuses controller from deviceio; rate-limited warnings in DeviceIOSession to reduce log spam.
• deviceio C++ files renamed to snake_case for consistency.

• CloudXR
• CloudXR env setup: fail early on missing env; use NV_CXR_RUNTIME_DIR (replacing XDG_RUNTIME_DIR where appropriate); ensure OpenXR env vars (XR_RUNTIME_JSON, etc.) are set in oxr_session.
• CloudXR Docker: run_cloudxr.sh can download Web SDK from NGC, build containers, and start services; cloudxr_image_ops.sh for flexible image load/save/pull with container_tag.
• Migrated CloudXR JS SDK download from manual PID to NGC CLI; one-time NGC setup documented.
• README improvements for CXR Runtime setup; HAProxy Dockerfile tidied (permissions).

• Build & CI
• Hunter cache (.hunter) to improve build speed for OAKD dependencies (#85).
• ccache (Ubuntu) and sccache (Windows) support (#29).
• CMake cleanup: redundant checks removed, inc aligned with target names, deviceio_py_utils library (#40).
• DepthAI/CMake: policy version set for compatibility with CMake ≥ 4.0 (#60).
• Ubuntu CI uses Ubuntu-22.04; GPU runner for GitHub Actions; native tests run in CI.
• Publish-wheel job for tagged releases (#70); multi-Python wheel builds (#89).
• Docker-based test fixture for CloudXR tests (local and CI); run_tests_with_cloudxr.sh improvements and documentation.

• Versioning & deps
• Single consolidated VERSION file; version includes git commit count (#83); bump to 0.3.x.
• PEP 440 versioning for alpha, rc, dev, local, and final releases (#87).
• NumPy 1.22+ (aligned across docs); optional update to 2.4.0 in examples.
• get_home_dir improved (Windows: USERPROFILE; runtime error instead of /tmp fallback).

• Security & supply chain
• npm install runs with --ignore-scripts to reduce supply chain risk (#9da1d58).
• HAProxy config ownership set to root to address security hotspot.

• Oak-D / camera
• Default size switched to match camera resolution to avoid unnecessary scaling and preserve FOV (#74).

• LeRobot
• Recorder fixes; dataset written relative to recorder.py location.

• Misc
• OpenXR SDK compatibility via sanitizers-cmake (#43).
• Removed internal references (#72); README and doc cleanups.
• Issue templates updated; unnecessary README sections removed.
• UV: specify Python version in Dockerfile.test; remove uv.lock files from repo.

---

Bug Fixes

• Use get_required_extensions correctly for OpenXR.
• Fix typo in load_cloudxr_images.sh.
• Remove duplicate file and ctor definitions from OpenXRSessionHandles; add default initializers (#67).
• LeRobot recorder fixes and path handling.

---

API / Behavior Changes

• Trackers: initialize() replaced by create_tracker(); is_initialized() removed from tracker interfaces and implementations.
• Versioning: VERSION file consolidated; format follows PEP 440; git commit count used in version string.
• CloudXR: Prefer NV_CXR_RUNTIME_DIR; stricter env checks unless disabled via TELEOPCORE_DISABLE_CXR_ENV_CHECKS.
• Plugin utils: Controllers…