Add camera projection sensor.
BEGIN_PUBLIC Add camera projection sensor. END_PUBLIC PiperOrigin-RevId: 564726114 Change-Id: I33b8e5562eff29c21538bf8f38ce3116c7dfc5a9
This commit is contained in:
Alessio Quaglino
committed by
Copybara-Service
Copybara-Service
parent
ccda87aafa
commit
8064ad59c8
+42
-1
@@ -3054,6 +3054,13 @@ and the +Y axis points up. Thus the frame position and orientation are the key a
|
||||
Vertical field of view of the camera, expressed in degrees regardless of the global angle setting. The horizontal
|
||||
field of view is computed automatically given the window size and the vertical field of view.
|
||||
|
||||
.. _body-camera-resolution:
|
||||
|
||||
:at:`resolution`: :at-val:`int(2), "1 1"`
|
||||
Resolution of the camera in pixels [width height]. Note that these values are not used for rendering since those
|
||||
dimensions are determined by the size of the rendering context. This attribute serves as a convenient
|
||||
location to save the required resolution when creating a context.
|
||||
|
||||
.. _body-camera-ipd:
|
||||
|
||||
:at:`ipd`: :at-val:`real, "0.068"`
|
||||
@@ -5412,7 +5419,6 @@ site frame. The output is a 3D vector.
|
||||
:at:`site`: :at-val:`string, required`
|
||||
The site where the sensor is attached.
|
||||
|
||||
|
||||
.. _sensor-rangefinder:
|
||||
|
||||
:el-prefix:`sensor/` |-| **rangefinder** (*)
|
||||
@@ -5441,6 +5447,39 @@ excluded; this is because sensor calculations are independent of the visualizer.
|
||||
:at:`site`: :at-val:`string, required`
|
||||
The site where the sensor is attached.
|
||||
|
||||
.. _sensor-camprojection:
|
||||
|
||||
:el-prefix:`sensor/` |-| **camprojection** (*)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
This element creates a camprojection sensor, which returns the location of a target site, projected onto a camera image
|
||||
in pixel coordinates. The origin of this system is located at the center of the top left pixel, so a target which
|
||||
projects exactly onto the corner of the image, will have value (-0.5, -0.5). Values are not clipped, so targets which
|
||||
fall outside the camera image will take values above or below the pixel limits. Moreover, points behind the camera
|
||||
are also projected onto the image, so it is up to the user to filter out such points, if desired. This can be done using
|
||||
a `framepos<sensor-framepos>` sensor with the camera as reference frame, then a negative/positive value in the
|
||||
z-coordinate indicates (respectively) a location in the front/back of the camera.
|
||||
|
||||
.. _sensor-camprojection-site:
|
||||
|
||||
:at:`site`: :at-val:`string, required`
|
||||
The site which is projected on to the camera image.
|
||||
|
||||
.. _sensor-camprojection-camera:
|
||||
|
||||
:at:`camera`: :at-val:`string, required`
|
||||
The camera used for the projection, its :ref:`resolution<body-camera-resolution>` attribute must be positive.
|
||||
|
||||
.. _sensor-camprojection-name:
|
||||
|
||||
.. _sensor-camprojection-noise:
|
||||
|
||||
.. _sensor-camprojection-cutoff:
|
||||
|
||||
.. _sensor-camprojection-user:
|
||||
|
||||
:at:`name`, :at:`noise`, :at:`cutoff`, :at:`user`
|
||||
See :ref:`CSensor`.
|
||||
|
||||
.. _sensor-jointpos:
|
||||
|
||||
@@ -6655,6 +6694,8 @@ if omitted.
|
||||
|
||||
.. _default-camera-fovy:
|
||||
|
||||
.. _default-camera-resolution:
|
||||
|
||||
.. _default-camera-ipd:
|
||||
|
||||
.. _default-camera-pos:
|
||||
|
||||
+14
-5
@@ -364,11 +364,11 @@
|
||||
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
|
||||
| | | | :ref:`name<body-camera-name>` | :ref:`class<body-camera-class>` | :ref:`fovy<body-camera-fovy>` | :ref:`ipd<body-camera-ipd>` | |
|
||||
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
|
||||
| | | | :ref:`pos<body-camera-pos>` | :ref:`quat<body-camera-quat>` | :ref:`axisangle<body-camera-axisangle>` | :ref:`xyaxes<body-camera-xyaxes>` | |
|
||||
| | | | :ref:`resolution<body-camera-resolution>` | :ref:`pos<body-camera-pos>` | :ref:`quat<body-camera-quat>` | :ref:`axisangle<body-camera-axisangle>` | |
|
||||
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
|
||||
| | | | :ref:`zaxis<body-camera-zaxis>` | :ref:`euler<body-camera-euler>` | :ref:`mode<body-camera-mode>` | :ref:`target<body-camera-target>` | |
|
||||
| | | | :ref:`xyaxes<body-camera-xyaxes>` | :ref:`zaxis<body-camera-zaxis>` | :ref:`euler<body-camera-euler>` | :ref:`mode<body-camera-mode>` | |
|
||||
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
|
||||
| | | | :ref:`user<body-camera-user>` | | | | |
|
||||
| | | | :ref:`target<body-camera-target>` | :ref:`user<body-camera-user>` | | | |
|
||||
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
|
||||
+------------------------------------+----+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| |_| body |br| |_| |L| | | .. table:: |
|
||||
@@ -835,6 +835,15 @@
|
||||
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
|
||||
+------------------------------------+----+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| |_| sensor |br| |_| |L| | | .. table:: |
|
||||
| :ref:`camprojection | \* | :class: mjcf-attributes |
|
||||
| <sensor-camprojection>` | | |
|
||||
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
|
||||
| | | | :ref:`name<sensor-camprojection-name>` | :ref:`site<sensor-camprojection-site>` | :ref:`camera<sensor-camprojection-camera>` | :ref:`cutoff<sensor-camprojection-cutoff>` | |
|
||||
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
|
||||
| | | | :ref:`noise<sensor-camprojection-noise>` | :ref:`user<sensor-camprojection-user>` | | | |
|
||||
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
|
||||
+------------------------------------+----+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| |_| sensor |br| |_| |L| | | .. table:: |
|
||||
| :ref:`rangefinder | \* | :class: mjcf-attributes |
|
||||
| <sensor-rangefinder>` | | |
|
||||
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
|
||||
@@ -1223,9 +1232,9 @@
|
||||
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
|
||||
| | | | :ref:`fovy<default-camera-fovy>` | :ref:`ipd<default-camera-ipd>` | :ref:`pos<default-camera-pos>` | :ref:`quat<default-camera-quat>` | |
|
||||
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
|
||||
| | | | :ref:`axisangle<default-camera-axisangle>` | :ref:`xyaxes<default-camera-xyaxes>` | :ref:`zaxis<default-camera-zaxis>` | :ref:`euler<default-camera-euler>` | |
|
||||
| | | | :ref:`resolution<default-camera-resolution>` | :ref:`axisangle<default-camera-axisangle>` | :ref:`xyaxes<default-camera-xyaxes>` | :ref:`zaxis<default-camera-zaxis>` | |
|
||||
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
|
||||
| | | | :ref:`mode<default-camera-mode>` | :ref:`user<default-camera-user>` | | | |
|
||||
| | | | :ref:`euler<default-camera-euler>` | :ref:`mode<default-camera-mode>` | :ref:`user<default-camera-user>` | | |
|
||||
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
|
||||
+------------------------------------+----+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
|
||||
| |_| default |br| |_| |L| | | .. table:: |
|
||||
|
||||
+6
-2
@@ -65,16 +65,20 @@ General
|
||||
12. Added :ref:`ls_iterations<option-ls_iterations>` and :ref:`ls_tolerance<option-ls_tolerance>` options for adjusting
|
||||
linesearch stopping criteria in CG and Newton solvers. This can be useful for performance tuning.
|
||||
13. Added ``mesh_pos`` and ``mesh_quat`` fields to :ref:`mjModel` to store normalizing transformation.
|
||||
14. Added camera :ref:`resolution<body-camera-resolution>` attribute and :ref:`camprojection<sensor-camprojection>`
|
||||
sensor. If camera resolution is set to positive values, the camera projection sensor will report the location of a
|
||||
target site, projected onto the camera image, in pixel coordinates.
|
||||
|
||||
Python bindings
|
||||
^^^^^^^^^^^^^^^
|
||||
|
||||
14. Fixed `#870 <https://github.com/google-deepmind/mujoco/issues/870>`__ where calling ``update_scene`` with an invalid
|
||||
15. Fixed `#870 <https://github.com/google-deepmind/mujoco/issues/870>`__ where calling ``update_scene`` with an invalid
|
||||
camera name used the default camera.
|
||||
|
||||
Bug fixes
|
||||
^^^^^^^^^
|
||||
|
||||
15. Fixed a bug that was causing the geom margins to be ignored during the midphase.
|
||||
16. Fixed a bug that was causing the geom margins to be ignored during the midphase.
|
||||
|
||||
|
||||
Version 2.3.7 (July 20, 2023)
|
||||
|
||||
@@ -577,6 +577,7 @@ typedef enum mjtSensor_ { // type of sensor
|
||||
mjSENS_TORQUE, // 3D torque between site's body and its parent body
|
||||
mjSENS_MAGNETOMETER, // 3D magnetometer
|
||||
mjSENS_RANGEFINDER, // scalar distance to nearest geom or site along z-axis
|
||||
mjSENS_CAMPROJECTION, // pixel coordinates of a site in the camera image
|
||||
|
||||
// sensors related to scalar joints, tendons, actuators
|
||||
mjSENS_JOINTPOS, // scalar joint position (hinge and slide only)
|
||||
@@ -1010,6 +1011,7 @@ struct mjModel_ {
|
||||
mjtNum* cam_poscom0; // global position rel. to sub-com in qpos0 (ncam x 3)
|
||||
mjtNum* cam_pos0; // global position rel. to body in qpos0 (ncam x 3)
|
||||
mjtNum* cam_mat0; // global orientation in qpos0 (ncam x 9)
|
||||
int* cam_resolution; // [width, height] in pixels (ncam x 2)
|
||||
mjtNum* cam_fovy; // y-field of view (deg) (ncam x 1)
|
||||
mjtNum* cam_ipd; // inter-pupilary distance (ncam x 1)
|
||||
mjtNum* cam_user; // user data (ncam x nuser_cam)
|
||||
|
||||
Reference in New Issue
Block a user