chenlin/Mujoco_WASM
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Specify camera parameters using standard robotics conventions.

Browse Source 
PiperOrigin-RevId: 569147555
Change-Id: I3443f95e5de56a17024f5e5c03f33888dd583edb
This commit is contained in:
Alessio Quaglino
2023-09-28 05:18:36 -07:00
committed by Copybara-Service
parent ca60de79c8
commit 36d2ffe4f2
28 changed files with 528 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/XMLreference.rst
+45 -3
View File
@@ -3053,6 +3053,35 @@ and the +Y axis points up. Thus the frame position and orientation are the key a
dimensions are determined by the size of the rendering context. This attribute serves as a convenient
location to save the required resolution when creating a context.
.. _body-camera-focal:
:at:`focal`: :at-val:`real(2), "0 0"`
Focal length of the camera in length units. It is mutually exclusive with :ref:`fovy <body-camera-fovy>`.
See :ref:`CCamera` for details.
.. _body-camera-focalpixel:
:at:`focalpixel`: :at-val:`int(2), "0 0"`
Focal length of the camera in pixel units. If both :at:`focal`: and :at:`focalpixel`: are specified, the former is
ignored.
.. _body-camera-principal:
:at:`principal`: :at-val:`real(2), "0 0"`
Principal point of the camera in length units. It is mutually exclusive with :ref:`fovy <body-camera-fovy>`.
.. _body-camera-principalpixel:
:at:`principalpixel`: :at-val:`real(2), "0 0"`
Principal point of the camera in pixel units. If both :at:`principal`: and :at:`principalpixel`: are specified, the
former is ignored.
.. _body-camera-sensorsize:
:at:`sensorsize`: :at-val:`real(2), "0 0"`
Size of the camera sensor in length units. It is mutually exclusive with :ref:`fovy <body-camera-fovy>`. If
specified, :ref:`resolution <body-camera-resolution>` and :ref:`focal <body-camera-focal>` are required.
.. _body-camera-ipd:
:at:`ipd`: :at-val:`real, "0.068"`
@@ -6700,8 +6729,6 @@ if omitted.
| All site attributes are available here except: name, class.
.. _default-camera:
.. _default-camera-fovy:
.. _default-camera-resolution:
@@ -6724,11 +6751,26 @@ if omitted.
.. _default-camera-user:
.. _default-camera-resolution:
.. _default-camera-focal:
.. _default-camera-focalpixel:
.. _default-camera-principal:
.. _default-camera-principalpixel:
.. _default-camera-sensorsize:
.. _default-camera:
:el-prefix:`default/` |-| **camera** (?)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| This element sets the attributes of the dummy :ref:`camera <body-camera>` element of the defaults class.
| All camera attributes are available here except: name, class.
| All camera attributes are available here except: name, class, mode, target.
.. _default-light:
doc/XMLschema.rst
+8 -4
View File
@@ -354,7 +354,9 @@
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
| | | | :ref:`xyaxes<body-camera-xyaxes>` | :ref:`zaxis<body-camera-zaxis>` | :ref:`euler<body-camera-euler>` | :ref:`mode<body-camera-mode>` | |
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
| | | | :ref:`target<body-camera-target>` | :ref:`user<body-camera-user>` | | | |
| | | | :ref:`target<body-camera-target>` | :ref:`focal<body-camera-focal>` | :ref:`focalpixel<body-camera-focalpixel>` | :ref:`principal<body-camera-principal>` | |
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
| | | | :ref:`principalpixel<body-camera-principalpixel>` | :ref:`sensorsize<body-camera-sensorsize>` | :ref:`user<body-camera-user>` | | |
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
+------------------------------------+----+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| |_| body |br| |_| |L| | | .. table:: |
@@ -1230,11 +1232,13 @@
| :ref:`camera | ? | :class: mjcf-attributes |
| <default-camera>` | | |
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
| | | | :ref:`fovy<default-camera-fovy>` | :ref:`ipd<default-camera-ipd>` | :ref:`pos<default-camera-pos>` | :ref:`quat<default-camera-quat>` | |
| | | | :ref:`fovy<default-camera-fovy>` | :ref:`ipd<default-camera-ipd>` | :ref:`resolution<default-camera-resolution>` | :ref:`pos<default-camera-pos>` | |
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
| | | | :ref:`resolution<default-camera-resolution>` | :ref:`axisangle<default-camera-axisangle>` | :ref:`xyaxes<default-camera-xyaxes>` | :ref:`zaxis<default-camera-zaxis>` | |
| | | | :ref:`quat<default-camera-quat>` | :ref:`axisangle<default-camera-axisangle>` | :ref:`xyaxes<default-camera-xyaxes>` | :ref:`zaxis<default-camera-zaxis>` | |
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
| | | | :ref:`euler<default-camera-euler>` | :ref:`mode<default-camera-mode>` | :ref:`user<default-camera-user>` | | |
| | | | :ref:`euler<default-camera-euler>` | :ref:`mode<default-camera-mode>` | :ref:`focal<default-camera-focal>` | :ref:`focalpixel<default-camera-focalpixel>` | |
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
| | | | :ref:`principal<default-camera-principal>` | :ref:`principalpixel<default-camera-principalpixel>` | :ref:`sensorsize<default-camera-sensorsize>` | :ref:`user<default-camera-user>` | |
| | | +-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+-----------------------------------------------------------------+ |
+------------------------------------+----+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| |_| default |br| |_| |L| | | .. table:: |
doc/changelog.rst
+11 -2
View File
@@ -106,17 +106,26 @@ General
19. Added camera :ref:`resolution<body-camera-resolution>` attribute and :ref:`camprojection<sensor-camprojection>`
sensor. If camera resolution is set to positive values, the camera projection sensor will report the location of a
target site, projected onto the camera image, in pixel coordinates.
20. Added :ref:`camera<body-camera>` calibration attributes:
- The new attributes are :ref:`resolution<body-camera-resolution>`, :ref:`focal<body-camera-focal>`,
:ref:`focalpixel<body-camera-focalpixel>`, :ref:`principal<body-camera-principal>`,
:ref:`principalpixel<body-camera-principalpixel>` and :ref:`sensorsize<body-camera-sensorsize>`.
- Visualize the calibrated frustum using the :ref:`mjVIS_CAMERA<mjtVisFlag>` visualization flag when these
attributes are specified. See the following
`example model <https://github.com/deepmind/mujoco/blob/main/test/engine/testdata/vis_visualize/frustum.xml>`__.
- Note that these attributes only take effect for offline rendering and do not affect interactive visualisation.
Python bindings
^^^^^^^^^^^^^^^
20. Fixed `#870 <https://github.com/google-deepmind/mujoco/issues/870>`__ where calling ``update_scene`` with an invalid
21. Fixed `#870 <https://github.com/google-deepmind/mujoco/issues/870>`__ where calling ``update_scene`` with an invalid
camera name used the default camera.
Bug fixes
^^^^^^^^^
21. Fixed a bug that was causing :ref:`geom margin<body-geom-margin>` to be ignored during the construction of
22. Fixed a bug that was causing :ref:`geom margin<body-geom-margin>` to be ignored during the construction of
midphase collision trees.
doc/includes/references.h
+5
View File
@@ -440,6 +440,7 @@ typedef enum mjtGeom_ { // type of geometric shape
mjGEOM_LINE, // line
mjGEOM_SKIN, // skin
mjGEOM_LABEL, // text label
mjGEOM_TRIANGLE, // triangle connecting a frame
mjGEOM_NONE = 1001 // missing geom type
} mjtGeom;
@@ -1010,6 +1011,8 @@ struct mjModel_ {
mjtNum* cam_mat0; // global orientation in qpos0 (ncam x 9)
int* cam_resolution; // [width, height] in pixels (ncam x 2)
mjtNum* cam_fovy; // y-field of view (deg) (ncam x 1)
float* cam_intrinsic; // [focal length; principal point] (ncam x 4)
float* cam_sensorsize; // sensor size (ncam x 2)
mjtNum* cam_ipd; // inter-pupilary distance (ncam x 1)
mjtNum* cam_user; // user data (ncam x nuser_cam)
@@ -1822,6 +1825,7 @@ struct mjvGLCamera_ { // OpenGL camera
// camera projection
float frustum_center; // hor. center (left,right set to match aspect)
float frustum_width; // width (not used for rendering)
float frustum_bottom; // bottom
float frustum_top; // top
float frustum_near; // near
@@ -2053,6 +2057,7 @@ struct mjvSceneState_ {
mjtNum* cam_fovy;
mjtNum* cam_ipd;
float* cam_sensorsize;
mjtByte* light_directional;
mjtByte* light_castshadow;
doc/modeling.rst
+25
View File
@@ -973,6 +973,31 @@ Here we describe the XML attributes common to all sensor types, so as to avoid r
:at:`user`: :at-val:`real(nuser_sensor), "0 0 ..."`
See :ref:`User parameters <CUser>`.
.. _CCamera:
Cameras
~~~~~~~
Besides the default, user-controllable, free camera, "fixed" cameras can be attached to the kinematic tree.
Extrinsics
By default, camera frames are attached to the parent body. The optional :ref:`mode<body-camera-mode>` and
:ref:`target<body-camera-target>` attributes can be used to specify camera that track (move with) or target (look at)
a body or subtree. Cameras look towards the negative Z axis of the camera frame, while positive X and Y correspond to
*right* and *up* in the image plane, respectively.
Intrinsics
Camera intrinsics are specified using :ref:`ipd<body-camera-ipd>` (inter-pupilary distance, required for
stereoscopic rendering and VR) and :ref:`fovy<body-camera-fovy>` (vertical field of view, in degrees).
The above specification implies a perfect point camera with no aberrations. However when calibrating real cameras,
two types of linear aberration can be expressed using standard rendering pipelines. The first is different focal
lengths in the vertical and horizontal directions (axis-aligned astigmatism). The second is a non-centered principal
point. These can be specified using the :ref:`focal<body-camera-focal>` and :ref:`principal<body-camera-principal>`
attributes. When these calibration-related attributes are used, the physical
:ref:`sensor size<body-camera-sensorsize>` and camera :ref:`resolution<body-camera-resolution>` must also be
specified. In this case, the rendering frustum can be visualized.
.. _CComposite:
Composite objects
doc/overview.rst
+4 -2
View File
@@ -515,8 +515,10 @@ Camera
Multiple cameras can be defined in a model. There is always a default camera which the user can freely move with the
mouse in the interactive visualizer. However it is often convenient to define additional cameras that are either
fixed to the world, or are attached to one of the bodies and move with it. In addition to the camera position and
orientation, the user can adjust the field of view and the inter-pupilary distance for stereoscopic rendering, as
well as create oblique projections needed for stereoscopic virtual environments.
orientation, the user can adjust the vertical field of view and the inter-pupilary distance for stereoscopic rendering,
as well as create oblique projections needed for stereoscopic virtual environments. When modeling real cameras with
imperfect optics, it is possible to specify separate focal lengths for the horizontal and vertical directions and a
non-centered principal point.
Light
^^^^^