Commit Graph

62 Commits

Author SHA1 Message Date
Yuval Tassa a04c2b1b4a Consolidate stack allocation calls in primal solvers down from 30 to 2 (dense) or 6 (sparse). Preparation for atomic allocation calls in threaded mode.
PiperOrigin-RevId: 902496039
Change-Id: I39df011951713505c4743c7475d74856ddbdf4f9
2026-04-20 01:47:30 -07:00
Yuval Tassa a2d0e33c0f 2-3x speedup of sparse matrix squaring.
Split symbolic and numeric phases for sparse `M'*diag*M` computation. Microseconds per call for the monolithic vs the split approach for the 100_humanoids and 2humanoid100 models:

```
+-------+------+----------+------------+---------+
| Model | Arch | Col (µs) | Split (µs) | Speedup |
+-------+------+----------+------------+---------+
| 2H100 | x86  | 238.3    | 74.5       | 3.2x    |
+-------+------+----------+------------+---------+
|       | ARM  | 111.6    | 53.2       | 2.1x    |
+-------+------+----------+------------+---------+
| 100H  | x86  | 1325.3   | 656.2      | 2.0x    |
+-------+------+----------+------------+---------+
|       | ARM  | 594.8    | 306.6      | 1.9x    |
+-------+------+----------+------------+---------+
```

PiperOrigin-RevId: 900154308
Change-Id: Ia6e9b8e196e2ed37b723a0faf60e9731303a9619
2026-04-15 07:19:41 -07:00
Yuval Tassa 8f3ed662eb Hoist solver stack allocations out of iteration loop
PiperOrigin-RevId: 900085248
Change-Id: I2bde8fafc8a4801a92c39aa9a13af0ea7475843d
2026-04-15 04:23:08 -07:00
Yuval Tassa fa044fe0e5 Use the word "primal" for things that relate to both Newton and CG.
PiperOrigin-RevId: 852634373
Change-Id: I2c7641bb92d9c01252f861b9c36e72ef6ddc6e98
2026-01-05 23:56:57 -08:00
Yuval Tassa 849a650143 Refactor mju_cholUpdateSparse to use a dense accumulator, 1.5-2x function speedup.
PiperOrigin-RevId: 846754379
Change-Id: Ie599658db2907a26095072d97d07b99a693050dc
2025-12-19 08:57:45 -08:00
Yuval Tassa 76e0a78b32 Use two-step sparse Cholesky in Newton solver.
17% overall speedup for `100_humanoids.xml` as measured by `testspeed`

PiperOrigin-RevId: 846723342
Change-Id: Ibc41487bfc576c992f14853640d0b6000f5b47b2
2025-12-19 07:20:09 -08:00
Yuval Tassa 45b0153067 Refactor sparse Cholesky factorization into symbolic and numeric phases.
The new symbolic function is a generalization of the function it replaces. In this CL it takes two unused temp arrays. The actual change in behavior happens in the followup.

New benchmark test output below ("L" is 2 humanoids and 100 free objects, "XL" is 100 humanoids). Note that `symbolic` is only ever called once per Newton iteration, while `numeric` is sometimes called multiple times (when the rank-1 update fails), hence timing them separately is valuable.

```
Benchmark               Time(ns)        CPU(ns)     Iterations
--------------------------------------------------------------
BM_old_L_mean              84382          84703          19547  11.807k items/s
BM_symbolic_L_mean         16345          16381          88414  61.055k items/s
BM_numeric_L_mean          10986          10994         120000  90.999k items/s
BM_old_XL_mean           1241208        1244212           1200  803.924 items/s
BM_symbolic_XL_mean       130917         131042          12720  7.631k items/s
BM_numeric_XL_mean         77004          76767          21116  13.029k items/s
```

PiperOrigin-RevId: 846704054
Change-Id: Ib0c365724d63bf2b81606ca5353756a6496c3a26
2025-12-19 06:11:55 -08:00
Yuval Tassa 2b9940bc67 Add private function mju_fillInt
PiperOrigin-RevId: 845217293
Change-Id: I604372ab7ea7f4d47d9427d36dc8cd78e60d84ca
2025-12-16 05:16:50 -08:00
Yuval Tassa ccbfc6c8bf Improve line search initialization logic and first Newton step acceptance.
PiperOrigin-RevId: 844316808
Change-Id: I0a7639a486c75f122db7f8b8c0df42ef810f060c
2025-12-14 01:50:52 -08:00
Yuval Tassa 769f37b653 Implement sleeping in engine
PiperOrigin-RevId: 829361787
Change-Id: I6f64d8e25c4248cf32c18cd94d37ff5def78946e
2025-11-07 03:33:07 -08:00
Yuval Tassa edbdb5195c Change MuJoCo engine source code function-spacing convention from 3 blank lines to 2
PiperOrigin-RevId: 813754244
Change-Id: I6836e41c3b021cb727e922c25c60f629b9814c93
2025-10-01 07:58:17 -07:00
Yuval Tassa 52da7586dc Clean up includes in src/engine
PiperOrigin-RevId: 807726978
Change-Id: I02800deb79bfb3c328d941861ded9d817e8f6e32
2025-09-16 09:36:44 -07:00
Yuval Tassa b9900db00e Extract memory allocation functions and core utilities
PiperOrigin-RevId: 801745499
Change-Id: Iaf05c3430769d3115743d8ab020d13148cb2eb59
2025-09-01 03:38:25 -07:00
Yuval Tassa 5fe4224ae7 Compute sparse Jacobian transpose inside solver.
PiperOrigin-RevId: 797745776
Change-Id: I2ec429120d0c6eb51bec340619ab178389482e69
2025-08-21 06:55:48 -07:00
Yuval Tassa 977f94e9df Delete sparsity structures from mjData, use the ones in mjModel
PiperOrigin-RevId: 795067950
Change-Id: If4f52e40ac49f7c6986b33ef9c8b534199dfd21a
2025-08-14 09:13:02 -07:00
Yuval Tassa 998769a995 Return the total non-zeros from mju_sqrMatTDSparseCount.
PiperOrigin-RevId: 772414653
Change-Id: Ic1cac868af0034196f33d08b2456be77c72cbd50
2025-06-17 04:27:15 -07:00
Yuval Tassa 45fc15b844 Speed up sparse supernode detection by combining it with transposition.
PiperOrigin-RevId: 760673008
Change-Id: I6d66580e675fd86e5b974859383f93f87482ca16
2025-05-19 10:14:15 -07:00
Yuval Tassa e8c566bee1 Remove use of diagnum in solver
PiperOrigin-RevId: 759021147
Change-Id: I9e5c85b6542380780fc487781398f406ddaad51b
2025-05-15 00:39:28 -07:00
Yuval Tassa e1baa15776 Remove unnecessary argument in mju_mulSymVecSparse
PiperOrigin-RevId: 758688536
Change-Id: Ife1099b948215363ddaa6d0b0edcf3492edcddcc
2025-05-14 08:12:15 -07:00
Yuval Tassa d8bebdc675 Remove unnecessary diagnum argument in mj_solveLD and mj_factorI
PiperOrigin-RevId: 758669364
Change-Id: Icfefc4a7a1e7d28da373725358ccf9d5f589f689
2025-05-14 07:13:27 -07:00
Yuval Tassa dd28b887d4 Rename C sparse structure to M in mjdata, improve docstrings.
PiperOrigin-RevId: 758636638
Change-Id: If78acc423601d2911f514929b27f7b6d0af9ef58
2025-05-14 05:26:19 -07:00
Yuval Tassa 1165018f71 Add island support to Newton solver
PiperOrigin-RevId: 758618486
Change-Id: I257d78c7dc9aa4dbf7cd6d8849d8ec48201edf24
2025-05-14 04:20:34 -07:00
Yuval Tassa a6c3a287d6 Prepare Newton solver for island support
PiperOrigin-RevId: 758600631
Change-Id: Iac354588ad91f404a30fa035893560db60db733f
2025-05-14 03:18:03 -07:00
Yuval Tassa 4186589738 Use mjData.M where appropriate
PiperOrigin-RevId: 758374391
Change-Id: I9de7af7be8e41b5c300d0a04ea99082b4cefdec6
2025-05-13 14:22:41 -07:00
Yuval Tassa 627fffdef9 Switch mjData.{qH,qLD} from full ("M") to reduced ("C") inertia matrix structure.
PiperOrigin-RevId: 758273074
Change-Id: If1a2e663ea70044694af985e0119afd6d58115ac
2025-05-13 10:20:32 -07:00
Yuval Tassa 755564a348 Remove unused code related to legacy island implementation.
Also fix a docstring.

PiperOrigin-RevId: 757939564
Change-Id: I3a970fd38c63886d34cd23b1fb45617a9ce5c147
2025-05-12 15:39:16 -07:00
Yuval Tassa ecb769fc3a Refactor islands to be memory contiguous.
PiperOrigin-RevId: 755803476
Change-Id: I41972b07e0d5ef5d0117c94f565b93367b87458b
2025-05-07 05:06:27 -07:00
Yuval Tassa c2ac0d724e Add mju_gather and mju_scatter, private engine functions.
PiperOrigin-RevId: 750245294
Change-Id: I3a14ccdd55a324d3fe206e388fb2a513772f5f5e
2025-04-22 10:33:33 -07:00
Yuval Tassa 2624d524ba Switch mj_solveM_island to use CSR representation
PiperOrigin-RevId: 717684720
Change-Id: I7d74a5d4aef4aa5b0c0acddf88c5b8591d63e88d
2025-01-20 18:36:04 -08:00
Yuval Tassa 893a993bf6 Fix typos.
PiperOrigin-RevId: 716211176
Change-Id: Ibadfe13db89ed9047832b977e918faa3c5c6d9f1
2025-01-16 06:28:54 -08:00
Yuval Tassa 00010f5848 Compute diagonal indices in mj_sqrMatTDSparse
PiperOrigin-RevId: 713714087
Change-Id: Icc8eae74e6e47ba1d12a6e9aa0d774ab006cfe38
2025-01-09 10:17:31 -08:00
Yuval Tassa 674d227050 Standardize names of sparse fill-in pre-counting functions
PiperOrigin-RevId: 712835830
Change-Id: I8e90dfa52af56ede917e1fd90b8d551898c244e5
2025-01-07 02:39:46 -08:00
Yuval Tassa 7eb8231fda Represent only the lower triangle in Newton solver's reduced dof-dof matrix.
PiperOrigin-RevId: 712488529
Change-Id: Iad91c72654376539791d7856765a0d0ac9088251
2025-01-06 04:58:55 -08:00
Yuval Tassa 2691887500 Add engine-internal convenience macro for allocating typed arrays, improve error message.
PiperOrigin-RevId: 705126655
Change-Id: I2bd8fada6d33a919d2fb82297f93ac57958355a4
2024-12-11 09:04:44 -08:00
Yuval Tassa 7ec94f46d4 Refactor Newton solver: move Hessian from arena back to the stack.
The changes in 9a0dc20821, moving the Hessian from the stack to the arena, should be rolled back: they prevent future threading over islands. Unlike the stack, arena allocations are not thread-friendly. The reason for the original move was to save memory, but the savings are small: `O(ctx->nH)` and only linear in `nv`. The significant reduction of the Cholesky factor, from quadratic in `nv` to quadratic in the largest dof island — the reduction afforded by 2dd518734f — remains in place.

Also refactor and improve readability.

PiperOrigin-RevId: 685717356
Change-Id: Ia9ec3e44a62d459a3b9cffb578cf6479d5aa1d7f
2024-10-14 08:34:16 -07:00
Yuval Tassa 2dd518734f Stop over-allocating memory in Newton solver.
PiperOrigin-RevId: 684797464
Change-Id: I3389416cb69a8564b5119f6e3944b579893ea511
2024-10-11 05:09:33 -07:00
Yuval Tassa 9a0dc20821 Move Newton Hessian memory allocation from stack to arena.
PiperOrigin-RevId: 684054446
Change-Id: I4be0b216810ebbd9d7fa119c6a84eb08d3c16b83
2024-10-09 08:54:54 -07:00
Yuval Tassa cf13413c0b Clean up dual solver functions.
PiperOrigin-RevId: 680587029
Change-Id: I20c78e96785af94634361d7823b1e5caba08800d
2024-09-30 08:31:40 -07:00
Yuval Tassa e4d4153352 Move sanitizer instrumentation to a separate header file. Fixes #2049.
PiperOrigin-RevId: 672986547
Change-Id: I42522f4925237a73168e965364a2dd65c0f067cb
2024-09-10 09:05:35 -07:00
Yuval Tassa 3437361bfb Add reduced sparse dof-dof matrix structures to mjData.
PiperOrigin-RevId: 670919263
Change-Id: I3592de802ae84648d97230115c2e9c3437424968
2024-09-04 04:24:02 -07:00
Yuval Tassa 772326e9eb Fix potential out-of-range write in solver statistics saving.
PiperOrigin-RevId: 569094007
Change-Id: I2e596a7c03d76523359cae7ae7f4501b06256a38
2023-09-28 00:58:13 -07:00
Yuval Tassa 2cc1205498 Add island support to CG solver.
BEGIN_PUBLIC

Add island support to CG solver.

END_PUBLIC

PiperOrigin-RevId: 565653886
Change-Id: Ib971ae37dd011b8f2cbe1f88d8875e0ed657c7af
2023-09-15 05:47:24 -07:00
Yuval Tassa 86d8b912f4 Rejigged solver statistics in preparation for islanding.
- See public description below.
- Stopped incrementing the iteration count in saveStats().
- `sizeof(mjSolverStat) == 40`, so this ends up costing 160KB, up from 40KB.

BEGIN_PUBLIC

Changed the size of `mjData.solver`, the structure used to collect solver diagnostic information. The array is now of length `mjNISLAND * mjNSOLVER`, where each row of length `mjNSOLVER` contains separate solver statistics for each constraint island. Until solver islanding is implemented, only row 0 is used.

- The new constant `mjNISLAND` is set to 20.
- `mjNSOLVER` is reduced from 1000 to 200.
- Added `mjData.solver_nisland`, the number of islands for which the solver ran.
- `mjData.solver_niter` (renamed from mjData.solver_iter) and `mjData.solver_nnz` are now integer vectors of length `mjNISLAND`.

END_PUBLIC

PiperOrigin-RevId: 565030093
Change-Id: I773e918805c6ced79f0dab5f19ea23956760c8c5
2023-09-13 06:18:37 -07:00
Google DeepMind 040d3efce2 Add options for controlling early termination criteria of CG/Newton linesearch.
Useful for performance tuning models.

PiperOrigin-RevId: 563175078
Change-Id: Id64a510d6080038ea8d5876dabbf0b2b311435bb
2023-09-06 12:02:58 -07:00
Saran Tunyasuvunakool 4929077211 Remove mjMARKSTACK and mjFREESTACK macros.
PiperOrigin-RevId: 562844184
Change-Id: Id2f57f5d132c47094dad75c6dab7297d7aa73458
2023-09-05 11:41:43 -07:00
Yuval Tassa d5292976de Add internal utility for zeroing int vectors.
Remove unnecessary branches in related functions.

PiperOrigin-RevId: 562768873
Change-Id: I3964a7a953d95bcd4e29137a7b590be5ffa6e311
2023-09-05 07:15:14 -07:00
Yuval Tassa 29aa5e4a41 Allow dot products with sparse vectors to specify that one vector uses uncompressed memory.
PiperOrigin-RevId: 561951337
Change-Id: I1310d7c09d9a85ad9b43877155844a9d0ce6edab
2023-09-01 07:40:16 -07:00
Saran Tunyasuvunakool b979a26b10 Rename mj_stackAlloc to mj_stackAllocNum.
PiperOrigin-RevId: 561040193
Change-Id: I49ea21ffb5b596471d2aacefb1961c11c5c905d0
2023-08-29 08:43:03 -07:00
Yuval Tassa 5f3e22a45f Refactor a loop in engine_solver.c
PiperOrigin-RevId: 560997065
Change-Id: I26540e4f750eb18d9497bf35396adbb15cc3a122
2023-08-29 05:17:14 -07:00
Matthew Bennice 59138af10b Refactor stack allocations to have a clean byte allocation interface.
PiperOrigin-RevId: 558881470
Change-Id: I08c45cb5fa25a1a9c39560e598e805cfa5141c3e
2023-08-21 13:18:29 -07:00