DRAFT: diffengine backend for ignore_dpp#3348
Draft
Transurgeon wants to merge 18 commits into
Draft
Conversation
|
Transurgeon seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
|
Benchmarks that have stayed the same: |
Collaborator
|
This looks promising, but could you add benchmark timings? There were some benchmarks used for the COO backend PRs, or you could make more. |
PTNobel
reviewed
Jun 2, 2026
dcecba7 to
098b456
Compare
…ackend Reverts changes that are not required to introduce the DIFFENGINE backend: - cone_matrix_stuffing.py: remove the top-level lower_and_order_constraints() helper. ConeMatrixStuffing.apply() now lowers + reorders the constraints once inline (as on master), then selects the backend afterward: the DIFFENGINE path is a small early-return branch, the standard path is otherwise unchanged. No duplicated lowering loop. - Drop the use_diffengine property; inline the `canon_backend == DIFFENGINE_CANON_BACKEND` check (also addresses the review question of why it differs from canon_backend). - solving_chain.py: move the use_quad/quad_obj block back to its original location; keep only the ignore_dpp -> DIFFENGINE wiring. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
098b456 to
f215ff9
Compare
These will be rewritten by hand later. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- DiffengineConeProgram: extract the shared x=0 matrix evaluation (q, d, A, b, P) into _extract_matrices, used by both from_problem and apply_parameters instead of duplicating it. Behavior unchanged. - convert_symbolic_quad_form / _lower_symbolic_power: condense the verbose docstrings/comments; logic unchanged (all sparse/dense/parametric/Power branches are exercised). - Drop the verbose DIFFENGINE explanatory comments in cone_matrix_stuffing.py and solving_chain.py. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both ParamProb backends (ParamConeProg and DiffengineConeProgram) already return concrete (q, d, A, b) from apply_parameters(), and the perspective aux problem is parameter-free. Use that single call instead of branching on the program type and hand-decoding ParamConeProg's embedded sparse tensors. Removes the isinstance branch, the DiffengineConeProgram import, and the brittle reshape(order="F") decode. A is kept sparse (sparse @ x_canon works and avoids embedding a dense constant in the enclosing problem). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
a6c6f51 to
f5924ce
Compare
Introduce DiffEngineExtractor (the non-DPP analog of CoeffExtractor) in the diff_engine package: it owns the C_problem and recovers the concrete cone matrices (q, d, A, b, P) by evaluating the autodiff graph at x=0, re-evaluating on parameter changes. It absorbs the _build_jacobian_csc / _build_hessian_csc / _extract_matrices helpers. DiffengineConeProgram becomes a thin ParamProb that holds a DiffEngineExtractor and delegates: apply_parameters calls extractor.update_parameters + extract, from_problem builds the extractor, and apply_restruct_mat shares it with the restructured instance. This mirrors the standard path (CoeffExtractor -> ParamConeProg) with (DiffEngineExtractor -> DiffengineConeProgram). Behavior-preserving; the ParamProb interface and the standard cone path are unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The diffengine cone backend's quad_form handling requires the dense-quadform bindings shipped in the 0.5.x line. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The diffengine cone backend is tightly coupled to the sparsediffpy binding ABI (dense make_quad_form), so pin the exact version rather than a range. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove the lb_tensor/ub_tensor placeholders (parametric bounds are not supported on the diffengine path and nothing reads them), the unused param_id_to_size, and the self-referential quad_obj attribute (the solver already keys quadratic-ness off `problem.P is None`). No external readers of any of these. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Nothing reads it (apply_parameters reads p.value directly rather than going through a param_id->col map). The other accessors stay: variables (read by the HIGHS/Gurobi/Xpress interfaces), var_id_to_col (perspective_canon + split_solution), and id_to_var (split_solution). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Drop the intra-method grouping blank lines in __init__, apply_parameters, apply_restruct_mat, and from_problem. No behavior change. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mirror the Jacobian pattern for the Hessian: capture the (fixed) lower-triangular Hessian sparsity once in build() via init_hessian_coo_lower_tri + get_problem_hessian_sparsity_coo, then on each re-solve fetch only the lower-tri values (eval_hessian_vals_coo_lower_tri) and mirror off-diagonals to the full symmetric P the cone solvers expect. Reuses existing bindings (no engine change). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The lazy-operand variant was confirmed unrelated to the matmul slowdown/segfault (reverting left both unchanged: Murray ~15.5s cold compile, OptimalAdvertising still segfaults). The slowdown/crash live in make_dense/sparse_left_matmul + the C engine, invoked identically by both forms. Keep the simpler eager version; convert_expr converts all args up front and convert_matmul consumes children. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Please include a short summary of the change.
Issue link (if applicable):
Type of change
Contribution checklist