Very cool! Will take a closer soon. (Also, thanks for fixing the MOSEK interface.)

Just one comment for now.

I do this because the suppfunc_canon function defined within the dcp2cone folder doesn't have access to the SuppFuncAtom object in question; it only has access to the args field of that SuppFuncAtom object!

Actually, the first argument to suppfunc_canon (expr) should be an instance of SuppFuncAtom.

Actually, the first argument to suppfunc_canon (expr) should be an instance of SuppFuncAtom

Wonderful news! I will change the implementation to reflect that.

Although, this now raises a question for me: what is the purpose of the second argument to suppfunc_canon, or any other canon method for that matter? If the first argument (expr) is the atom in question, then we should be able to retrieve args directly from that object.

Okay, now that the hack with suppfunc_canon is fixed, I think this PR is in much better shape.

Here are some points to consider in the review:

1. SuppFuncAtom overrides essentially all Atom and Expression functions. It seems to me that support functions are too general for us to automatically determine properties like is_incr or is_nonpos, and so it is best to return pessimistic results for those queries.

2. The convex set described by a SuppFunc object cannot contain Parameters (a-la DPP programming). At least, not in the current implementation.

3. If this PR is merged, the features should be described (in some detail) on the website. Using these features, you can construct conjugate functions, explicitly solve dual problems, and do some standard operations in robust convex optimization.

One consistent thought I had throughout this process, is that it would be very useful if convex sets were first-class objects within cvxpy. If you'd like to discuss this more, let me know.

Although, this now raises a question for me: what is the purpose of the second argument to suppfunc_canon, or any other canon method for that matter? If the first argument (expr) is the atom in question, then we should be able to retrieve args directly from that object.

The args passed to a canon method are canonicalized args, ie, they are affine expressions representing the epigraph variables for the args. You’re right that the semantics of these arguments should be documented. You can look at the Canonicalization reduction to see what exactly is passed to canonicalizers.

This is a really cool change! I agree with / am okay with all of your three points.

@rileyjmurray can you say more about what you mean by convex sets are not first class objects? I added an Indicator transform at one point to convert constraints to atoms of the form f(x) = 1 if x satisfies constraints, +\infty otherwise. Would that be helpful?

[edit] I guess what I mean is that I view constraints as the current implementation of convex sets. Anyway this is a very cool PR! It would be great to have examples of the different operations you mentioned.

@SteveDiamond Constraint objects completely handle the problem of membership in a set. However, in convex analysis, we often want to do more than just enforce membership. We want to transform sets, and build more complicated sets from simpler sets. Common operations include applying a linear operator, taking set-intersection, or computing a Cartesian product. Less-common (but still fundamental) operations include taking Minkowski sums, polar bodies, conic hulls, or dual cones. Most of these operations have some implementation in terms of Constraint objects (or using the indicator transform), but these implementations are often clunky, unnatural, or result in cvxpy Problem instances with very slow compile times.

There are different ways that a ConvexSet class could be introduced. In one scenario, ConvexSet objects are only used in the background, as a way to facilitate more complicated transforms I've described above. In another scenario, ConvexSet objects could be exposed to the user. I think there are compelling mathematical and performance reasons to expose a ConvexSet class to users. I give one performance reason below.

A possible (end-user) use-case of a ConvexSet class.

Consider a scenario where a user has a repeated constraint pattern they want to apply to every expression in a long expr_list. In particular, suppose the user conceptualizes their set as S, and they want to require expr in S for every expr in expr_list.

Right now, a user can do this in a clean way defining a function cvxpy_cons_for_S, which takes in an Expression expr, and returns a list of cvxpy Constraints which are satisfiable iff expr.value belongs to their conceptualization of S. The user would then write:

constrs = []
for expr in expr_list
    constrs += cvxpy_cons_for_S(expr)

The downside to that approach, is that cvxpy has no idea that so many of the constraint objects are just slight variations on one another.

Now consider the same scenario where the user has a ConvexSet class at their disposal. They would simply construct the convex set S as a cvxpy object, and then write something like

constrs = [expr in S for expr in expr_list]

If each of the Constraint objects con = (expr in S) referenced the ConvexSet object S (much like how SuppFuncAtom objects in this PR reference a parent SuppFunc object), then the canonicalizer can now see the underlying structure in the problem.

@SteveDiamond as to examples, let me know if I need to add some in order to get this PR merged. There are several tests which could be converted into examples without too much trouble.

@rileyjmurray no need, it looks good. Your comments on convex sets as a first class object were very clarifying.

@SteveDiamond I've resolved your comments. Should be ready to merge, assuming @akshayka doesn't want to make additional comments.

Looks good! I merged it in.