Skip to content

Don't break SI-2066 (non-trans) #328

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
larsrh merged 5 commits into from
Apr 25, 2013
Merged

Don't break SI-2066 (non-trans) #328

larsrh merged 5 commits into from
Apr 25, 2013

Conversation

@S11001001
Copy link
Member

@S11001001 S11001001 commented Apr 10, 2013

Someday SI-2066 may be fixed. We shouldn't assume it won't be.

This removes some instances of SI-2066 incompatibility.

However, the trickery to solve the variance issues revealed by traverse don't seem to apply to the Hoist and Cohoist instances; an attempt to solve this by making MonadTrans take a F[_[+_], _] eventually threatened to demand that Monad require variance again, making it highly distasteful; see S11001001/scalaz@8d9c4c1.

What else can we do to satisfy the demand for variance when constructing transformer stacks?

(I believe StateTLiftIO and others like it are also unsound, but won't worry about that here.)

@tonymorris
Copy link
Member

tonymorris commented Apr 10, 2013

I am considering reverting to a position that I once held and have since
for a long time now, been on the fence.

For practical purposes, do not use variance annotations, not ever. The
Scala type system is simply too broken and narrow-minded.

On 10/04/13 12:58, Stephen Compall wrote:

Someday SI-2066 https://issues.scala-lang.org/browse/SI-2066 may be
fixed. We shouldn't assume it won't be.

This removes some instances of SI-2066 incompatibility.

However, the trickery to solve the variance issues revealed by
|traverse| don't seem to apply to the |Hoist| and |Cohoist| instances;
an attempt to solve this by making |MonadTrans| take a |F[[+], _]|
eventually threatened to demand that |Monad| require variance again,
making it highly distasteful; see S11001001/scalaz@8d9c4c1
S11001001@8d9c4c1.

What else can we do to satisfy the demand for variance when
constructing transformer stacks?

(I believe |StateTLiftIO| and others like it are also unsound, but
won't worry about that here.)

    You can merge this Pull Request by running

git pull https://github.com/S11001001/scalaz topic/hk-ev-variance

Or view, comment on, or merge it at:

#328

    Commit Summary

Tony Morris
http://tmorris.net/

@runarorama
Copy link
Member

runarorama commented Apr 10, 2013

The fact is that subtyping and variance are an inherently difficult
(undecidable?) problem. On the other hand, going without variance
annotations in Scala can get you into a lot of manual typing that could
otherwise be inferred.

When I made the changes to add variance annotations everywhere in scalaz I
found that it was possible to strike a balance (with some caveats as has
become apparent here). I found that we could add variance for data types
(and type arguments to methods) without doing the same for type classes.
This is because in Scala's kind system the type Foo[A] means that Foo is
invariant in A, but Bar[F[_]] does not restrict the variance of the
argument to F at all.

That said, I believe that the current state of Scala's kind system should
make one extremely wary of variance annotations and subtyping in general.
Unfortunately, subtyping is used all over the standard library. I also
believe that if Scala's kind system were extended to have real kind
inference and kind polymorphism, these problems would be alleviated. Does
anybody know how feasible it would be to modify the compiler in this way?

On Tue, Apr 9, 2013 at 11:14 PM, tonymorris notifications@github.comwrote:

I am considering reverting to a position that I once held and have since
for a long time now, been on the fence.

For practical purposes, do not use variance annotations, not ever. The
Scala type system is simply too broken and narrow-minded.

On 10/04/13 12:58, Stephen Compall wrote:

Someday SI-2066 https://issues.scala-lang.org/browse/SI-2066 may be

fixed. We shouldn't assume it won't be.

This removes some instances of SI-2066 incompatibility.

However, the trickery to solve the variance issues revealed by
|traverse| don't seem to apply to the |Hoist| and |Cohoist| instances;
an attempt to solve this by making |MonadTrans| take a |F[[+], _]|
eventually threatened to demand that |Monad| require variance again,
making it highly distasteful; see S11001001/scalaz@8d9c4c1
<
S11001001@8d9c4c1
.

What else can we do to satisfy the demand for variance when
constructing transformer stacks?

(I believe |StateTLiftIO| and others like it are also unsound, but
won't worry about that here.)

You can merge this Pull Request by running

git pull https://github.com/S11001001/scalaz topic/hk-ev-variance

Or view, comment on, or merge it at:

#328

Commit Summary

  • Remove some unused variance, with a little remaining breakage in Free.
  • Backtrack a lot; scalac tricked me.
  • Avoid illegal variance in bitraverseImpls.
  • Avoid illegal variance in traverseImpls.
  • Remove more unused variance.

File Changes

Patch Links:

Tony Morris
http://tmorris.net/

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/328#issuecomment-16153116
.

@markhibberd
Copy link
Contributor

markhibberd commented Apr 10, 2013

On Wed, Apr 10, 2013 at 1:46 PM, Rúnar notifications@github.com wrote:

The fact is that subtyping and variance are an inherently difficult
(undecidable?) problem. On the other hand, going without variance
annotations in Scala can get you into a lot of manual typing that could
otherwise be inferred.

Is this really true if you hide all the sub-types? The only evidence of
type inference failure I have seen demonstrated is when the sub-types are
allowed to escape, so why not encourage them to be hidden?

The obvious issue is that hiding the sub-types involves significant
boilerplate, but perhaps there are ways to alleviate that for 2.10+ by
generating companion apply methods which return the super type.

When I made the changes to add variance annotations everywhere in scalaz I
found that it was possible to strike a balance (with some caveats as has
become apparent here). I found that we could add variance for data types
(and type arguments to methods) without doing the same for type classes.
This is because in Scala's kind system the type Foo[A] means that Foo is
invariant in A, but Bar[F[_]] does not restrict the variance of the
argument to F at all.

There are a few more annoyances that it introduces.

One example is trying to implement something like
https://gist.github.com/markhibberd/5351682#file-blat-scala-L25 in the face
of covariance (either can't be tail-recursive, or can't be implemented
inline depending on how you define the method). I have been trying to work
out a better way to deal with it in argonaut for a while, if you look at
https://github.com/markhibberd/argonaut/blob/master/src/main/scala/argonaut/DecodeResult.scala#L13you
can see that we currently exploit some cognitive dissidence in the
compiler by shifting the implementation, but this is not really ok.

Another (weaker) case that comes up is that the inference
is aggravatingly broken in some situations when variance is encouraged.
This piece of code in Either,
https://github.com/scalaz/scalaz/blob/scalaz-seven/core/src/main/scala/scalaz/Either.scala#L17,
is an example that is unusable because the inference breaks down on the
return type. Perhaps this can be disregarded as it could be solved with a
simpler construct on either, but it is a pattern that comes up regularly
and (I think) it demonstrates that the issue with letting the sub-types be
exposed.

I think my greatest grievance is simply that if you use scalaz it forces
you to use variance annotations, there is no opt-out (The DecodeResult
example is certainly only declared contravariant to play nicely with
scalaz) and you run into some data types where you have to make a choice
about what part of scalaz you want to be excluded from using which sucks.

I would be happy if there were some straight forward way to handle the
above situations, so if anyone knows please correct me.

That said, I believe that the current state of Scala's kind system should
make one extremely wary of variance annotations and subtyping in general.
Unfortunately, subtyping is used all over the standard library. I also
believe that if Scala's kind system were extended to have real kind
inference and kind polymorphism, these problems would be alleviated. Does
anybody know how feasible it would be to modify the compiler in this way?

I don't, but it sounds like a nice world, I would definitely subscribe to
the newsletter.

Cheers
Mark.

@markhibberd
Copy link
Contributor

markhibberd commented Apr 10, 2013

And I only just realised this was a github issue, and not on the mailing list. Sorry for the rant.

@S11001001
Copy link
Member Author

S11001001 commented Apr 11, 2013

runarorama writes:

I found that we could add variance for data types (and type arguments to methods) without doing the same for type classes. This is because in Scala's kind system the type Foo[A] means that Foo is invariant in A, but Bar[F[_]] does not restrict the variance of the argument to F at all.

Which is wonderful and useful, but it also seems to accept Baz[F[+_], A] as the T arg to Quux[T[_[_], _]], which is unsound.

Thinking about it now, it should be the opposite: an M[_[_], _] should be suitable as the T arg to TC[T[_[+_], _]]. I think. But scalac 2.9.2 rejects it.

@S11001001
Copy link
Member Author

S11001001 commented Apr 25, 2013

These are fine; let's split out issue of whether to remove variance on transformers or adapt S11001001/scalaz@8d9c4c1 to not be terrible.

larsrh added a commit that referenced this pull request Apr 25, 2013
@larsrh
Merge pull request #328 from S11001001/topic/hk-ev-variance
db00080 
Don't break SI-2066 (non-trans)
@larsrh larsrh merged commit db00080 into scalaz:scalaz-seven Apr 25, 2013
@larsrh larsrh mentioned this pull request Jun 2, 2013
Merged
@S11001001 S11001001 deleted the topic/hk-ev-variance branch July 25, 2013 04:13
@S11001001 S11001001 mentioned this pull request Oct 9, 2013
Merged
@S11001001 S11001001 mentioned this pull request Nov 24, 2013
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

7.1

Development

Successfully merging this pull request may close these issues.

5 participants

@S11001001 @tonymorris @runarorama @markhibberd @larsrh