Add net stdlib module with URL support - #1834

Why not use the JDK's java.net.URLEncoder and java.net.URLDecoder?

URLEncoder/URLDecoder implement application/x-www-form-urlencoded (as stated in their javadoc), not RFC 3986.
For example URLEncoder will encode ~ which is in unreserved in 3986. It also encodes to +.
It could replace our PercentEncoder.encode/decodeForm, but this is a trivial implementation once you already have all the machinery.

Yeah, I'm referring to encodeForm and decodeForm specifically.

According to WHATWG, the characters to encode also includes U+0021, U+0027-U+0027, and U+007E, but this implementation isn't doing that.

But also, URLEncoder/URLDecoder have already been optimized and battle-tested, so I feel like we might as well use it.

URLEncoder is not RFC 3986 compatible so we can't use it (it would encode ~, for example, which is in the unreserved set).
URLDecoder seems to be compatible, from the tests I did, but I still don't think it's a good idea to replace it. Specially because we'll use our own encoder. Also, it allocates a full input size string builder every time, where we short circuit the happy path where there's nothing to encode. So I'm not convinced it's going to be any faster.

External properties already behave as if they're fixed. We don't combine these modifiers anywhere else in the stdlib.

The external modifier has no relation to fixed. The only contract for external properties is that they can't have a default body (it comes from Java).

local m = (import("pkl:math")) {
  minInt = 42
}

bogusMinInt = m.minInt

bogusMinInt = 42

The fact we don't put external + fixed together in other places is because most external properties are in a primitive class (which can't be amended), in an external class (which also can't be amended), or in a module (which can be amended, but doesn't change the original module).

This implementation (and others) will still create an IndirectCallNode even if it doesn't need it (e.g. it's already in extra storage).

I played around with this and I think an elegant solution here is to introduce a specialized truffle node for it; e.g. this:

package org.pkl.core.ast.internal;

import com.oracle.truffle.api.dsl.Cached;
import com.oracle.truffle.api.dsl.Specialization;
import com.oracle.truffle.api.frame.VirtualFrame;
import com.oracle.truffle.api.nodes.IndirectCallNode;
import com.oracle.truffle.api.source.SourceSection;
import org.pkl.core.ast.PklNode;
import org.pkl.core.runtime.VmTyped;
import org.pkl.core.stdlib.net.UrlFactory;
import org.pkl.core.stdlib.net.UrlParser;

public abstract class GetParsedUrlNode extends PklNode {
  protected GetParsedUrlNode(SourceSection sourceSection) {
    super(sourceSection);
  }

  @Specialization(guards = "url.hasExtraStorage()")
  protected UrlParser.Parsed evalCached(VmTyped url) {
    return (UrlParser.Parsed) url.getExtraStorage();
  }

  @Specialization
  protected UrlParser.Parsed eval(VmTyped url, @Cached("create()") IndirectCallNode callNode) {
    return UrlFactory.read(url, callNode);
  }

  public abstract UrlParser.Parsed execute(VirtualFrame frame, VmTyped url);
}

Then this implementation turns into:

  public abstract static class resolve extends ExternalMethod1Node {
    private @Child GetParsedUrlNode getParsedUrlNode = GetParsedUrlNodeGen.create(sourceSection);

    @Specialization
    protected Object evalString(VirtualFrame frame, VmTyped self, String ref) {
      var base = getParsedUrlNode.execute(frame, self);
      return resolve(base, UrlFactory.parseOrThrow(ref, this));
    }

    @Specialization
    protected Object eval(VirtualFrame frame, VmTyped self, VmTyped ref) {
      var base = getParsedUrlNode.execute(frame, self);
      var parsed = getParsedUrlNode.execute(frame, ref);
      return resolve(base, parsed);
    }

    @SuppressWarnings("MethodNameSameAsClassName")
    private Object resolve(Parsed base, Parsed ref) {
      if (base.scheme() == null) {
        CompilerDirectives.transferToInterpreter();
        throw exceptionBuilder()
            .evalError("cannotResolveAgainstRelativeUrl", base.serialize())
            .build();
      }
      return UrlFactory.create(UrlParser.resolve(base, ref));
    }
  }

It doesn't seem great that you can define valid data in Pkl and have that blow up when mapped to Java, although it's definitely an edge case.

It's unfortunate, but we can't make Pkl support every language URL library out there. Go won't have a problem (as it supports RFC 6874).

Yeah, I'm referring to encodeForm and decodeForm specifically.

According to WHATWG, the characters to encode also includes U+0021, U+0027-U+0027, and U+007E, but this implementation isn't doing that.

But also, URLEncoder/URLDecoder have already been optimized and battle-tested, so I feel like we might as well use it.