Add net stdlib module with URL support - #1834

Why not use the JDK's java.net.URLEncoder and java.net.URLDecoder?

URLEncoder/URLDecoder implement application/x-www-form-urlencoded (as stated in their javadoc), not RFC 3986.
For example URLEncoder will encode ~ which is in unreserved in 3986. It also encodes to +.
It could replace our PercentEncoder.encode/decodeForm, but this is a trivial implementation once you already have all the machinery.

Yeah, I'm referring to encodeForm and decodeForm specifically.

According to WHATWG, the characters to encode also includes U+0021, U+0027-U+0027, and U+007E, but this implementation isn't doing that.

But also, URLEncoder/URLDecoder have already been optimized and battle-tested, so I feel like we might as well use it.

External properties already behave as if they're fixed. We don't combine these modifiers anywhere else in the stdlib.

The external modifier has no relation to fixed. The only contract for external properties is that they can't have a default body (it comes from Java).

local m = (import("pkl:math")) {
  minInt = 42
}

bogusMinInt = m.minInt

bogusMinInt = 42

The fact we don't put external + fixed together in other places is because most external properties are in a primitive class (which can't be amended), in an external class (which also can't be amended), or in a module (which can be amended, but doesn't change the original module).

This implementation (and others) will still create an IndirectCallNode even if it doesn't need it (e.g. it's already in extra storage).

I played around with this and I think an elegant solution here is to introduce a specialized truffle node for it; e.g. this:

package org.pkl.core.ast.internal;

import com.oracle.truffle.api.dsl.Cached;
import com.oracle.truffle.api.dsl.Specialization;
import com.oracle.truffle.api.frame.VirtualFrame;
import com.oracle.truffle.api.nodes.IndirectCallNode;
import com.oracle.truffle.api.source.SourceSection;
import org.pkl.core.ast.PklNode;
import org.pkl.core.runtime.VmTyped;
import org.pkl.core.stdlib.net.UrlFactory;
import org.pkl.core.stdlib.net.UrlParser;

public abstract class GetParsedUrlNode extends PklNode {
  protected GetParsedUrlNode(SourceSection sourceSection) {
    super(sourceSection);
  }

  @Specialization(guards = "url.hasExtraStorage()")
  protected UrlParser.Parsed evalCached(VmTyped url) {
    return (UrlParser.Parsed) url.getExtraStorage();
  }

  @Specialization
  protected UrlParser.Parsed eval(VmTyped url, @Cached("create()") IndirectCallNode callNode) {
    return UrlFactory.read(url, callNode);
  }

  public abstract UrlParser.Parsed execute(VirtualFrame frame, VmTyped url);
}

Then this implementation turns into:

  public abstract static class resolve extends ExternalMethod1Node {
    private @Child GetParsedUrlNode getParsedUrlNode = GetParsedUrlNodeGen.create(sourceSection);

    @Specialization
    protected Object evalString(VirtualFrame frame, VmTyped self, String ref) {
      var base = getParsedUrlNode.execute(frame, self);
      return resolve(base, UrlFactory.parseOrThrow(ref, this));
    }

    @Specialization
    protected Object eval(VirtualFrame frame, VmTyped self, VmTyped ref) {
      var base = getParsedUrlNode.execute(frame, self);
      var parsed = getParsedUrlNode.execute(frame, ref);
      return resolve(base, parsed);
    }

    @SuppressWarnings("MethodNameSameAsClassName")
    private Object resolve(Parsed base, Parsed ref) {
      if (base.scheme() == null) {
        CompilerDirectives.transferToInterpreter();
        throw exceptionBuilder()
            .evalError("cannotResolveAgainstRelativeUrl", base.serialize())
            .build();
      }
      return UrlFactory.create(UrlParser.resolve(base, ref));
    }
  }

Needs a call node

It doesn't seem great that you can define valid data in Pkl and have that blow up when mapped to Java, although it's definitely an edge case.

Yeah, I'm referring to encodeForm and decodeForm specifically.

According to WHATWG, the characters to encode also includes U+0021, U+0027-U+0027, and U+007E, but this implementation isn't doing that.

But also, URLEncoder/URLDecoder have already been optimized and battle-tested, so I feel like we might as well use it.

Optimization: we can skip the allocation overhead if there's no % to begin with.

I strongly feel that this should throw, not kept as-is. Otherwise, a malformed percent encoding would silently be passed through as a decoded string.