Optimize new Gson()#2864
Conversation
41b38eb to
6c2e3b2
Compare
|
I'm not really convinced here. It might have been better if this had been the API from the outset, but now we have 17 years of client code that uses However, it's true that every invocation of
The last item is potentially controversial, since it does mean that some memory would end up being retained even when there are no active Whether or not we do the last item, I think the first two would be worthwhile, if only because it would eliminate the duplication whereby the default value for every Gson property is specified in both the |
|
I will drop |
e4f3797 to
f4ba10b
Compare
|
I need help about Java 9 modules... |
It looks like there is something wrong with how the adapters are looked up. It is trying to use the reflective adapter for types such as I have made some changes to this code here and proposed them as MukjepScarlet#1; that also fixes the test failures. If you are fine with them, you can merge them and they will directly appear in this PR here then. But @MukjepScarlet, could you please edit the title of this PR here (click the "Edit" button at the top right) and rename it to something like "Reduce overhead of default |
|
@eamonnmcmanus, what do you think about the current state of this PR? |
The outline looks correct, but I think there is a subtle change in behaviour. When I ran this against Google's internal tests, I saw failures that looked like this: I guess this code is running with |
|
This PR does change If you revert those |
|
I'm interesting about that too. If you think the change of |
|
I think this is the reason because stuffs like |
|
I think I am able to reproduce this locally:
Fails with the same stack trace as above. Replacing the method references in In that case maybe we could indeed replace the method references in Interestingly this only seems to affect public class SystemClassLoader extends ClassLoader {
public SystemClassLoader(ClassLoader parent) {
super(parent);
List<Supplier<?>> unused = Arrays.asList(
ArrayList::new,
LinkedHashSet::new,
TreeSet::new,
ArrayDeque::new,
// LinkedTreeMap::new,
() -> new LinkedTreeMap<>(),
LinkedHashMap::new,
TreeMap::new,
ConcurrentHashMap::new,
ConcurrentSkipListMap::new
);
}
@SuppressWarnings("SystemOut")
public static void main(String[] args) {
System.out.println("done");
}
}Edit: It seems the reason why this issue only occurs for ...
At least that is my understanding of it. But this also seems to highly depend on JDK and compiler implementation details. But given that usage of lambdas here worked before apparently, that seems to be a reasonable solution, albeit brittle. Though using Gson in general from within the constructor of a custom system class loader seems to be brittle. |
eamonnmcmanus
left a comment
There was a problem hiding this comment.
Thanks! This is basically what I had in mind. But it looks like there are a lot of changes in here that are not really related to the important change, which is eliminating duplication between Gson and GsonBuilder. Would it possible to make it so that that is the only thing that changes?
Sorry, might also have been my fault to some extent due to the changes I had proposed to @MukjepScarlet. The changes to The change to @eamonnmcmanus, is that what you had in mind? And @MukjepScarlet what do you think about this? Do you think this is a reasonable way to split this PR, and would you be able to do that? |
|
Splitting the PR is reasonable, as certain changes within it were deliberately intended to serve as separate prerequisite PR. I have no objection. @Marcono1234 |
5b04e3c to
1db03ac
Compare
|
(I'm still looking for a PR that has no changes other than the ones needed to remove the duplication between |
|
@eamonnmcmanus, most of the changes have been moved by @MukjepScarlet into #2951, which is kind of a prerequisite for this PR here now I think. If possible could you please have a look at that PR? Once that is (hopefully) merged and this PR here is updated, the remaining changes here should be only1 about the improvements for the Gson default instance. Footnotes
|
Co-authored-by: Marcono1234 <Marcono1234@users.noreply.github.com>
650d2e3 to
4560d87
Compare
|
@eamonnmcmanus I think this is ready for review now. |
eamonnmcmanus
left a comment
There was a problem hiding this comment.
Sorry for the long delay here. It looks like this PR is now focused on the one change of reducing duplication in Gson and GsonBuilder. I have one very minor comment, but it can potentially be addressed in a follow-up.
|
|
||
| private static Gson createGson() { | ||
| GsonBuilder gsonBuilder = new GsonBuilder(); | ||
| gsonBuilder.excluder = CUSTOM_EXCLUDER; |
There was a problem hiding this comment.
I think it would be a bit cleaner to set these values through the public API of GsonBuilder.
There was a problem hiding this comment.
Yes, better readability, just like how we use it outside of this package.
|
Thanks again, @MukjepScarlet, for seeing this PR through to completion! I think it's a great simplification of the code. |
…ip ci] Bumps [com.google.code.gson:gson](https://github.com/google/gson) from 2.13.2 to 2.14.0. Release notes *Sourced from [com.google.code.gson:gson's releases](https://github.com/google/gson/releases).* > Gson 2.14.0 > ----------- > > What's Changed > -------------- > > * Add type adapters for `java.time` classes by [`@eamonnmcmanus`](https://github.com/eamonnmcmanus) in [google/gson#2948](https://redirect.github.com/google/gson/pull/2948) > > When the `java.time` API is available, Gson automatically can read and write instances of classes like `Instant` and `Duration`. The format it uses essentially freezes the JSON representation that `ReflectiveTypeAdapterFactory` established by default, based on the private fields of `java.time` classes. That's not a great representation, but it is understandable. Changing it to anything else would break compatibility with systems that are expecting the current format. > > With this change, Gson no longer tries to access private fields of these classes using reflection. So it is no longer necessary to run with `--add-opens` for these classes on recent JDKs. > * Remove `com.google.gson.graph` by [`@eamonnmcmanus`](https://github.com/eamonnmcmanus) in [google/gson#2990](https://redirect.github.com/google/gson/pull/2990). > > This package was not part of any released artifact and depended on Gson internals in potentially problematic ways. > * Validate that strings being parsed as integers consist of ASCII characters by [`@eamonnmcmanus`](https://github.com/eamonnmcmanus) in [google/gson#2995](https://redirect.github.com/google/gson/pull/2995) > > Previously, strings could contain non-ASCII Unicode digits and still be parsed as integers. That's inconsistent with how JSON numbers are treated. > * Fix duplicate key detection when first value is null by [`@andrewstellman`](https://github.com/andrewstellman) in [google/gson#3006](https://redirect.github.com/google/gson/pull/3006) > > This could potentially break code that was relying on the incorrect behaviour. For example, this JSON string was previously accepted but will no longer be: `{"foo": null, "foo": bar}`. > * Remove `Serializable` from internal `Type` implementation classes. by [`@eamonnmcmanus`](https://github.com/eamonnmcmanus) in [google/gson#3011](https://redirect.github.com/google/gson/pull/3011) > > The nested classes `ParameterizedTypeImpl`, `GenericArrayTypeImpl`, and `WildcardTypeImpl` in `GsonTypes` are implementations of the corresponding types (without `Impl`) in `java.lang.reflect`. For some reason, they were serializable, even though the `java.lang.reflect` implementations are not. Having unnecessarily serializable classes could *conceivably* have been a security problem if they were part of a larger exploit using serialization. (We do not consider this a likely scenario and do not suggest that you need to update Gson just to get this change.) > * Add `LegacyProtoTypeAdapterFactory`. by [`@eamonnmcmanus`](https://github.com/eamonnmcmanus) in [google/gson#3014](https://redirect.github.com/google/gson/pull/3014) > > This is not part of any released artifact, but may be of use when trying to fix code that is currently accessing the internals of protobuf classes via reflection. > * Make AppendableWriter do flush and close if delegation object supports by [`@MukjepScarlet`](https://github.com/MukjepScarlet) in [google/gson#2925](https://redirect.github.com/google/gson/pull/2925) > > Other less visible changes > -------------------------- > > * Add default capacity to EnumTypeAdapter maps by [`@MukjepScarlet`](https://github.com/MukjepScarlet) in [google/gson#2959](https://redirect.github.com/google/gson/pull/2959) > * refactor: move derived adapters from Gson to TypeAdapters by [`@MukjepScarlet`](https://github.com/MukjepScarlet) in [google/gson#2951](https://redirect.github.com/google/gson/pull/2951) > * Optimize `new Gson()` by [`@MukjepScarlet`](https://github.com/MukjepScarlet) in [google/gson#2864](https://redirect.github.com/google/gson/pull/2864) > > New Contributors > ---------------- > > * [`@ThirdGoddess`](https://github.com/ThirdGoddess) made their first contribution in [google/gson#2944](https://redirect.github.com/google/gson/pull/2944) > * [`@lmj798`](https://github.com/lmj798) made their first contribution in [google/gson#2988](https://redirect.github.com/google/gson/pull/2988) > * [`@Eng-YasminKotb`](https://github.com/Eng-YasminKotb) made their first contribution in [google/gson#3005](https://redirect.github.com/google/gson/pull/3005) > * [`@andrewstellman`](https://github.com/andrewstellman) made their first contribution in [google/gson#3006](https://redirect.github.com/google/gson/pull/3006) > > **Full Changelog**: <google/gson@gson-parent-2.13.2...gson-parent-2.14.0> Commits * [`3ff35d6`](google/gson@3ff35d6) [maven-release-plugin] prepare release gson-parent-2.14.0 * [`a3024fd`](google/gson@a3024fd) Bump the maven group with 13 updates ([#3002](https://redirect.github.com/google/gson/issues/3002)) * [`5689ffe`](google/gson@5689ffe) Bump the github-actions group across 1 directory with 3 updates ([#3018](https://redirect.github.com/google/gson/issues/3018)) * [`48db33c`](google/gson@48db33c) Add `LegacyProtoTypeAdapterFactory`. ([#3014](https://redirect.github.com/google/gson/issues/3014)) * [`53d703e`](google/gson@53d703e) Update outdated comment regarding serializable types ([#3012](https://redirect.github.com/google/gson/issues/3012)) * [`0189b72`](google/gson@0189b72) Remove `Serializable` from internal `Type` implementation classes. ([#3011](https://redirect.github.com/google/gson/issues/3011)) * [`f4d371d`](google/gson@f4d371d) Fix duplicate key detection when first value is null ([#3006](https://redirect.github.com/google/gson/issues/3006)) * [`27d9ba1`](google/gson@27d9ba1) Fix typo in README (JPMS dependencies section) ([#3005](https://redirect.github.com/google/gson/issues/3005)) * [`1fa9b7a`](google/gson@1fa9b7a) Validate that strings being parsed as integers consist of ASCII characters (#... * [`b7d5954`](google/gson@b7d5954) Add iterator fail-fast tests for LinkedTreeMap.clear() ([#2992](https://redirect.github.com/google/gson/issues/2992)) * Additional commits viewable in [compare view](google/gson@gson-parent-2.13.2...gson-parent-2.14.0) [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
## Release notes Sourced from com.google.code.gson:gson's releases. Gson 2.14.0 What's Changed Add type adapters for java.time classes by @eamonnmcmanus in google/gson#2948 When the java.time API is available, Gson automatically can read and write instances of classes like Instant and Duration. The format it uses essentially freezes the JSON representation that ReflectiveTypeAdapterFactory established by default, based on the private fields of java.time classes. That's not a great representation, but it is understandable. Changing it to anything else would break compatibility with systems that are expecting the current format. With this change, Gson no longer tries to access private fields of these classes using reflection. So it is no longer necessary to run with --add-opens for these classes on recent JDKs. Remove com.google.gson.graph by @eamonnmcmanus in google/gson#2990. This package was not part of any released artifact and depended on Gson internals in potentially problematic ways. Validate that strings being parsed as integers consist of ASCII characters by @eamonnmcmanus in google/gson#2995 Previously, strings could contain non-ASCII Unicode digits and still be parsed as integers. That's inconsistent with how JSON numbers are treated. Fix duplicate key detection when first value is null by @andrewstellman in google/gson#3006 This could potentially break code that was relying on the incorrect behaviour. For example, this JSON string was previously accepted but will no longer be: {"foo": null, "foo": bar}. Remove Serializable from internal Type implementation classes. by @eamonnmcmanus in google/gson#3011 The nested classes ParameterizedTypeImpl, GenericArrayTypeImpl, and WildcardTypeImpl in GsonTypes are implementations of the corresponding types (without Impl) in java.lang.reflect. For some reason, they were serializable, even though the java.lang.reflect implementations are not. Having unnecessarily serializable classes could conceivably have been a security problem if they were part of a larger exploit using serialization. (We do not consider this a likely scenario and do not suggest that you need to update Gson just to get this change.) Add LegacyProtoTypeAdapterFactory. by @eamonnmcmanus in google/gson#3014 This is not part of any released artifact, but may be of use when trying to fix code that is currently accessing the internals of protobuf classes via reflection. Make AppendableWriter do flush and close if delegation object supports by @MukjepScarlet in google/gson#2925 Other less visible changes Add default capacity to EnumTypeAdapter maps by @MukjepScarlet in google/gson#2959 refactor: move derived adapters from Gson to TypeAdapters by @MukjepScarlet in google/gson#2951 Optimize new Gson() by @MukjepScarlet in google/gson#2864 New Contributors @ThirdGoddess made their first contribution in google/gson#2944 @lmj798 made their first contribution in google/gson#2988 @Eng-YasminKotb made their first contribution in google/gson#3005 @andrewstellman made their first contribution in google/gson#3006 Full Changelog: google/gson@gson-parent-2.13.2...gson-parent-2.14.0 Gson 2.13.2 The main changes in this release are just newer dependencies. ... (truncated) ## Changelog Sourced from com.google.code.gson:gson's changelog. Change Log The change log for versions newer than 2.10 is available only on the GitHub Releases page. Version 2.10 Support for serializing and deserializing Java records, on Java ≥ 16. (google/gson#2201) Add JsonArray.asList and JsonObject.asMap view methods (google/gson#2225) Fix TypeAdapterRuntimeTypeWrapper not detecting reflective TreeTypeAdapter and FutureTypeAdapter (google/gson#1787) Improve JsonReader.skipValue() (google/gson#2062) Perform numeric conversion for primitive numeric type adapters (google/gson#2158) Add Gson.fromJson(..., TypeToken) overloads (google/gson#1700) Fix changes to GsonBuilder affecting existing Gson instances (google/gson#1815) Make JsonElement conversion methods more consistent and fix javadoc (google/gson#2178) Throw UnsupportedOperationException when JsonWriter.jsonValue is not supported (google/gson#1651) Disallow JsonObject Entry.setValue(null) (google/gson#2167) Fix TypeAdapter.toJson throwing AssertionError for custom IOException (google/gson#2172) Convert null to JsonNull for JsonArray.set (google/gson#2170) Fixed nullSafe usage. (google/gson#1555) Validate TypeToken.getParameterized arguments (google/gson#2166) Fix #1702: Gson.toJson creates CharSequence which does not implement toString (google/gson#1703) Prefer existing adapter for concurrent Gson.getAdapter calls (google/gson#2153) Improve ArrayTypeAdapter for Object[] (google/gson#1716) Improve AppendableWriter performance (google/gson#1706) ## Commits 3ff35d6 [maven-release-plugin] prepare release gson-parent-2.14.0 a3024fd Bump the maven group with 13 updates (#3002) 5689ffe Bump the github-actions group across 1 directory with 3 updates (#3018) 48db33c Add LegacyProtoTypeAdapterFactory. (#3014) 53d703e Update outdated comment regarding serializable types (#3012) 0189b72 Remove Serializable from internal Type implementation classes. (#3011) f4d371d Fix duplicate key detection when first value is null (#3006) 27d9ba1 Fix typo in README (JPMS dependencies section) (#3005) 1fa9b7a Validate that strings being parsed as integers consist of ASCII characters (# b7d5954 Add iterator fail-fast tests for LinkedTreeMap.clear() (#2992) Additional commits viewable in compare view  Issue-ID: CIMAN-33 Signed-off-by: dependabot[bot] <support@github.com> Change-Id: I3707533b94e77d221389af97baae133d90bdb985 GitHub-PR: #37 GitHub-Hash: dc2f307754a500e1 Signed-off-by: onap.gh2gerrit <releng+onap-gh2gerrit@linuxfoundation.org>
Optimizes
new Gson()closes #2863