Skip to content

problem with serializing strong params (3.11.0+) #929

Description

@ellmo

Problem

Hello there

Seems like I have an issue with oj (after updating to 3.16.4) not properly being used in Rails when it comes to ActionController::Parameters.

Since the users params is passed further to a Sidekiq worker, Sidekiq uses JSON.generate(object) to serialize parameters.

JSON.generate is one of three different ways of doing this incorrectly:

users = params["users"]
#=> [#<ActionController::Parameters {"email"=>"test@test.com", "first_name"=>"joe", "last_name"=>"test"} permitted: false>]

JSON.generate users
#=> "[\"{\\\"email\\\"=>\\\"test@test.com\\\", \\\"first_name\\\"=>\\\"joe\\\", \\\"last_name\\\"=>\\\"test\\\"}\"]"

JSON.dump users
#=> "[\"{\\\"email\\\"=>\\\"test@test.com\\\", \\\"first_name\\\"=>\\\"joe\\\", \\\"last_name\\\"=>\\\"test\\\"}\"]"

Oj.generate users
#=> "[\"{\\\"email\\\"=>\\\"test@test.com\\\", \\\"first_name\\\"=>\\\"joe\\\", \\\"last_name\\\"=>\\\"test\\\"}\"]"

None of those approaches returns a properly serialized object.
If you try to parse it back as JSON, it will return:

reverse = Oj.load(Oj.generate users)
#=> ["{\"email\"=>\"test@test.com\", \"first_name\"=>\"joe\", \"last_name\"=>\"test\"}"]

So now, instead of an array of hashes, we end up with an array of strings. And those strings - mind you - cannot be further parsed:

reverse.map {|x| Oj.load(x)}
#=> Oj::ParseError: unexpected character (after email) at line 1, column 9 [parse.c:762]

The only module/method combination that works is:

Oj.dump users
#=> "[{\"email\":\"test@test.com\",\"first_name\":\"joe\",\"last_name\":\"test\"}]"
Oj.load _
#=> [{"email"=>"test@test.com", "first_name"=>"joe", "last_name"=>"test"}]

Question

So here's my question: Is there an option for Oj (3.16.4) and Rails 6.1+ that I'm missing?
This was working as expected up to oj 3.10.18 and it breaks with anything over that.

My current config for Oj is this:

MultiJson.use(:oj)

Oj.default_options = {
  bigdecimal_as_decimal: true,
  bigdecimal_load: :auto,
  mode: :custom,
  second_precision: ActiveSupport::JSON::Encoding.time_precision,
  time_format: :xmlschema,
  use_as_json: true,
}
Oj.optimize_rails

Activity

  1. ohler55 commented on Jul 16, 2024

    @ohler55
    Owner

    It looks like ActionController::Parameters is not encoding as a json object. I'm not really set up to test this right now so would you be able to help?

    The first thing to try would be to see if there is an #as_json or #to_json method on ActionController::Parameters. If so that does it return?

  2. ellmo commented on Jul 17, 2024

    @ellmo
    Author

    prying into the controller (using 3.11.8)

    params
    #=> #<ActionController::Parameters {"url"=>"(...)", "users"=>[#<ActionController::Parameters {"email"=>"test@test.com", "first_name"=>"joe", "last_name"=>"test"} permitted: false>], "uuid"=>"something-something", "format"=>"json", "controller"=>"api/v1/users", "action"=>"create"} permitted: false>
    
    users = params[:users]
    #=> [#<ActionController::Parameters {"email"=>"test@test.com", "first_name"=>"joe", "last_name"=>"test"} permitted: false>]
    
    users.as_json
    #=> [{"email"=>"test@test.com", "first_name"=>"joe", "last_name"=>"test"}]
    users.to_json
    #=> "[{\"email\":\"test@test.com\",\"first_name\":\"joe\",\"last_name\":\"test\"}]"

    and yet, after sending users to a sidekiq worker:

    users
    #=> ["{\"email\"=>\"test@test.com\", \"first_name\"=>\"joe\", \"last_name\"=>\"test\"}"]

    Is this what you needed?

  3. ohler55 commented on Jul 17, 2024

    @ohler55
    Owner

    What I'm trying to determine is if the ActionController::Parameters.as_json exists and is being called. It kind of looks like #to_json is being called by Oj instead of #as_json.

  4. ellmo commented on Aug 6, 2024

    @ellmo
    Author

    Hey, circling back to this – any way I can help?

  5. ohler55 commented on Aug 6, 2024

    @ohler55
    Owner

    What does the json gem emit when calling JSON.generate without Oj?

    It appears as if the #to_json method is being called on the params instead of #as_json. It's as if the :use_as_json option is not set which would be the normal case for the JSON gem.

  6. alecho commented on Aug 29, 2024

    @alecho

    I'm having the same issue. I can confirm that version 3.10.18 does not have this issue. Setting Oj.default_options = { mode: :compat, use_as_json: true } in a later version also doesn't fix it nor does Oj.default_options = { mode: :compat, compat_bigdecimal: true }.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions