Skip to content

WIP: [KEYCLOAK-7353]#5206

Closed
chicco785 wants to merge 2 commits intokeycloak:masterfrom
martel-innovate:KEYCLOAK-7353
Closed

WIP: [KEYCLOAK-7353]#5206
chicco785 wants to merge 2 commits intokeycloak:masterfrom
martel-innovate:KEYCLOAK-7353

Conversation

@chicco785
Copy link
Contributor

No description provided.

@pedroigor pedroigor self-assigned this May 16, 2018
@pedroigor pedroigor self-requested a review May 16, 2018 23:29
@pedroigor
Copy link
Contributor

pedroigor commented May 17, 2018

@chicco785, I've merged #5203. Can you rebase, please ?

I've also sent this PR #5211 that might be interesting to consider in this PR.

Copy link
Contributor

@pedroigor pedroigor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chicco785, it seems we have failures because of:

Unrecognized field "policy_endpoint" (class org.keycloak.authorization.client.representation.ServerConfiguration)

Maybe you missed changing ServerConfiguration to include policy_endpoint property?

@chicco785
Copy link
Contributor Author

chicco785 commented May 18, 2018 via email

@pedroigor
Copy link
Contributor

@chicco785, it seems it is still failing. If could make sure you can successfully run ResourceManagementWithAuthzClientTest locally, I think most of the failures will be gone.

@pedroigor
Copy link
Contributor

@chicco785, regarding roadmap, I will be including this change in our next sprint (which will start next week). More time to align and discuss the last bits before merging. Will spend more time next sprint looking this and #5211.

@chicco785
Copy link
Contributor Author

sorry, last 2 days have been at an event, so could work on this properly, will try to fix everything by monday EOB

@chicco785 chicco785 force-pushed the KEYCLOAK-7353 branch 3 times, most recently from 606588d to 677e0b1 Compare May 19, 2018 10:17
@chicco785
Copy link
Contributor Author

pff.... getting crazy with rebases XD

@chicco785 chicco785 force-pushed the KEYCLOAK-7353 branch 3 times, most recently from 99fcf25 to 6ec1599 Compare May 21, 2018 15:45
@chicco785
Copy link
Contributor Author

chicco785 commented May 21, 2018

@pedroigor the build should be now clean.

not 100% complete, and probably can be improved (apologies, but I haven't coded java in the last 10 years... and i am getting lost in all these novelties from Java 7 / 8 / 9)

how to proceed to integrate your PR? i think it would be good if my code leverage directly on that (I need to made few changes).

PS: currently, i decoupled completely the JSon representation from the "Inner" one. this because, i found quite complex to override the JSon getters and setters (e.g. I had issues in the change of type from Set<String> policies; to Set<UmaPolicyRepresentation> policies;) or remove them when not needed. It should be anyhow fine, given that this representation is purely to simplify the JSon and it is not persisted.

@chicco785
Copy link
Contributor Author

@pedroigor did cherry-pick of your commit and integrated on top of that

@chicco785 chicco785 force-pushed the KEYCLOAK-7353 branch 4 times, most recently from b75a0d7 to 3780e49 Compare May 22, 2018 10:25
@pedroigor
Copy link
Contributor

@chicco785, thanks for the updates. I've fixed the issue you pointed out in #5211. I would appreciate if you could rebase your branch again.

Will cherry-pick your changes now and perform some tests. Hope I can get the review done tomorrow.

@chicco785 chicco785 force-pushed the KEYCLOAK-7353 branch 2 times, most recently from 65b9031 to e36c70c Compare May 22, 2018 21:26
@chicco785
Copy link
Contributor Author

@pedroigor branch re-based

@pedroigor
Copy link
Contributor

pedroigor commented May 23, 2018

@chicco785, In general I think changes look good. Would also be nice if you could provide some tests, basically some test that invoke the new endpoint to manage UMA permissions.

I've also added some comments that represent some concerns about the new API design and implementation. Basically, I'm not if we need UmaPolicyRepresentation so we can simplify the both contract and implementation.

The endpoints in PolicyService is also missing the requester to which we are going to manage permissions to access a resource.

If you want me to cherry-pick your changes and implement what I have mind, we can then show you my proposal which is not very different to what you did, but just some adjustments.

@chicco785
Copy link
Contributor Author

@pedroigor where did you add the comments?

@chicco785
Copy link
Contributor Author

surely feel free to update my code :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chicco785, I'm not sure if we need this representation but just reuse the representations we have for each type of policy.

I think that the payload should change to something like this:

{
    "scopes": [
        "read"
    ],
    "policies": [
	{
            "type": "group",
            "groupsClaim": "groups",
            "groups": [
                {
                    "id": "abc",
                    "path": "abc",
                    "extendChildren": true
                }
            ]
        }
     ]
}

If we do that way, I think it is easier to map the different configuration options for each policy type as well avoid unecessary parsing from UmaPolicyRepresentation to a GroupPolicyRepresentation (considering the example above).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in this way we would not "simplify" the payload as discussed in first instance. i understand that this may simplify lot of coding, but may be less friendly from the point of view of creating policies. as you prefer.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure either :) Let me think a bit more about that, maybe we can come up with something even easier. For instance, if we are supporting user, role, group and client policies, we could just use something like:

{
    "scopes": ["read"],
    "users": ["a", "b"],
    "groups": ["c", "d"],
    "roles": ["e", "f"]
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

while at creation time many things can be implied, the internal representation is still richer, and some how we have to deal with that (or with assumptions, or with "optional fields").

currently, for example to share with more groups, you will create something like that via the api:

{
    "scopes": [
        "read"
    ],
    "policies": [
	{
        "type": "group",
        "subjects": [
            "a759a3a2-95b9-4f88-9c5e-c2b9ea2f1211",
            "d14d66e5-3341-439a-9da4-60944487a158"
        ]
    }
    ]
}

or for users

{
    "scopes": [
        "read"
    ],
    "policies": [
	{
        "type": "user",
        "subjects": [
            "a759a3a2-95b9-4f88-9c5e-c2b9ea2f1211",
            "d14d66e5-3341-439a-9da4-60944487a158"
        ]
    }
    ]
}

The response will include the enriched json (including optional parameters - but probably we can have a way to not return them, unless they are different from the "default"):

{
    "logic": "POSITIVE",
    "decisionStrategy": "UNANIMOUS",
    "scopes": [
        "read"
    ],
    "policies": [
        {
            "id": "94c61cd8-6d64-4e08-91e7-7066809db190",
            "type": "group",
            "logic": "POSITIVE",
            "decisionStrategy": "UNANIMOUS",
            "subjects": [
               "a759a3a2-95b9-4f88-9c5e-c2b9ea2f1211",
               "d14d66e5-3341-439a-9da4-60944487a158"
            ],
            "options": {
                "groupsClaim": [
                    "groups"
                ]
            }
        }
    ],
    "id": "d59d7e08-1448-4a65-894a-ee921987b348"
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'll agree with you. Let's keep this structure.

+1 to not return "options" if they are basically the defaults. And I think does not make sense to return "id" and "decisionStrategy". The "id" because if you want to update the "group" policy above we could just update the existing "group" policy, right ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we remove UmaPolicyRepresentation we probably don't need this method.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also avoid this method if we remove UmaPolicyRepresentation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we could just return the representation that we already have for each policy if we remove UmaPolicyRepresentation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is true, but current representation is very verbose, that was the reason to have an intermediary representation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you override getType you don't need to use setType here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To create a UMA permission/policy, you should first create a PermissionTicket and then mark it as granted. Once you do that, the policy will be created automatically so you could just update the policy associated with the ticket accordingly.

My PR already deals with the management of UMA permissions, so you just need to worry about creating permission tickets and grant/revoke/delete them accordingly in order to have the policy updated.

Don't we also need to know the requester to which the permission will grant access to the resource ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the idea is that uma-policy api allows to create policies from the owner point of view.
which means the requester is not needed. e.g.

  1. if owners of resource a create a policy, there is no an actual requester, but still, of course we can assume the requester to be the user to which the owner granted access.
  2. if the policy does not deal with authorising a user, but for example a role or a group, who should be the requester?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we have now two types of UMA permissions:

  • Those created through a permission ticket as part of an authorization flow using UMA
  • Those created by the resource server on behalf of an user

My main concern here is how we are going to handle permissions created by the resource server when the user is managing his permissions using Account Services ? How they will be revoked ? What will happen if you have a UMA permission created from a ticket and others created by the resource server ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that probably the solution is as follow:

  1. if user create a permission via ticket, when the permission is removed also the ticket is removed.
  2. if user modifies a permission created via a ticket, the change is rejected.

of course in case a permission is created directly using APIs, there is no issue in "revoking" them. right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My point is that the user won't be able to revoke permissions created using APIs if they are using Account Services. By Account Services I mean this https://www.keycloak.org/docs/latest/authorization_services/index.html#_service_authorization_api_aapi.

If you look that documentation you`ll see that we allow users to manage permissions to their resources through Keycloak Account Service. But there we only allow users to revoke the permissions created as consequence of approving/granting a permission ticket.

Makes more sense now ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand now, you are referring the user Account UI not "Service Accounts" auth flow :)
UI should also support management of other user defined policies.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah and that is a big thing, you know :)

And there are some things to consider that are not so simple or don't make sense:

  • How the UI will look like in order to allow users to manage these additional policies ?
  • Does it makes sense to allow users (we are talking about end-users) to manage role, group and client policies. Think about it, as a end-user I really don't need to know about roles in the application I'm using.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • client, i agree, it does not make sense in the ui.
  • role, may have, depending on the complexity of roles in your app.
  • groups, makes definitively sense.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why pass the policyId if user-managed resources have a single UMA permission associated with it ?

To update a UMA permission (considering you are creating accordingly with my comments to the create method) you basically need to grab the policy from any permission ticket with status granted.

Don't we also need to know the requester to which the permission will grant access to the resource ?

Copy link
Contributor Author

@chicco785 chicco785 May 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if i recall correctly from the gdoc, we agreed that for flexibility a resource could have more "permissions" associated. so that each permission in the end is evaluated like an "or".

Copy link
Contributor

@pedroigor pedroigor May 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, we can just update the "decision strategy" of the user-managed permission to "affirmative" as ti will work like an "or".

Copy link
Contributor Author

@chicco785 chicco785 May 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

correct, but then either we support (which we agreed to remove) aggregated policies, or we won't be able to say things as:

authorise if

  • permission1 user role A

    or

  • permission2 user role B and group Z

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is fine because you are creating a uma permission type. This type of policies should be associated with multiple "policies". To achieve the "or" behavior you just set the decision strategy to AFFIRMATIVE when creating the "uma" permission type.

In other words, you don't need to manage multiple permissions for a single resource, but just a single one by updating the scopes and associated policies.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mmm, i am confused now...

I tested the behaviour of your code.

  1. If user A ask for READ permission on resource Z of user B. A UMA permission is created attached to resource Z.
  2. If user C ask for WRITE permission on resource Z of user B. An additional UMA permission is created attached to resource Z.

so if i want to see, remove, or edit only one of the two created uma permissions for resource Z i need the id that identify such permission.

also, if we would have a single uma permission per resource, how we would be able to assign different scopes for the same resource to different users?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My changes are all about the "normal" UMA flow, where a requester is asking access to the owner of a resource.So, yeah, we have different UMA permissions for Resource Z each one specific for User A and User C. It is easier to identify those permissions, as you just need to know the resource and the requester, so you can obtain a ticket which is then associated with the UMA permission.

That is what I have commented here about adding the requester to your /uma-policy, so you could manage a specific policy for a specific requester/user. But that won't work for you, as you need to define additional policies to a resource that are not necessarily linked to a requester. For instance, if you define group or role policies.

Now think about this scenario. You have created a UMA permission using /uma-policy granting access to a specific role or group. Now you want to check if a user has access to the resource associated with this permission. As you know, you can ask for permissions using a ticket (UMA flow), using a permission request and passing the resource(s) or just obtain all permissions from the server. The two first methods are OK since you know the resource you need to check for permissions. But what about the last method ? Obtain all permissions from the server ? How we would perform this evaluation without iterating over all existing resources in the server ? That may impact performance ....

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. we can add requester for ticket permission generated UMA permissions as query parameter.
  2. we can use in a "tricky" way requester for the other policies. e.g. if it is not a uma permission, requester is used to find all the policy where related to "requesterId" via user, group or role.

Copy link
Contributor

@pedroigor pedroigor May 24, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I understand 1. And that would allow users to manage their policies created using UMA flow through a permission ticket (or sharing via Account Services).

But did not understand 2.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i proposed that to have a quick way from a ui to see which resources "using" uma a user is related to so that they can be displayed in the ui without obtaining all permissions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why pass the policyId if user-managed resources have a single UMA permission associated with it ?

Also when you delete a policy, does that means you are also revoking access to a requester ?

Copy link
Contributor Author

@chicco785 chicco785 May 23, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. see above
  2. if the policy is part of a ticket, then the ticket should be removed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also allow querying by requester. For example, give me all uma permissions associated with this resource and related with the requester XPTO.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. make sense in case the policy is originated from a PermissionTicket

@pedroigor
Copy link
Contributor

pedroigor commented May 24, 2018

@chicco785, could you take a look at my changes here https://github.com/pedroigor/keycloak/tree/KEYCLOAK-7353 ?

As an initial round of changes, what I did was to use a single UMA permission for a single resource, where this permission is defined with AFFIRMATIVE decision type. Note that the name of the permission is the resource id and the name is used as the key to query the permission from the database.

There is a test here that you can check this working pedroigor@89ab282#diff-4ea1fcdef84ba376fb4edbb21c46f5d0.

What do you think ?

@chicco785
Copy link
Contributor Author

@pedroigor
i have to test, but did you check the following flow:

  1. user create ticket permission, owner of the resource approve.
  2. owner of the resource add additional policy for the above resource (it will be attached to the uma policy created at 1.)
  3. ticket is cancelled (and should affect only policy associated with user request in 1), what happens to the policy created at step 2?

@chicco785
Copy link
Contributor Author

@pedroigor fyi following the change of the permssion ticket on your side, i had to introduce a change in the direct permission creation. you may need to rebase.

@pedroigor
Copy link
Contributor

pedroigor commented May 25, 2018

@chicco785, I've added a test for this scenario. What happens is:

  • Ticket is created, granted, and a UMA permission is created granting access to user.
  • User can now access owner's resource.
  • Owner creates an additional policy granting access to any users with role role_a using /uma-policy endpoint.
  • User can still access owner's resource. For two reasons: the owner granted access and he is granted with role_a.
  • Owner decides to revoke access to the user, ticket is cancelled and associated policy is removed.
  • User can still access owner's resource. He is granted with role_a
  • Owner decides to change (or delete) additional policy and allow access only to users granted with role_b.
  • User can no longer access the resource

See https://github.com/pedroigor/keycloak/blob/5986cdb6734b54f11a62ceb4a8fb2b42404aabff/testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/authz/UmaPolicyServiceTest.java#L149.

@pedroigor
Copy link
Contributor

pedroigor commented May 25, 2018

@chicco785, I think I have a plan to complete this spike:

  • Test different scenarios other than those you already suggested to make sure we are managing and evaluating permissions correctly. Hope we can now start writing bunch of tests (like the one I have started) considering these scenarios.
  • Make sure my changes to [KEYCLOAK-7367] - User-Managed Policy Provider #5211 are OK and merge ASAP. I think this PR provides a lot of improvements to UMA policies, not really related with this PR but with future improvements we can do like caching of decisions, additional config to UMA policies, etc.
  • Change Account Services to allow users to revoke UMA policies defined via /uma-policy endpoint. Now that we have only two types of UMA policies (those created via ticket and those created via uma-policy it will be easier to support that.
  • Make sure the uma-policy contract is stable and good enough to make it public.

I think that once we have these things done, we are OK to merge this PR. I hope we can achieve this next week. Until there we have a plenty of time to implement changes and fix issues.

What do you say ?

@chicco785
Copy link
Contributor Author

chicco785 commented May 25, 2018 via email

@pedroigor
Copy link
Contributor

@chicco785, I think we have to define some names for both policies: those managed via ticket vs those managed via user-policy endpoint :)

The ticket is associated with a UMA policy, but the policy originated from the ticket when it was granted. The "additional" policy is not associated at all with the ticket, just a "uma" policy type associated with the resource/scopes.

@chicco785
Copy link
Contributor Author

not sure i understood, so there will be more than 1 uma policy per resource:

  • 1 uma policy for each "ticket" (so n requests n uma policies)
  • 1 uma policy for all user created policies

@chicco785
Copy link
Contributor Author

as regards the plan, I can review your changes over the weekend. as regards working out tests and other things next week, I am quite packaged (big deadline for a customer next friday), things will be easier the following one. hope is not an issue.

@pedroigor
Copy link
Contributor

Yeah, that is correct. A policy originated from a ticket is limited to define access to a resource for a specific requester/user, we can't reuse it to also define "user created policies". The same is true for "user created policies", they are not really related with a specific requester, but with different subjects (user, group, role, client, etc) that can access a resource/scope.

@pedroigor
Copy link
Contributor

@chicco785, no problem. You are helping a lot and I appreciate your contribution. As I said, I can take some changes for me and you can just review. Most important thing is to get the requirements well set and understood. Think we have now a better understanding of these changes and what needs to be done.

@pedroigor
Copy link
Contributor

@chicco785, I was wondering something different about the payload of /uma-policy. Instead of using the current structure with subjects, options, etc, to represent the different policies we support from this endpoint, why not just use Javascript to write policies ?

I see some benefits using JS policies such as:

  • We already have an Evaluation API that exposes everything you need to write policies
  • You will be able to write more richer policies using different access control mechanisms
  • We don't restrict policies to be only role, group or client, but any of these or combination of different access control mechanisms, including ABAC (using attributes from identity and evaluation context)
  • Your policies could base their decisions based on claims pushed from applications (pushed claims)

Much more flexible and simple, don't you think ?

@chicco785
Copy link
Contributor Author

chicco785 commented May 29, 2018 via email

@pedroigor
Copy link
Contributor

pedroigor commented May 29, 2018

@chicco785, the owner would never need to worry about JS, but your application which is creating policies on behalf of its users, right ?

The JS code to check if an user has a role or a group is as simple as:

var context = $evaluation.getContext();
var identity = context.getIdentity();

if (identity.hasRealmRole('keycloak_user')) {
    $evaluation.grant();
}

Or:

var realm = $evaluation.getRealm();
if (realm.isUserInGroup('marta', 'Group C')) { 
    $evaluation.grant(); 
};

People writing these policies could even use the Policy Evaluation Tool in admin console to implement and test these policies.

My main points with this are:

  • From an API perspective, we can not limit what resource servers can do
  • We should be flexible enough to support different policy implementations, using different access control mechanisms
  • JS is a well-known language is easier for developers to write policies on top of it.

In fact, I think that API would be more simple because you don't need to worry about options for instance in cases you want to define a specific behavior to a role, group or client policy. And users can just send something like:

{
    "scopes": ["scope-a", "scope-b"],
    "description": "Grant access to X, Y, B",
    "policy": "// js code"
}

No need to add multiple policies as you can just use a single policy with different access control mechanisms (role + group + client + whatever).

@pedroigor
Copy link
Contributor

@chicco785, I've changed the payload to allow defining roles, groups, clients in addition to custom policies using JS. The structure now looks as follows:

{
    "scopes": ["scope_a", "scope_b"],
    "roles": ["role_a"],
    "groups": ["group_a"],
    "clients": ["client_a"],
    "condition": "// some code using JS and Evaluation API",
}

With this structure we are still keeping the API simple for common cases where roles, groups and clients are used to define policies. But you can also provide a condition if you want more fine-grained access control.

All those properties are optional. But at least one must be defined.

@chicco785
Copy link
Contributor Author

chicco785 commented May 30, 2018 via email

@pedroigor
Copy link
Contributor

pedroigor commented May 30, 2018

Cool. So I'm going to prepare a PR and start the review process. I think I can finish this until the end of the week. I need to add more tests.

Just not sure about how to proceed with the PR

Do you want to rebase with my branch once I finish all changes or close this one so I can send another PR (keeping your changes)?

@chicco785
Copy link
Contributor Author

chicco785 commented May 30, 2018 via email

@pedroigor pedroigor closed this Jun 1, 2018
@pedroigor
Copy link
Contributor

Closed in favor of #5240.

@chicco785, I've kept your commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants