Skip to content

Conversation

@Youngwb
Copy link
Contributor

@Youngwb Youngwb commented Mar 31, 2025

Why I'm doing:

##57024
Implement user identity impersonation for the Iceberg REST catalog using the OIDC token + Iceberg token exchange flow.
I have already conducted tests using OKTA + Nessie.

example:

CREATE EXTERNAL CATALOG `nessie`
PROPERTIES ("aws.s3.access_key"  =  "AK******CY",
"aws.s3.secret_key"  =  "Gf******hK",
"iceberg.catalog.oauth2.audience"  =  "localhost:19120/iceberg",
"aws.s3.enable_path_style_access"  =  "true",
"iceberg.catalog.client.region"  =  "us-west-2",
"iceberg.catalog.uri"  =  "http://127.0.0.1:19120/iceberg/main",
"iceberg.catalog.security"  =  "oauth2",
"iceberg.catalog.vended-credentials-enabled"  =  "false",
"type"  =  "iceberg",
"iceberg.catalog.oauth2.credential"  =  "xxx:xxxxx",
"client.factory"  =  "com.starrocks.connector.share.iceberg.IcebergAwsClientFactory",
"iceberg.catalog.oauth2.scope"  =  "catalog_read catalog_write",
"iceberg.catalog.oauth2.server-uri"  =  "https://xxxx.okta.com/oauth2/default/v1/token",
"aws.s3.region"  =  "us-west-2",
"iceberg.catalog.warehouse"  =  "warehouse",
"iceberg.catalog.type"  =  "rest"
)

What I'm doing:

  1. refactor connnector metadata, add paramter ConnectContext
  2. copy RESTSessionCatalog from iceberg 1.7.1 and made some modifications to be compatible with the OKTA token exchange flow.
   // 1. use basic auth (base64 encode credential) as Authorization header instead of bearer token
   // 2. do not add actor_token/actor_token_type in token exchange request if enable_actor_token is false
   // 3. do not refresh Authorization header when refresh token
  1. add oauth2.audience properties

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.5
    • 3.4
    • 3.3
    • 3.2
    • 3.1

@Youngwb Youngwb requested a review from a team as a code owner March 31, 2025 07:49
@Youngwb Youngwb changed the title Implement user identity impersonation for the Iceberg REST catalog using the OIDC token + Iceberg token exchange flow [Feature] Implement user identity impersonation for the Iceberg REST catalog using the OIDC token + Iceberg token exchange flow Mar 31, 2025
return icebergCatalog.tableExists(context, dbName, tblName);
}

@Override
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The most risky bug in this code is:
Inconsistent Use of ConnectContext in alterView

You can modify the code like this:

    Database db = getDb(context, stmt.getDbName());

return new SessionCatalog.SessionContext(sessionId, context.getQualifiedUser(), credentials, ImmutableMap.of(),
context.getCurrentUserIdentity());
}
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The most risky bug in this code is:
The convertContext method overwrites credentials in a way that might not include required keys if context.getAuthToken() is non-empty, potentially breaking authentication.

You can modify the code like this:

private SessionCatalog.SessionContext convertContext(ConnectContext context) {
    String sessionId = format("%s-%s", context.getQualifiedUser(), context.getSessionId());

    Map<String, String> baseCredentials = Maps.filterKeys(securityProperties.get(), key -> Set.of(TOKEN, CREDENTIAL).contains(key));
    
    Map<String, String> credentials;
    if (Strings.isNullOrEmpty(context.getAuthToken())) {
        credentials = baseCredentials;
    } else {
        credentials = ImmutableMap.<String, String>builder()
                .putAll(baseCredentials)
                .put(OAuth2Properties.ACCESS_TOKEN_TYPE, context.getAuthToken())
                .build();
    }

    return new SessionCatalog.SessionContext(sessionId, context.getQualifiedUser(), credentials, ImmutableMap.of(),
            context.getCurrentUserIdentity());
}

@Youngwb Youngwb linked an issue Mar 31, 2025 that may be closed by this pull request
@sonarqubecloud
Copy link

sonarqubecloud bot commented Apr 1, 2025

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@DorianZheng DorianZheng marked this pull request as draft April 1, 2025 06:17
@github-actions
Copy link

github-actions bot commented Apr 1, 2025

[Java-Extensions Incremental Coverage Report]

pass : 0 / 0 (0%)

@github-actions
Copy link

github-actions bot commented Apr 1, 2025

[FE Incremental Coverage Report]

pass : 107 / 126 (84.92%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 com/starrocks/qe/StmtExecutor.java 0 1 00.00% [973]
🔵 com/starrocks/connector/iceberg/glue/IcebergGlueCatalog.java 0 2 00.00% [120, 160]
🔵 com/starrocks/connector/iceberg/rest/OAuth2SecurityProperties.java 1 2 50.00% [43]
🔵 com/starrocks/connector/iceberg/rest/IcebergRESTCatalog.java 22 35 62.86% [127, 161, 168, 183, 218, 219, 230, 231, 233, 283, 319, 320, 321]
🔵 com/starrocks/connector/unified/UnifiedMetadata.java 2 3 66.67% [258]
🔵 com/starrocks/server/MetadataMgr.java 6 7 85.71% [403]
🔵 com/starrocks/connector/iceberg/IcebergCatalog.java 7 7 100.00% []
🔵 com/starrocks/connector/iceberg/rest/OAuth2SecurityConfig.java 6 6 100.00% []
🔵 com/starrocks/connector/iceberg/CachingIcebergCatalog.java 19 19 100.00% []
🔵 com/starrocks/server/LocalMetastore.java 10 10 100.00% []
🔵 com/starrocks/connector/iceberg/hive/IcebergHiveCatalog.java 1 1 100.00% []
🔵 com/starrocks/connector/iceberg/IcebergMetadata.java 19 19 100.00% []
🔵 com/starrocks/connector/iceberg/jdbc/IcebergJdbcCatalog.java 1 1 100.00% []
🔵 com/starrocks/connector/iceberg/IcebergAlterTableExecutor.java 1 1 100.00% []
🔵 com/starrocks/connector/CatalogConnectorMetadata.java 5 5 100.00% []
🔵 com/starrocks/connector/iceberg/hadoop/IcebergHadoopCatalog.java 1 1 100.00% []
🔵 com/starrocks/qe/DDLStmtExecutor.java 6 6 100.00% []

@github-actions
Copy link

github-actions bot commented Apr 1, 2025

[BE Incremental Coverage Report]

pass : 0 / 0 (0%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Impersonation for external catalog

1 participant