Ahmad Hesam (Jan 26 2024 at 14:32): Ahmad Hesam (Jan 26 2024 at 14:32):Hey everyone, at a particular Dataverse instance (https://ssh.datastations.nl) users are able to login with an institutional account (SURFconext, which supports both OIDC and SAML). For a service I'm developing for this DV instance I want to use this institutional account to make API calls for that user (mainly: does user have access to this data? If so, allow to download). I understand the user can do this himself using the API-key (generated via a webinterface).
I want to avoid my service having any knowledge of the API-key and instead use the (federative) institutional login to authenticate the service to fetch data from this DV instance on his behalf. AFAICT from the docs, this is not possible.
Two questions: 1) is there a way to do this currently in DV (have their been use cases before perhaps?), 2) If not, there must be some link between the institutional account <-> DV internal account. Is there a way to programmatically get a handle this?
Ahmad Hesam (Jan 26 2024 at 14:34):To give a bit of context: this service is a collaborative effort between the hosting DV party and the institute I work at. If there needs to be consensus on what this service is allowed to do on behalf of the user; that can be agreed upon
Philip Durbin 🚀 (Jan 26 2024 at 14:42):Hmm, would bearer tokens help here? https://guides.dataverse.org/en/6.1/api/auth.html#bearer-tokens
Philip Durbin 🚀 (Jan 26 2024 at 14:42):@Johannes D or @Oliver Bertuch know more about them than I do. :sweat_smile:
Johannes D (Jan 26 2024 at 14:49):Yap, that more or less our use case. We enabled only OIDC signup (ensure that each user is an OIDC user); enabled OIDC bearer tokens via feature flag and can happily interact with dataverse using access tokens from our IdP. The open challenges are signup (the user must create an account via dataverse UI), and propagation of property changes within the IdP (they are only read once during account creation...). Besides those issues it works really great :).
Ahmad Hesam (Jan 26 2024 at 14:58):Very cool, Johannes! That sounds very much like what I was looking for. So as long as the OIDC user has signed in once in Dataverse (i.e. registered) through the IdP, any service that support that same OIDC user can make requests to DV on behalf of that user?
Johannes D (Jan 26 2024 at 14:59):Yes.
Ahmad Hesam (Jan 26 2024 at 15:02):Cool! That's already very useful. Do you know if there as a way to make the registration part of the authentication process: "if a user doesn't already have a dataverse account, click here to make the link". That would save a user to separately go to the DV portal, log in again with OIDC and go back to the 'on-behalf service'
Johannes D (Jan 26 2024 at 15:19):You can use this API to create a user account . However, this call need admin rights and hence cannot be invoked by the user needing the account.
Johannes D (Jan 26 2024 at 15:22):@Philip Durbin : We could extend the BearerTokenAuthMechanism to create an account if not existing. Possible hiding the function behind a configuration flag. What do you think about that?
Philip Durbin 🚀 (Jan 26 2024 at 15:26):I like the idea of hiding it behind a feature flag for now. @Johannes D would you create a PR?
Johannes D (Jan 26 2024 at 15:33):Sure do, I have some free time this weekend :) Would that approach help you @Ahmad Hesam ?
Philip Durbin 🚀 (Jan 26 2024 at 15:33):I pointed Jim Myers to this thread and he indicated he's giving a talk at #community > #dataverse2024 in Mexico entitled "Using OIDC to Support Dataverse Integrations in a Research Commons Environment"
Philip Durbin 🚀 (Jan 26 2024 at 15:33):Here's the abstract: "Modern research environments, like research “commons”, allow users to combine data and compute resources to pursue their research goals. In these settings, the interaction between applications can occur asynchronously rather than only when the user is logged in. Dataverse’s API keys provide one mechanism for supporting asynchronous authentication by a trusted application to, for example, retrieve restricted data for use in a computation. However, API keys are non-standard and potentially insecure, and there is no automatic way to transfer a key. Offline tokens, as defined in the OpenID Connect (OIDC) standard, are a potential alternative. OIDC is one of the available options for authentication in Dataverse and is expected to become the preferred option when using the new single-page front-end. The Qualitative Data Repository (QDR) at Syracuse University already uses OIDC and the Keycloak OIDC Broker to support single-sign-on between Dataverse and QDR’s content management system. It is also now using OIDC’s offline token mechanism instead of API keys as a means of supporting long-term and asynchronous access to the QDR Dataverse instance from within a Gen3-based Data Commons environment developed to support the US National Institutes of Health (NIH) Helping to End Addiction Long-term® (HEAL) Initiative. Offline tokens allow a user to log into QDR from the HEAL platform and for the platform to request and store an offline token which can then be used as needed over time to allow the platform to access QDR data on behalf of the user. This talk will overview the OIDC offline token mechanism, describe how it has been configured and is being used by QDR to support HEAL use cases, and outline the usability and security benefits OIDC offline tokens have relative to API keys. We will also discuss how the combination of OIDC offline tokens and Dataverse’s ability to generate signed URLs for less trusted applications, may ultimately be a replacement for Dataverse’s API key mechanism."
Johannes D (Jan 26 2024 at 15:35):Good to here that more people are using the Tokens now!
Ahmad Hesam (Jan 26 2024 at 15:41):So just to be sure I understand the PR correctly: a Dataverse admin would be able to configure globally that any incoming requests through the OIDC bearer token will be executed, regardless if a user has an DV account, because the account will now be automatically created based on the user data coming from the IdP?
Johannes D (Jan 26 2024 at 15:45):Yes, the idea is: If a valid OIDC Token is presented to the system but no associated user account is present in dataverse (i.e valid IdP user that never used dataverse before). A dataverse account is generated for that users (identified by the Token) on the fly.
Ahmad Hesam (Jan 26 2024 at 15:47):Great, then we are talking about the same thing indeed! I believe the DV instance I work with also has a T&A that a user needs to agree with. Would this then implicitly be accepted? Or is there a way to forward this to the 'on-behalf service'?
Philip Durbin 🚀 (Jan 26 2024 at 15:53):Yes, every Dataverse instance has an application-wide terms of use and when you create an account via the web interface, you have to check a box to agree with it.
Johannes D (Jan 26 2024 at 15:57):Good point, implicit accepting the T&C isn't a good idea. This means the sketched idea does not work as the user must actively accept the T&C. Hence, we need another approach...
Ahmad Hesam (Jan 26 2024 at 16:01):Hmm there must be a way to direct the user to a T&C page before granting access to the API. It's similar to how I just made my Zulip account: Github asked me if I accept sharing information with Zulip (could've been a T&C) and I had to accept it before my Zulip account could be made
Philip Durbin 🚀 (Jan 26 2024 at 16:01):Well, that Create an Authenticated User API you pointed out also doesn't allow users to accept terms. It's morally equivalent.
Ahmad Hesam (Jan 26 2024 at 16:05):True.. although the registration request in that case would explicitly be coming from an admin if I understood Johannes correctly.
Ahmad Hesam (Jan 26 2024 at 16:07):Ideally ( I think) you'd want to keep a whitelist of approved services to auto-agree the T&C step on the user's behalf, with the trust that the service offers the T&C from their side to be accepted
Johannes D (Jan 26 2024 at 16:23):@Ahmad Hesam What you described is the [promt=consent](https://openid.net/specs/openid-connect-basic-1_0.html#RequestParameters). Maybe its a good idea to integrate the flow into the API usecase...
Ahmad Hesam (Jan 26 2024 at 16:29):Ah nice, this is something Dataverse could impose on the auto-registration step indeed. I'm just not clear on how this consent would manifest itself on the service side. Would the user be directed to a 'consent page' that contains the T&C statement and checkbox?
Philip Durbin 🚀 (Jan 26 2024 at 16:35):It must be a common story. Lots of services have some sort of T&C you must agree to.
Johannes D (Jan 26 2024 at 16:37):I never looked into this part, but this looks quite good. This at least means the user approved a screen in keycloack...
Ahmad Hesam (Jan 26 2024 at 16:46):What exactly is the role of keycloak in this case? Because keycloack can act as either an OIDC provider or as a kind of proxy to an OIDC client
Johannes D (Jan 26 2024 at 16:52):Sorry, we (my usecase) uses keycloak as a IdP in production. Similarly keycloak is used during testing/development of dataverse (see and here). Hence, I looked into the options of that ODIC implementation.
Ahmad Hesam (Jan 26 2024 at 16:56):Ah gotcha. But then wouldn't a solution to this T&C prompt have to work for other IdPs as well? Or does the OIDC standard impose the same behavior across various IdPs?
Ahmad Hesam (Jan 26 2024 at 16:59):If the latter is true, then I see a nice way to facilitate this flow :)
Johannes D (Jan 28 2024 at 18:02):I've played around with the code and it could be implemented, but it's not as easy as I originally thought.
However, we should not create an account without the explicit consent of the user.
The [OIDC spec] (https://openid.net/specs/openid-connect-basic-1_0.html#RequestParameters) includes `promt=consent' for these use cases. We can easily build authentication requests, including the Promt parameter, using the
of the builAuthzUrl method.
The user is redirected to the IdP, has to agree to the use of properties for the registered client and we are done. At least the user has actively agreed.
The tricky part is handling the IdP response, which is implemented here.
The code flow is bound to the JSF pages and shows a special login page. We want a different behaviour for the API.
I'm not sure how to do this without changing or duplicating too much code....
Johannes D (Jan 28 2024 at 18:06):@Oliver Bertuch I believe you have an idea :)
Johannes D (Jan 29 2024 at 08:57):Nevermind, I got a PoC working. . For the PoC I used a dedicated URL to trigger the signup process.
Johannes D (Jan 29 2024 at 09:06):PoC_OIDC_signup.mov
The code is here
Philip Durbin 🚀 (Jan 29 2024 at 13:15):Not too much code, really. Nice.
Philip Durbin 🚀 (Jan 29 2024 at 13:15):The video is great.
Philip Durbin 🚀 (Jan 29 2024 at 13:16):@Oliver Bertuch have you seen this?
Johannes D (Jan 29 2024 at 13:27):@Philip Durbin : Sadly it's duplicated and security relevant code. Therefore, we should refactor the codebase a bit to avoid the duplication.
Oliver Bertuch (Jan 29 2024 at 13:59):Great to see more activity around OIDC here!
Just a few comments:
1) Note that OIDC bearer tokens are hidden behind a feature flag at the moment. They are not considered mature enough for production but enable delivery of feature previews, use at your own risk.
2) It's nice to initialize the code auth flow using the API endpoint. Yet, I am not sure it fully covers the intended goal: an API endpoint is also intended to be used with a service not running in a browser, right? (For example: some Python script)
3) I like that this code extension allows skipping the signup form. This can probably be useful in the non-API-endpoint "user clicks on login link" scenario as well.
4) This should be coordinated or at least brought to the attention of @Guillermo Portas as this certainly is SPA relevant
Oliver Bertuch (Jan 29 2024 at 15:21):@Johannes D we just noticed in the docs that there are no docs for the API Bearer Token flag... Would you be able to provide a PR?
Johannes D (Jan 30 2024 at 07:10):@Oliver Bertuch : Concerning 2) It't is the best we can do. The only alternative in the OAuth2/OIDC context would be a device auth flow. Do we really expect API clients that cannot open a browser and do we wan't them to create user accounts? Besides, how can an API client authenticate users via OIDC if it cannot open a browser? The only options I know, are the device auth flow and resource owner credentials flow. The latter is deprecated and shall not be used according to this document. and the former requires a browser session potentially on another device.
Oliver Bertuch (Jan 30 2024 at 07:47):I agree - best effort we can make. Just saying we should manage expectations and document the limitations.
Last updated: Nov 01 2025 at 14:11 UTC