storing "data transparency" in consent records

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

storing "data transparency" in consent records

Peter Schober
More text with little dev content, sorry. Maybe user would have been a
better choice for this:

* Peter Schober <[hidden email]> [2018-05-25 07:18]:
> Are there any plans to provide a UI within the IDP to manage (i.e.,
> display and optionally revoke) previously created and still active
> consent records in whatever storage service is active? E.g.:

Why is that important to me now: I'm currently working on a clone of
the attribute-release intercept, for "informational / transparency"
purposes (i.e., it looks like consent but isn't, as the legal basis is
not the individual's consent. The subject is merely informed about
what's going on. Plus the fact that you could chose not to click "ok"
to continue provides an added failsafe the EU data protection
authorities call "unconditional opt-out" which may be part of what
makes global use of R&S legal, at least for EU/EEA-based institutions.)
The reason for that clone is to:
* use separate velocity templates for "inform" vs. attribute-release*;
* easily trigger "attribute-release" vs. "inform"
  postAuthenticationFlows in the relying party configuration.
  E.g. R&S and CoCo SPs get "inform" (RelyingPartyByTag), locally
  managed SPs may not get any intercept (RelyingPartyByGroup), and
  everything else gets the "attribute-release" intercept via the
  (unmodified) DefaultRelyingParty.

Now, using the attribute-release intercept (or a clone thereof) for
informational purposes is simple and adds the desired "memory" so that
people don't get "informed" on every single access to the same SP.

But if the IDP introduced a management UI for consent storage records
(think oauth grant revocation UIs) it would be undesirable to also
show those "consent records" that were the result of clicking "ok" on
my "inform" intercept flow -- it makes no sense to "revoke" the fact
that you have been informed and said "Thanks, I don't need to be
informed about this SP again in the future".

So it's with these thoughts that I'm pondering how best to treat these
two intercepts *right* *now* (with GDPR coming into effect today):
* Use RDBMS storage for consent and localStorage for inform?
  (But then if you have RDBMS storage to make consent Just Work across
  devices why bother with device-specific localStorage for the
  "inform" nag screens?)
* Use separate RDBMS tables for consent vs. inform (somehow)?
* Don't bother about this and let people "revoke" "consent", explaining
  in The Fine Local Documentation that this also covers "re-activating
  those friendly information screens" for R&S and CoCo services?

Any comments?
-peter
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: storing "data transparency" in consent records

Rod Widdowson
> So it's with these thoughts that I'm pondering how best to treat these
> two intercepts *right* *now* (with GDPR coming into effect today):
> * Use RDBMS storage for consent and localStorage for inform?
>   (But then if you have RDBMS storage to make consent Just Work across
>   devices why bother with device-specific localStorage for the
>   "inform" nag screens?)
> * Use separate RDBMS tables for consent vs. inform (somehow)?
> * Don't bother about this and let people "revoke" "consent", explaining
>   in The Fine Local Documentation that this also covers "re-activating
>   those friendly information screens" for R&S and CoCo services?

I'm taking this as read that this cannot be pushed back to the IDM?  It seems to fit the "admin-writes, user-reads" paradigm...

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: storing "data transparency" in consent records

Peter Schober
* Rod Widdowson <[hidden email]> [2018-05-25 11:21]:

> > So it's with these thoughts that I'm pondering how best to treat these
> > two intercepts *right* *now* (with GDPR coming into effect today):
> > * Use RDBMS storage for consent and localStorage for inform?
> >   (But then if you have RDBMS storage to make consent Just Work across
> >   devices why bother with device-specific localStorage for the
> >   "inform" nag screens?)
> > * Use separate RDBMS tables for consent vs. inform (somehow)?
> > * Don't bother about this and let people "revoke" "consent", explaining
> >   in The Fine Local Documentation that this also covers "re-activating
> >   those friendly information screens" for R&S and CoCo services?
>
> I'm taking this as read that this cannot be pushed back to the IDM?
> It seems to fit the "admin-writes, user-reads" paradigm...

Not sure I understand: How am I going to push the "this" you're
suggestiong to push back to the IDM if it's data the IDP's storage
service has put into the subject's HTML5 localStorage?
Please see my other mail "manage (view, revoke) consent in IDP" about
that, unless I'm mistaken.

The above was about either using only the "attribute-release"
intercept and adapting it dynamically to the "inform" vs. "consent"
use-cases (i.e., only one intercept at play, hence one storage
option), or cloning the attribute-release intercept into an "inform"
intercept (which is what I have done for now).  Either way, both mode
would use the same idp.consent.StorageService and so records would be
lumped together.
In that context I was wondering about consequences and alternatives.

I guess I could create a copy of the
shibboleth.consent.AttributeReleaseFlow bean I see in
system/conf/profile-intercept-system.xml and adapt that to an "inform"
flow, with a separate storage service configuration.

Now whether that effort is justified is an open question and relates
to the other question I've asked about potential (later) managing of
those consent records in a web UI -- if such a web UI was on the table
maybe I should make the effort to avoid lumping "inform" and "consent"
records in to the same storage.

-peter
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: storing "data transparency" in consent records

Cantor, Scott E.
In reply to this post by Peter Schober
> But if the IDP introduced a management UI for consent storage records (think
> oauth grant revocation UIs) it would be undesirable to also show those
> "consent records" that were the result of clicking "ok" on my "inform" intercept
> flow -- it makes no sense to "revoke" the fact that you have been informed and
> said "Thanks, I don't need to be informed about this SP again in the future".

I'm not sure I would say that's an obvious answer. I think it makes a lot of sense to treat them both as consent.

I'm not actually sure I get the difference. How are "inform" and "consent" different? By default, consent is not per-attribute, and you don't get control. You're just informed. You either approve or not. I don't yet follow how that's different than what you're suggesting you want to do.

> * Use RDBMS storage for consent and localStorage for inform?
>   (But then if you have RDBMS storage to make consent Just Work across
>   devices why bother with device-specific localStorage for the
>   "inform" nag screens?)

For sure.

> * Don't bother about this and let people "revoke" "consent", explaining
>   in The Fine Local Documentation that this also covers "re-activating
>   those friendly information screens" for R&S and CoCo services?

I'm just not getting the distinction. As long as you can tailor the text to suit your needs, which you can certainly do brute force and we could obviously make more clean if need be, how are these actually different models? If you're displaying the attribute data at all, that's just consent as the IdP thinks of it.

-- Scott

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: storing "data transparency" in consent records

Peter Schober
* Cantor, Scott <[hidden email]> [2018-05-25 19:28]:
> I'm not actually sure I get the difference. How are "inform" and
> "consent" different?

I agree:
https://lists.refeds.org/sympa/arc/attribute-release/2018-02/msg00000.html

> By default, consent is not per-attribute, and you don't get
> control. You're just informed. You either approve or not. I don't
> yet follow how that's different than what you're suggesting you want
> to do.

You'd ask for consent when it's the right legal basis (e.g. not for
R&S or CoCo SPs) and you'd avoid making it look like consent when
the subjects consent is not the jegal justification for processing.
(And I have not heared anywhere else that our current way of doing
it -- including not doing per-attribute consent -- would fail the
legal requirements. That may involve looking at the bigger context,
though, how we already pre-minimize attribute requirements during
registration to Identity Federations, etc.)

Now, completely indepedent from legal justifications there's a general
information duty in GDPR. In case of e.g. R&S or CoCo (where the legal
basis is not thought to be consent, but legitimate interests) trying
to satisfy that requirement may just happen to look very much like
consent, when it isn't. In the words of the CoCo 2.0 draft, lines 585f:

  No user consent is expected before release. (However, given how web
  browsers work, the user may have to click a CONTINUE button in order
  to continue in the sequence.)

So yeah. We must avoid legalese and provide clear and simple language
to explain whats going on, but we have to also provide interfaces for
2 very different legal purposes that will end up looking to much alike
it's doubtful we can pull off making the difference simple and clear.


> I'm just not getting the distinction. As long as you can tailor the
> text to suit your needs, which you can certainly do brute force and
> we could obviously make more clean if need be, how are these
> actually different models? If you're displaying the attribute data
> at all, that's just consent as the IdP thinks of it.

I tend to agree, which is why one attempt of mine essentially copied
the attribute-release flow but used the same storage configuration, so
even there to the IDP it's the same thing.

The alternative I haven't yet explored in practice (but will be doing
so next, and I'll move that to the users list where it belongs) would
be labelling e.g. CoCo and R&S services (maybe the way Keith has done
here, http://shibboleth.net/pipermail/users/2018-February/039201.html)
and then exposing that info to the velocity templates and handling the
differences between "consent" and "inform" solely with if/elseif/else
conditionals.
That may be simpler for deployments, or it may not (no new flows, but
still new beans to be defined, so maybe same difference, effort wise).
Doing it all in one flow may be simpler to understand/style (only one
velocity template, so UI improvements done to one case also apply to
the other), or may make things less easy for designers etc, having to
handle the conditionals with one template.

-peter
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: storing "data transparency" in consent records

Mike Schwartz
Did you look at the Kanata consent receipts standard? It could help
define the interface to the consent record system.


https://kantarainitiative.org/confluence/display/infosharing/Consent+Receipt+Specification

Mike


------------------------
Michael Schwartz
Gluu
Founder / CEO
[hidden email]
https://www.linkedin.com/in/nynymike/
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: storing "data transparency" in consent records

Cantor, Scott E.
On 5/26/18, 12:02 PM, "dev on behalf of Mike Schwartz" <[hidden email] on behalf of [hidden email]> wrote:

> Did you look at the Kanata consent receipts standard? It could help
> define the interface to the consent record system.

Is there evidence of any uptake? Defining things is easy, getting somebody to use them is something else. I have some experience in that area.

-- Scott


--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: storing "data transparency" in consent records

Cantor, Scott E.
In reply to this post by Peter Schober
On 5/26/18, 8:53 AM, "dev on behalf of Peter Schober" <[hidden email] on behalf of [hidden email]> wrote:

> You'd ask for consent when it's the right legal basis (e.g. not for
> R&S or CoCo SPs) and you'd avoid making it look like consent when
> the subjects consent is not the jegal justification for processing.

To me that's UI. I get it, but I would *hope* that it's doable within the existing flow if we don't get too caught up in the internals. That's really what I meant: tweaking the verbage in specific cases should be well within our capabilities.

> So yeah. We must avoid legalese and provide clear and simple language
> to explain whats going on, but we have to also provide interfaces for
> 2 very different legal purposes that will end up looking to much alike
> it's doubtful we can pull off making the difference simple and clear.

Probably so, but I don't think we need to contaminate the technical layer with that problem if we can avoid it.

> I tend to agree, which is why one attempt of mine essentially copied
> the attribute-release flow but used the same storage configuration, so
> even there to the IDP it's the same thing.

I was just suggesting condtional logic in the templates, but you basically said that's your alternative, so yes.

> That may be simpler for deployments, or it may not (no new flows, but
> still new beans to be defined, so maybe same difference, effort wise).

I think it will be simpler in practice but if there's something else we need to do, we can do it. If it's just a case of adding some legalistic metadata so to speak to the storage record, no big deal.

-- Scott


--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: storing "data transparency" in consent records

Peter Schober
* Cantor, Scott <[hidden email]> [2018-05-28 17:29]:
> I was just suggesting condtional logic in the templates, but you
> basically said that's your alternative, so yes.

Right, so I'll follow up on the users list about how to label entities
with e.g. "isRandS" or "infoOnly" or whatever (possibly following the
example Keith shared a while back) and then look at getting at those
variables from within velocity (possibly via springConext).
Metadata-driven config flags will heavily factor into this, I guess.

> > That may be simpler for deployments, or it may not (no new flows, but
> > still new beans to be defined, so maybe same difference, effort wise).
>
> I think it will be simpler in practice but if there's something else
> we need to do, we can do it. If it's just a case of adding some
> legalistic metadata so to speak to the storage record, no big deal.

Amending the existing storage record (to differentiate between, say,
"consent" and "transparency duty") would only be necessary if a UI
were available, or if the APIs existed to access those records to
build such a UI.
(I doubt someone will implement a Consent Manager based on an
decidedly undocumented format.)

-peter
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: storing "data transparency" in consent records

Cantor, Scott E.
> Right, so I'll follow up on the users list about how to label entities with e.g.
> "isRandS" or "infoOnly" or whatever (possibly following the example Keith
> shared a while back) and then look at getting at those variables from within
> velocity (possibly via springConext).
> Metadata-driven config flags will heavily factor into this, I guess.

Typically what I would do is define beans based on the EntityAttributePredicate class in OpenSAML that know how to evaluate the tags, and then create different ones to check for the different conditions involved and inject them via the custom object hook we have for all the templates.

Then you just have to do something like #if ($custom["RSCondition"].apply($rpmd))

I don't know offhand whether the RP metadata is right in the template or not, but it's accessible without much work.

> Amending the existing storage record (to differentiate between, say, "consent"
> and "transparency duty") would only be necessary if a UI were available, or if
> the APIs existed to access those records to build such a UI.
> (I doubt someone will implement a Consent Manager based on an decidedly
> undocumented format.)

No, but if people want to record something new now with what's already built, it would be there later for reporting.

-- Scott

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: storing "data transparency" in consent records

Tom Zeller-3
In reply to this post by Peter Schober
> if the APIs existed to access those records

Well, just noting that Java APIs exist in the IdP (actually OpenSAML)
to access consent (or any kind of) StorageRecords. I imagine a
REST-ish profile flow could be written to access them as well, but
that assumes the records are stored server-side, which is not the
IdP's default. Sometimes I think that consent records should be stored
both client-side and server-side in order to facilitate a UI/UX as
well as to follow the user across devices. But I guess one issue there
would be syncing the records.

Tom
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: storing "data transparency" in consent records

Cantor, Scott E.
> > if the APIs existed to access those records
>
> Well, just noting that Java APIs exist in the IdP (actually OpenSAML) to access
> consent (or any kind of) StorageRecords. I imagine a REST-ish profile flow could
> be written to access them as well, but that assumes the records are stored
> server-side, which is not the IdP's default.

There's already a REST API to the storage layer, but the format of the records for consent in that layer are the undocumented part, as are all the bits we store there.

-- Scott

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: storing "data transparency" in consent records

Tom Zeller-3
> There's already a REST API to the storage layer,

Oh, right, I forgot. That also facilitates OIDC.

> but the format of the records for consent in that layer are the undocumented part

Sounds like I should add some documentation.

Tom
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: storing "data transparency" in consent records

Tom Zeller-3
In reply to this post by Mike Schwartz
> Did you look at the Kanata consent receipts standard?
>
> https://kantarainitiative.org/confluence/display/infosharing/Consent+Receipt+Specification

Thanks for that link. My first thought was : "it won't fit in a
cookie" (which is the IdP's default client-side storage mechanism).
Maybe a minimum set of JSON fields would.

Tom
--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: storing "data transparency" in consent records

Cantor, Scott E.
In reply to this post by Tom Zeller-3
> Sounds like I should add some documentation.

No, it's deliberately undocumented because then we're free to change it.

-- Scott

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: storing "data transparency" in consent records

Peter Schober
In reply to this post by Cantor, Scott E.
* Cantor, Scott <[hidden email]> [2018-06-01 18:47]:
> Typically what I would do is define beans based on the
> EntityAttributePredicate class in OpenSAML that know how to evaluate
> the tags, and then create different ones to check for the different
> conditions involved and inject them via the custom object hook we
> have for all the templates.

OK, so following
https://wiki.shibboleth.net/confluence/display/IDP30/VelocityVariables
and the example provided in
https://wiki.shibboleth.net/confluence/display/IDP30/Logging+Inside+Views
and using the bean defintion shared by Keith in
http://shibboleth.net/pipermail/users/2018-February/039201.html
... I added the following to conf/global.xml:

<util:map id="shibboleth.CustomViewContext">
    <entry key="RandS">
        <bean class="org.opensaml.saml.common.profile.logic.EntityAttributesPredicate">
            <constructor-arg>
                <list>
                    <bean class="org.opensaml.saml.common.profile.logic.EntityAttributesPredicate.Candidate"
                        c:name="http://macedir.org/entity-category"
                        p:values="http://refeds.org/category/research-and-scholarship" />
                </list>
            </constructor-arg>
        </bean>
    </entry>
    <entry key="CoCo">
        <bean class="org.opensaml.saml.common.profile.logic.EntityAttributesPredicate">
            <constructor-arg>
                <list>
                    <bean class="org.opensaml.saml.common.profile.logic.EntityAttributesPredicate.Candidate"
                        c:name="http://macedir.org/entity-category"
                        p:values="http://www.geant.net/uri/dataprotection-code-of-conduct/v1" />
                </list>
            </constructor-arg>
        </bean>
    </entry>
</util:map>

> Then you just have to do something like #if ($custom["RSCondition"].apply($rpmd))
>
> I don't know offhand whether the RP metadata is right in the
> template or not, but it's accessible without much work.

I wasn't able to esily find anything on 'rpmd', so I added the code
from the example provided here to my template:
https://wiki.shibboleth.net/confluence/display/IDP30/VelocityVariables#VelocityVariables-LocatingtheOpenSAMLEntityDescriptorfortheRelyingParty

  #set ($outboundContext = $profileRequestContext.getOutboundMessageContext())
  #set ($samlPeerContext = $outboundContext.getSubcontext('org.opensaml.saml.common.messaging.context.SAMLPeerEntityContext'))
  #set ($metadataContext = $samlPeerContext.getSubcontext('org.opensaml.saml.common.messaging.context.SAMLMetadataContext'))
  #set ($spEntity = $metadataContext.getEntityDescriptor())

and then was able to check for RandS (or CoCo, respectively) using
your suggested example:

  #if ($custom["RandS"].apply($spEntity))
  #end

Amazingly that all worked, esp considering how little (if anything) I
understand about Spring and beans and all that.

I also think there could be easier ways to get there, I suggested one
idea here though that may not be the easiest yet:
https://issues.shibboleth.net/jira/projects/IDP/issues/IDP-1301

Thanks for the hand-holding,
-peter
--
To unsubscribe from this list send an email to [hidden email]