Scriptable DataSealerKeyStrategy Question

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Scriptable DataSealerKeyStrategy Question

Snyder, Bill

Hi,

 

I’m trying to backport the changes that are part of https://issues.shibboleth.net/jira/browse/JSPT-87 to the current release version with the goal of phasing this out once version 4.0 is released.  While I have to include the changes made in this commit (https://git.shibboleth.net/view/?p=java-identity-provider.git;a=commit;h=c235024c26e507a36baa12cbca1e1ba70da04636) I’m trying to avoid making any custom changes to the idp-conf/src/main/resources/system/conf/global-system.xml file as part of my solution.  Therefore, can someone please provide guidance on the best place to define the bean that needs to be specified as a property in idp-conf/src/main/resources/conf/idp.properties?  Along those lines, I’m assuming the class for that bean should be set to “net.shibboleth.utilities.java.support.security.impl.ScriptedKeyStrategy“ and that we should be passing in parameters to that bean in its definition.  I have looked but have not found an example of doing this (I have seen the test harness).  Am I misunderstanding how this is ultimately supposed to be used?  If not, is there a complete example somewhere showing the bean definition, the corresponding script, as well as where to place that script?

 

Thanks,

Bill


--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Scriptable DataSealerKeyStrategy Question

Cantor, Scott E.
On 10/15/19, 5:41 PM, "dev on behalf of Snyder, Bill" <[hidden email] on behalf of [hidden email]> wrote:

> I’m trying to avoid making any custom changes to the idp-conf/src/main/resources/system/conf/global-system.xml file
> as part of my solution.

It is impossible to do this without redefinition of all of the higher level beans up at least 1-2 levels from where this one is injected. It's a waste of time. I would pull in the changes from the master branch in this specific area and use the new property I defined for this purpose to do it, so the new bean can be defined by itself.

For the purposes of the intended audience (InCommon TAP container deployments), any future patches can be applied underneath a layered commit/change on top to keep it working until it's part of a release. I don't have any reason to think this specific change is going to be revisited unless it doesn't work properly, and that will be much more likely to be found out if it gets tried out now.

> Along those lines, I’m assuming the class for that bean should be set to
> “net.shibboleth.utilities.java.support.security.impl.ScriptedKeyStrategy“ and that we should be passing in parameters to
> that bean in its definition.  I have looked but have not found an example of doing this (I have seen the test harness).  Am
> I misunderstanding how this is ultimately supposed to be used?  If not, is there a complete example somewhere
> showing the bean definition, the corresponding script, as well as where to place that script?

I don't have an example, I will have to draft one tomorrow or so.

Scripts can go anywhere. I put mine in conf/ or in whatever folder the file referencing it is in, but it doesn't really matter to the code. I don't imagine a lot of people put things outside the main installation tree, but I'm sure some do.

-- Scott


--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Scriptable DataSealerKeyStrategy Question

Cantor, Scott E.
In reply to this post by Snyder, Bill
Untested example but should be close enough. There are some gaps in the c'tors to create the EvaluableScript objects that I'll patch now (e.g. using a File to build one requires specifying the script language also), but it's nothing fatal for the time being. I forgot that class was a bit low level, we haven't used it directly much.
 
<bean id="MyDataSealerKeyStrategy"
        class="net.shibboleth.utilities.java.support.security.impl.ScriptedKeyStrategy"
        p:updateInterval="%{idp.sealer.updateInterval:PT15M}">
        <property name="keyScript">
                <bean class="net.shibboleth.utilities.java.support.scripting.EvaluableScript">
                        <constructor-arg name="engineName">javascript</constructor-arg>
                        <constructor-arg name="scriptSource">
                                <bean class="java.io.File" c:pathname="%{idp.home}/conf/keyStrategyScript.js" />
                        </constructor-arg>
                </bean>
        </property>
</bean>

The idp.sealer.keyStrategy property should be used to enable it with the global-system.xml change that's necessary for this to be pluggable.

I think I described the script contract I defined before, but if I didn't please tell me and I'll put something together.

The best place to write something up on this is in the KB wiki space as a how-to.

-- Scott


--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Scriptable DataSealerKeyStrategy Question

Snyder, Bill
Hi Scott,

Thanks much for the example, this has proven quite useful.  A few things I've found...

If I include the p:updateInterval attribute on the bean, I get the following exception during servlet initialization:

Caused by: java.lang.IllegalStateException: Cannot convert value of type 'java.lang.String' to required type 'java.time.Duration' for property 'updateInterval': no matching editors or conversion strategy found

Also, after much testing over the last 36 hours, I'm not convinced that anything beyond a simple example can be used with the current solution.

First, I attempted to write the script using JavaScript.  The underlying JavaScript engine included with java (Nashorn) appears to be severely limited in its ability to import and use third party modules.  In my case, I need to be able to import the AWS JavaScript SDK.   I tried unsuccessfully to do this using the typical Node.js way (by leveraging https://github.com/nodyn/jvm-npm)  or by using the minified, stripped down version of the AWS SDK.  In both cases, it was unable to load the SDK.

After having exhausted that effort, I switched to using Jython.  While this initially seemed more promising, I've finally come to the conclusion that Jython as a scripting engine is quite limited.  Apparently there are 2 ways one can run a script in Jython.  The first is via calling exec() and the second is via calling eval() (as is done by the underlying EvaluableScript.java class).  The issues with these 2 are that the former returns "None" for all executions so there is no way to return data to Java and the latter is only intended to run expressions, not statements.  So, simple things like "2*3", not multi-line functions and programs. For more information about eval (and the limitations of using compiled code in eval), see the following 2 links:

https://docs.python.org/2/library/functions.html#eval
https://docs.python.org/2/library/functions.html#compile

While I'm certainly open to trying more things or other scripting engines, I fear that the flexibility/extensibility that the scripting based solution is intending to provide is actually making things more difficult for anything but the simplest use cases.  These issues, coupled with the fact that we still need to define our own bean inside of the global-system.xml file, lead me to believe that writing a custom java class that implements DataSealerKeyStrategy is much simpler and more straight forward.  If anyone has any thoughts of suggestion, I'm all ears.

Thanks,
Bill

On 10/16/19, 8:14 AM, "dev on behalf of Cantor, Scott" <[hidden email] on behalf of [hidden email]> wrote:

    Untested example but should be close enough. There are some gaps in the c'tors to create the EvaluableScript objects that I'll patch now (e.g. using a File to build one requires specifying the script language also), but it's nothing fatal for the time being. I forgot that class was a bit low level, we haven't used it directly much.
     
    <bean id="MyDataSealerKeyStrategy"
    class="net.shibboleth.utilities.java.support.security.impl.ScriptedKeyStrategy"
    p:updateInterval="%{idp.sealer.updateInterval:PT15M}">
    <property name="keyScript">
    <bean class="net.shibboleth.utilities.java.support.scripting.EvaluableScript">
    <constructor-arg name="engineName">javascript</constructor-arg>
    <constructor-arg name="scriptSource">
    <bean class="java.io.File" c:pathname="%{idp.home}/conf/keyStrategyScript.js" />
    </constructor-arg>
    </bean>
    </property>
    </bean>
   
    The idp.sealer.keyStrategy property should be used to enable it with the global-system.xml change that's necessary for this to be pluggable.
   
    I think I described the script contract I defined before, but if I didn't please tell me and I'll put something together.
   
    The best place to write something up on this is in the KB wiki space as a how-to.
   
    -- Scott
   
   
    --
    To unsubscribe from this list send an email to [hidden email]
   

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Scriptable DataSealerKeyStrategy Question

Cantor, Scott E.
On 10/17/19, 1:18 PM, "dev on behalf of Snyder, Bill" <[hidden email] on behalf of [hidden email]> wrote:

> If I include the p:updateInterval attribute on the bean, I get the following exception during servlet initialization:

The old code can't handle the newer Duration types. We probably used longs before with a @Duration annotation we defined to handle the conversions. I didn't consider that but it's a consequence of the backport.
 
> While I'm certainly open to trying more things or other scripting engines, I fear that the flexibility/extensibility that
> the scripting based solution is intending to provide is actually making things more difficult for anything but the
> simplest use cases.  These issues, coupled with the fact that we still need to define our own bean inside of the global-> system.xml file, lead me to believe that writing a custom java class that implements DataSealerKeyStrategy is much
> simpler and more straight forward.  If anyone has any thoughts of suggestion, I'm all ears.

If Jython's that broken that's definitely disappointing. I don't think anything related to node.js is relevant, but that doesn't mean there are workarounds there either. Nashorn is awful, and I tend to use Rhino, but I don't think it would make much difference. I only know how to reference Java code from them, not other javascript. I would definitely think that has to be embedded inline to work.

Java's fine, but that just means it will be a third party extension jar (and whatever other jars are needed) to add to the system. What people choose to add to their local deploys is not a concern of ours as long as they understand any conflicts introduced aren't things we can necessarily resolve.

In any case, that API is the same, AFAIK, between V3 and V4. There are issues such as the Duration thing that will be internal to the plugin, so I doubt it will be a final end state to have the same code working with both, but the changes would be pretty minor.

-- Scott


--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Scriptable DataSealerKeyStrategy Question

Snyder, Bill
I ended up going with the custom java class solution and it appears that things are working (at least to some degree).  To be clear, I've created a class that implements the DataSealerKeyStrategy interface as defined here:

http://git.shibboleth.net/view/?p=java-support.git;a=blob_plain;f=src/main/java/net/shibboleth/utilities/java/support/security/DataSealerKeyStrategy.java;hb=HEAD

I'm trying to test it thoroughly and could use some help understanding the lifecycle of cookies signed with the sealer key.  To test the IdP, I'm using the shibboleth SP and an LDAP identity store like I2 has published here:

https://github.internet2.edu/docker/shib-idp/blob/master/test-compose/

In order to make manual testing reasonable, I've gone ahead and tweaked the SP's session timeout to 15 seconds as is documented here:

https://wiki.shibboleth.net/confluence/display/SP3/Sessions

The behavior I see is that I can successfully get logged in at the IdP and redirected back to the SP.  If I keep refreshing the SP page inside of the 15 second timeout window, the page keeps reloading properly and no calls to the IdP are made.  This is the behavior I would expect.

If I let the page sit for longer than the 15 second timeout, the browser gets redirected back to the IdP which appears to successfully redirect the user back to the SP without requiring them to login again.  What is confusing me is that, in this situation, I would expect there to be a call to my class's implementation of the getKey() method, passing in the version id of the sealer key previously used, but the calls that are made are to the getDefaultKey() method.   Is this the expected behavior?

If I let things sit for a while (maybe hours), I will see calls to the getKey() method be made but, in this scenario, the user seems to always get redirected back to the login screen.  My goal is to be able to definitively test that I can keep X number of sealer key versions and have those sessions still work and also verify that once a sealer key version is "rotated out", that any sessions using that sealer key will be terminated.

I think I could use some help understanding how the sealer key lifecycle works and what timeout parameters (beyond the one mentioned above) I might play with to make testing this easier.  I could also use some help understanding when the getKey() and getDefaultKey() methods should be getting called.

Thanks,
Bill

On 10/17/19, 11:38 AM, "dev on behalf of Cantor, Scott" <[hidden email] on behalf of [hidden email]> wrote:

    On 10/17/19, 1:18 PM, "dev on behalf of Snyder, Bill" <[hidden email] on behalf of [hidden email]> wrote:
   
    > If I include the p:updateInterval attribute on the bean, I get the following exception during servlet initialization:
   
    The old code can't handle the newer Duration types. We probably used longs before with a @Duration annotation we defined to handle the conversions. I didn't consider that but it's a consequence of the backport.
     
    > While I'm certainly open to trying more things or other scripting engines, I fear that the flexibility/extensibility that
    > the scripting based solution is intending to provide is actually making things more difficult for anything but the
    > simplest use cases.  These issues, coupled with the fact that we still need to define our own bean inside of the global-> system.xml file, lead me to believe that writing a custom java class that implements DataSealerKeyStrategy is much
    > simpler and more straight forward.  If anyone has any thoughts of suggestion, I'm all ears.
   
    If Jython's that broken that's definitely disappointing. I don't think anything related to node.js is relevant, but that doesn't mean there are workarounds there either. Nashorn is awful, and I tend to use Rhino, but I don't think it would make much difference. I only know how to reference Java code from them, not other javascript. I would definitely think that has to be embedded inline to work.
   
    Java's fine, but that just means it will be a third party extension jar (and whatever other jars are needed) to add to the system. What people choose to add to their local deploys is not a concern of ours as long as they understand any conflicts introduced aren't things we can necessarily resolve.
   
    In any case, that API is the same, AFAIK, between V3 and V4. There are issues such as the Duration thing that will be internal to the plugin, so I doubt it will be a final end state to have the same code working with both, but the changes would be pretty minor.
   
    -- Scott
   
   
    --
    To unsubscribe from this list send an email to [hidden email]
   

--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Scriptable DataSealerKeyStrategy Question

Cantor, Scott E.
On 11/4/19, 4:31 PM, "dev on behalf of Snyder, Bill" <[hidden email] on behalf of [hidden email]> wrote:

> If I let the page sit for longer than the 15 second timeout, the browser gets redirected back to the IdP which appears to
> successfully redirect the user back to the SP without requiring them to login again.  What is confusing me is that, in this
> situation, I would expect there to be a call to my class's implementation of the getKey() method, passing in the version
> id of the sealer key previously used, but the calls that are made are to the getDefaultKey() method.   Is this the expected
> behavior?

Yes. The data is cached in the container session and is tagged with a simple version number (not the key version, it's a record version) that enables the code to detect that the data in the session cache is the same as the data in the client and doesn't need to be reloaded, so the read step is skipped. The data is generally updated each time and written back to the client under the default key with the updated record number, thus the calls to getDefaultKey() each time.

> If I let things sit for a while (maybe hours), I will see calls to the getKey() method be made but, in this scenario, the user
> seems to always get redirected back to the login screen.  My goal is to be able to definitively test that I can keep X
> number of sealer key versions and have those sessions still work and also verify that once a sealer key version is
> "rotated out", that any sessions using that sealer key will be terminated.

Clear the JSESSIONID cookie in between or restart the container and the read step will be forced back through the client and have to upload and decrypt the data again with whatever key was used.

Rotating out a key will not end any sessions, it simply makes them unsable by fiat, but as long as the sessions would have been expired anyway then the impact is the same.

Writes always happen under the default key, reads under whatever key was used.

Turn up logging to TRACE for some of the calling classes and you'll see more low level operations.

-- Scott


--
To unsubscribe from this list send an email to [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Scriptable DataSealerKeyStrategy Question

Snyder, Bill
Thanks much for the tips!  That's exactly what I needed to know so that I could fully test my solution.

Thanks,
Bill

On 11/4/19, 4:10 PM, "dev on behalf of Cantor, Scott" <[hidden email] on behalf of [hidden email]> wrote:

    On 11/4/19, 4:31 PM, "dev on behalf of Snyder, Bill" <[hidden email] on behalf of [hidden email]> wrote:
   
    > If I let the page sit for longer than the 15 second timeout, the browser gets redirected back to the IdP which appears to
    > successfully redirect the user back to the SP without requiring them to login again.  What is confusing me is that, in this
    > situation, I would expect there to be a call to my class's implementation of the getKey() method, passing in the version
    > id of the sealer key previously used, but the calls that are made are to the getDefaultKey() method.   Is this the expected
    > behavior?
   
    Yes. The data is cached in the container session and is tagged with a simple version number (not the key version, it's a record version) that enables the code to detect that the data in the session cache is the same as the data in the client and doesn't need to be reloaded, so the read step is skipped. The data is generally updated each time and written back to the client under the default key with the updated record number, thus the calls to getDefaultKey() each time.
   
    > If I let things sit for a while (maybe hours), I will see calls to the getKey() method be made but, in this scenario, the user
    > seems to always get redirected back to the login screen.  My goal is to be able to definitively test that I can keep X
    > number of sealer key versions and have those sessions still work and also verify that once a sealer key version is
    > "rotated out", that any sessions using that sealer key will be terminated.
   
    Clear the JSESSIONID cookie in between or restart the container and the read step will be forced back through the client and have to upload and decrypt the data again with whatever key was used.
   
    Rotating out a key will not end any sessions, it simply makes them unsable by fiat, but as long as the sessions would have been expired anyway then the impact is the same.
   
    Writes always happen under the default key, reads under whatever key was used.
   
    Turn up logging to TRACE for some of the calling classes and you'll see more low level operations.
   
    -- Scott
   
   
    --
    To unsubscribe from this list send an email to [hidden email]
   

--
To unsubscribe from this list send an email to [hidden email]